Exposure Duration Doesn't Appear in Dark master

rgsalinger

Member
Mar 10, 2018
7
0
After creating a dark master the resulting xisf file does not appear to have the duration or the temperature in the header. I am using the FITS header process as I can't find an xisf header display process. I must be doing something wrong but making a dark master is so trivial that I just can't find my error. Any ideas of what mistake I'm making?
 

Juan Conejero

PTeam Member
Sep 2, 2004
7,595
103
56
Valencia, Spain
pixinsight.com
As for sensor temperatures, this should only happen if the individual dark frames have either different values of sensor temperature in their metadata, or no metadata at all. For FITS, our ImageIntegration tool supports the CCD-TEMP header keyword, so if your acquisition software writes different keywords, here is the reason. Typical FITS interoperability problem.

For exposure times, we support the EXPTIME and EXPOSURE keywords, which are used by all acquisition applications as far as I know. Exposure times are added for all integrated frames and the integrated image receives an EXPTIME keyword with the total sum. If this does not happen in your case, then the only reason I can figure out is that there are no valid EXPTIME or EXPOSURE keywords in your frames, which would be weird.

To help you further, if possible, we need a sample of the dark frames you are integrating.
 

bulrichl

PTeam Member
Nov 2, 2016
855
65
La Palma, Canary Islands
I am using the FITS header process as I can't find an xisf header display process.
The data in the XISF header can be viewed with PixInsight's View Explorer, section 'properties'.


It would be helpful if you specified the camera model and the file format (proprietary raw format of a regular camera or FITS format).

I integrated some FITS files (same exposure time: 0.0055 s) with the 'EXPOSURE' FITS keyword. The integration result did not contain a FITS keyword 'EXPTIME'. I also integrated some CR2 files (same exposure time: 302.0 s). The integration result did not contain a FITS keyword 'EXPTIME' either. So it is true that the current ImageIntegration process (module version: 1.2.33) does not carry on the exposure time to the integration result.

Regarding temperature, Juan has answered your question for files in FITS format.

For files in a proprietary raw format of a regular digital camera, the camera temperature is recorded in the EXIF metadata of some camera models. It is not trivial to extract the temperature from the EXIF metadata because no standard EXIF tag ID exists for temperature, i.e. different camera makes and even different camera models can have different EXIF tag IDs for temperature. Currently PixInsight does not support this feature.

Bernd
 
Last edited:

Juan Conejero

PTeam Member
Sep 2, 2004
7,595
103
56
Valencia, Spain
pixinsight.com
Oh, sorry, I've given wrong information in my previous post regarding exposure time metadata in integrated images. Exposure time metadata is now not generated in the integrated result. We decided to remove it because it is wrong: we cannot assume that an integrated image is equivalent to a single image acquired with the sum of exposure times in the integrated subframes. So an EXPTIME keyword or an Instrument:ExposureTime property in the integrated image would be misleading.

We record the starting and ending times of acquisition in the integrated image as Observation:Time:Start and Observation:Time:End properties, as well as DATE-OBS and DATE-END keywords, respectively. The values of these items are gathered from the set of integrated subframes, and the ending time of acquisition includes the exposure time of the last acquired frame.

Sorry for my mistake forgetting recent changes. Too many things to keep in my brain at the same time...
 

airscottdenning

Active member
Oct 26, 2012
26
1
I noticed this too. For calibration frames (e.g., flat frames), it's important to preserve EXPTIME in the integrated result in order to match other frames (e.g., flat darks).

You can easily add your own keyword via the File->FITS Header dialog. Or you can just be super careful to include this information in the filenames.

It does seem weird that with all the HISTORY being added to integration files, such a basic piece of information is being discarded.
 

Juan Conejero

PTeam Member
Sep 2, 2004
7,595
103
56
Valencia, Spain
pixinsight.com
It is being discarded by the ImageIntegration tool because generating it on a regular basis would be wrong. An integrated image is not equivalent to an image acquired with an exposure time equal to the sum of exposure times in the integrated subframes, or, in case all integrated subframes have the same exposure time, equal to that exposure time. In both cases an EXPTIME keyword or an Instrument:ExposureTime property would be wrong and misleading. I have explained this in my previous post above.

However, for master frames the EXPTIME keyword (not the Instrument:ExposureTime property, since its definition in the XISF standard forbids this use) is being generated by the WeightedBatchPreprocessing script for convenience (although it should not be generated for integrated light frames, IMO). For a master dark or flat frame generated manually you can encode exposure times in file names (as WBPP does for example) and/or in other metadata items that you can define ad-hoc for your specific needs, including a (loosely defined, strictly wrong) EXPTIME keyword.

We have already been considering new XISF properties such as Integration:TotalExposureTime and Integration:FrameExposureTime, which could be included in a future revision of the XISF standard.
 

rgsalinger

Member
Mar 10, 2018
7
0
Well, what's confusing me is that I understand that I need to match all exposure parameters when calibrating my light frames. I have a QHY600 and I have found that trying to just use a long exposure master dark does not result in proper calibration. So, I take care to match my dark frame duration. All well and good. What I don't understand is why I'm getting that warning message and whether or not it matters. I just integrated a stack this AM with WBBP and the results look really nice which is good. I'm just wondering if my "zero" exposure dark master is actually doing anything. The camera is so clean that sometimes I wonder about taking darks at all.
 

rgsalinger

Member
Mar 10, 2018
7
0
Warning message is probably not the correct term of art. When I add the master dark what happens is that the duration is shown as 0.00s. That means that PI doesn't think that it is the correct dark to use with my (in this case) 300 second light frames. When I click on "Diagnostics" in WBPP I get this message


** Warning: Light frames (filter = BLUE, binning = 2, exposure = 300.00s, length = 6) will be calibrated using a master dark with a different exposure of 0 sec.

So, what I'm trying to figure out is whether or not this actually matters. I have to say that I'm really happy with my results but the QHY600 camera has super low noise and between that and drizzling, it's not clear that I even need dark frames.

Rgrds-Ross
 

bulrichl

PTeam Member
Nov 2, 2016
855
65
La Palma, Canary Islands
In PixInsight, warnings shall point to a potential problem. If you are convinced that what you are doing is correct (in this case: you compared the dark and the bias and don't see a significant difference), you can always ignore a warning.

Bernd