Files corrupted during BatchPreProcessingFD

[ColinR]

Hi, I hope somebody can help me solve this unusual problem!

I processed 30 lights, (including darks, flats and dark flats), using BatchProprocessingFD. Before uploading these 30 lights they were in two separate folders of 15 files each, but with different prefixes as SharpCap always returns to 0001 when starting a new imaging run and therfore new folder.

BatchPreprocessing FD failed at the start of Integration of Light Frames, with the a dialog box "Error in Intergrating Light Frames".
Examination of the Process Console showed the following:
"Zero or insufficient signal detected (empty images?)"
After investigation if was clear the 15 files from one folder had no problems and 15  files in the other folder all had the problem!

Both folders contained identical looking acceptable single images and identical FITs headers. Both set were taken within half and hour of each other. The "corruption" took place at Calibration. Please see the images attached.

Finally I continued to BatchPreprocessFD with the remaining 15 good light files, (and the master dark, master flat and master dark flat successfully process from the BatchPreprocessingFD run that failed), with no problems!

Baffled

Any help would be greatly appreciated - If I don't know what caused this I cannot be sure it won't happen again.


Colin   

[bulrichl]

Hi Colin,

screenshots will not help in this case.

Perhaps you can upload an uncalibrated frame from each folder plus the MasterDark?

Bernd

[bulrichl]

Hi Colin,

the two light frames (OK_File: 14.5 °C, exposure time: 100 s and Not_OK_file: 14.1 °C, exposure time 100 s) are not corrupted. In the histogram, the peak widths are quite similar, but the peak of the "Not_OK_file" is shifted by about 500 ADUs towards lower intensity. This is visible also in Statistics (take a look at the mean and median values):

            OK_File      Not_OK_file      dark_BINNING_1_EXPTIME_100_integration
count (%)   100.00000    100.00000        100.00000
count (px)  19845120     19845120         19845120
mean        1524.1       1028.6           1160.411
median      1536.0       1040.0           1146.667
variance    194954.3     202463.8         33695.827
stdDev      441.5        450.0            183.564
avgDev      178.1        183.1            29.459
MAD         142.3        166.1            12.651
minimum     656.0        160.0            848.000
maximum     65504.0      65504.0          65504.000

The MasterDark seems to be appropriate for the "OK_File", but not for the "Not_OK_file". In the latter case, severe clipping will occur when ImageCalibration is performed.

Unfortunately, the capturing software (in this case SharpCap) didn't populate much FITS keywords, neither gain nor offset values are stored in the FITS header. I can only imagine that the two sets of light frames were captured at different offset values, so the darks suit to one series of light frames and not to the other. It is critical to capture light frames and dark frames with the same camera settings (gain, offset, temperature and exposure time).

Regarding temperature: is this a cooled camera and you didn't turn on the TEC or is this a camera without cooling?

Bernd

[ColinR]

Bernd,
Once again, thanks massively for you help. What I think may have happened, (however, see "Finally" below), is that somehow or other I have accidently shifted or not noticed the shift in gain or off-set, or black point it think it is called in SharpCap.  So if that is the case I will have to be more observant of settings between imaging runs. (I like to periodically check focus hence the 25 to 30 minutes or so maximum imaging runs.) 

I'm just getting into understanding ADUs / unity gain etc - gain 400 is unity I believe for a 183c camera. 500 ADU's seems  big difference between the two imaging runs. Maybe I'll request Dr Robin Glover to consider gain and off-set in the FITs headers of SharpCap!

Also, thanks for reminding me of the Statistic process, although I have an engineering based background in statistics, !would not have realised that the answer lay in these numbers. Just to note that these images are 12 bit. But I suspect Ihat the message is the same! So for my own learning I take it that the mean/median of an image should be higher than the mean/median of the dark master?

As for the sensor temperature, yes I have the 183C Pro TEC, however, for the following reasons I chose not to turn it on, (normally I set it to -20C);

1. I have recently moved house and I have not yet installed mains power to my back garden. I was on battery power and chose not to turn the cooling system on to maximise available time.
2. The primary purpose of this imaging session was to test out my new PrimaLuceLab Sesto Senso robotic focuser and to grapple with SharpCap Pro's excellent Smart Histogram and Brain capabilities. I chose M31 core because it was quick and easy to find.

Finally, the Not OK frames were taken before to OK frames? Hope that it is not a fault in the camera. More likely a fault in the operator!     

So thanks again - I take my hat off to you depth of knowledge.

Colin