Script: BasicCCDParameters, Flats Subs

F

Farzad_k

Well-known member
Hi.

When Flats are used to make corrections to images there are many rules here and there on median pixel value of the Flat image and also a minimum length of exposure time for capturing those flats because I think in parts we don't want Flats captured as fast as a bias frame may be captured. I don't know how true all this is but my question is about quality of Flat frames in order for them to be useful in determining a sensor's parameters using the subject script.

Considering these Flats will not be used for calibration of an image, would it be sufficient if they are captured with a histogram being kind of to the left?

Does anyone know what role the Flats play in calculating sensor properties?

Thanks in advance.

Farzad
 
there are 2 reasons for avoiding short flat exposures. one is shutter effects - some cameras have mechanical shutters which do not evenly illuminate the sensor as they open and close, so too short of an exposure can leave a shutter image artifact in the flat. the other is that if you are using a light panel that flickers for whatever reason (60Hz bulb or ~20KHz LED PWM) the panel itself can create artifacts unless the shutter is left open long enough to average those out.

as for the minimum length, that's really about making sure the flats are "well exposed", meaning that enough light was captured so that the SNR of the flat sub is reasonably high, and also that the flat duration was not so long that the sensor departs the linear range. (for most of the pixel well depth, the ADU reported by the A/D converter is linearly correlated with the number of photons that hit the detector. but there is a point after which the ADU reported is no longer linearly correlated with the # of photons.) this is important for properly correcting the image (see below). if the flat is overexposed the corrections will not work properly, since the lights are almost always within the linear range of the sensor (with the exception of stars which tend to be overexposed, but that's OK cause the true data is lost anyway.)

in calibration, the flat has 2 purposes - to eliminate vignetting and dust shadowing effects, and to normalize the quantum efficiency of all the pixels. each pixel responds to light very slightly differently. the QE differences are probably not important for pretty pictures but important in scientific work like spectroscopy and maybe exoplanet transits.

i think for the purposes of the script you should make sure the flats are appropriate for calibration, meaning you should make sure they are artifact free and linear as described above.

rob
 
pfile said:
there are 2 reasons for avoiding short flat exposures. one is shutter effects - some cameras have mechanical shutters which do not evenly illuminate the sensor as they open and close, so too short of an exposure can leave a shutter image artifact in the flat. the other is that if you are using a light panel that flickers for whatever reason (60Hz bulb or ~20KHz LED PWM) the panel itself can create artifacts unless the shutter is left open long enough to average those out.

as for the minimum length, that's really about making sure the flats are "well exposed", meaning that enough light was captured so that the SNR of the flat sub is reasonably high, and also that the flat duration was not so long that the sensor departs the linear range. (for most of the pixel well depth, the ADU reported by the A/D converter is linearly correlated with the number of photons that hit the detector. but there is a point after which the ADU reported is no longer linearly correlated with the # of photons.) this is important for properly correcting the image (see below). if the flat is overexposed the corrections will not work properly, since the lights are almost always within the linear range of the sensor (with the exception of stars which tend to be overexposed, but that's OK cause the true data is lost anyway.)

in calibration, the flat has 2 purposes - to eliminate vignetting and dust shadowing effects, and to normalize the quantum efficiency of all the pixels. each pixel responds to light very slightly differently. the QE differences are probably not important for pretty pictures but important in scientific work like spectroscopy and maybe exoplanet transits.

i think for the purposes of the script you should make sure the flats are appropriate for calibration, meaning you should make sure they are artifact free and linear as described above.

rob
Click to expand...

Thanks for the explanation. I am not sure about the last statement: "artifact free". Flats will show vignetting and shadows associated with dust motes, etc. I am not sure what other artifacts there could be.

So for a camera that does not have a mechanical shutter, and if light source if un-flickering, any exposure length would be adequate as long as the sensor is in linear raanges.

Farzad
 
by artifact-free, i'm talking about shutter artifacts and other things not related to the optical train itself.

some CMOS cameras have problems with really short exposures, so that would be one further complication. but yes if your light source is flicker free then the range of acceptable exposures is wider.

rob
 
Great, thanks.

I have another question: under Camera Properties we have Readout Depth, A/D bits and auto-populating Maximum ADU. The maximum ADU is always reported in SGP as if the image is 16 bits (65535). Is readout depth the same as the bit depth of the sensor's ADC which is 12 in case of ASI1600?
 
well that one can get complicated. generally what happens is that because computers use 8-bit bytes, any data wider than a byte is represented by 16, 32 or 64 bit integers, whichever is wide enough to just fit the data. so if your ADC has 12- or 14- or 16- bit outputs, 16-bit integers will usually be used to represent the data.

so then the next question is, if the capture program is aware of the bit depth of the camera's ADC converter, it might try to scale the data so that the data "fills" the 16-bit space. in the case of the 12 bit ADC it might shift the data leftward 4 bits so that the numbers that get saved to the file all have 4 trailing 0-valued bits. so if the ADC outputs the number 1, the data file will have 0000_0000_0001_0000, which is the decimal number 16.

SGP could be doing this. it definitely does it for DSLR data that is 14-bit, because DCRAW (which it uses to support NEF/CR2) has the option to expand the data into 16-bits. although this eliminates the "pink stars" problem that's sometimes seen, it also makes SGP-generated .fits files from a DSLR incompatible with the way PI opens CR2/NEF.

this is one reason why mixing capture programs between lights and darks/bias/flats can be a huge headache - different programs do different things when the camera data is up for interpretation.

rob
 
For this line of work mixing capture software is definitely craziest thing that can be done.

Yes, as soon as we start taking about but depth, ADU, etc., things start getting complicated. Unfortunately the field also suffers from a lack of consistency because this very challenging work is based primarily in passion and we accept almost anything that is produced ad try to make the most of it.

Do you know what the intent of the BasicCCDParameter script is for the two values: Readout Depth and Maximum ADU? It seems to me that the Maximum ADU is meant to be the bit depth of the image being worked on. If the image is 16 bit then the Maximum ADU is automatically calculated as 2 to the power of 16. Then the Readout Depth might just be the bit depth of the sensor, 12 bits in case of ASI1600 and 14 bits in case of ASI294.
 
i looked at the source code for the script and realized that there are tooltips for those 3 UI controls. hover your mouse over them and see what they say - i think the descriptions are helpful.

the script makes reference to a PDF which does not seem to actually have the chapter of the book in question, but i found that this errata PDF does:


the last few pages of the pdf describe the method that the script uses... however i'd have to read the source code of the script more carefully to see what those controls really do. when testing CCDs you need to first convert ADUs to # electrons, which is dependent on the gain and the ADUs read out, so the pdf talks about that but not specifically how those 3 script inputs are used internally.

rob
 
Thanks for going through the trouble.

I have seen the tooltips and find them to be unclear - at least to my eyes that are not as experienced as yours perhaps. The Readout Depth is described as "bit depth for scaling standardized...e.g. 16 for Cannon", and A/D bits is described as "bit depth of camera A/D resolution...e.g. 14 for Canon...". These definitions may be incorrect or the tooltips may need to be switched. It makes more sense for the A/D bit instead to be the Readout Depth or simply Image bit depth of 16 (something that doesn't require a PhD degree to understand) and the Readout Depth actually be ADC value of the sensor which is 12 bits for ASI1600 and 14 bits for ASI294. I can verify that when I compare the results with published ZWO curves.
 

Attachments

  • 2020-10-25_16-49-23.jpg
    2020-10-25_16-49-23.jpg
    239.1 KB · Views: 42
well like i said i should go read the source to see how it maps to the paper... but you could too of course. sounds like perhaps those two things are backwards but i can't say for sure...

rob
 
pfile said:
well like i said i should go read the source to see how it maps to the paper... but you could too of course. sounds like perhaps those two things are backwards but i can't say for sure...

rob
Click to expand...

That would be a good way for me to learn how script codes work. I will look into it.

Farzad
 
You must log in or register to reply here.
Back
Top