How to evaluate calibration frames

[gboyle]

I can see some older posts that refer to some of this but was prompted to start a new one...

Which statistic and in which process or script should I be focusing on the determine the quality of bias dark and flat frames?
For each type of calibration frame should I be focussed on MAD on std Deviation or some other stats entirely?

The lack of stars in these images has me wondering how the programme knows at all what is noise and not the intended signal but that is a bigger personal issue I guess !!

[pfile]

well, i guess to start off, the statistical properties of an image don't really correlate with the subject matter directly. when PI is doing noise evaluation it's not really trying to detect what features of the image are signal and what is noise.

imagine a grey image where all pixels have the value 0.5. this image is noise-free; there is no variation in pixel values. if you went in and randomly added and subtracted random values from that image, then you'd start to be able to talk about what the mean value is, what the median value is, what the standard deviation of the noise is, and all kinds of other statistical properties. obviously the image with all pixels 0.5 have these statistical properties, but they are not very interesting as they basically just tell you the image is noise-free.

as far as your calibration frames go, you should strive to minimize the noise in the master frame. for bias and darks this really means simply integrating a lot of subs. darks usually need some kind of data rejection, since the odds that a cosmic ray hits your sensor during a long exposure is reasonably high. for your flats, you should strive to make the brightest part of the flat as bright as possible while still remaining in the "linear" range of the sensor. this range differs from design to design, so you may need to measure it for yourself. a rule of thumb that people use is that the brightest part of the image should be at 1/2 well, meaning if the ADC gives you values of 0-65535, then you are looking for 32767 or thereabouts. generally most sensors remain linear past that point, but it's a good idea to check.

rob

[gboyle]

Thanks Rob
Is there a Patricial particular process and metric I should use to evaluate noise in the darks and flats?
I'm thinking subframe evaluation process and use MAD out Standard deviation or some other metric?
Cheers
Gavin

[pfile]

for darks, i guess i've never really tried to evaluate them until something went wrong. mainly what you are looking for is light leaks, so if you are sure that there's no way for stray light to enter your camera while taking darks, you are probably OK.

for flats, i guess you can at least make sure that the median value is what you intended when you took the flats and that the brightest pixels in the image are still in the linear range of the sensor (typically people use 1/2 well depth but many cameras are linear far beyond that point...)

rob

[jerryyyyy]

Any comments on running TGV denoise on Flats Masters?  Noise in ImageEvaluation goes down.  Saw this somewhere. 

[pfile]

i know morten advocated for this but i would never do noise reduction on a flat. part of the purpose of a flat is to fix any small differences between quantum efficiency between pixels. admittedly probably a very small effect but regardless, that's one purpose of a flat. if you smooth the flat with some kind of noise reduction you just killed any hope of normalizing your QE.

flats are well-exposed images comparable to daytime images with high SNR to begin with. read noise is vanishingly small compared with the signal. i don't see any reason not to just take more flats if you feel your flat master's SNR is not good enough...

rob

[mschuster]

flats are well-exposed images comparable to daytime images with high SNR to begin with. read noise is vanishingly small compared with the signal. i don't see any reason not to just take more flats if you feel your flat master's SNR is not good enough...

Flat SNR is limited by the shot noise in the signal, read noise is way smaller than shot noise in a well exposed flat. A flat on my camera has a SNR of about 130 (about 130^2 or 18k electrons/pixel). I understand a good rule of thumb is SNR 1000 for a master flat. On my camera this requires about 60 exposures (1000/130)^2.

Mike

[Geoff]

Flat SNR is limited by the shot noise in the signal, read noise is way smaller than shot noise in a well exposed flat. A flat on my camera has a SNR of about 130 (about 130^2 or 18k electrons/pixel). I understand a good rule of thumb is SNR 1000 for a master flat. On my camera this requires about 60 exposures (1000/130)^2.

Mike

This surprised me initially, but the I did the calculations for my setup and got in the same ball park--about45 flats needed.  I've only been doing 20, so I should definitely raise the number.
Geoff

[pfile]

flats are well-exposed images comparable to daytime images with high SNR to begin with. read noise is vanishingly small compared with the signal. i don't see any reason not to just take more flats if you feel your flat master's SNR is not good enough...

Flat SNR is limited by the shot noise in the signal, read noise is way smaller than shot noise in a well exposed flat. A flat on my camera has a SNR of about 130 (about 130^2 or 18k electrons/pixel). I understand a good rule of thumb is SNR 1000 for a master flat. On my camera this requires about 60 exposures (1000/130)^2.

Mike

maybe what i said was poorly worded, but this is what i was trying to express.

rob

[mschuster]

I received a PM questioning my post, how to calculate flat SNR and where the SNR 1000 comes from.

To estimate flat SNR, measure the mean or median of your flat (either is OK for an estimate), multiply by your camera's gain, and then take the square root. This is a good estimate of flat SNR, assuming a bright exposure where shot noise dominates.

For my setup flat median is 36k DN, gain is about 0.5 e-/DN, product is 18k e-, SNR is square root of 18k or about 130. Full well depth on my sensor is 25k e-, 18k e- is about 70% of full well. I don't want to expose any brighter than that to avoid anti-blooming problems.

Gain and full well numbers should be available from your camera manufacturer, at least for mono cameras. My camera writes the gain value in the FITS header.

I think the SNR 1000 suggestion for master flats comes from a Ron Wodaski/Russ Croman book or talk, I don't remember which. IMO 1000 may be overkill, and for my setup is a bit time consuming for narrowband panel flats.

To get SNR 1000 you need to collect 1000^2 or 1 million electrons. For my setup one flat is 18k, so I need a total of about 60 flats to get 1 million. My previous calculation of (1000/130)^2 amounts to the same thing.

Mike

[mschuster]

When integrating biases and darks, you are driving down read noise but not fixed pattern noise. When you measure noise on a resulting master, you are measuring a combination of both. So measured noise does not go down as fast as you might think it should due to the invariant pattern noise.

One way around this problem is to create two masters each with the same number but different frames. Subtract the two and add 0.5 in PixelMath with rescaling turned off, measure the noise in the result, and then divide the measurement by square root of 2. Fixed pattern noise cancels in the subtraction, and so you are better estimating residual read noise, the square root 2 accounts for the accumulation of two doses of independent read noise in the two masters. Adding the 0.5 avoids clipping problems in the subtraction that would otherwise invalidate the measured result.

This measurement will decrease more rapidly with more frames, but still maybe not quite as fast as you think it should. Why this is true, at least on my setup, I don't know.

I like to use lots of biases and darks for my masters, at least 100 and a lot more if possible. Actually I don't use bias masters in my calibrations, only dark masters. My narrowband darks are 40 mins, so 100+ frame dark masters take a long time. I just run the camera for 4 or 5 days once a month or so.

Mike

[mschuster]

If you use the NoiseEvaluation script to measure the noise in a frame, it will give output something like this:

Code: [Select]

* Channel #0
?K = 1.967e-004, N = 2030959 (97.54%), J = 4

The noise measurement is the first number, in normalized units RMS. To convert to electrons RMS multiply by both 65535 DN and your camera's gain.

On this particular frame (a binned 24 second panel flat-dark), gain is about 1.06 e-/DN, so measured noise is

1.967e-004 * 65535 DN * 1.06 e-/DN = 13.7 e- RMS.

The NoiseEvaluation script usually does a good job measuring noise while ignoring non-noise structures on my frames, but it is not absolutely fool proof due to its Gaussian noise only assumption.

Mike

[mschuster]

I wrote a script to estimate flat signal to noise ratio. Here is a link to the script and some test images if you want to try it. It is a beta version, please report bugs.

Thanks,
Mike

https://dl.dropboxusercontent.com/u/109232477/PixInsight/Scriptbox/FlatSNREstimator.0.1.zip

The dialog below shows an example measurement. Input is two uncalibrated flats and a flat dark master. A bias master would work also. Actually, a single flat dark or bias frame suffices also.

Signal is the average of the two flat dark subtracted flat medians. Noise is measured using robust local statistics by first subtracting the two flats to remove both fixed pattern noise and fixed pixel sensitivity variations.



The dialog below shows an example measurement on flat masters. Both masters are integrations of 64 different uncalibrated flats. You can see that noise is about 8 times lower and SNR is about 8 times higher, as expected.



These results can be double checked using photon statistics. As an exmaple, using a known gain of about 1.06 e-/DN, the total number of electrons captured in each flat master is about 26342.6 DN * 1.06 e-/DN * 64 or 1.787 million. Taking the square root gives SNR 1337, nearly equal to the script's estimate.

[AstroScience]

Hi Mike,
is this applicable to DSLR frames?

[mschuster]

Hi Sergio,

The script works only on single channel images. For DSLR frames you may

1) Debayer all frames. Extract red, green, and blue channels. Run the script once on corresponding frames from each channel.

or 2) Run the script once on all non-debayered frames. The bayer pattern will be seen as strong fixed pattern noise, which should be accounted for in the flat difference. However, this may not give good results (untested). If you try it, compare the results to 1) as a sanity check.

Mike