Question about SNR statistics in Process Console

[astrovienna]

I'm comparing several different levels of clipping settings with linear fit clipping (47 images in stack).  I get the best median noise reduction with low/high of 6/6, a reduction of 1.9425.  With no rejection, I get median noise reduction of only 1.8384.  I thought the noise reduction should be maximized with no rejection?

6/6 gets the highest median noise reduction.  Both 6/4 and 6/8 have lower median noise reductions.  Can anyone explain?  Except for the no rejection stack, they all seem to eliminate all hot pixels quite well.

Kevin

[RickS]

Hi Kevin,

I've seen results that don't seem quite right occasionally too.  I have always put that down to the fact that the noise estimate is only an estimate, and that sometimes it may not be completely accurate.  I mostly go by the rejection percentages.  I just get them as low as I can without introducing any artefacts and then I stop.

Cheers,
Rick.

[Juan Conejero]

Hi Kevin,

I thought the noise reduction should be maximized with no rejection?

The effective noise reduction function (ENR, see ImageIntegration documentation) depends on scaled noise estimates, that is, it is a function of estimates of the standard deviation of the noise and statistical scale estimates (dispersion). Pixel rejection may affect dispersion in a way that our robust estimation routines can detect accurately. As the documentation describes, ENR values should not be compared among different rejection algorithms; for example, with and without rejection. Strictly, comparisons among different ENR values are only valid for the same pixel rejection algorithm. Your goal is to maximize ENR for a given pixel rejection algorithm while you achieve the required rejection of outliers.

It is very important to point out that ENR values are not signal-to-noise estimates. SNR evaluation is nonrobust by nature, so SNR values must always be taken with a grain of salt. They are just indicative values. On the contrary, ENR, as implemented in our code base, is a robust and very accurate method. Scaled noise estimates computed with the default multiscale noise evaluation algorithm (MRS) are typically uncertain by a 1%.

[astrovienna]

Thanks, Juan and Rick.  I forgot that different rejection methods can't be directly compared for the noise stats.

I'm still not sure why a high clipping setting of 6 gets a better ENR result than 8.  Shouldn't ENR rise as more pixels are included in the integration?  Or does it go down as the setting goes to 8 because hot pixels are reappearing in the integrated image, and the formula regards those as noise?  I'm not noticing any hot pixels, but maybe they're showing up regardless.  I'm trying to figure out how to determine what clipping settings are best, and whether there's a quantitative method vs just relying on my eyes.  To my eyes, all the stacks using clipping looked the same.

Kevin

[Juan Conejero]

Hi Kevin,

Yes, this is unusual under normal working conditions. Can you upload a set of integrable images where I can check out this? A possible explanation is that a significant fraction of background pixels is being rejected when you use a more restrictive clipping point. Since ENR measures the noise on low-SNR regions, an excessive rejection of noise pixels can lead to this apparent improvement. But I need to take a look at the data to confirm this hypothesis.

[astrovienna]

Thanks Juan.  The folder (big - it's 47 frames) is on Dropbox:

https://www.dropbox.com/s/elcpqdj5fncjz14/2017-10-27_012407_258.4deg_1200sec_1x1_Ha_frame10_c_cc_r.zip?dl=0

Please let me know if you have any problems downloading that.  Uploading it seemed a bit easier than I expected!  I used frame 29 for registration in BatchPreProcessing, if that matters.

Kevin