Maximizing SNR in a stack

[Ignacio]

This topic has always puzzled me. Although I think I understand the concept and theory behind it, its algorithmic implementations and results many times left me wondering what is going on exactly.

I am fairly aware of the difficulties in finding a robust metric for SNR, particularly on the signal side of that equation (ie, scale).  It is often the case, when I use SubframeSelector to look at a data set, or when watching the weights being calculated during ImageIntegration, that higher weighted subs don't generally coincide with what I would consider the better subs upon visual inspection (sharper, darker background, etc.) and other metrics (fwhm, star support, estimated noise, etc.).

We have discussed here before, the effects of significant and changing gradients on estimating signal strength that often skew scale estimates, and the way to handle this during processing. But many times, even with no apparent gradients, I also see unexpected results. Could it be subtle changes in transparency? of maybe bloated stars due to poor guiding/focus read as better signal? or a bug? I don't know.

Lately, I have been trying to combine two data sets of M33: one from a couple of years back and of lesser quality (shorter integration, worse fwhm due to seeing/focus/guiding, worse camera), with another from last new moon. I calibrated and stack both sessions independently, removed background gradients and color-calibrated both stacks. Then I registered one to the other with no problems.

To my eyes, the latter stack looks better than the old one (sharper, more detail, and better low-scale contrast), although from a distance they both look similar (after a STF stretch). But when I try to combine them (using ImageIntegration with SNR weighting), I see higher weights on the older stack (say, 1.35 vs 1), and I have tried many if not all of the scale estimator methods available in II.

Then I thought that maybe the fact that the old camera was 12-bit (canon 1000D) and the new one is 14-bit (canon 6D), was getting in the way. So I proceed to apply a LinearFit of one stack to the other, as a sort of rescaling. Then ImageIntegration went bananas, attributing a weight between 100 and 170 to the RGB channels of the worse stack!

By close inspection of the console output, I see noise estimations an order of magnitude smaller than what I get with the NoiseEvaluation script, but only in the older (ie, 1000D) stack.

One last note: since ImageItegration cannot work with only two images, I had to duplicate both entries in order for it to run. Hope this is not causing an unexpected problem.

I am uploading both stacks to the Endor server (under the folder "M33 SNR"), so that you can try to replicate the problem, and maybe help me understand what is going on. Be aware that each file is 119 MB.

Thanks,
Ignacio

 


 

[mschuster]

Hi Ignacio,

I didn't look at your stacks, but I can make suggestions.

Default weights don't include FWHM measures. Try custom weighting. Check out Reply #6 in this thread.

Using StarSupport, NoiseEstimate, or BackgroundMean is not always a good idea. Better FWHM often results in smaller StarSupport assuming all else equal (more tiny stars rejected as hot pixels). Better transparency often means higher NoiseEstimate and BackgroundMean (more sky background photons).

Higher SNR estimate by itself is not always better (worse gradients or more high thin cloud bright star scatter).

There is no perfect measure, must use good judgement.

Mike

[Ignacio]

Thanks, Mike. I do use custom weights most of the time, but in a very ad-hoc way. I would like to find a more systematic approach to this. More so now, when the uploaded examples, where seemingly comparable images get a weight 100 vs 1. I would like to understand why this happens. I suspect that there is a bug.

Juan: would you look at those two stacks, please?

Thanks,
Ignacio

[Ignacio]

No one?

[jerryyyyy]

Have to chime in here.  I have been scratching my head on this a lot recently and have gone back to eyeballing.  I am familiar with that old thread and it helped me understand the basics. 

What has driven me to drink recently is if I try to run Subframe selector on a combination of H-alpha and RGB images it rejects all the H-alpha even though to my eye they are way better than the subframes seen with RGB in my light polluted area. 

Now, in rereading the old thread I understand that using an expression like:

FWHMSigma

Will see the two types of images as apples and oranges since the noise will be calculated across all the images...  Duh...

I have to do the two types of images separately... I guess.

Can someone post what they are using as values for both values under the "Expressions" in subframe selection.

I am using:

Approval:  FWHMSigma < 2 && SNRWeightSigma > -2 && Eccentricity <0.51

Weighting:  FWHMSigma

[Warhen]

Mike, thanks so much for your incredible work on this and other scripts! I'd like to ask a general question of you, Juan, and perhaps others who might have an opinion. Since ImageIntegration's use of a quality reference image is supposed to provide a robust point of comparison for weighting other subs to it, how much better is weighting subs beforehand with the SubframeSelector script? Thanks very much all!

[vpcirc]

I'm testing that Warren!!

[Warhen]

Hello Michael! I noticed you didn't get a response to a similar question on another thread. This is really the crux of what I'd like to know, whether it's worth the effort to grade/write to all the subs with the script, or good enough to merely determine a higher quality SNR sub to use as the reference image for image integration. You and I never got to finish up talking about that, and I'm anxious to hear your report. I would also like to hear a response from Juan and Mike. Thanks and have a good holiday everyone!

[vpcirc]

Well here's some results
I used the same bad reference image and stacks. The files had all been weighted in the sub frame selector. In the first integration I used noise evaluation. In the second integration I used fits keyword "weight"
The weighted stack from what I can see is a better quality master. It appears to not place nearly as much value on the reference image if the weight in the fits header applied by the script Screen shots and links to actual masters are posted below

https://dl.dropboxusercontent.com/u/75904861/integration%20test%20no%20weight.fit
https://dl.dropboxusercontent.com/u/75904861/integration%20test%20fits%20key%20word%20weight.fit
https://www.dropbox.com/s/tkq44cbzt61oops/Screenshot%202014-12-21%2014.34.15.png?dl=0
https://www.dropbox.com/s/xr1v03j44ft2rni/Screenshot%202014-12-21%2014.24.19.png?dl=0

[vpcirc]

In the final test, I took the best reference image, then used the key word weight. Changing the reference image did not seem to make any difference from what I can see

https://www.dropbox.com/s/zhl4wwfg6tk04xf/Screenshot%202014-12-21%2014.50.50.png?dl=0

[Warhen]

Mike, thanks for your time! What I'd like to see is a comparison between ImIn using 'Noise Evaluation' with the highest quality sub (SNR, etc.) as reference; and with the Weight keyword and subs having been ordered by the SFS script. I'd then compare the two results with the NoiseEvaluation script, Process Console, etc. to see if the script betters the SNR result.

We already know that choosing a poor sub as reference isn't good practice, so I'd think this would be a solid test.

[vpcirc]

I'm saying if you weight the images, then use fits keyword weight in normalization, then choosing the reference image doesn't seem to matter. This is not so much about noise reduction, and more about getting a proper normalization.

[Warhen]

I do wish Juan would 'weigh in' on this. :>) The documentation states that ImIn's reference image is used to weight other subs in the stack with regard to normalization and SNR. This has nothing to do with noise reduction. Read http://pixinsight.com/doc/tools/ImageIntegration/ImageIntegration.html#description_002
under Usage/Input Images/Set Reference.

The purpose of ordering subs with the SFS script would be to achieve the highest quality master. If I'm going to take the time to do that, I want to know that it accomplishes that better than ImIn assigning weights based on comparison to the reference image.

[vpcirc]

The results you get will depend on which images you reject from the stack and understanding what most of the meanings of the terms are. For example I think I understood that the SNR graph isn't a good example to use for rejection since thin clouds can make it appear better than it is in the graph.  FWHM and Elasticity seem to be the best but I'm learning as I go. Blink is a good secondary check, but hopefully this complex script will do a better job than our eyes in getting the best result. Of course I'd like to just push a button and the script says yea-ney!  I recall going to Adam's class and him saying any image with less xx percent weight he excludes. There is some heavy duty math going on here, so I think once I can figure out what to reject and what to accept, it will be as Juan said in another post "my best friend"

[vpcirc]

In reading the documentation, I can already see things I'm doing incorrectly for the best results. I was using registered images but not cropped

"In the Target Subframes section, add all target subframes. Subframes may be raw, calibrated or registered, but all should be the same type, have their overscan regions (if any) applied and cropped, be compatible for registration (if unregistered) and be compatible for integration. Measurements are typically most accurate on calibrated but unregistered subframes."

That would make sense has a registered image would have areas needed for cropping.

I did not create a PSF model either as described below which would further corrupt the model

"Select a Point spread function (PSF). The PSF functions are defined in the DynamicPSF process documentation. Star images will be fit to the selected PSF model. The quality of the fit on each subframe is provided by the StarResidual and StarResidualMeanDev properties once the measurement process has completed. A map of residuals for each fitted star in each subframe may also be generated by clicking the Output Maps button."

At this point I'm wondering if the small improvement I saw in normalization is worth the amount of work required compared to just using a reference image? The further you read, the more complex it becomes