ImageIntegration...What consitutes a "better result" in the final stacked image?

[joelshort]

I'm delving a little deeper into learning PI and have some questions regarding ImageIntegration.  Nearly all of the resources and tutorials I've come across say that the pixel rejection algorithm used is very important and that one should "experiment with the different rejection algorithms and rejection parameters to find the best result." 

However none of those tutorials really explain what "best result" means.  When I have experimented with the rejection parameters I see little if any difference in the resulting images.  Is there a way to emperically measure the result of what constitutes a "good" result?  Obviously the best result would be the highest signal with lowest noise, but the differences are so subtle as to be impossible to see. 

[dnault42]

What you want from ImageIntegration is a stack that is clean of hot and cold pixels as well as transient data like satellite trails or cosmic ray strikes while maintaining the highest signal to noise ratio possible.  Outlier rejection is the first part of that and should be reasonably obvious as you either have outliers left after stacking or you don't.  The goal is to set the rejection so that as few pixels are rejected as possible.  The second part is to try to maximize SNR.  Comparing to an image stacked with a straight average is a good place to start, although we frequently encounter conditions that make a straight average less ideal than a weighted average, for example, when you have lower SNR subs that would bring down the overall SNR.  Typically I use a stack with no rejection that is weighted using noise evaluation as my reference for SNR.  To compare the various passes of ImageIntegration I use a preview that covers a section of background sky and examine the standard deviation of both the reference and test stack (I use the PropagatePreviews script to duplicate the preview from the reference image to the test stack).  The lower the standard deviation the lower the noise in the image, so I target the parameters that give me good outlier rejection while getting as close to the background standard deviation of the reference image as possible.

Regards,
David

[chrisvdberge]

The 'best result' is up to you to decide
I would say its a combination of good SNR, low Noise, good StarSupport and good FWHM, while still getting rid of all artefacts.
Without the last requirement you would always be best off using no rejection at all.
In my experience you can get the best results using Linear Rejection when using 25+ frames.

[joelshort]

Thanks for the replies guys.  David, that's some good info and exactly what I was looking for.  Using your info, I went back and did some test stacks and it looks to me like when using winsorized sigma clipping I need to raise the sigma low and sigma high rejection from the default settings (which conversely means that there is less overall rejection of outliers).  In my tests it was obvious that this resulted in getting rid of trails and spurrious data while maintaining the highest signal to noise ratio.

Great, now I need to go back and reprocess a bunch of images because I now believe I was throwing away a bunch of data due to too aggressive of pixel rejection.

[dnault42]

Chris made a good point about other metrics like FWHM as well.  I use SubframeSelector to modify the weighting of each file and with that you can focus on FWHM over SNR or eccentricity or star support or whatever you choose.  In general I weight FWHM very high as I'm willing to take additional subs to reach a desired SNR.  Using those techniques the includes 4 steps: SubframeSelector, StarAlignment, ImageIntegration & DrizzleIntegration.  I've started using DrizzleIntegration every time because of it's superior interpolation algorithm.  You don't have to use it to scale up, so for example, if your data is already well sampled you can run it with a scale of 1 and drop shrink of 1.00.  If your data supports it, you can actually run it with a Scale of 1 and a smaller drop shrink to further improve FHWM, but you really need a lot of subframes for this to work well.

Regards,
David

[jkmorse]

Folks,

If you want to understand how to maximize ImageIntegration, you need to check out guidance from a master at this link:

http://www.astrosurf.com/jordigallego/articles.html

That will take you to Jordi Gallego’s powerpoint entitled: “Image integration techniques: IncreasingSNR and outlier rejection with PixInsight.”  Title sort of says it all and it deserves a look.  It takes you into the guts of the process and sets out testable goals for getting the most out of the tool.

Best,

Jim

[joelshort]

That's awesome Jim, thanks for that link.  Dang, I've been doing this integration thing all wrong and no doubt have not been maximizing my integration results. 

[jerryyyyy]

For my two cents, do not forget subframe selection (script).  Garbage in Garbage out.  I swear I get better images rejecting sometimes 50% of the images due to my high humidity near San Francisco.  Key for me is to look at the plots and I see the number of stars supported start to fade... that is where I set my Sigma for star support. 

You also need to do this separately for each filter and not lump them all together.

IMHO, I never got much more from linear and drizzle after I read on how to do the noise evaluation... more mileage from careful subframe selection in my hands....

Cheers & good discussion...

[jkmorse]

Jerry,

Gotta try that.  I have been operating on the assumption even weaker subs add to the SNR, but that is likely a wrong assumption.  We test.

Thanks,

Jim

[joelshort]

I agree Jerry.  I've been using subframeselector for a while now and like you I throw about usually 10-25% of my images.

[jkmorse]

Joel,

Gotta ask, as a Big Ten guy (Hoosiers '79 and '82; those were good years  ) your website name suggests you are from THE Ohio State.  Any truth in that rumor?   

Best,

Jim

[joelshort]

Ha, yes Jim I'm from Ohio and am a big OSU fan (mostly football - last year was a good one!).  Currently live in Indiana surrounded by a bunch of Purdue fans...

[jkmorse]

Ugh, what mafia boss did you piss off?   

[jerryyyyy]

Jerry,

Gotta try that.  I have been operating on the assumption even weaker subs add to the SNR, but that is likely a wrong assumption.  We test.

Thanks,

Jim

Just integrate two different stacks, one with the good ones only and one with the bad + good and see what you get....  I have found even one or two bad ones screw things up....  In particular the star counts (real numbers) are telling... imagine if you have images with 5000 stars merged with images with 3000 stars....  This is despite all the math behind the stacking...

[jkmorse]

I have been using the star count as a confirmation of my best image for reference in Star Alignment and Image Integration, but need to take that further as you suggest.  Will post my results.

Thanks,

Jim