New Reference Documentation Released: ImageIntegration

[Juan Conejero]

Hi,

A new version of this document is now available online and as a regular update.

The formal description of the Winsorized Sigma Clipping algorithm was incredibly incorrect! it is fixed now. Along with that, I've made a number of fixes and corrections to the text; this is a complex document requiring several re-readings for polishing.

I would add that one of these occasions is integration of Bias and dark frames to create masters???

Good point, Jack. At the end of the Image Normalization section, I've added a table with the recommended normalization methods for light and master bias, dark and flat frames.

Under hints, I was expecting some kind on quick reference for settings to start out with for light frames, dark, flat and bias.

Max, I think this is well covered in Jordi's presentation, which is referenced in the first hint. Along with that, we have Vicent's tutorial on calibration frames acquisition and processing, which is also referenced from this documentation.

Thanks to all again for your encouragement. I'm starting the documentation for StarAlignment right now.

[martin_magnan]

Very comprehensive and well balanced documentation. not to geeky

Bravo !!

Martin

[Josh Lake]

Juan, this level of documentation is a masterpiece, thank you. This one is particularly timely for me as I am beginning work on a new data set from an observatory with spotty light frame consistency. Being able to run the different pixel rejection algorithms and optimize the results (as per Jordi's excellent presentation) and actually understand what I'm doing will be very useful.

For myself, I never experimented or even really looked at Linear Fit Clipping... I always just went with Winsorized. It looks like it could be very promising and can compete with WSC in a number of instances!

Thanks again, this is brilliant work.

[sreilly]

Juan,

While the math is waaaay over my head, I appreciate the detail as I know others will certainly find it interesting. Having worked a good deal through this documentation I think now is a good time to ask the question that's been in the back of my mind each time I process an image. As stated here:

"Set Reference

The ImageIntegration tool uses a reference image. By convention, the first (enabled) file in the input list is the reference image. The reference image is used for:

Image normalization. All input images will be normalized (both pixel rejection normalization and output normalization) to be compatible with the statistical properties of the reference image.
Image weighting. All image weights will be computed relative to the reference image, which will be assigned unit weight by convention.
Signal-to-noise improvement calculations. When the corresponding option is selected, SNR improvements are computed relative to the reference image and to the mean of all noise estimates at the end of the integration process."

I've often wondered about how to choose the reference image. What procedure/test would you use to make sure your reference image is the proper image to choose? I expect this is fairly important as the normalization and weights of the other images are based on this reference image. As of now I use a third party software to get statistical data such as FWHM to judge whether to include the image or not as well as a visual look at the image to be sure it doesn't contain any oddities but it doesn't tell me which image has the highest SNR. What should we be doing to determine this reference image from our usable data?

This is great information and should certainly quiet the "PI doesn't have any documentation" crowd. Keep up the great work!

Thanks,

[RBA]

This is great information and should certainly quiet the "PI doesn't have any documentation" crowd. Keep up the great work!

Are you kidding me? There will be a few who might praise the effort but overall I can already hear many of them singing: "Yes, great and detailed documentation, BUT in practical terms, that's too much... the most effective documentation is the one that you don't need to sit and read for 1 hour... Sometimes too much is not good. Don't these guys know how to synthesize?? Baaaaah beeeeeh baaaaahh...". Not to mention those who will go "This still doesn't teach you how to process" which by the way, it's what every one of us has been saying all along... Nay-sayers are what they are, it's better to turn deaf ears on them than trying to please them.

[Juan Conejero]

Hi Steve,

I've often wondered about how to choose the reference image.

It actually doesn't matter. ImageIntegration will always weight and evaluate each image correctly, irrespective of the reference image. The reference image is just a convention; the result is (should!) be independent on it.

That said, choosing the 'best' image as reference seems a good idea in practice, as in this way all numbers are easier to understand and interpret. If you want to use noise estimates as a criterion to select the reference image, you can run the integration process once, take a look at the weights on the console, and select the image with the highest weight as reference.

As of now I use a third party software to get statistical data such as FWHM

Why don't you use the DynamicPSF tool in PixInsight? It is now documented so you have no excuses

 http://pixinsight.com/doc/tools/DynamicPSF/DynamicPSF.html

[Juan Conejero]

Don't these guys know how to synthesize??

Just out of curiosity, is this a supposition or have you actually read/heard that? I don't ask you who/where, just 'if'

There will be a few who might praise the effort but overall I can already hear many of them singing: ...

Well, actually there are quite a bunch of PI users who have congratulated us, both on this forum and privately, which is a big encouragement and everything I want to hear Despite the fact that we are delaying development of new tools (we already have talked about this here), we are glad writing these documents, and I honestly think they are turning out quite well. The task is well worth the effort and we are progressing faster than I'd expected, considering the complexity of these documents.

Nay-sayers are what they are, it's better to turn deaf ears on them than trying to please them.

I agree completely. Don't worry, my ears can be deaf and my skin is thick. On the other hand, I sometimes enjoy N-S comments (la cosa tiene su morbo, ya sabes )

[mmirot]

Under hints, I was expecting some kind on quick reference for settings to start out with for light frames, dark, flat and bias.

Max, I think this is well covered in Jordi's presentation, which is referenced in the first hint. Along with that, we have Vicent's tutorial on calibration frames acquisition and processing, which is also referenced from this documentation.

[/quote]

Jordi's recommendation are excellent. The ppt document can't be loaded from the link in the PI doc browser .  It might be wise to have quick reference table.

Max
This is a FAQ in the forums

[RBA]

Don't these guys know how to synthesize??

Just out of curiosity, is this a supposition or have you actually read/heard that? I don't ask you who/where, just 'if'

No, I haven't heard/read anything like that, yet. But wait long enough and we'll see if I was wrong.

Well, actually there are quite a bunch of PI users who have congratulated us, both on this forum and privately, which is a big encouragement and everything I want to hear

When I said "There will be a few who might praise the effort, but..." I was referring to people who haven't been too "kind" in the past, and now will "praise the effort"... not to actual PixInsight users.

On the other hand, I sometimes enjoy N-S comments

Will pass them onto you when I feel they're morbid enough