Image Intergration

[sreilly]

As I get more comfortable with PI I start to explore more of it's features. What I've seen that has really impressed me probably the most is the Histogram Transformation. It replaces what I use to do with levels and curves and gives a more finite control to the user, especially with track and preview turned on. In fact I have had little reason to use the CurvesTransformation tool.

I've been exploring the different combine methods in Image Integration and understand Average and Median combine but Maximum and Minimum are new to me. I've used Sum in other programs and that isn't a choice here, at least by that name. What do the Maximum and Minimum combine methods do for the image? I understand that Average gives a smoother overall image but for me leaves a bit of cleaning up to do that the Median method doesn't. I confess I usually use the Median combined image for the master. Keep in mind though I hardly ever have an image that has less than 9-12 images per color channel and usually 20-40 luminance frames that go into making the master frames. Most final images are 12-16 hours or more in usable data. I expect that different methods might benifit by the number of frames being used. Until Juan mentioned the Linear Clip Fitting rejection algorithm, I was using Windsorized Sigma Clipping.

Steve

[Carlos Milovic]

Hi Steve

Usually Average gives better SNR than Median, for large set of data (>10). Depending on the noise, Median may give best results with few images.
Average and Sum are the same thing.
Minimum and Maximum methods look for the lower or highest pixel value in the set. Not too useful with common astronomical images, but it may work well to do star trails or to produce a background image.

[Juan Conejero]

Hi Steve, Carlos,

I've used Sum in other programs and that isn't a choice here

The equivalent is average combination in PixInsight's ImageIntegration tool. Adding and averaging are equivalent operation as for SNR improvement.

Usually Average gives better SNR than Median, for large set of data (>10).

Average combination is always better in terms of SNR improvement, even for three images, which is the smallest possible set of images that can be integrated. For normal imaging purposes, there is no reason to use median combination in PixInsight. Always use average combination with the appropriate pixel rejection, and fine tune your rejection parameters.

For very small data sets, say from 3 to 5 images, use percentile clipping or averaged (Poisson based) sigma clipping rejection. For intermediate sets (from 5 to 10 images) sigma clipping and averaged sigma clipping are normally the best options. For large sets (from 10 to 20 images) use Winsorized sigma clipping or linear fit clipping. For very large sets (more than 15-20 images) linear fit clipping should perform better than Winsorized s.c. This is a sort of 'rule of thumb'; don't take it as written on stone and carry out your own tests with your data.

Other pixel rejection algorithms have been implemented for completion/research and for special applications. An example is min/max clipping, which can be useful in special cases but should be avoided for normal applications, as it performs an unconditional rejection of a fixed number of pixels without any statistical basis.

Hope this helps.

[sreilly]

Thank you Carlos. My usual process at this point is to calibrate the images in the same software that I use to acquire the data, MaxImDl. The usual dark and flat calibrations are done and then I use the De-Bloomer plug-in by Ron Wodaski and remove hot pixels using a threshold of 20%. The images are saved in a new folder, Cal, using IEEE Float size format. At that point everything else is done in PI and saved as 32 bit IEEE 754 Floating point. Those processes are usually star register, combine (either Median or Average combined using either Windorsized Sigma or Linear Fit Clipping rejection), use of Screen Transfer Function to see where to use the Dynamic Crop tool, Dynamic Background Extraction if needed, and the Histogram Transformation to stretch the image. If there seems some benefit I'll use HDRWavelets to dig out some additional details. I have been trying to use the Star Mask function to work on everything but the stars, not sure I'm being very successful there yet. I only save a final version as 16 bit when I am completely done and want an image to bring into Photoshop and that's only done to create the different size JPEGs for my website.

Having said all that, my intent was to give you the background as to what happens to the data before PI gets it. Maybe there is a better process in PI to remove the hot pixels and give a cleaner Average combine result? Is my processing flow flawed or could it use some other order of processing?

Thanks for your time and patience.

Steve

[sreilly]

Hello Juan,

This is all great information for me. I noticed that there are three choices under Pixel Rejection 1 for normalization. Scale and Zero Offset is what I have used for lack of knowing any better. Would Equalize Fluxes be a better choice. I realize that just trying this would tell me something but it may be so subtle I may not even notice it. As for fine tuning the rejection settings are you referring to the High/Low settings in Pixel Rejection 2? In Pixel Rejection 3 for Noise Model, is this the information that is in the fits header of by SBIG ST10 images and is there any reason to model this separately?

Again thanks for the help. I'm using PI almost exclusively now that my feet are wet. Still have a long way to go in exploring all the relevant tools, some of which I suspect would only be used under rare occasions but good they are there.

Steve

[Carlos Milovic]

Hi Steve

I think Juan is going to take the flag here He wrote the process, and he has a lot more experience with data sets than me.

[Niall Saunders]

I only save a final version as 16 bit when I am completely done and want an image to bring into Photoshop and that's only done to create the different size JPEGs for my website

Hi Steve,

There is no real need to transfer to PS to achieve this - PixInsight is more than capable of creating JPG images for use on the web - in fact it may even be better than PS, because PI can embed an ICC profile within the JPG so that the image, if viewed on an appropriate browser, actually looks the same on someone else's monitor as it did on yours. I have no idea whether PS has this capability but, even if it does, you do NOT need to resort to PS in order to get the JPG that you need.

If you are not sure how to achieve this, start a new question thread, and we can take it from there.

[Carlos Milovic]

I think that the only drewback in PI's jpeg writing system is that it does not predict the size, and does not let you preview the compression artifacts...

[Emanuele]

I think that the only drewback in PI's jpeg writing system is that it does not predict the size, and does not let you preview the compression artifacts...

I totally agree with that.

[sreilly]

I use PS because of the reasons others have replied. Being able to see the compression affect is important to me. I'd like to keep the file down to a reasonable size but would rather a larger file that looks better than a small file that destroys the image itself. PS gives these options very nicely and I did pay $400 for the program so I might as well use it where it can do me some good. Each image seems to be able to take a different quality compression. I suspect it's based largely on the S/N of the final image.

[Carlos Milovic]

Yes, the SNR and the high frecuency components it has. In a nutshell, the JPEG algorithm is deleting some frecuency components that he thinks are not usefull, for each 8x8 square over the image. So, the quality factor is in fact how many, and how much are you deleting them.

[Niall Saunders]

Yes, but there is very little control a user actually has as far as defining what the JPG algorithm will actually 'do' to the source image.

Sure, PS may be able to 'predict' what the final file size is 'likely' to be - but, if memory serves me right, it doesn't actually 'change' the in-memory, and on-screen, image to reflect what the image would look like if it was saved and then re-loaded.

Which is, in fact, all that is actually needed to 'check' the JPG compression settings in the first place. 14 years of experience saving JPG images has taught me that there really is very little to be 'seen' as far as saving with different compression factors - for 14 years I have tended to always save just using the same factor!!

Every so often I choose to save with full-quality (usually for some very specific reason, like when it is actually the JPG file-type I need, not the reduction in file size) and, conversely, sometimes I compress very hard, to minimise file size (although, in reality, far better file-size reduction can be achieved by compressing less, but by reducing the pixel count first, again something that PI does very well)

The point I am trying to make is that you don't 'need' PS to be able to achieve this task - and this is an important point that others, who might not already have PS, need to be aware of. Fine, if you already have a $400 investment in PS, you may feel inclined to justify that investment. However, I take the other stance - my investment in PS was about as useful as my investment in Enron (I still tend to use Corel for every other type of graphics manipulation that I cannot achieve in PixInsight, and don't even have my copy of PS loaded in any of my machines).

To me, it seems like using a sledgehammer to crack a nut, if you have to output an image, through some intermediate process (to be able to get it loaded into PS) if you then only use PS to convert it into a JPG. Sorry, I am not trying to offend or upset anybody, I just know that - for me - the process is FAR easier to achive within the confines of PixInsight.