New script for denoising linear monochannel images

[Roger B]

noise reduction has always been by sore point for some reason , but this looks like it's my answer. 


Thanks

Roger

[mschuster]

A second follow-up.  Can you expand a bit on the following statement:

Warning: Do not combine denoised images. Signal-to-noise ratio will be enhanced by combining noisy images and denoising the result.

Are you saying that the single frame image denoising is NOT an option if you are working with a stack?  If so, are there any limitations in running ImageIntegration beyond using an averaging method?   For example, are there any limitations on which rejection method we can use, whether we are better off with normalization set at additive or additive with scaling, or which scale estimator to use?

Hi Jim,

Yes. If stacking frames do not denoise the individual frames, rather denoise the resulting stack. If you denoise the individual frames you risk the loss of weak signals that rise above noise only after combination. Basically a weak signal might look like noise in the frame and be removed, but after combination the signal is stronger than noise and would not be removed. I have observed this loss and it can be significant, the deeper the stack the worse the loss.

Denoising a single frame is useful for survey work, for example, where you capture one frame and want to see how it might look with more time.

There are no limitations or preference on rejection method. No limitations or preference on scale estimator. For output normalization I suggest Additive with scaling. The ImageIntegration documentation recommends Additive with scaling for lights. I am experimenting with a scaling only option (no additive correction, not yet supported by ImageIntegration) that may turn out to be an good alternative.

Also, the script doesn't work properly for combinations of images captured at different pixel resolutions with different equipment.

Thanks,
Mike

[chris.bailey]

Mike

It seems to work better if its the first operation post stacking (other than perhaps cropping dither borders) but certainly doing it before DBE seems to me to yield better results than the other way around. Is that to be expected ?

Chris

[mschuster]

Yes Chris that is expected. Thank you for reporting your experience. The script's noise estimates will be less accurate on processed images.

Thanks
Mike

[aworonow]

Just tried it on a frame with a dozen faint galaxies that I got last night. Pretty impressive!

Thanks.....
Alex W

[jkmorse]

Mike,

I ran a quick test and got great results as shown on the links below to my before and after images from the Pickering's Triangle portion of my Western Veil image.  Really happy with the result, especially since I really didn't do any fine tuning of the inputs. 

Now my question.  At 2.59 arcsec/pixel, this image just screams for drizzle treatment but I do not see a way to use both drizzle and MureDeNoise.  Am I missing a step that will allow me to use both?  Of course, even if I can't it will be perfect for images from my Planewave CDK where the image scale ranges from 0.43 to 0.73 arcsec/pixel depending on my CCD.

One last point.  I assume in the DarkBiasNoiseEstimator, the number to use is the temporal noise and not the offset.

Thanks,

Jim

https://drive.google.com/file/d/0ByrTommGQM20Vy1pcVFjU1A0U3M/view?usp=sharing

https://drive.google.com/file/d/0ByrTommGQM20NWpfUUJGZGRYNDA/view?usp=sharing

[gvanhau]

Hello Mike
Thanks for sharing this script.
I run a quick test las night on a single and on a a stack of monochrome images form my atik 383L cam.

The noise reduction on the single image is really dramatic.
On the 41 images stack there was an improvement, but not so dramatic as in the single image I asume that this is because SNR is already very improved.

Regards
Geert

[mschuster]

Hi Alex,
Thank you for your report!
Mike

[mschuster]

Hi Jim,

Thank you for posting images! I am glad you got great results.

Drizzle combinations are not supported, no workaround is available. I am working on it, but not yet sure of success.

Yes, use the temporal noise number. I hope to release an update to DarkBiasNoiseEstimator soon with consistent labeling to make it less confusing.

Thanks,
Mike

[mschuster]

Hi Geert,

Thank you for testing!

Yes, you are correct, the stack has enhanced SNR and hence there is less noise to remove. My head post shows an example, at 1x actual size, the difference is harder to see in the 8 image combination.

Thanks,
Mike

[mschuster]

Hi Geert,

More info on stack SNR:

When you stack 100 frames, the usual rule says noise goes down by 10x, the square root of frame count. The top blue line below shows this as a function of count. At count 100 noise is scaled by 0.1.

But when you stack 100 frames, 99 of them were registered. The interpolation method used by registration averages adjacent pixels, introduces correlation, and hence reduces noise some. Curves show this for three StarAlignment pixel interpolation methods.

With drizzle the correlation is larger. Adjacent, smaller output pixels are more influenced by the same input pixels due to the fact that they are smaller. The dashed two curves show this. Increasing the drop size increases the correlation and further reduces noise. Also, noise does not drop as fast with drizzle due to the partial coverage.

All of this is relevant to the script, it needs to estimate (guess) how much noise is present in order to remove it.

Thanks,
Mike

[aworonow]

Mike, et al.

Wouldn't most of the problem Mike describes disappear if the noise reduction was applied to the individual images before stacking? Clearly, that order of operations carries a computational penalty, but keeps pixel-to-pixel correlated noise out of the stack, as much as possible. Remediating correlated noise seems far more difficult than does simply avoiding it as much as possible. Also correlated noise fosters the introduction of artifacts by other processing tools that presume it is just not there. It's like trying to get OSC/debayered noise out...not easy because of the long-range correlation of noise that debayering introduces.

Can a script be applied to an image container?

Alex

[mschuster]

Hi Alex,

I agree remediating correlated noise is difficult and best avoided.

Consider a 0.5 SNR signal in an individual image, a signal likely confused as noise by pre stack denoising, and removed. But in a 100 frame stack of noisy images the signal would be 5 SNR, not confused as noise by post stack denoising, and not removed. Pre stack denoising is not recommended for this reason. In my data this is not a theoretical problem.

With testing (thank you everyone) it appears that MureDenoise underestimates the amount of noise in a stack, and I think this is due to the correlation. And I do not have a solution. The only alternative at this point is to allow the Amount parameter to be larger than one. But this interferes with ease of use because now you have something that needs to be tweaked for stacks. This would allow you to get "more" denoising, but it comes at a cost of potential loss of signal due to the correlation.

Thanks,
Mike

[chris.bailey]

Mike

I have thrown a lot of images at it over the past day or so and I would say it works VERY well where the images in the stack are of similar S/N weighting say in the range 0.95 to 1.0. It does a really fantastic job on some of my long sub-frame Ha image stacks! Where it starts to underestimate the noise is when the contributing images are more variable e.g where some of the contributing images have a weighting of less than say 0.85. Now there is an argument these should not be in the stack in the first place but 30 minute Ha subs are hard won when you only get one clear night a month! On these sort of images I have tried re-integrating with weights all set to unity and it does seem to improve NR but with other unwanted side effects. In my simple logic, when the S/N contribution of images in a stack is variable, the quadrature theory starts to break down so this somehow needs to be taken into account.

Chris

[mschuster]

Hi Chris,

Thank you for your feedback. I agree that this is an issue that needs to be addressed. I am working on it. There is a similar issue due to flat fielding, I am working on this as well.

Also, on the noise correlation issue discussed above, I have made progress on this problem, and was able to fix some bugs. I hope to have a new release of the script in 10 days or so.

Thanks,
Mike