New script for denoising linear monochannel images

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
MureDenoise Version 1.21

Thanks,
Mike Schuster

The MureDenoise script denoises linear monochannel images corrupted by mixed Poisson-Gaussian noise. MureDenoise is applicable to single frame images and average combinations of equally exposed and registered frames.

The script supports an astronomical image processing workflow in which the denoising step occurs immediately after the calibration and optional average combination steps and prior to other linear or nonlinear processing steps.

The script applies an interscale wavelet mixed noise unbiased risk estimator (MURE) to find a denoised output image that minimizes an estimate of the oracle mean-squared error (MSE), or "risk", between the denoised output image and the unknown noise-free image.

Note: For linear multichannel images from monocolor detectors, run the monochannel denoiser on each channel separately. The script does not work properly for images from one shot color (OSC) detectors.

Warning: The script is adapted to denoise linear monochannel images mainly corrupted by shot noise, read noise, and dark current noise which is typically the case for astronomical data. The script does not work properly for other noise distributions, for saturated images, for debayered images, for linearly or nonlinearly processed images, for median combinations, or for drizzle combinations.

Warning: Do not combine denoised images. Signal-to-noise ratio (SNR) will be enhanced by combining noisy images and denoising the result. Combined images must be equally exposed, have the same pixel resolution, and be registered by projective transformation with no distortion correction.

More more information and examples, please see the script's documentation.
 

pfile

PTeam Member
Nov 23, 2009
5,709
116
Mike, this is fantastic. i've used it already to denoise some narrowband masters and the results are impressive.

thanks for all your continued development

rob
 

msmythers

PTeam Member
Jan 16, 2013
1,179
1
www.astrobin.com
Mike

Very nicely done. Makes me wish I had real equipment and a good location to shoot from. Since I don't have either I wanted to see how I could adapt your script for my needs. I use a 12 bit Sony Nex-5 Mirrorless camera, don't laugh, and shoot from the coast of east Florida. A lot of my images are widefield with a 45mm lens.

Anyway since I don't have the gain or noise numbers for my camera I tried a little experimenting to see how I could use the script. That wasn't so bad.

The images I am showing are a crop of Cygnus with my 45mm lens. The image is 4608x3072. It is 23, 2 minute exposures stacked. I used ABE and then AutoHistogram with the average mean of the images 3 channels. Not a major stretch but enough to get a decent color balance. I then extracted the 3 channels for use with MureDenoise. After the different denoise tries or no denoise I then combined the channels and processed the image. No more noise reduction was applied after stretching.

There are 3 separate images. One with no denoise. One with my MureDenoise settings which included 23 combination count and 8 Cycle-spin count. The last was 2 passes of MureDenoise with the previous settings. All 3 images were then combined and the same exact processing was done to each combined image.

Thanks Mike!



Mike
 

Attachments

chris.bailey

Well-known member
Jan 6, 2010
235
0
Wow! I have just thrown about a dozen Ha stacks at it and in every case the results are better than I have been getting with MLT noise reduction, which has been my linear weapon of choice recently but have always had to follow this with a round of ACDNR post stretch. This seems to do it in one pass.

Does the Auto pixel interpolation setting pick up the setting used during star alignment?
 

jkmorse

Well-known member
Mike,

Can you explain a little more about the cycle-spin count.  From what you say in your note, it looks like it operates not as an intensity multiplier (like iterations in Deconvolution) but instead like the iterations in Masked Stretch where more means a better solution, not more intensity.  As such, if you are willing to trade time for quality, does it make sense to up the number beyond 8 and, if so, is there a practical limit?

Btw, the script looks fantastic!  Its the ability of talented people like you to supplement its workings that makes PI so much fun to use.

Thanks,

Jim 
 

jkmorse

Well-known member
Mike,

A second follow-up.  Can you expand a bit on the following statement:

Warning: Do not combine denoised images. Signal-to-noise ratio will be enhanced by combining noisy images and denoising the result.

Are you saying that the single frame image denoising is NOT an option if you are working with a stack?  If so, are there any limitations in running ImageIntegration beyond using an averaging method?  For example, are there any limitations on which rejection method we can use, whether we are better off with normalization set at additive or additive with scaling, or which scale estimator to use?

Thanks,

Jim



 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
cdesselles said:
Would it be possible to adapt this script to OSC images shot with a DSLR?
Yes, msmythers is correct. For linear multichannel images, run the monochannel denoiser on each channel separately.

For linear OSC images, denoise the CFA channels not the de-Bayered RGB channels. Recombine the denoised channels and then de-Bayer. And as msmythers mentioned, be sure to use the correct detector parameters as they typically vary with ISO settings.

For a monochrome detector, again just denoise the filtered channels L, R, G, B, S, H, O, etc and combine the results as usual.

Thanks,
Mike
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
msmythers said:
Anyway since I don't have the gain or noise numbers for my camera I tried a little experimenting to see how I could use the script. That wasn't so bad.
Thank you Mike for posting feedback, your comments and images! I am glad to hear that your experiments worked!

If you can run the script twice and see improvements, chances are that the detector parameters are not as accurate as they could be. That is OK, no problem. With more accurate parameters, typically you can not run the script twice. Doing so results in obvious artifacts and loss of detail.

Also, I suggest running denoising before ABE or any other processing. Anything that modifies the signal modifies the noise statistics also. MureDoise does not know these transformations were performed, and so its noise estimates may be less accurate.

Thanks,
Mike
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
RickS said:
That's very cool, Mike!  I have 50+ hours on a very dim Oiii target and I'll give this a try.
Thank you Rick! I hope you get good results. FYI: my 3nm H-alpha targets run 8 to 20+ hours and I am getting really good results on all of them.

Thank,
Mike
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
chris.bailey said:
Wow! I have just thrown about a dozen Ha stacks at it and in every case the results are better than I have been getting with MLT noise reduction, which has been my linear weapon of choice recently but have always had to follow this with a round of ACDNR post stretch. This seems to do it in one pass.

Does the Auto pixel interpolation setting pick up the setting used during star alignment?
That is excellent news Chris, thank you so much for posting!

Only choose Auto in the script when you selected Auto in StarAlignment. If you selected something else in StarAlignment, be sure use the same setting in the script. The settings need to match.

Thanks,
Mike
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
jkmorse said:
Can you explain a little more about the cycle-spin count.  From what you say in your note, it looks like it operates not as an intensity multiplier (like iterations in Deconvolution) but instead like the iterations in Masked Stretch where more means a better solution, not more intensity.  As such, if you are willing to trade time for quality, does it make sense to up the number beyond 8 and, if so, is there a practical limit?
Thank you Jim for your kind comments.

Yes, a larger cycle-spin count means more iterations and a better solution. You can specify up to 32, at which point the results are at nearly highest quality possible in my testing. More than that give negligible improvements to the underlying risk estimator. Here is the idea:

The method uses a wavelet transform that subsamples at each level of refinement, each successive level having dimensions half the size of the prior level. The orthogonality of this transform allows the method to denoise each level independently, yet still achieve a globally optimal solution in terms of SNR across all levels. But the solution is shift variant, i.e. it depends on the locations of the subsampling boundaries. The solution to this problem is cycle-spinning: Run the method multiple times with different image shifts or translations and average the results. The final result is then nearly shift invariant.

Thanks,
Mike
 

Roger B

Member
Nov 2, 2013
13
0
noise reduction has always been by sore point for some reason , but this looks like it's my answer.  :D


Thanks

Roger
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
jkmorse said:
A second follow-up.  Can you expand a bit on the following statement:

Warning: Do not combine denoised images. Signal-to-noise ratio will be enhanced by combining noisy images and denoising the result.

Are you saying that the single frame image denoising is NOT an option if you are working with a stack?  If so, are there any limitations in running ImageIntegration beyond using an averaging method?  For example, are there any limitations on which rejection method we can use, whether we are better off with normalization set at additive or additive with scaling, or which scale estimator to use?
Hi Jim,

Yes. If stacking frames do not denoise the individual frames, rather denoise the resulting stack. If you denoise the individual frames you risk the loss of weak signals that rise above noise only after combination. Basically a weak signal might look like noise in the frame and be removed, but after combination the signal is stronger than noise and would not be removed. I have observed this loss and it can be significant, the deeper the stack the worse the loss.

Denoising a single frame is useful for survey work, for example, where you capture one frame and want to see how it might look with more time.

There are no limitations or preference on rejection method. No limitations or preference on scale estimator. For output normalization I suggest Additive with scaling. The ImageIntegration documentation recommends Additive with scaling for lights. I am experimenting with a scaling only option (no additive correction, not yet supported by ImageIntegration) that may turn out to be an good alternative.

Also, the script doesn't work properly for combinations of images captured at different pixel resolutions with different equipment.

Thanks,
Mike
 

chris.bailey

Well-known member
Jan 6, 2010
235
0
Mike

It seems to work better if its the first operation post stacking (other than perhaps cropping dither borders) but certainly doing it before DBE seems to me to yield better results than the other way around. Is that to be expected ?

Chris
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
Yes Chris that is expected. Thank you for reporting your experience. The script's noise estimates will be less accurate on processed images.

Thanks
Mike