New script for denoising linear monochannel images

[mschuster]

The new MureDenoise version 1.8 is available in the head post.

The default denoising performed is unchanged.

The new Variance scale parameter lets you decrease or increase the default amount of denoising. Note that excessively large variance scale values risk the generation of denoising artifacts and the loss of signal-to-noise ratio (SNR).

The variance scale parameter can also be set based on information from an ImageIntegration process log file. The log file is a manually created .txt file containing a copy of the Process Console log generated by ImageIntegration. This feature results in more accurate denoising by taking into account the image output normalization and weighting performed by the combination process. Chris, this feature should help address the issues you see.

In the head post I added a warning concerning the script's inability to remove correlated noise introduced by the image registration process. For combinations, the script estimates the amount of correlated noise introduced by the registration pixel interpolation method, and then reduces the nominal amount of denoising so as to not inadvertently reduce SNR.

Thanks,
Mike

[chris.bailey]

Mike

I have run a few of the previous 'problem' images through it this morning using the variance scale set from the Image Integration Log and it is a vast improvement. Typically the Variance Scale on these was in the 1.1 to 1.2 range but that was enough to make the difference. Doing the same on some of the better image run resulted in a Variance Scale of less than one (as low as 0.8) but the end result was better than setting it at unity. Variance scale is VERY sensitive to very small changes so setting it using the Integration Log seems the best approach based on the limited testing I have done, the extra step of cutting and pasting is well worth it.

Chris

[mschuster]

Thank you Chris, this is helpful information.

Please post a 0 or near 0 log file in dropbox or inline in a post. I will investigate.

Thanks,
Mike

[chris.bailey]

Mike

Apologies. For some reason the forum changed zero point nine to a zero point smiley face! I have not seen a variance scale of less that zero point eight five in about 20 images.

Chris

[mschuster]

OK Chris, thanks.

The range of values you are seeing is roughly what I would expect. On my data I see maybe 0.9+ to 1.2+.

Note that the numbers you see will depend on which image you select as a reference in ImageIntegration. This is because images will be scaled differently by the process with a different reference.

Thanks,
Mike

[starhopper]

Hello Mike,
thank you very much for this great work. I installed and used it with success. I was able to examine nearly all parameters for my images, even the variance scale. But I am not familiar with the DarkBiasNoiseEstimation script. What image should I provide for darkbias1 and 2?

The standard deviation of detector additive white Gaussian noise parameter is set to an estimate of the quadrature sum of read noise and dark current noise in a 40 minute dark provided by the DarkBiasNoiseEstimator script. Detector offset is set to 0 because the image is dark-subtracted.

I there a possiblility to determine the StarAlignment pixel interpolation. I have set it to auto, and does not find it in the fits header. Is this information lost?

Thomas

[mschuster]

Hi Thomas,

Thank you very much.

For DarkBiasNoiseEstimator, please use two good bias images. The script will report "Temporal noise", which is an estimate of the "read noise" or "system noise" of the detector in DN units. Use this value for "Gaussian noise" in MureDenoise.

Regarding StarAlignment pixel interpolation, in my testing Auto means Lanczos-3 when aligning images with the same pixel resolution. So select Lanczos-3 in MureDenoise. You can double check this choice by looking at the StarAlignment process log, where the actual interpolation method used is listed. Note: some PI documentation says Auto means Lanczos-4 for the no rescaling situation, but in my testing Lanczos-3 is listed in the process log. Probably best to double check.

Thanks,
Mike

[starhopper]

Hello Mike,

thanks for your explanation. Look it seems to work fine with my image. It is a stack of 11x5min OIII frames, registerered with lanzos3 and stacked with average sigma clipping.

[mschuster]

Hi Thomas,

Thank you for posting your results, I am glad to know it is working!

Regards,
Mike

[cs_pixinsight]

Mike, all I can say is WOW!  That last example posted by starhopper is amazing 

Great work as usual.

Craig

[mschuster]

Thank you Craig!

The new MureDenoise Version 1.9 is available in the head post.

This version includes a new large scale flatfield compensation option, which improves denoising quality by compensating for spatial noise variance scaling due to flatfielding.

Flatfield compensation is useful for telescopes with more than ~10% optical vignetting. For telescopes with less vignetting, flatfield compensation results in negligible denoising quality improvement.

The denoising performed by the new version is unchanged unless you use the new option.

Thanks,
Mike

Update: Here is an FSQ flatfield. The thick lines mark the location of the mean exposure, the center is ~2% brighter, the corners up to ~4% dimmer. SNR is lower in the corners (fewer photons). Flatfielding equalizes the exposure, SNR remains unchanged, and so the noise level ends up being relatively higher in the corners. Given a flatfield, the new MureDenoise option varies the amount of denoising across the image appropriately.

[Philippe B.]

Hi Mike
Very nice work ! Thank you !

[jkmorse]

Mike,

I have been playing around with the script and making some interesting findings I want to confirm with you:

1.  While only 19 subs deep, the data set I used for testing is very good, well focused, etc, from my F16M - Planewave CDK12.5 system;
2.  The "default" settings you recommend are not coming up with the best fit solution but by making adjustments in the settings, I am able to get excellent results (almost miraculous actually given that it cleans both the background and the target so well);
3.  The two most sensitive settings for me are Gaussian noise and variance scale (which is probably to be expected).  I get my best results in getting the Gaussian noise set first, then fine tuning using the variance scale;
4.  Here is an example using a year old M81-M82 data set of 19 10 minute images (working in blue):
     a.  Reported gain on my F16M is 1.5 but I get better results lowering to 0.8
     b.  Gaussian noise as measured using two 10 minute dark subs is 26.98 but I have to lower that to around 10
     c.   Measured variance scale shows 1.312 but at that level (and in fact for anything above around 0.85 which I am using), I get "blocky" artifacts, almost like it is pixelating the image.

To put this into visual context, attached in this and the next two posts are excerpts representing my base, recommended settings and adjusted settings images.

Would love to get your thoughts on this experiment.

Thanks,

Jim

p.s. If I forgot to say thank you for this incredible tool, THANKS!!!!

[jkmorse]

Recommended settings image:

[jkmorse]

Adjusted settings version: