New script for denoising linear monochannel images

I notice that the number of interpolation algorithms in the script's drop-down do not match all of the options that star alignment.  eg Bicubic spline is available, but SA has bicubic spline and bicubic b-spline and so on.  I've been using b-spline is SA.  Results using just bicubic spline as closest match in Mure Denoise appear ok.  Just curious if that's right, or should we limit use of SA algorithms to match the options in MureDN?
 
Troy, what you are doing is fine. When using any of the missing methods, check for excessive denoising artifacts (eg, pixel checker-boarding) and reduce Variance Scale slightly if you see problems. Sorry for the lack of support, I didn't spend the necessary development time thinking that these other methods were not often used.
 
MureDenoise 1.22 release, should appear as an auto-update soon.

When you apply several processes to a preview, the preview's base image is normally used as input to each process. The results from previous processes are normally discarded, unless you use the command Preview > Store.

Up until this new release, MureDenoise did not work this way. If you applied MureDenoise multiple times to a preview (say to test different parameter settings) previous results were not discarded and you ended up denoising a previously denoised result, a bad thing to do. The new release fixes this problem. Thank you ngc1535 very much for finding and reporting this bug.

Also, the new release has updated documentation and tooltips for several of the script's parameters. This should help with ease of use.

 
I have just released the corresponding updates for MureDenoise 1.22 and its documentation.

Mike, thank you so much for your continued excellent work. Thank you also Adam for your help and support.
 
Mike,

Just a new thank you.  I was struggling using MureDN with my SBIG 6303E but I has recently switched it out for an FLI Microline 16803 and Mure is working beautifully again.  It is just a just a joy using such a well designed script.

Best,

Jim
 
Jim,

Thank you, I am glad you are getting better results.

On the SBIG, have you seen significant variation of the median bias level in bias frames? I saw this on my SBIG sometimes. This can make signal and noise harder to distinguish, which can cause trouble in the script.

Mike
 
I cant get this to make any difference to an Ha image I took with my ASI1600mono.  I have the gain as 200 (that's what was set in SGPro), the offset as o (since it is already calibrated and flat frame subtracted etc) and the noise, I took the 1.2e figure quoted on ZWO website and divided by the Gain (200) but the before & after images look exactly the same, am I doing something wrong?
 
blinky,

This "ZWO gain 200" value has dB units, so it is not detector gain, rather it is IMO some sort of camera control parameter.

Looking at the charts on this page astronomy-imaging-camera.com/product/asi1600mm-cool, for "ZWO gain 200", detector gain is about 0.5 e-/DN and detector read noise is about 1.5 e-.

So I suggest for MureDenoise you try values 0.5 e-/DN for detector gain and 1.5 / 0.5 = 3 DN for detector gaussian noise.

As a double check, if you have two bias frames, try using the script DarkBiasNoiseEstimator. Select the two bias frames. The provided "Temporal noise" value is an estimate of detector gaussian noise in DN that you can use in MureDenoise. Of course, it should be about 3 DN assuming your camera is operating as expected.

Also, if you have two well-exposed flat frames (ie, neither underexposed nor overexposed), try using the script FlatSNREstimator. Select the two flat frames, and also one bias frame. The provided "Gain" is an estimate of detector gain in e-/DN which you can use in MureDenoise. Again, assuming everything is as expected the value should be about 0.5 e-/DN. Note it is very important that your flats are well-exposed (if not, garbage in garbage out).

Mike
 
Only thing is, if I load the variance scale from the stack, then the denoise does not work, the before & after images are the same, leave it at 1 and its fine though
 
blinky said:
Only thing is, if I load the variance scale from the stack, then the denoise does not work, the before & after images are the same, leave it at 1 and its fine though

If the script appears not to be working, check the "Method noise" value in the process console log. If non-zero, then the script is in fact doing something. Depending on the image, the current STF stretch, and especially for very deep stacks where the noise level has been driven down significantly, the difference can be hard to see or distinguish from typical background variations due to unresolved stars, galaxies, and nebulae.

Regarding the loaded variance scale, the loaded value is an estimate that unfortunately does not completely account for the image normalization and weighting operations done by ImageIntegration. Feel free to adjust this value. As a double check, you can redo your processing using Nearest Neighbor interpolation in StarAlignment, and both output normalization and image weighting turned off in ImageIntegration. Doing so basically removes all of PI's pixel level modifications and hence best preserves image noise statistics, and so makes the script's job more straightforward.
 
MureDenoise 1.23 should appear soon as an auto update. Change log:

  • Addressed a PixInsight 1.8.8 metadata compatibility issue involving load variance scale.
  • Removed support for load variance scale from a process log file.
  • Improved the handling of ill-conditioned linear systems that may arise when optimizing the denoising process.
  • Included an optional gradient classifier to provide a minor increase in the adaptivity and accuracy of the denoising process, but also increases processing time by about 50%.

Include gradient classifier is a new checkbox in the dialog. At least on my projects, its denoising improvements are minor and typically visible only with boosted STF stretch. Two examples are shown below. The differences may be more visible if you view them on your computer in a darkened room.

Example 1: 20 hour integration
light_integration_30_crop.png


Example 1: Include gradient classifier disabled
light_integration_30_early2017_crop.png


Example 1: Include gradient classifier enabled
light_integration_30_late2019_crop.png


Example 2: 15 hour integration
light_integration_22_crop.png


Example 2: Include gradient classifier disabled
light_integration_22_early2017_crop.png


Example 2: Include gradient classifier enabled
light_integration_22_late2019_crop.png

 
Here is another example of the Include gradient classifier option. A 20 hour Ha integration of a higher contrast target, for which the gradient classifier appears to be effective. The crops are shown at boosted stretch.

Original 20 hour integration, showing the crop location
2019_09.png


Crop of original
light_integration_31_crop1.png


Include gradient classifier disabled
light_integration_31_early2017_crop1.png


Include gradient classifier enabled
light_integration_31_late2019_crop1.png
 
MureDenoise 1.24 should appear soon as an auto update.

MureDenoise 1.24 uses a new scheme to account for ImageIntegration's normalization, weighting, and output rescaling operations. Rather than load Variance scale, you enable the option Use image metadata. When enabled, the new scheme estimates and applies a linear image transformation for improved accuracy in the accounting.

When Use image metadata is enabled, Variance scale remains unchanged. Start with a value of 1, and then as in previous versions, you may increase or decrease its value to control the amount of denoising. Combination count will be loaded.

The following ImageIntegration settings are supported: Image Integration > Combination: Average. Image Integration > Normalization: No normalization, Additive, or Additive with scaling. Image Integration > Weights: all values. The script is not smart enough to handle other settings, it will apply no transformation in these cases.

Thank you Juan for providing the metadata that makes this improvement possible.

2013_11_light_integration_8_dialog.png
 
Here are two tips now how to use the latest version of MureDenoise. The UI has changed (which can be annoying sometimes), so you might find this helpful:

1) The normal case, do this:
  • Select the image
  • Check Use image metadata
  • Verify that Combination count is set correctly
  • Set Interpolation method correctly
  • Set Variance scale to 1
  • Denoise
Increase or decrease Variance scale to modify the amount of denoising.

For increased accuracy, but at a price of increased processing time, do one or both of these:
  • Check Include gradient classifier
  • Increase Cycle-spin count
2) A special case where you used Load variance scale... in a previous version of the script and want to recreate that result as close as possible for some reason:
  • Do 1) above, the normal case
  • Look in the process log for the line Use image metadata: true [scale: <scale number>, offset: <offset number>]
  • Uncheck Use image metadata
  • Uncheck Include gradient classifier
  • Set Variance scale to the square of the scale number (eg, if scale number is 1.02, set Variance scale to 1.02 * 1.02 = 1.04)
  • Denoise
Note that the result of 2) will not be as accurate as the result of 1) due to the new version's improvements.
 
I found a bug in the new version 1.24 Use image metadata option, a typo in a variable name.

I apologize for the problem. I will release a new version asap.

Mike
 
MureDenoise 1.25 should appear soon as an auto update.

Fixed bug in Use image metadata due to a typo.

Increased maximum Cycle-spin count to 48.

The text Use image metadata [none] indicates either unavailable image metadata or unsupported ImageIntegration options.
 
Hello Mike,

Thanks for the update.

Just wanted to make sure one point; are we still entering the detector data like the gain and the gaussian noise, or is the script getting those from the image somehow ?


Cheers

Sedat
 
Hello Sedat,

Yes, you still need to enter detector Gain, Gaussian noise, and Offset.

Here are tips on how to find these numbers:

1) Camera manufacturer data: Manufacturers sometimes provide gain in e-/DN and read noise in e-. Use this gain value and for Gaussian noise use read noise divided by gain. For example, if gain is 0.5 e-/DN and read noise is 8 e-, use 8 / 0.5 = 16 DN for Gaussian noise. Set Offset to 0 DN (because the image to be denoised has been calibrated). Note that some cameras allow you to switch between low and high gain, or change the amount of binning, and manufacturers may not provide data covering all of these cases. In situations like this you may need to follow 3) below.

2) Camera manufacturer gain parameter charts: Manufacturers sometimes provide gain parameter charts, here the gain parameter is not actually detector gain but rather a camera control parameter. You have to read gain in e-/DN and read noise in e- from charts. Then you calculate Gaussian noise in DN as in 1) above. Set Offset to 0 DN. See this post for an example.

3) Scripts Image Analysis > FlatSNREstimator and DarkBiasNoiseEstimator: These two scripts provide Gain in e- and Gaussian noise in DN using calibration frames you supply. You need two bias frames and two flat frames. DarkBiasNoiseEstimator provides a value which it calls Temporal noise in DN. Use this value for Gaussian noise. DarkBiasNoiseEstimator also provides Offset, but don't use this value, use 0 DN instead as above. FlatSNREstimator provides Gain, use this and ignore the other provided values.

For long exposures where dark current is more of a factor, you can supply to two dark frames to DarkBiasNoiseEstimator rather than two bias frames. This will give you a Gaussian noise value that accounts for the dark current noise. On the other hand, for FlatSNREstimator, a bias frame along with two well-exposed flat frames suffices.
 
Back
Top