New script for denoising linear monochannel images

troypiggo

Well-known member
Jul 25, 2011
258
0
Brisbane, AUSTRALIA
I notice that the number of interpolation algorithms in the script's drop-down do not match all of the options that star alignment.  eg Bicubic spline is available, but SA has bicubic spline and bicubic b-spline and so on.  I've been using b-spline is SA.  Results using just bicubic spline as closest match in Mure Denoise appear ok.  Just curious if that's right, or should we limit use of SA algorithms to match the options in MureDN?
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
Troy, what you are doing is fine. When using any of the missing methods, check for excessive denoising artifacts (eg, pixel checker-boarding) and reduce Variance Scale slightly if you see problems. Sorry for the lack of support, I didn't spend the necessary development time thinking that these other methods were not often used.
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
MureDenoise 1.22 release, should appear as an auto-update soon.

When you apply several processes to a preview, the preview's base image is normally used as input to each process. The results from previous processes are normally discarded, unless you use the command Preview > Store.

Up until this new release, MureDenoise did not work this way. If you applied MureDenoise multiple times to a preview (say to test different parameter settings) previous results were not discarded and you ended up denoising a previously denoised result, a bad thing to do. The new release fixes this problem. Thank you ngc1535 very much for finding and reporting this bug.

Also, the new release has updated documentation and tooltips for several of the script's parameters. This should help with ease of use.

 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
Jim,

Thank you, I am glad you are getting better results.

On the SBIG, have you seen significant variation of the median bias level in bias frames? I saw this on my SBIG sometimes. This can make signal and noise harder to distinguish, which can cause trouble in the script.

Mike
 

blinky

Well-known member
Jan 14, 2012
81
0
I cant get this to make any difference to an Ha image I took with my ASI1600mono.  I have the gain as 200 (that's what was set in SGPro), the offset as o (since it is already calibrated and flat frame subtracted etc) and the noise, I took the 1.2e figure quoted on ZWO website and divided by the Gain (200) but the before & after images look exactly the same, am I doing something wrong?
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
blinky,

This "ZWO gain 200" value has dB units, so it is not detector gain, rather it is IMO some sort of camera control parameter.

Looking at the charts on this page astronomy-imaging-camera.com/product/asi1600mm-cool, for "ZWO gain 200", detector gain is about 0.5 e-/DN and detector read noise is about 1.5 e-.

So I suggest for MureDenoise you try values 0.5 e-/DN for detector gain and 1.5 / 0.5 = 3 DN for detector gaussian noise.

As a double check, if you have two bias frames, try using the script DarkBiasNoiseEstimator. Select the two bias frames. The provided "Temporal noise" value is an estimate of detector gaussian noise in DN that you can use in MureDenoise. Of course, it should be about 3 DN assuming your camera is operating as expected.

Also, if you have two well-exposed flat frames (ie, neither underexposed nor overexposed), try using the script FlatSNREstimator. Select the two flat frames, and also one bias frame. The provided "Gain" is an estimate of detector gain in e-/DN which you can use in MureDenoise. Again, assuming everything is as expected the value should be about 0.5 e-/DN. Note it is very important that your flats are well-exposed (if not, garbage in garbage out).

Mike
 

blinky

Well-known member
Jan 14, 2012
81
0
Only thing is, if I load the variance scale from the stack, then the denoise does not work, the before & after images are the same, leave it at 1 and its fine though
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
blinky said:
Only thing is, if I load the variance scale from the stack, then the denoise does not work, the before & after images are the same, leave it at 1 and its fine though
If the script appears not to be working, check the "Method noise" value in the process console log. If non-zero, then the script is in fact doing something. Depending on the image, the current STF stretch, and especially for very deep stacks where the noise level has been driven down significantly, the difference can be hard to see or distinguish from typical background variations due to unresolved stars, galaxies, and nebulae.

Regarding the loaded variance scale, the loaded value is an estimate that unfortunately does not completely account for the image normalization and weighting operations done by ImageIntegration. Feel free to adjust this value. As a double check, you can redo your processing using Nearest Neighbor interpolation in StarAlignment, and both output normalization and image weighting turned off in ImageIntegration. Doing so basically removes all of PI's pixel level modifications and hence best preserves image noise statistics, and so makes the script's job more straightforward.
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
MureDenoise 1.23 should appear soon as an auto update. Change log:

  • Addressed a PixInsight 1.8.8 metadata compatibility issue involving load variance scale.
  • Removed support for load variance scale from a process log file.
  • Improved the handling of ill-conditioned linear systems that may arise when optimizing the denoising process.
  • Included an optional gradient classifier to provide a minor increase in the adaptivity and accuracy of the denoising process, but also increases processing time by about 50%.

Include gradient classifier is a new checkbox in the dialog. At least on my projects, its denoising improvements are minor and typically visible only with boosted STF stretch. Two examples are shown below. The differences may be more visible if you view them on your computer in a darkened room.

Example 1: 20 hour integration


Example 1: Include gradient classifier disabled


Example 1: Include gradient classifier enabled


Example 2: 15 hour integration


Example 2: Include gradient classifier disabled


Example 2: Include gradient classifier enabled

 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
Here is another example of the Include gradient classifier option. A 20 hour Ha integration of a higher contrast target, for which the gradient classifier appears to be effective. The crops are shown at boosted stretch.

Original 20 hour integration, showing the crop location


Crop of original


Include gradient classifier disabled


Include gradient classifier enabled
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
MureDenoise 1.24 should appear soon as an auto update.

MureDenoise 1.24 uses a new scheme to account for ImageIntegration's normalization, weighting, and output rescaling operations. Rather than load Variance scale, you enable the option Use image metadata. When enabled, the new scheme estimates and applies a linear image transformation for improved accuracy in the accounting.

When Use image metadata is enabled, Variance scale remains unchanged. Start with a value of 1, and then as in previous versions, you may increase or decrease its value to control the amount of denoising. Combination count will be loaded.

The following ImageIntegration settings are supported: Image Integration > Combination: Average. Image Integration > Normalization: No normalization, Additive, or Additive with scaling. Image Integration > Weights: all values. The script is not smart enough to handle other settings, it will apply no transformation in these cases.

Thank you Juan for providing the metadata that makes this improvement possible.

 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
Here are two tips now how to use the latest version of MureDenoise. The UI has changed (which can be annoying sometimes), so you might find this helpful:

1) The normal case, do this:
  • Select the image
  • Check Use image metadata
  • Verify that Combination count is set correctly
  • Set Interpolation method correctly
  • Set Variance scale to 1
  • Denoise
Increase or decrease Variance scale to modify the amount of denoising.

For increased accuracy, but at a price of increased processing time, do one or both of these:
  • Check Include gradient classifier
  • Increase Cycle-spin count
2) A special case where you used Load variance scale... in a previous version of the script and want to recreate that result as close as possible for some reason:
  • Do 1) above, the normal case
  • Look in the process log for the line Use image metadata: true [scale: <scale number>, offset: <offset number>]
  • Uncheck Use image metadata
  • Uncheck Include gradient classifier
  • Set Variance scale to the square of the scale number (eg, if scale number is 1.02, set Variance scale to 1.02 * 1.02 = 1.04)
  • Denoise
Note that the result of 2) will not be as accurate as the result of 1) due to the new version's improvements.
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
I found a bug in the new version 1.24 Use image metadata option, a typo in a variable name.

I apologize for the problem. I will release a new version asap.

Mike
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
MureDenoise 1.25 should appear soon as an auto update.

Fixed bug in Use image metadata due to a typo.

Increased maximum Cycle-spin count to 48.

The text Use image metadata [none] indicates either unavailable image metadata or unsupported ImageIntegration options.
 

bilgebay

Well-known member
Oct 6, 2011
88
2
59
T?rkiye - Marmaris and Istanbul
Hello Mike,

Thanks for the update.

Just wanted to make sure one point; are we still entering the detector data like the gain and the gaussian noise, or is the script getting those from the image somehow ?


Cheers

Sedat
 

mschuster

PTeam Member
Nov 27, 2011
1,137
4
San Francisco
Hello Sedat,

Yes, you still need to enter detector Gain, Gaussian noise, and Offset.

Here are tips on how to find these numbers:

1) Camera manufacturer data: Manufacturers sometimes provide gain in e-/DN and read noise in e-. Use this gain value and for Gaussian noise use read noise divided by gain. For example, if gain is 0.5 e-/DN and read noise is 8 e-, use 8 / 0.5 = 16 DN for Gaussian noise. Set Offset to 0 DN (because the image to be denoised has been calibrated). Note that some cameras allow you to switch between low and high gain, or change the amount of binning, and manufacturers may not provide data covering all of these cases. In situations like this you may need to follow 3) below.

2) Camera manufacturer gain parameter charts: Manufacturers sometimes provide gain parameter charts, here the gain parameter is not actually detector gain but rather a camera control parameter. You have to read gain in e-/DN and read noise in e- from charts. Then you calculate Gaussian noise in DN as in 1) above. Set Offset to 0 DN. See this post for an example.

3) Scripts Image Analysis > FlatSNREstimator and DarkBiasNoiseEstimator: These two scripts provide Gain in e- and Gaussian noise in DN using calibration frames you supply. You need two bias frames and two flat frames. DarkBiasNoiseEstimator provides a value which it calls Temporal noise in DN. Use this value for Gaussian noise. DarkBiasNoiseEstimator also provides Offset, but don't use this value, use 0 DN instead as above. FlatSNREstimator provides Gain, use this and ignore the other provided values.

For long exposures where dark current is more of a factor, you can supply to two dark frames to DarkBiasNoiseEstimator rather than two bias frames. This will give you a Gaussian noise value that accounts for the dark current noise. On the other hand, for FlatSNREstimator, a bias frame along with two well-exposed flat frames suffices.