Author Topic: Care to share your workflow?  (Read 12032 times)

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: Care to share your workflow?
« Reply #15 on: 2012 February 27 10:55:41 »
Quote
I noticed the S value and Adaptive Values are applied less and less agressively with each successive layer with the MultiscaleMedianTransform.

Good observation. At larger dimensional scales, the SNR is larger and hence less noise reduction is required. The successive smaller thresholds reflect the fact that the fraction of coefficients due to noise at each transformation layer is inversely proportional to the scale.

Quote
I was never sure what the adaptive parameter does. Can you expand on this?

Each layer of a MMT transform contains coefficients that represent the structures described by the layer. MMT coefficients are real numbers that can be positive, negative, or zero. This happens also in the wavelet transform.

In each layer, a number of coefficients are due to the noise. The set of noise coefficients is always characterized by relatively small absolute values. The threshold noise reduction parameter specifies a limiting value (in sigma units) to kill noise coefficients: all coefficients with absolute values less than the threshold are set to zero, so their contributions to the reconstructed image (generated by an inverse MMT transform) are eliminated. This simple procedure is known as hard thresholding.

We don't implement hard thresholding in the ATrousWaveletTransform tool because there are other techniques, known as soft thresholding, that work better for wavelets. However, soft thresholding doesn't work for the MMT. For the MultiscaleMedianTransform tool we implement something slightly better than standard hard thresholding using the Amount parameter: each coefficient below the threshold limit is multiplied by 1 - Amount, so when Amount=1 we have pure hard thresholding, and Amount < 1 allows for some fine control.

So in theory we could reduce or remove the noise completely at each layer by just finding the threshold value that divides the coefficients into noise and significant. However, when it comes to noise we always have uncertainty: as we get close to the threshold limit we cannot say for sure if a given coefficient is due to the noise or is supporting a significant image structure. The consequence of this uncertainty is that we always have to set the noise threshold somewhat below its optimum value, in order to be sure that nothing significant will be damaged.

So in practice some noise coefficients always survive after thresholding. If no noise survives, then we are destroying significant data for sure. Surviving noise can be identified as isolated bright pixels on the sky background and other dark regions of the images. These 'artificial hot pixels' are typical artifacts of the first two or three MMT layers. The Adaptive parameter controls a special adaptive noise reduction filter applied to the layer coefficients after the thresholding process. This filter looks for isolated, bright and small structures and selectively removes them. The higher the Adaptive parameter, the more aggressive adaptive filtering effect.

The noise reduction part of the MMT tool is still undergoing extensive development. Some or even all of the noise reduction parameters in MMT can change in future versions. In addition, a new algorithm will be available on this tool as an option: the wavelet-median transform, which is intended to have the best of both worlds.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline sleshin

  • PixInsight Old Hand
  • ****
  • Posts: 431
Re: Care to share your workflow?
« Reply #16 on: 2012 February 28 14:43:49 »
Juan,

A couple of questions regards another of your excellent tutorials.

In step 7 you used the I channel of the HSI color space for the lightness mask. Any special reason for doing it this way rather than using the extracted Cie L component? Or are you just showing another way to make a Lum Mask?

In Step 11 for the Selective noise addition, it appears you are using ACDNR to make the mask. I am unable to figure out how you can apply that mask to the image and then use a different tool, in this case the NoiseGenerator. Explain please.

Steve
Steve Leshin

Stargazer Observatory
Sedona, Arizona

ruediger

  • Guest
Re: Care to share your workflow?
« Reply #17 on: 2012 February 29 06:06:09 »
I would also like to add a question regarding the SCNR step (removal of green residual noise). In every processing example I've seen so far, this one here included, this step is done very late in the processing on stretched data.

With very noisy data, a very late SCNR has impact on overall color balance (background tends to get bluish), so I would expect to do the SCNR and all other noise suppression before background neutralization and color calibration?

RĂ¼diger
« Last Edit: 2012 February 29 07:39:08 by ruediger »

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: Care to share your workflow?
« Reply #18 on: 2012 February 29 11:30:18 »
Quote
In step 7 you used the I channel of the HSI color space for the lightness mask. Any special reason for doing it this way rather than using the extracted Cie L component? Or are you just showing another way to make a Lum Mask?

No special reason at all. You're very smart in detecting that I just wanted to show an alternative way to do this :) Actually, since we are working with a linear image, the CIE L* component makes no sense (CIE L*a*b* is a perceptual space, hence a nonlinear space; there's nothing visually perceptible in a linear image). I could have used simply the green channel, for example, with similar results.

Quote
In Step 11 for the Selective noise addition, it appears you are using ACDNR to make the mask. I am unable to figure out how you can apply that mask to the image and then use a different tool, in this case the NoiseGenerator. Explain please.

ACDNR has a nice mask generation feature integrated. You enable it by checking the Preview option in the Lightness Mask section. When you apply ACDNR to an image with this option enabled, it generated an inverted lightness mask instead of performing its nominal noise reduction task. Just a trick :) We have to write this simple mask generation feature as an independent tool.

Quote
With very noisy data, a very late SCNR has impact on overall color balance (background tends to get bluish), so I would expect to do the SCNR and all other noise suppression before background neutralization and color calibration?

If you apply SCNR with its default average neutral protection method, its impact on a neutral background should be small, and its impact on the white balance should be marginal. However, its impact will always be larger for nonlinear data, so you're right: strictly, it should be applied before neutralizing the background. However, usually one detects that green noise is a problem during late processing stages, when going back to the beginning just for that reason is overly expensive. We have a good solution to this problem though: just apply AutoHistogram with the appropriate target median values and its default gamma stretch method. It will neutralize the background without altering the overall color balance.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline sleshin

  • PixInsight Old Hand
  • ****
  • Posts: 431
Re: Care to share your workflow?
« Reply #19 on: 2012 March 04 09:27:49 »
Thanks Juan for the reply. The ACDNR "trick" to create a permanent copy of the mask is particularly useful. Whenever you reveal one of these cool PI features or show us a practical example of how to use a tool,  I'm always amazed and appreciative. So, please keep the "tips" coming.

Thanks again,

Steve
Steve Leshin

Stargazer Observatory
Sedona, Arizona

Offline skoop

  • Newcomer
  • Posts: 11
Re: Care to share your workflow?
« Reply #20 on: 2012 March 24 07:45:00 »
...  I ask as I have 5 hours of Luminance sat on the hard drive!!

Can you please post the L frames too.
We have only snow, clouds and rain here :) I like to try out Juan's work flow.
Thank you.