Noise Reduction: Linear or Non-Linear

[iksose7]

Hi guys,

I finally went ahead and got myself the PI commercial license today. I got hooked on the trial a couple months back and have refused to process anything until i had it back so i have a few images to work through starting tomorrow.

I have a nice dusty image of NGC 1333 that i am going to be playing with over the coming weeks. The data looks good with only the STF and ABE applied and although i have 19 hours exposure its still pretty noisy as i am using a DSLR.

My question is regarding at what point do you guys think it is best to apply noise reduction. Obviously it can be applied at different stages multiple times, and different users will have different preferences on when they will apply it. During my trial i was applying NR during the linear stage, my thinking that otherwise when it comes to going non-linear i will only be stretching noise. Is my thinking correct? Are there any advantages to waiting to the non-linear stage to apply NR?

Thanks
Callum

[jkmorse]

Callum,

I was never happy with linear noise reduction until the MultiscaleLinearTransform (MLT) tool came along.  It has a built in mask and produces wonderful results in the linear stage where it is designed to act.  Give it a try and I think you will be pleased with the result.

Best,

Jim

[msmythers]

Callum,

I second what Jim said. MultiscaleLinearTransform has changed my processing. Most of the time I have no need for any non-linear noise reduction after using MLT before stretching. Very powerful but not difficult to use. The M81/M82 example is a great place to start with learning this tool.

http://www.pixinsight.com/examples/M81M82/index.html#Noise_Reduction


Mike

[jkmorse]

Take a close look at the settings in the tutorial Mike noted.  Those are a great place to start and will give you very good results.  Only thing I have changed from tutorial and tool notes is that I use the K-Sigma Noise Thresholding, with starting settings of threshold 3 and amount 80%, with both the soft Thresholding and Multi-resolution Support boxes checked.  I have found that this produces even better results so worth a try.

Best,

Jim

[iksose7]

Thanks for confirming my suspicions guys, linear is what i was thinking. I'll have a read through that M81+M82 tutorial tonight Mike and have a play next image i process, i'll try your suggestion also Jim.

I just got done processing an image of M33 yesterday, i was using ATWT but ended up only applying a micro dose to the background rather than the entire image as i am frightened of over doing it like i have in the past! I also used MLT but again only in a micro dose.

Thanks again
Callum

[Corries]

I have been following this thread and I have a question.  I have been using ATrousWaveletTransform (at the linear stage) for noise reduction.  How does this method compare with MultiscaleLinearTransform process?  I would appreciate any feedback as I am rather new to PI.

[msmythers]

tcorridan

Here is what Juan wrote about MultiscaleLinearTransform.

* The new MultiscaleLinearTransform (MLT) tool replaces ATrousWaveletTransform (which is now deprecated but maintained for compatibility). MLT introduces a new linear multiscale transform based on multiple Gaussian filters, besides the starlet transform (aka à  trous wavelet transform).

* The MultiscaleMedianTransform (MMT) tool introduces a new median-wavelet transform that splits the multiscale representation optimally into wavelets for nonsignificant structures and the median transform for high-contrast structures.

This can be found here.  http://pixinsight.com/forum/index.php?topic=6972.msg47043#msg47043



Mike

[iksose7]

So basically MLT is a new and superior NR routine than ATWT but ATWT has been kept just in case anybody needs it for any purpose?

I may have been confused in my last post, when i said i used MLT as a second micro dose of noise reduction i may have been meaning MMT. I will have a go today at using MLT as a primary NR followed by MMT.

Callum

[msmythers]

Callum,

As tools become superseded by completely new processes they could be removed from the software. I've seen this many times with other types of engineering software. The problem arises when someone needs to work on an old 'project'. If the tools are no longer in the latest software they must rethink and expend much time to recreate the project, sometimes having to start from scratch.  In PI we can save complete 'projects'. Every image during the making of the project and a detailed order of the every step used are kept in the project. That order includes every parameter used by a tool during the project. So when we recall a project the tools that were used when the project was created should be available. If the tool is no longer available in the latest software revision the project won't work as it did. You loose backwards compatibility. This is a problem. Since the tools are kept in PixInsight I can call up a project from 2 years ago when I was using ATWT and operate within the project. I can then opt to change that process within the project to MLT and resave the project or just continue with ATWT.

This is a major part of the power of  the PixInsight software package. This type of flexibility and compatibility.

MMT is normally used for what you would call sharpening. That's it strong suit. Even though both are similar in many functions it's how they operate in the code. MMT is excellent at handling ringing which can be a problem when trying to get the best details in an image.

Mike 

[iksose7]

Hi Mike,

Thanks for that explanation. It makes sense.

Going a little off topic from the noise reduction here but may as well just ask another question while i'm here and the topic has kind of moved to flexibility of PI, granted the question may be a stupid one as i am new and dont know much about this.

If i was to process an image say with Deconvolution near the start of the process then finish the image, could i go into the history explorer and remove that application of deconvolution from the workflow without disturbing any of the processes that followed? Or would all the processes after the decon also disappear from the stack, kind of like removing the basement from a building and having the rest of the floors come crashing down.

I can understand if this is not a possibility. To be honest i really cant see how it would work as the following processes would be based on that application of deconvolution but i thought i would ask anyway!

Thanks
Callum

[msmythers]

Callum,

Go to that point before the deconvolution, make a clone of the image, then apply each step from the history of the original to the clone. If you were to make any change to the original and resaved the project you could lose your original project steps. At least when I've made that mistake that's what's happened. Beware though project files can become very, very large because every step and that image from that step which is normally in temporary memory is saved in that project file.


Mike

[iksose7]

Thanks again Mike, i've taken what you have said on board and will try it out in the future, probably tomorrow!

Callum

[Warhen]

Wow, this is great! If Juan keeps this up, RBA and I may have to go back to work. I always got excellent linear results with ATWT settings suggested in PI-2 of our Part-2 tutorials http://www.ip4ap.com/pixinsight_part2.htm, but I see that MLT may work a tad better! And, Jim Morse, your idea does seem very good. Thanks a lot guys!

Do we know if MLT then becomes the recommended sharpener (Bias push) as well, rather than ATWT?

[JoLo]

I have played with all the noise reduction tools PI has to offer, based on various tutorials and workflows I have run across, and my own experience.   I tried to get TGVDenoise to work for me, and have difficulty in mastering it, although many point to its superiority as a tool for NR.

I ran across Kayron Mercieca's workflow on his blog (link below) and find it to be the best solution i have encountered.  Both ATWT and MMT are used in the linear state, the images (Lum/Ha and RGB) are then stretched using HT, and ACDNR is applied as a final touch in the non-linear state.  It works so well for me that I rarely have to change the defaults he uses in the tutorial, sometimes having to tone down the application of ACDNR a bit.

I always struggled with getting the right mix of smoothing vs ending up with the curdled or plastic looking background.  The judicious applicaton of masks (step by steps included in the workflow) with the three NR tools leaves me with a nicely smoothed image, a clean, non-lumpy background, and the high SNR target with a bit of noise left in.  I really don't use any other workflow anymore, worth a try!

Joe

http://lightvortexastronomy.blogspot.com/2014/07/tutorial-post-processing-technique.html

[Warhen]

JoLo, I get great results with TGVD nonlinearly. Can't seem to escape mottling linearly, so I've had great success with ATWT, now MLT as in the above posts.