PixInsight Forum (historical)

PixInsight => Release Information => Topic started by: Juan Conejero on 2013 May 16 11:26:46

Title: TGVDenoise Example: Gamma Cygni Region
Post by: Juan Conejero on 2013 May 16 11:26:46
Philippe Bernhard has kindly uploaded a set of six linear images (http://pixinsight.com/forum/index.php?topic=5606.msg38424#msg38424) for us to test TGVDenoise out. In this first example I'm going to show how easily we can achieve a very good noise reduction when the data are excellent.

The original image is an integration of 20 light frames of the Gamma Cygni region:

(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/01-tn.jpg) (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/01.jpg)
Click to view a full size version (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/01.jpg)

The six images that Philippe has uploaded are of excellent quality, as you'll see in the next screenshots. This is the linear output of the ImageIntegration process, so we need ScreenTransferFunction (STF) in order to see the image on the screen. For newbie users, the STF tool in PixInsight applies a histogram transformation to the data that are sent to the screen, but does not change the actual image. This way we can work with linear images without stretching them. This is very important because data linearity is necessary for some essential image processing algorithms and techniques, such as color calibration, background modeling and deconvolution. There are also some tools that can be applied to both linear and nonlinear images, but perform much better with linear data.

For noise reduction of linear images, it is very important to carefully adjust STF settings, so that we can see the background, and at the same time, bright areas without saturation. Usually the automatic STF stretch is too aggressive for this purpose. The next screenshot shows the original image enlarged 4:1 with the custom STF that I've used in this example.

(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/02-tn.jpg) (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/02.png)
Click to view a full size version (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/02.png)

And without more preambles, this is the same area after TGVDenoise:

(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/03-tn.jpg) (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/03.png)
Click to view a full size version (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/03.png)

As you probably have already discovered if you have tried out this tool, the default TGVDenoise parameters have been optimized for nonlinear images. For linear images, the default parameters are way too aggressive, and usually have to be lowered by one order of magnitude or even more.

Something that will cause some surprise is the fact that I haven't used a local support in this example. This will give you a better idea of the power of TGVDenoise: For images with relatively high SNR, this tool is able to remove virtually all of the noise, preserving significant structures remarkably well without any help from external data.

Once you get some experience with the tool, finding the correct parameters becomes very easy, especially for high-quality data. It has taken me about five minutes to achieve the noise reduction that I was looking for with this image. I'll show you the result on two additional previews for comparison.

Original image:
(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/04-tn.jpg) (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/04.png)
Click to view a full size version (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/04.png)

After TGVDenoise:
(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/05-tn.jpg) (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/05.png)
Click to view a full size version (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/05.png)

Original image:
(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/06-tn.jpg) (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/06.png)
Click to view a full size version (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/06.png)

After TGVDenoise:
(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/07-tn.jpg) (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/07.png)
Click to view a full size version (http://forum-images.pixinsight.com/20130516/TGVDenoise/01/07.png)

And as usual, some crops of special interest zoomed 3:1 (left: original image, right: after TGVDenoise).

(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/crop01-original.png)(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/crop01-final.png)

(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/crop02-original.png)(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/crop02-final.png)

(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/crop03-original.png)(http://forum-images.pixinsight.com/20130516/TGVDenoise/01/crop03-final.png)

Now this image can be printed at a size of 1 meter by 80 cm at 300 dpi without problems (about 40 x 30 inches), and navigating through it onscreen zoomed 4x is a real joy.

Thank you Philippe for allowing me to work with your nice data.
Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: Nocturnal on 2013 May 16 11:34:03
Nice!
Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: Philippe B. on 2013 May 16 23:48:58
Hi Juan

not bad, not bad  :cheesy: :cheesy: :cheesy: :cheesy:

I tried yesterday with local support but didn't find something nice. So I come back without it.
I have worked on the importance of the couple "strength" + "edge", decrease one while increase other and I can see differences in rendition in the dark structures. but tends to get a "fuzzy" effect not really nice, whereas a completely smooth dark areas is not nice too !!!!  ???

Feel free to play with my images and post some examples (if you want, I have the rest of the serie because I gave you the less signal ones taken with S-II or O-III filter. My Ha layers are much better.

Thanks for this new function which will fill the next rainy week-end  :'(
Cheers
Philippe

 
Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: Philippe B. on 2013 May 17 00:49:37
me again

Some examples in 1:1 format

Original
(http://www.pixinsight.fr/fichiers/0.png)

Juan, with your settings, I got some "fuzzy" pattern in the darkest areas)
(http://www.pixinsight.fr/fichiers/1.png)

with 500 iterations, it is much better but smoother
(http://www.pixinsight.fr/fichiers/2.png)

with a mask (not a local support), it is less smoother and a very little noise is still there which is not bad at all
(http://www.pixinsight.fr/fichiers/3.png)

4 images Animated with 1s interval
 (http://www.pixinsight.fr/fichiers/TGV2.gif)

Philippe
Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: Philippe B. on 2013 May 17 00:58:34
with 200% original and last one give :

(http://www.pixinsight.fr/fichiers/TGV3.gif)
Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: Juan Conejero on 2013 May 17 03:02:29
Hi Philippe,

For noise reduction of linear images, the STF is critical. In your tests, you've used a much more aggressive STF than the one I used in my example. It's not that your STF is incorrect, by no means; it's just that it reveals much more noise in low-SNR areas of the image, as a result of a stronger nonlinear stretch.

I have tried to emulate the STF you've used, and have repeated my example using it:

(http://forum-images.pixinsight.com/20130517/TGVDenoise/01/01-tn.jpg) (http://forum-images.pixinsight.com/20130517/TGVDenoise/01/01.png)
Click to view a full size version (http://forum-images.pixinsight.com/20130517/TGVDenoise/01/01.png)

The image shown above represents a heavy nonlinear stretch that would require some dynamic range compression (IMO) with the HDRMultiscaleTransform tool.

On this screenshot you can also see the TGVDenoise parameters that I've used in this second example. This time I've enabled the local support, using the target image and the same midtones balance and shadows clipping point values that I have set for STF. In other words, the local support is just what you see on the screen.

Also note that I have decreased the smoothness parameter to 0.3. This has allowed me to control the degree of smoothness on low-SNR regions.

As for the number of iterations, beyond 200 iterations or so the results don't change visibly with these parameters, so the process converges quickly.

I think the results are very nice with this configuration. Here are three comparisons zoomed 2:1 without interpolation.

(http://forum-images.pixinsight.com/20130517/TGVDenoise/01/crop01-original.png)(http://forum-images.pixinsight.com/20130517/TGVDenoise/01/crop01-final.png)

(http://forum-images.pixinsight.com/20130517/TGVDenoise/01/crop02-original.png)(http://forum-images.pixinsight.com/20130517/TGVDenoise/01/crop02-final.png)

(http://forum-images.pixinsight.com/20130517/TGVDenoise/01/crop03-original.png)(http://forum-images.pixinsight.com/20130517/TGVDenoise/01/crop03-final.png)

In my opinion, all of the nebular structures that can be seen on the denoised images over dark regions are real, in the sense that they have been recorded in your data set. The noise reduction process just makes them more visible, which is a very nice result. You can verify this visually by comparing the before and after crops.

As for mixing the original and denoised images with a mask, it's clearly an option, but I think it isn't necessary with advanced noise reduction algorithms such as MultiscaleMedianTransform and TGVDenoise. A drawback of reinserting some of the original noise in the denoised image is that it creates staircase artifacts, i.e. relatively hard transitions at the edges of low-SNR structures. Personally, I lost the need for comfort noise (http://en.wikipedia.org/wiki/Comfort_noise) long time ago (that happens when you design noise reduction tools :) ), and definitely prefer smooth gradients.
Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: Philippe B. on 2013 May 17 03:12:57
Juan, I like when you go deeper in your analysis  >:D >:D >:D >:D >:D
You see when you want !!!  :laugh: :laugh: :laugh:
I like your result too  :-*

Glad to see there can be a discussion on noise reduction technics and what is the final goal we want to achieve

For example, here, I wanted a hard result without increase the noise. So STF was very intensive, that's right !
I just got this result of the final image (reduced 50%)



(http://www.astroccd.eu/images/GammaCygne2_MR.jpg)
Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: Juan Conejero on 2013 May 17 04:07:32
Gorgeous image. Sorry, I couldn't resist  O:)

(http://forum-images.pixinsight.com/20130517/TGVDenoise/01/test-1.jpg)

(disclaimer: I haven't paid attention to the stars, so the brightest ones are slightly concave)

It's nice to see how the same data can lead to different interpretations, different ways to understand the properties and limitations of the image with good analysis and processing tools, but always respecting the data. The people that paint their images (you know, brushes, pencils, 'selective sharpening' by hand and all that stuff) really don't know how much are those practices blocking their creativity.
Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: cs_pixinsight on 2013 May 21 11:21:04
Philippe, is there an issue with your website?  None of the images you post seem to show up, including your Avatar.  :sad:

Craig
Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: Nocturnal on 2013 May 21 12:01:25
All images on this page load fine here including Philippe's avatar.
Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: Philippe B. on 2013 May 21 23:50:37
It should work fine as right now
Maybe there was a shutdown during some minutes (it can happen sometimes)


Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: mads0100 on 2013 June 05 18:56:21
So, I can use TGVDenoise to reduce noise prior to stretching the image?  Is that what you're all hinting at?
Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: Nocturnal on 2013 June 05 19:55:43
Well, by saying it works on linear images there is no hinting going on. It can't be more explicit and clear :)
Title: Re: TGVDenoise Example: Gamma Cygni Region
Post by: Philippe B. on 2013 June 05 23:31:12
TGV is the BEST denoising algorithm for LINEAR image !

What else ?