TGVDenoise 1.0 Released

Juan Conejero

PixInsight Staff
Staff member
Hi everybody,

I am proud to announce that we have just released a new tool: TGVDenoise, written by PTeam member Carlos Milovic. All users have now access to this tool as a regular update to PixInsight 1.8.0 RC7.

TGVDenoise is a noise reduction tool based on total generalized variation (see references [1...5] at the end of this post), a novel signal processing concept with important applications to image regularization and restoration. As I anticipated in a previous post, TGVDenoise is the first one of a series of TGV-based tools that we have been working on during the last year. Other members of this series are a multiscale, spatially adaptive TGV noise reduction tool (SATGV), and an image restoration (deconvolution) tool. Hopefully we'll be releasing these modules during the next weeks and months. I would like to reiterate my congratulations to Carlos Milovic for his excellent research and development work on these tools.

In the rest of this post I'll describe briefly the main TGVDenoise parameters. We'll be posting noise reduction examples on this board during the next days.

Overview of the TGVDenoise tool

TGVDenoise.png

The TGVDenoise tool has two separate sections: noise reduction parameters at the top side, and local support parameters at the bottom side. While we write an official documentation, let's describe briefly these sections and their corresponding parameters.

Noise Reduction Parameters

* Color working mode.

For color images, TGVDenoise provides two working modes: RGB/K and CIE L*a*b*. In RGB/K mode, the same set of parameters is applied to each nominal RGB channel, for color images, or to the nominal grayscale channel, for monochrome images. Note that for grayscale images the color working mode is irrelevant.

In CIE L*a*b* mode, the target image is first converted to the CIE L*a*b* color space. Lightness parameters (first tab) are then applied to the CIE L* component, and chrominance parameters (second tab) are applied to the CIE a* and b* components. Finally, the image is converted back to the RGB space. This mode allows you to apply a different (usually more aggressive) noise reduction procedure to the chrominance, which makes the tool more flexible and adaptable to a variety of typical denoising problems. You can disable one of the parameter sets, so you can apply noise reduction just to either the lightness or chroma components.

* Strength

This parameter controls the strength of the diffusion process that smoothens the image. Large values are more aggressive, while small values tend to give results that are more similar to the original image. You have to find a good balance between the strength and edge protection parameters (see below).

* Edge protection

This is by far the most critical parameter of the TGVDenoise tool. This parameter works as a threshold for the regularization process, preventing diffusion over edge features stronger than the threshold value. Hence, you use this parameter to prevent the noise reduction process from damaging significant image detail and features. Smaller values are more aggressive to protect edges against blurring caused by the diffusion process. Larger values are more permissive and lead to smoother images, but may cause some detail loss.

* Smoothness

Controls the continuity degree of smooth regions. Smaller values tend to preserve small-scale structures such as sharp edges, but also may generate staircase artifacts. The default value of 2 usually works well with most images, and you normally won't need to change it.

* Iterations

TGVDenoise emulates a diffusion process, following a dynamic evolution scheme. Each iteration can be seen as a time sample, where the image reaches a new state. The process converges to an optimal solution when no significant diffusion can happen between neighbor pixels. This usually requires a large number of iterations. Convergence is evaluated by calculating the norm of the difference between two consecutive iteration steps. This is an estimate of global error, or mean square change. Usually, once this difference drops below 1/255, little or no changes can be perceived in the displayed image.

Upon execution, TGVDenoise writes its final global error estimate to the console as a 'delta' value. You can watch these estimates to get an idea of how well the process is converging to an acceptable result. The default value of the iterations parameter (100 iterations) is a good starting point for most images.

* Automatic convergence

When automatic convergence is enabled, the process will be stopped if either the difference between two successive iterations (see the iterations parameter above) is smaller than the convergence parameter (see below), or if the maximum number of iterations is reached; whichever happens first. If this parameter is disabled, the specified number of iterations will always be performed, ignoring the convergence parameter.

Be aware that when automatic convergence is enabled, the TGVDenoise process is not previewable. In other words, since convergence is specific to the data being processed, there is no guarantee that the result obtained on a preview be the same as the result obtained on its mother image with the same parameters. Depending on the image, the differences can be significant. For this reason the automatic convergence parameter is disabled by default.

* Convergence

When automatic convergence is enabled, the TGVDenoise process will be stopped when the difference between two successive iterations becomes smaller than this value. See the information given for the iterations and automatic convergence parameters.

Local Support Parameters

A local support image changes the rate of the diffusion process, creating a spatially dependent function. This allows to better preserve high SNR areas, or to protect specific image edges. For deep-sky astronomical images, a local support is advisable to achieve optimal results, especially for linear images.

It is very important to point out that the local support image is not a mask. The support image should represent the signal-to-noise ratio of the target image, where higher pixel values will provide stronger protection. This effectively controls the dynamic properties and evolution of the TGV regularization process, but does not work by mixing processed and unprocessed pixels, so it has nothing to do with a mask. Of course, the TGVDenoise process is fully maskable, so you can apply it through a mask if you find it necessary, in addition to using (or not) a local support image.

* Preview

Enable this option to get a preview of the working local support image that will be used by TGVDenoise. This is a control mode that must be disabled to perform an actual noise reduction process.

* Support image

By default, the local support image is disabled. TGVDenoise performs outstandingly well for most images without the help of a local support. When enabled, the local support is the intensity component (in the HSI color system) of the target image by default, but you can select any grayscale image with the same geometry as the target view (or its mother image in the case of a preview). When SNR is proportional to illumination, as happens with deep-sky images, a local support can be of great help to drive the TGVDenoise process, as noted above.

* Noise Reduction

This is the number of small-scale wavelet layers that will be removed from the working local support image. For very noisy images, smoothing the local support may help to achieve more accurate results because this makes it less dependent on spurious small-scale variations.

* Midtones

This is the midtones balance parameter of a histogram transformation that will be applied to the working local support image. Use it to fine tune protection on moderately bright areas.

* Shadows

This is the shadows clipping point parameter of a histogram transformation that will be applied to the working local support image. Increase it to decrease protection on dark areas.

* Highlights

This is the highlights clipping point parameter of a histogram transformation that will be applied to the working local support image. Decrease it to increase protection on bright areas.

Important Notes

* TGVDenoise is a memory intensive process. Internally it requires 13 working duplicates of one channel of the target image in 32-bit floating point format, so it may easily cause out-of-memory errors on the 32-bit version of PixInsight for Windows. If this happens to you, the only solution is a 64-bit version of PixInsight running on a 64-bit operating system.

* TGVDenoise is processor-intensive. We have done our best to achieve the best optimized parallel implementation we are capable of, but despite this, the algorithm requires significant processing time for moderately large images. Since the process is fully previewable (please note: if the automatic convergence parameter is disabled), you should use relatively small previews defined on special interest areas of the image to find an optimal set of parameters, which you can apply to the whole image after testing them thoroughly.

* The strength, edge protection and smoothness parameters have been implemented as exponential controls on the TGVDenoise interface. We have already used exponential controls in the AdaptiveStretch tool. These controls allow you to work in scientific notation, which gives you quick and easier control for parameters with very large ranges. Each parameter is divided into four fields that work in tandem: a large edit field where you can enter the parameter's value directly, a small edit field to enter the coefficient in the range from 1 to 9.99, a slider to change the coefficient value with the mouse, and a spin box where you can define the exponent. In future versions of the TGVDenoise tool, most of these parameters will probably be implemented differently, once we improve our knowledge of their practical ranges.

References

[1] Florian Knoll, Kristian Bredies, Thomas Pock and Rudolf Stollberger. Second order total generalized variation (TGV) for MRI. Magnetic Resonance in Medicine 65(2):480-491, 2011.

[2] Kristian Bredies, Karl Kunisch and Thomas Pock. Total generalized variation. SIAM Journal on Imaging Sciences, 3(3):492-526, 2010.

[3] Kristian Bredies. Recovering piecewise smooth multichannel images by minimization of convex functionals with total generalized variation penalty. Submitted for publication, 2012.

[4] Kristian Bredies, Yiqiu Dong and Michael Hinterm?ller. Spatially dependent regularization parameter selection in total generalized variation models for image restoration. Submitted for publication, 2012.

[5] Chambolle, A., Pock, T. A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120?145, 2011.
 
Yet another poorly documented tool... It's such a shame the only advice we get is "trial and error". I'm sure that for a certain image scale, seeing, pixel size = i.e. measurable or known quantities, and desired size of the protected features (in pixels or arcseconds), there are at least recommended starting values.

Pix is such a powerful software, plagued by sub standard documentation. Yes, after a while you get a "feeling", but in the meantime it torture.
 
look at the dates on those reference papers. look at the journal titles.

this is cutting-edge stuff. no one has any experience with this tool on astronomical images besides Juan and Carlos. how about trying to figure it out and post some ideas? should PITeam keep new processes a secret until they can write a cookbook?

cripes.
 
Excellent piece of software, I just tried it on an old OSC image of M33 and the results are dramatic..
 
multiweb said:
Can't seem to se PixInsight 1.8.0 RC7 in the update repository yet. Has it been posted yet? I'm on RC6.

rc7 should be there, i've been using it for a couple of weeks... what platform are you on? come to think of it maybe it was not released for linux?


 
Get RC7 from 'download central.' Uninstall RC6 (save preferences as an icon first if you wish). Once RC7 is installed it will update from the update repository.
 
Juan and Carlos,

Thanks for another cutting edge tool. I look forward to putting it through its paces.

Alin, you might be more comfortable with a different software package. For example, Photoshop has a large number of 1000 page books available on Amazon.

Sean
 
Thanks for your words. They are very encouraging. :) there is much left to do, so these next weeks/months will be highly active in the development department.
We'll be expecting your thoughs and results with the new tool!
 
Wow!  The following are small sections of the original images.  There are too few subs and therefore very noisy.  My attempt at boosting the central blue saturation is crude but I was reasonably happy.  Image01 is before, Image02 is after TVG!
 

Attachments

  • Rho_Ophiuchi_Nebula_Image01.jpg
    Rho_Ophiuchi_Nebula_Image01.jpg
    228 KB · Views: 361
  • Rho_Ophiuchi_Nebula_Image02.jpg
    Rho_Ophiuchi_Nebula_Image02.jpg
    158.7 KB · Views: 369
i was trying this out on some terrestrial photos that were shot at high ISO - lots of shot noise. the results were encouraging but i kept losing detail. i'll have to keep fiddling with the parameters.

thanks for your hard work on this stuff, it is very interesting.

 
Try lowering the strength. For the noise amplitude, as a starting point you may use the std dev of an uniform area in the image. Also using a local support may help.
 
i had used a strength < 9 to begin with, but then found that i had to increase the smoothing way too much to compensate for large areas of noise that the algorithm thought was structure. eventually i moved the strength up to about 15 and reduced the smoothing. but even at lower strengths i lost a lot of detail.

i did use local support and that helped a lot.

i think it is a very difficult image, iso 6400 and 1/250s at f/2.8 from a canon 7D.
 
You may also try increasing the number of iterations. I think that 100 is quite a low number. 500 is much closer to the real convergence point of the algorithm. This may change in future releases, with multiscale aproaches. But, right now, try more iterations.
 
is there any advantage to working on a linear image with this process? the image in question has already been autostretched by lightroom and exported as tiff.
 
I don't know :D

Mathematically, this algorithm (and most other ones) assumes constant gaussian noise. Linear astronomical images suffer from poisson noise mainly. So, in that case is mandatory to use a local support to help the algorithm to discriminate between different SNR zones. In stretched images something similar happens, but now the noise distribution is not as easy described. Also, noise amplitudes in the shadows are larger than on the highlights. So, fine tunning the parameters may be trickier, to protect real details in high snr areas. Again, the local support can make the difference, since it acts as a modulator of the strength parameter (inversally proportional, and in aspect similar to an snr map).

The bottom line. Using LS, both approaches should work. It remains to be seeing wich ones proves to be more user friendly, with easier parameters to fine tune. The next algorithm that is under production should create internally this support maps, and require only a first guess od the strenght that is not critical for the result. There is also an open development field, regarding spatially variable noise amplitudes. We'll get into that once all the current development codes are implemented as official releases, and there is still left some work to do.
 
Back
Top