Incredibly frustrated with PI and Noise Reduction

[J_N_Dubya]

This has probably been discussed and whined about ad nauseam, but, I cannot for the life of me figure out a technique for effectively removing noise . My main beef is that no matter which algorithm I use (MLT, MMT, TGV Denoise, ATWT, ACDNR), the noise is always shifted from fine grain noise to larger scale blobs. This makes my images look far worse than if no NR was used. I have used a variety of tutorials from Light Vortex, to Jon Rista, to Harry's Astroshed, to Pixinsight's own tutorials and the results are invariably the same. When I look at Astrobin, it seems that others have figured out some technique to minimize noise to create and attractive and smooth background. Maybe my data is lacking in some regard (LP or insufficient signal). I have tried to remedy this by drowning the problem with data.

I tried the Iris Nebula and I took 350 x 60s Lum exposures  and 150 x 180s (50 each) RGB exposures which is over 12 hours!  I am using a ASI 1600 Pro at a gain of 76 and offset of 15. The masters look fine as far as I can tell. Here are a link to the files with no NR. Maybe someone can look at them and either have better luck or give me a heads up that the images suck. I'd appreciate it either way.

https://www.dropbox.com/s/r037adhvrnzmdmm/Iris_lum_ABE.xisf?dl=0

https://www.dropbox.com/s/gl2i24bh9r1x9zx/Iris_RGB_ABE_BN_PCC.xisf?dl=0

[ngc1535]

I would be interested in responding to this question... but without an appreciation of *your* opinion on what is "good" or "objectionable"... I feel it risky.
So... do you have an example of what you think is your best noise reduction attempt?

Concerning your acquisition- there is likely diminishing returns on taking so many (short) exposures. In terms of the requisite statistical analysis... it is certainly overkill. 20-30 exposures of your longest exposure time you can do would likely help. However, as I understand it, your camera has variable gain- and that really confuses me in terms of figuring out what the best exposure length really is to be read-noise limited.

One more thing- TGVDenoise is really very good. Did you discover you get different results when using the separate Chrominance and Luminance method?
There are also methods that combine say MLT with TGVDenoise. (This happens to be my method of choice.)
It is unfortunately because of the variable Gain that MureDenoise might be not easily used. It is also very powerful.

-adam

[chris.bailey]

Important to consider it as noise reduction and not noise removal. The latter will tend to lead to a blobby background.

Personally I don't find the fine grain noise in those two images at all objectionable and if it were my data I would try to take a very light handed approach to noise removal.

Judicious use of TGV DeNoise does a pretty good job of knocking the fine grain noise down a little (at least in the L Image posted) without the dreaded clumping in the background. Follow it up with a very light touch of ACDNR will tame it a bit further.

[dave_galera]

As a general rule SNR is not reduced that much beyond 35 integrated exposures, I would have 35 longer exposures and not zillians of short exposures

Dave

[Geoff]

Try this site https://jonrista.com/the-astrophotographers-guide/pixinsights/effective-noise-reduction-part-1/
Really good.

[aworonow]

One principal objective of noise suppression is to give an image the perception of sharpness. Online discussions of sharpness emphasize that it depends on factors external to an image as well as the properties of the image itself. Principally, viewing distance and enlargement factor affect our impression of image sharpness. An image that appears wonderfully sharp when viewed from 1m away and displayed as 8x10 may look noisy and blurry if viewed from a closer distance or at greater enlargement. Some internet sites provide equations to help with this relationship, but the point is, at some distance or at some size, virtually every image will be unsatisfactorily noisy. So, target the image processing for the intended viewing conditions.

Next issue is that noise reduction, good noise reduction anyway, tries to smooth regions that are smooth (subject to viewing conditions) while preserving edges. What you see as large-scale noise is what the algorithm sees as edges around homogeneous regions. So, the game is to blur or otherwise smooth-out noise in the regions between 'true' edges but leave the edges themselves unscathed. To do this the algorithm needs to work with some definition of edge--some definition that says, "a rapid change in color or intensity at a particular scale (or larger) is not noise--it's signal!"

Generally, the parameters you set in the PI procedure tell the algorithm what the change-over scale between noise and edge is--unless it's a primitive algorithm that just smooths everything without regard for edges. So, decide how the image is to be view, and proceed accordingly--and inspect your image just as closely as you intend to display it, and no closer!

BTW TGV is an 'edge-aware' algorithm, some others are not.

[CharlesW]

You didn’t mention Mure Denoise but I have found it to be extremely effective and extremely simple. I do it right after Integration.

[ngc1535]

You didn’t mention Mure Denoise but I have found it to be extremely effective and extremely simple. I do it right after Integration.

I mentioned it above...but with the variable gain, I am not convinced it is a good choice. Mr. Schuster will know the answer.
-adam

[dave_galera]

Looking at the images you are on a hiding to nothing trying to reduce the noise with any tool......the images are all noise and no signal, they need longer exposure times

Dave

[wadeh237]

I mentioned it above...but with the variable gain, I am not convinced it is a good choice. Mr. Schuster will know the answer.
-adam

I use it on my variable gain ASI1600MM and get excellent results.

It does require that I calcuate gain and read noise numbers for each gain/offset setting that I use.  In practice, I really only have two gain/offset strategies.  I have a broad band setting that allows me to optimize for dynamic range, and a narrow band setting were I sacrifice dynamic range in exchange for significantly lower read noise.

After using MureDenoise for a while, I can't imagine not using it with the ASI1600.  It significantly reduces speckle noise in the low signal areas, without introducing smoothing or destroying faint and small scale details.  It is the only noise reduction that I use any more on linear data.

[STEVE333]

Hi J_N - As mentioned earlier you do have quite a bit of noise in this image. I too often have significant noise because of my location and the fact I can't always get as much data as I would like.

One thing I've noticed is that the more noise an image has the less the noise can be reduced. It sounds odd, but, since noise isn't eliminated but just reduced, when you have a lot of noise and reduce it too much the result is either a lumpy or "plasticised" background.

I usually use MultiscaleLinearTransform (with the Linear Mask active) to reduce the noise slightly while the data is still linear. Once the data is stretched I use TGV with the data protected with an inverted Luminance mask. Again the noise can't be reduced too far or you will get the undesirable artifacts mentioned earlier.

You may just be expecting the "Noise Reduction" to reduce the noise more than is reasonable for your data. I downloaded your RGB data and processed it using my normal process. I didn't include the Luminance data because I don't normally use Luminance. The results looked reasonable to me.

By the way, your stars all seem to have some "halos" around them. Don't know if this is a focus issue or high clouds or what.

Hope this helps. I can share my "typical" MLT and TGV starting values if you are interested.

I it's OK with you I can post your image with my processing so you can see the noise reduction I was able to implement.

Steve