Help with approach to noise reduction

[javajunkie2121]

Hi All:

I've been watching tutorials and trying out various noise reduction processes that are available (ATrousWT, TGVDEnoise, ACDNR, Multiscale linear transform, SCNR) but as a beginner I'm having trouble establishing a workflow with these...linear vs. nonlinear state, single application, multiple applications etc.

while I'm always trying to gather more and better data to improve SNR, the reality for me in light polluted bay area with my OSC CCD is a lot of noise in background in images from stacked subs that are often less than 5 min or so..I aim for at least an hour's worth of data but I do not always achieve that given weather and time limitation and satellite trails etc...

I am struggling to get backgrounds to not look blotchy or too black (be it a galaxy or a planetary nebula or a glob cluster).

I'd appreciate some advice on which noise reduction process(es) and where in the workflow is best for beginners to try and use?   assume that I've started with cropped OSC RGB image and have already gotten through DBE, Background Neutralization, Color Calibration

jeff

[pfile]

i always struggle with this as well. for whatever reason the magic formula has eluded me. if i use TGVDenoise, i've had the best luck on nonlinear images. same with ACDNR of course. i usually mask the high-snr parts of the image and concentrate on the background. even in this age of TGVDenoise i sometimes still use ACDNR.

a cheat that i've employed before is to add gaussian noise to the background. that at least makes it look uniform...

for large-scale color noise in the background, i think Alejandro has a method where you use AtrousWavelets with the Target set to Chrominance and remove the largest scales thru a mask that only exposes the background. you have to be careful with this because it really kills the color… and it will remove the color from your stars if they are not well protected.

rob

[NGC7789]

While not a direct answer to you noise reduction question, it may be that your subs are too long and your total integration is too short. While counterintuitive your skyglow may make exposures as long as 5 minutes untenable. But these short subs mean you need even more total integration to achieve the same signal to noise ratio. See this link:

http://www.samirkharusi.net/sub-exposures.html

[javajunkie2121]

Hi: 
thanks for response.  you make an excellent point..when I first started with all this I did calculate time to overwhelm readout noise based on this site:
http://starizona.com/acb/ccd/advtheoryexp.aspx

supposedly it's at 7 min for me without filter but I had used LP filter (astronomik CLS CCD) and subs were much less than this...

[NGC7789]

In my example my nearest darkish site allows for a 6 to 8 minute exposure depending on conditions. At my house its more like 3 minutes. This means that I need 2 to 3 times as many subs at home to get the same SNR. Plus there will be some targets I can't image at home at all. If the skyglow is brighter than the target it's my understanding that no amount of subs will bring it out. It's simply not there. Another important point is the use of low ISO when imaging in light polluted skies. It's total integration time that matters. Low ISO in light polluted skies allows for longer and therefore fewer subs. Just remember that when calculating the relationship between dark and light polluted sites matters. So in the above example if my light polluted subs were at half the ISO I would need 4 to 6 times the total integration time! In the end nothing is free. But if you find a way to image without travel you may be able to collect more data.

[Jason Tackett]

Here is a great resource which lists the different noise reduction processes and whether they are intended for linear or non-linear images (slide 10, but the whole PPT is very informative).

"Noise reduction in astrophotography: Tools in PixInsight" Presentation by Jordi Gallego:
http://www.astrosurf.com/jordigallego/articles.html

Most folks recommend using SCNR to remove greens late in image processing. In my limited experience (I'm still a rookie), I have found Harry Page's advice to reduce the Amount of SCNR works well.

Also check out  Juan Carlos Moreno's blog where he shares processing examples imaged in a light polluted environment.
http://astrofotourbana.blogspot.com.es/2014/05/m104-detalle-del-procesado.html

I too have been experimenting with all sorts of noise reduction techniques based on the substantial amount of tutorials out there. One thing that struck home recently was how Juan Conejero reduced background noise in his M81 & M82 tutorial without producing blobs (http://pixinsight.com/examples/M81M82/index.html). In the past I would begin background noise reduction on wavelet layers 1 and 2 with MMT and/or MLT, using an inverse luminance mask. The mightily noisy background would organize itself into smooth, unattractive blobs. But in Juan's tutorial, he applies MLT noise reduction on all 5 wavelet layers, with noise reduction on each successive layer to a lesser degree. While experimenting this morning, I finally realized that it is applying noise reduction through those higher wavelet layers that eliminates those blobs. Of course there is the usual disclamer that this worked on my image, but may need tweaking to work on other images.

Similar to your situation, most of my imaging is done in my light polluted backyard with a CLS-CCD filter as my tool. The philosophy I am beginning to adopt (again, in my rookie experience) is to only apply mild noise reduction to my images where the SNR isn't the best - which is still most of my images at this point. When I first started I would aggressively reduce the background noise in the linear image using an inverse luminance mask that I stretched to strongly select the background. This would give me a nice smooth background that transitioned into a scattered noisy mess (i.e., the DSO that I really care about). Then when I tried a strong noise reduction on the DSO that would blend across the transition, I would lose too many details. It seems I was too generous in what I was calling "strong signal areas". So in my latest images I have taken the philosophy that I will not hide that my image is noisy, but instead apply mild noise reduction to the whole image, using a non-stretched inverse luminance mask that allows some noise reduction in the DSO and background.

Hope those resources help!

Best,
Jason

Edited by the Forum Administrator: Fixed link to point to relevant PixInsight related information.

[Juan Conejero]

As Jason has pointed out, the multiscale approach to noise reduction (with the MultiscaleLinearTransform and MultiscaleMedianTransform tools) is very powerful and efficient for linear deep sky images.

Another worth-mentioning point is the fact that we are doing noise reduction, not noise suppression. Noise is uncertainty in the data, so trying to suppress it completely is a conceptual error: since the difference between significant structures and noise is uncertain by nature, noise reduction has to provide for a safety margin where one can be reasonably sure that nothing significant will be removed. The stronger the noise in the image, the larger this safety margin should be.

[javajunkie2121]

thanks for all the replies..thanks also for patience with beginner questions..this software is really amazing but lots of bells and whistles to try and learn all at once

Jason: those were really helpful links.  they don't mention the more recent TGVDeNoise.  My understanding is that can be applied to linear and non-linear.

so as a beginner, trying to master a limited set of processes and approaches at a time to apply to OSC ATIK CCD images (I use the astronomik CLS CCD filter as well..at least when I remember to insert it before imaging), couple last questions for now:

1. seems good to attack noise in linear state. previously I tried ATWT or MLT in linear state (not sure I understand Multiscale linear transformation vs multiscale median transformation , but ok) ..which seemed better than not doing NR in linear state and waiting until non-linear to use ACDNR (which seems better in my hands with noise in chrominance).  Now that TGVDeNoise is available, is it better for beginners to use TGVDeNoise at the initial linear state vs ATWT and MLT/MMT?

2. better to stick to one process for NR in linear, and expect most of the time to apply that process just once?

3. if you use TGVDeNoise in linear, are there disadvantages to re-using it in non-linear?

4. I understand SCNR is best before color saturation...is it best to apply SCNR in linear state early on, or in non-linear state late in processing before color saturation?

jeff

[Jason Tackett]

Hi Jeff,

While I have very little experience with TGVDeNoise, Kayron Mercieca has a tutorial giving a practical example on how to use the tool:

http://lightvortexastronomy.blogspot.com/2014/04/tutorial-post-processing-technique.html

As far as when to attack noise, Jordi Gallego's presentation and many other references I've encountered propose that it is best to apply noise reduction in a linear state. On the other hand, check out the processing examples on the PixInsight Resources webpage (http://www.pixinsight.com.ar/en/processing-examples.html) and you will find many cases where noise reduction is applied well after non-linear stretches...and those examples turn out great. Of course, they start off with very good SNR to begin with.

Jason