Implementing Doug Germans's Noise Reduction with PI

[STEVE333]

The link below is to a short tutorial showing how to implement a noise reduction technique originally implemented in Photoshop. I first saw this technique presented by Doug German.

http://www.astronomyforum.net/astrophotography-image-processing-forum/236689-pixinsight-implementation-doug-germans-noise-reduction-technique.html


The link below is to a set of PI Process Icons (DropBox) that perform this noise reduction.

https://www.dropbox.com/s/xpt1clh2cjby5fp/Surface_Blur_Noise_Reduction.xpsm?dl=0

Comments and suggestions welcome.

Steve

[Niall Saunders]

Hi Steve,

An interesting approach - but one that does not sit well with me, if only for the primary reason that I started using PixInsight in the first place. And that is your use of the CloneStamp tool to effectively build a form of mask - in otherwords, the result of the CloneStamp action is one pirely born of the 'artistic interpretation' of the mind of the user and, as such, has little 'statistical' or 'scientific' value.

PixInsights foundations sit firmly on a scientific, statistical and mathematical treatment of noise - but not to their total exclusion (otherwise there would be no such tool as the CloneStamp Process).

Now, that said, of course there are no 'rules' that would control how any user chooses to process their image within PixInsight (or, indeed, whether users would seek recourse to external software to complete any part of their processing steps).

However, it is always interesting to hear when users feel that they do need to seek assistance from external software, or when they seek to duplicate external methodology (as in your case) to achieve what they cannot (apparently) achieve within PixInsight. Either the PTeam can learn from these solutions, or the solutons themselves can give rise to explanations of how to improve on the approach, or how to use existing tools that have been provided that will accomplish the same end result (or better).

Please, understand that this is in no way meant as a criticism of your efforts (and thanks for taking the time to detail and share them with everyone). Every day is a 'learning day' and ypur discussion will definitely prompt users to consider your suggestions.

[STEVE333]

Hi Niall - Thanks for taking the time to respond.

You and I clearly have different views about PI. I view it as a "tool" to aid me in doing AP image processing. How it performs the various processes is less important to me than how well it performs them. I guess I'm more of a pragmatist (if it works - use it!). Because this novel approach works better for me in some circumstances than the PI noise reduction processes (maybe because of my level of expertise) it will certainly remain in my "bag of tools" along with the PI Processes.

It will be interesting to see if there are any other points of view on this topic.

All the best,

Steve

[Niall Saunders]

Hi Steve,

As I said,

Please, understand that this is in no way meant as a criticism of your efforts

You are perfectly entitled to select any tool to achieve your desired aim - and, if you have the correct skill in using it, the end result will be perfectly acceptable.

Once, when touring the remotest islands off the coast of British Columbia in Canada, I had the pleasure of watching a skilled craftsman attack a massive 80cm (30+") diameter, 8m (24') long, log with a chainsaw that ought never to have been allowed off the set of "The Texas Chainsaw Massacre". Over the period of a couple of days this log was totally transformed into a terrifyingly life-like crocodile.

The analogy here is that, perhaps there might have been 'better' (or, at least, 'other' - and definitely 'safer'  ) tools that he could have used to achieve his masterpiece - but, in his hands, the chainsaw was absolutely the correct tool for the job!

Your methods are no different - in your hands, with your experience, you are able to use the process you described to your advantage.

Yes, my approach to creating the crocodile might have involved several months on a 3D modelling application, and then a terrifingly expensive CNC router to perform the wood-whittlng itself.

Neither method is 'bettter' (although the likelihood of me being able to replicate my work might perhaps have been higher - owing mostly to the fact that some of my limbs might not have had to be stored in a pickle-jar above the fireplace  )

[msmythers]

Man, now I need look in my grandmothers pickel jars on the mantle


Mike

[STEVE333]

Points taken.

By the way, did you just liken my AP image processing skills to a man with a chainsaw?  You must have seen some of my work.   Just kidding. 

Thanks again for your interest.

Steve

[msmythers]

Steve

I learned a long time ago to respect the man with the chainsaw in his hands!


Mike

[STEVE333]

Good advice. Never know when that will come in handy.

[RickS]

Hi Steve,

You could modify the process a little to use RangeSelection to mask the bright areas rather than making "arbitrary" decisions with CloneStamp.  I'm not above using a little CloneStamp myself, but it's more of an exception rather than something that's part of my standard workflow.  I won't think any the less of you for using it as much as you like, of course 

In the longer term you'll almost certainly find that the conventional PixInsight noise reduction techniques work very well once you learn to drive them.  The combination of a mask which progressively protects brighter features with multiscale tools to target the noise at different scales is very powerful.

Cheers,
Rick.

[STEVE333]

Hi Rick - Thanks for joining in the discussion and for your understanding approach.

As for the PI noise reduction techniques, I continue to learn, practice, and use them (especially MLT and TGVDenoise).

Because of my bad light pollution, restricted imaging times, and, DSLR noise, my stacked images often have quite high noise. If I try to reduce this noise to a "tolerable" level using the PI processes it usually leads to an unnatural looking background. If I don't reduce the noise enough it is quite distracting to the final image.  The one big advantage of this technique is that, in some cases, it can actually remove the noise  while leaving an unblemished background. There is, of course, a small loss of low-level nebulosity and/or low-level details. Whether or not this is acceptable depends on the specific target and noise.

The purpose of the CloneStamping is to replace the Galaxy/Nebula/Bright-Star with the nearby background before the blurring occurs. This cloning thus replaces the bright object with background similar to the surrounding background.  Thus, when the blurring occurs, the "bright object" (now replaced with surrounding background) won't "distort" the resulting smoothed background.  If there is a way to use the RangeMask to accomplish this I am very interested.  I use the RangeMask often, when making a Galaxy mask, but, don't see how to use it in this instance. Any suggestions would be appreciated.

Appreciatively,

Steve

[msmythers]

Steve

Could you make the original stacked image available. What I'm looking for is the image before any post processing is done. Maybe myself or others could show you other PI methods that you haven't seen or thought of. Over the years everyone develops their own methods. I shoot only from a Bortle 7/8 location with some very extreme, moving LP due to high humidity and heat. I have very poor equipment also and because of the LP only shoot very short exposures. I use TGVDenoise for most of my noise reduction and while still linear. I also have what I call a background smoothing routine that is sort of unique. It uses a combined structure mask and range mask along with MorphologicalTransformation tools.

Here is a link to another posting you might find interesting. There are number of links within the posting that are worth a look.
https://pixinsight.com/forum/index.php?topic=10119

There was a statement made by someone smart in this forum sometime ago that really stuck with me about noise reduction. It went like this. Noise Reduction is not noise removal. Noise Reduction is about preserving all the data with the least amount of acceptable noise.



Mike
http://www.astrobin.com/users/msmythers/
 



Mike

[STEVE333]

Mike -

I will be glad to make an original stacked image available. I was up late last night, and, early this morning so, it will be later today before I am able to do this.

I certainly appreciate your interest and involvement with this topic.

I'll definitely look at the link you provided.

That's a good quote about noise. One to keep in mind.

Steve

[RickS]

The purpose of the CloneStamping is to replace the Galaxy/Nebula/Bright-Star with the nearby background before the blurring occurs. This cloning thus replaces the bright object with background similar to the surrounding background.  Thus, when the blurring occurs, the "bright object" (now replaced with surrounding background) won't "distort" the resulting smoothed background.  If there is a way to use the RangeMask to accomplish this I am very interested.  I use the RangeMask often, when making a Galaxy mask, but, don't see how to use it in this instance. Any suggestions would be appreciated.

Hi Steve,

It might be a little tricky to stop the bright stuff bleeding into the surroundings with such an extreme Convolution blur, but I'll have a play with your test data when you make it available and see what I can do.  I have a couple of ideas...

If you don't like the look of the background produced by the other noise reduction processes then you can always mask it and use Convolution.  I have used Convolution to (slightly) blur noisy colour data before combining it with luminance in the past.

Cheers,
Rick.

[STEVE333]

Here is a link to the stacked data. All that has been done to the data is Crop/ABE/Neut/ColorCorrection. The image is from the S110 collection and is S87 (NGC 6818 - the Little Gem nebula) with Barnard's Galaxy as a surprise bonus.

https://www.dropbox.com/s/kvxxpih24sbhnqk/S87%20ABE_Neut_cc_crop.xisf?dl=0

Anyone can feel free to use this for processing with my blessing. Thanks for any feedback.

Steve

[STEVE333]

I want to apologize. The link in the previous post was to the wrong image. It is indeed a noisy image, but, not the one used in the noise reduction tutorial.

Below is a link to the same image used in the noise reduction tutorial (M36 M38 wide FOV image). This is the stack straight out of DSS. I wasn't yet using Flats, so, sorry for the nonuniform illumination.

https://www.dropbox.com/s/ybpucazmnhegdvk/M36%20M38%20DSS.TIF?dl=0

Steve