Local Normalization (batting 500)

[ngc1535]

Hi,

So my excitement for Local Normalization is truly astronomical. I would like to present some examples during my AIC talk coming up at the end of the month. However, I have been successful in one case and not so in another (which means I do not yet fully understand). Please help!

Below is an example where I had 18 images. 12 of them had a spurious glow on the left side of the frame- the other 6 (taken on a different night) did not have it to the same degree. So I created a combined reference from the 6 good frames for Local Normalization and created (view execution only) xnml files. Then I loaded them into ImageIntegration with the associated data and combined. It worked out well!

Here is the normally combined (no LN) image:


and here is the image using LN:


You can see it greatly reduced the glow (to the degree that was present in the reference image).

After normal DBE the image looks like:


So now I wanted to choose an example that is much more dramatic. I have in another data set 23 good images that are fine and 7 images that do not have flats! This means these 7 images have dust donuts everywhere in the image. If it was just one or two images... maybe I could force the rejection (large scale) to help- but it doesn't do it. I thought LN would help. However when I do the procedure above- the final combined image still shows all of the dust mote shadows. The difference is that dust motes can be slightly darker or brighter than the background- so perhaps there is a different strategy for parameters of LN (and rejection). The maps generated do not, very strongly, show the donuts- but they are there somewhat.

Please please... any help would be greatly appreciated. I am certain others will want to know how to use this very cool tool.

-adam

[Herbert_W]

Hi Adam,

some data to try and test would be useful.

Vicent - Juan - ...!!!

[Juan Conejero]

Hi Adam,

That's a nice example of local normalization, thank you for posting it.

I have in another data set 23 good images that are fine and 7 images that do not have flats! This means these 7 images have dust donuts everywhere in the image.

Local normalization cannot help with these problems. Normalization does not mean artifact suppression. LN can simplify gradients and equalize relatively small exposure differences in a data set, but it cannot do what a master flat does.

Local normalization is a multiscale algorithm that works at large scales. A dust donut is a complex image structure, with large-scale and small-scale components. If you look at it as a whole, it is relatively large. However, its inner and outer borders are represented at small dimensional scales, typically from one to 4 pixels. These small things are completely out of the scope of local normalization. Our current implementation works at a minimum scale of 32 pixels, and the default LN scale is 128 pixels.

LocalNormalization is a powerful tool able to solve difficult problems efficiently, but unfortunately, it is also an extremely dangerous tool. This is because the local normalization problem is, as most inverse problems, inherently ill-posed. That's why I have implemented many control and analysis features in the LN tool, including complex graphical representations that can be very useful to judge the correctness of each particular application.

I am working on a tutorial that describes the LocalNormalization algorithm and its current implementation in PixInsight 1.8.5, with practical usage recommendations and examples. Please bear with me while I complete this tutorial, since it is very complex and unfortunately I have other priorities now. Of course, I'll be glad to answer your questions here, preferably with samples of the data you are trying to normalize.

[ngc1535]

Wow...thank you for the excellent response Juan.
I now feel a bit better guided about what to expect. I did not understand the scale to which this tool is best applied (or how sensitive it might be).


1. In the example above- the "x" glow is minimized really well (basically it is minimized to the degree that it appears in the reference). That is why I got so excited.
2. Concerning the set of unflatted images- OK- no go.
3. I am still interested in finding a way to get rid of dust donuts that show up in frames that are properly flatted. (these dust motes fall on the chip after the exposure is taken, later in the night, so are in the flat- but not in the original data). They will be in many frames (so large/small scale rejection might- but probably not, get rid of them). If there is a good solution for this as part of a current process... please let me know.
  -- Having a way to select regions for rejection seems the easiest, and I do not think this runs afoul of philosophical constraints of processing. Kind of like a "rejection map" for individual images instead of a defect map.

[Juan Conejero]

Hi Adam,

My pleasure. As for the dust donuts, we have implemented the large-scale pixel rejection feature of ImageIntegration to solve these problems—when they are solvable. In this case we are dealing with dark artifacts. Have you tried this way?