NormalizeScaleGradient: Bookmark website now!

Status
Not open for further replies.
rockenrog said:
Hi John,
You are working too hard, and I am going around in circles with my today problem: I have defined 10% bands at top/bottom without any detected stars. There are many stars visible in the 10% areas. My images are calibrated, CC, debayered (to separate channels), and registered. There are thousands of stars in the subs. I am using NSG V2.0.1.

At first I believed this is due to registration of images from different nights imaging, because one night of images had about a (huge) 10% downward shift, corresponding to a 10% band vs non shifted. But upon further checking the reference image and the ones with the 10% bands (same night) are only different by a few dithering pixels.

So now I am asking your advice on these bands. For example...
I select the last 4 images which are from the same night and with only small dithering distances. These are higher PSF images. Any combination of reference and selected image have the approx 10% no detected stars bands. I also tried NSG1.6 with same result. The star detection slider shifted to -3.0 has same bands. Does NSG already know I have 10% registration difference on other subs?

Some other combinations of different (lower PSF) images have smaller bands.

Thanks,
Roger
Click to expand...
Hi Roger
If an image contains a very large number of stars, NSG does not need all of them. In these situations, NSG limits the star detection to a rectangular region to improve performance. This is important for people with large cameras taking large numbers of wide field images, as these can take a long time to process.
Regards, John Murphy
 
Hi John,
Thanks for explaining. It makes sense....
1. If I choose a lower PSF image than the reference, I get the detected stars to cover the frame.
2. The images with larger bands of un-detected stars are the best quality images (high PSF). The best image has 3767 detected stars, and one with slightly lower PSF (.87) has 2444 detected stars. FYI, my PSF's go from 1.0 down to about .2. I have the full range!:(

1. Is there going to be gradient correction in the no detected stars areas?
2. Is a good strategy to select, as reference, a lower PSF image (but with good gradient) that has full coverage of detected stars? Because if I select the largest PSF image and have large bands (10% top/bottom seems large to me), and no images get gradient correction in these areas.
3. Alternately If I change the Star Detection from -1.0 to -.6 the detected stars spread out over the entire frame.
Strangely though, the detected stars increased from 2444 to 2557. Further adjusting Star Detection to -.3 I get a decline down to 1865 stars as expected.
4. My Workflow conclusion for frames with many stars is to find a high PSF image, set it as reference, then check if detected stars in the reference image cover the entire image. Adjust Star Detection upward until full coverage is achieved. Then find the simplest gradient image with pretty high PSF.
John, you definitely should be a speaker at AIC in May 2022. But it is a long way to go. I guess you can 'settle' for Adam to be your representative!... Sorry, just a little bad American humor.
Roger
 
rockenrog said:
1. Is there going to be gradient correction in the no detected stars areas?
Click to expand...
Yes, the whole image will be fully gradient corrected.

Currently NSG is designed to work on images where the scale difference between the reference and target image is constant across the whole image. If this assumption is valid, the scale difference measured in a central rectangle will be valid for the whole image. Avoiding the edges and corners may even improve accuracy because these areas may contain aberrations.

I am planning to (when I have time...) add an extra section to provide variable scale support. This would involve creating a surface spline from each pair of measured stars. However, this has always been a low priority, because it is not as useful as you might think. Why? A variable scale factor is only needed if an image has been partially affected by a cloud. The part of the image that is obscured needs a different scale correction compared to the area that was not obscured. However, this can produce rapid changes in scale that will be difficult to model accurately. It may be better to discard the image.

In the following situations, plenty of gradient correction is required, but the scale factor is constant across the whole image:
  • Even thin cloud.
  • Broken clouds that drift across the image. This tends to average out the scale to a single scale factor.
  • Light pollution, Moonlight, Twilight and sky glow. These only affects the light pollution gradient. They has no affect on the scale.
  • A change in humidity that reduces transparency. This affects all parts of the image equally.
  • Flat calibration errors. These usually affect both the reference and the target images equally, so the relative scale factor remains constant across the image. Note that NSG only applies relative scale and gradient corrections so that the target image matches the reference.
A variable scale factor should only be used when really needed because a single scale factor can be calculated more accurately. When I finally do add a variable scale section, it will default to off. When switched on, the star detection will not be limited to the central region so that the whole image can benefit from the variable scale factor.


rockenrog said:
4. My Workflow conclusion for frames with many stars is to find a high PSF image, set it as reference, then check if detected stars in the reference image cover the entire image. Adjust Star Detection upward until full coverage is achieved. Then find the simplest gradient image with pretty high PSF.
Roger
Click to expand...
Now that I have explained what NSG is doing, you will understand that this is not necessary. The scale will be accurately calculated from a central region.

Hope this helps, John Murphy
 
John, since it looks like installing updates will require manual input from now on, please remind me of the process. I went to your website and have downloaded the three files including the .js file. What do I do with these? I will have to make some notes so I do not have to keep asking.
Larry
BTW, can you share why Juan does not want this incuded?
 
Larry Phillips said:
John, since it looks like installing updates will require manual input from now on, please remind me of the process. I went to your website and have downloaded the three files including the .js file. What do I do with these? I will have to make some notes so I do not have to keep asking.
Larry
BTW, can you share why Juan does not want this incuded?
Click to expand...
The install instructions are included on the NormalizeScaleGradient website:
sites.google.com

NormalizeScaleGradient

I have created a new website to host both NormalizeScaleGradient (NSG) and PhotometricMosaic (PMM): https://www.astroprocessing.com The NSG PixInsight Repository URL has also changed (see my new website for more information) Thank you!
sites.google.com sites.google.com
I have also copied them here:

Install NormalizeScaleGradient script
  • Unzip the script to a folder of your choice (but don't use a PixInsight folder).
  • In the PixInsight SCRIPTS menu, select 'Feature Scripts...'
  • Select 'Add' and navigate to the folder.
  • Select 'Done'
The script will now appear in 'SCRIPTS > Batch Processing > NormalizeScaleGradient 2.0'

Install Reference Documentation
Unzip the contents of ReferenceDocumenation.zip to the PixInsight documentation folder. For example, on Windows 10, unzip to:
C:\Program Files\PixInsight\doc\scripts\NormalizeScaleGradient

This folder should then contain:
  • NormalizeScaleGradient.html
  • An 'images' folder
Install NSGXnml C++ module
This module is optional. It allows NSG to create '.xnml' normalization files. This makes NSG compatible with DrizzleIntegration, and rescaling the corrected images is no longer necessary. For more information, see this post: https://pixinsight.com/forum/index....ble-with-drizzleintegration.16507/post-109500
  • Copy NSGXnml-pxm.dll to any folder (but don't use a PixInsight folder)
  • In the PixInsight PROCESS menu, select:
  • PROCESS -> Modules -> Install Modules...
  • Browse to the folder. Note that the folder contents are not displayed.
  • Press 'Search'. A dialog should display '1 additional PixInsight module(s) were found...
  • Press 'Install'
To activate the license: PROCESS -> ImageCalibration -> NSGXnml

Then use NSG as usual, but with the new 'Normalization data' check box selected.

The NormalizeScaleGradient website explains how to purchase a NSGXnml permanent single user license, valid on all your machines, and for all supported OS. It also entitles you to all upgrades.

Hope this helps, John Murphy
 
I have released NormalizeScaleGradient 2.0.2
https://sites.google.com/view/normalizescalegradient

For this release, I have rewritten the code I was using to read the FITS headers. The code is now both robust and fast. This should significantly reduce the time required to load images.

The gradient calculation should also run more quickly.

For best performance, I would recommend using the 'Overwrite existing files' option.

See the previous post for install instructions.
Regards, John Murphy
 

Attachments

  • NormalizeScaleGradient_2.0.2.zip
    127.3 KB · Views: 111
Last edited:
Hello,
As I usually leave the computer to run NSG over night, is it possible to have some kind of option that after the NSG is done and 'Image integration' is automatically set to also run the integration?
 
Grim408 said:
Hello,
As I usually leave the computer to run NSG over night, is it possible to have some kind of option that after the NSG is done and 'Image integration' is automatically set to also run the integration?
Click to expand...
This seems like a reasonable request (for example, WBPP runs C++ module processes in the background, and can run ImageIntegration). It has now been requested by two people, so I have decided to add this to the next NormalizeScaleGradient release (probably later on today).
 
ClearAmbientSkies said:
When do you anticipate MacOS support for NSGXnml will become available? Looking forward to using drizzle data with NSG on MacOS.
Click to expand...
I have just finished code reviewing NormalizeScaleGradient 2.1.0, which has some significant improvements.
My next task is providing MacOS and Linux support for NSGXnml. I will start working on this tomorrow.
Regards, John Murphy
 
NormalizeScaleGradient 2.1.0

1646329756143.png

  • The Auto exit and run ImageIntegration checkbox can both launch and run ImageIntegration after normalizing the target images. ImageIntegration remains open so it can be saved as a process icon, and so that it can be run again with adjusted settings. This option should be particularly useful when NSG is run unattended.
  • A few months ago, @ngc1535 (Adam Block) requested the ability to display the corrected target image. The Gradient graph dialog now has this ability:
1646330126286.png

  1. Select Gradient graph to display the dialog. See image above.
  2. Check that all the really bright stars have rejection circles (medium and faint stars do not need rejection circles). In this case there are five stars that need rejection circles. If necessary, use the Sample Generation dialog to add manual rejection circles.
  3. Deselect Rejection circles and graph lines to unclutter the display.
  4. Toggle the Reference checkbox to blink between the reference image and the corrected target. Adjust the Gradient smoothness until the images match.
  5. Toggle the Correct target (scale and gradient) to blink between the uncorrected and corrected target image.
Note that the type of image being viewed is displayed in the dialog's title bar:
  • Reference image
  • Target image (corrected for scale and gradient)
  • Target image
  • Gradient Graph 1:1

To use the graph display (this can be faster for large images):
  • Select Rejection circles and graph lines.
  • Double click on the display to set the horizontal and vertical line positions (shown in green). The gradient graphs display the data along these lines.
  • Select Graph to view the graph. Use the Vertical checkbox to switch between the horizontal and vertical lines.
  • Adjust the Gradient smoothness until the graph line follows the data but not the noise.
If the reference image has no gradient, or a smooth gradient, the gradient smoothness can be reduced to produce a very accurate fit. However, if the reference image has a 'lumpy' gradient, a smoother fit that only follows the gradient trend may produce better results. The lumpiness may then average out when the images are integrated.

I have attached the script, but it is also available on the NormalizeScaleGradient website:

sites.google.com

NormalizeScaleGradient

I have created a new website to host both NormalizeScaleGradient (NSG) and PhotometricMosaic (PMM): https://www.astroprocessing.com The NSG PixInsight Repository URL has also changed (see my new website for more information) Thank you!
sites.google.com sites.google.com

Install NormalizeScaleGradient script
  • Unzip the script to a folder of your choice (but don't use a PixInsight folder).
  • In the PixInsight SCRIPTS menu, select 'Feature Scripts...'
  • Select 'Add' and navigate to the folder.
  • Select 'Done'
The script will now appear in 'SCRIPTS > Batch Processing > NormalizeScaleGradient 2.0'

Regards, John Murphy

[Edit: NSGXnml is now also available for Linux]
 

Attachments

  • NormalizeScaleGradient_2.1.0.zip
    128.6 KB · Views: 101
Last edited:
This is perhaps more of a generic question than NSG specifically but....

NSG is very single streamed, and I have a lot of cores. I went to do a LRGB-HA set and decided to just keep hitting "PI - New Instance" and running it for a different filter, so I could set all 5 running at once.

I managed to get the system CPU 33% busy which is much better than one at a time!!

But... for each new instance I had to install both NSG and the XNML option separately. Is there a way to make that "stick"? Or is that just the price of admission for running parallel instances?

I like the "and run integration" -- I don't think any of these will finish before I crash for the night, that way another part will be all done! Thanks.
 
Status
Not open for further replies.
Back
Top