NormalizeScaleGradient: Impoved weight calculation.

jmurphy

PTeam Member
Jun 13, 2010
234
112
Basingstoke, England
I am using 1.1 Beta 2 and every time I try ti use the script I getthe following error message: I am using 1.1 Beta 2

*** Error [022]: /Applications/PixInsight/src/scripts/JohnMurphy/NormalizeScaleGradient/NormalizeScaleGradient.js, line 74: TypeError: (intermediate value).getExposure is not a function

I am using this on a Mac Book Pro
Can you reproduce this with just two files? If so, could you send me a link to the files so I can try to reproduce it.
This is urgent because version 1.1 is about to be released.
Thanks, John Murphy
 

jmurphy

PTeam Member
Jun 13, 2010
234
112
Basingstoke, England
Unfortunately I cannot reproduce it with my data.
Please also try it with this version (1.1 Beta4):

See Beta5, #124
 
Last edited:

nhey

Member
Oct 25, 2019
10
1
65
Sudbury, ON, Canada
I am right now running this script. Thank you for putting in the running time/progress!! I wish the whole PI process did that!!

My first question was already answered--I am running this on OSC data. Adam Block's video series on this is what got me started; I suspect I am going to be using it for all of my images now.

I will be reading your posts above much more carefully, and the documentation file. You are to be congratulated for all the amazing work this script and your PhotometricMosaic scripts involve. Even I, a very inexperienced user, can get great results easily (at least so far!) from PMM. I suspect the same will be true of NSG.
 
  • Like
Reactions: jmurphy

tdgm

Well-known member
Nov 26, 2014
321
4
First off thanks for getting back to me.

But A big OOPh on my part I did not add the script properly (I just place .js file in scr john Murphy folder not the whole NormalScaleGradient folder plus not adding the script file in featured scripts) When i went back and tried it with proper installation (with 2 `files no error message and everything works perfectly.

So sorry about my bad

Thanks agian for a great script
 

jmurphy

PTeam Member
Jun 13, 2010
234
112
Basingstoke, England
First off thanks for getting back to me.

But A big OOPh on my part I did not add the script properly (I just place .js file in scr john Murphy folder not the whole NormalScaleGradient folder plus not adding the script file in featured scripts) When i went back and tried it with proper installation (with 2 `files no error message and everything works perfectly.

So sorry about my bad

Thanks agian for a great script
I am very pleased to hear the script is ok. The new version has already been uploaded to PixInsight, and Juan will be creating an update soon...
 
  • Like
Reactions: Astrophoto_ANdy

limeyx

Member
Sep 17, 2020
6
3
Will this work with the CFA Drizzle workflow, or does it somehow replace that ? If not, would I generally be better off doing this than CFA drizzle ?
 

jmurphy

PTeam Member
Jun 13, 2010
234
112
Basingstoke, England
Will this work with the CFA Drizzle workflow, or does it somehow replace that ? If not, would I generally be better off doing this than CFA drizzle ?
Unfortunately NSG is not yet compatible with drizzle. Until it is updated for drizzle, personally, I would use NSG rather than drizzle. I think the best thing to do is to experiment with both and compare the results. Compare them at the size you would normally view the image. I think the less perfect the observing conditions, the more important it will be to use NSG.
Regards, John Murphy
 
  • Like
Reactions: rbotero

KevinB

New member
Apr 11, 2020
3
0
Hi John,

Outstanding work on this script...I have tried it on a number of past registered target frames to see if I could get superior results and I have as I needed less post processing to get very good results. I think I may include this as part of my normal workflow as I honestly think the normalization and weighting is superior to what is part of ImageIntegration as is.

Anyway a question...I have downloaded the 1.1 beta4 above and basically used it to replace the matching script directory with contents of the zip and it came up with the new features perfectly fine BUT if I check the ImageIntegration section leaving all defaults the script runs perfectly fine but doesn't start ImageIntegration at the end.

Curious what I am doing wrong?

Regards,
Kevin
 

jmurphy

PTeam Member
Jun 13, 2010
234
112
Basingstoke, England
Outstanding work on this script...I have tried it on a number of past registered target frames to see if I could get superior results and I have as I needed less post processing to get very good results. I think I may include this as part of my normal workflow as I honestly think the normalization and weighting is superior to what is part of ImageIntegration as is.

Anyway a question...I have downloaded the 1.1 beta4 above and basically used it to replace the matching script directory with contents of the zip and it came up with the new features perfectly fine BUT if I check the ImageIntegration section leaving all defaults the script runs perfectly fine but doesn't start ImageIntegration at the end.

Curious what I am doing wrong?

Regards,
Kevin
ImageIntegration should open after you exit from NSG, provided that some target images have been normalized.
Are there any errors shown in the console?
An image of the console showing the summary section might be useful. Also, what OS are you using, and PixInsight version?
 

limeyx

Member
Sep 17, 2020
6
3
I first need to port the code to C++.
It should then be possible to create a drizzle option.
Since this is all done in my spare time, there is likely to be a significant wait...

Regards, John Murphy
Thanks so much, this is awesome.

I do have a question that I didn't see asked above.
I'm stacking multiple nights but I cannot stack it all at once because its 500 frames, so I am stacking each night and then integrating the nightly images

It would seem like it makes sense to try to find a best reference frame across all the nights and use that for all the Normalizations rather than pciking a different reference for each night's data, does that make sense. I assume the reference frame does NOT have to be in the file list of files to normalize ?
 

jmurphy

PTeam Member
Jun 13, 2010
234
112
Basingstoke, England
Thanks so much, this is awesome.

I do have a question that I didn't see asked above.
I'm stacking multiple nights but I cannot stack it all at once because its 500 frames, so I am stacking each night and then integrating the nightly images

It would seem like it makes sense to try to find a best reference frame across all the nights and use that for all the Normalizations rather than picking a different reference for each night's data, does that make sense. I assume the reference frame does NOT have to be in the file list of files to normalize ?
Yes, you are correct. The reference frame does not have to be in the target list. You can use the dark folder icon to specify the reference frame instead of the 'Set reference' button
1624128311934.png
I agree with you, it makes sense to normalize all the images to the same reference frame, even if they are stacked in smaller groups. This also has the additional advantage that all the weights will be comparable.
Note that ImageIntegration is only populated with images from the 'Target Images' list. Hence if you have multiple runs, you would only want to include the reference image in the Target Images list for one of these runs.
Regards, John Murphy
 

gmeyer1

Member
Feb 16, 2021
7
0
Unfortunately I cannot reproduce it with my data.
Please also try it with this version (1.1 Beta4):
Unfortunately I cannot reproduce it with my data.
Please also try it with this version (1.1 Beta4):
When i download this zipfile....how do I get it into PI? I go to Scripts, feature scripts, add....but it doesn't show up when I point to it???
 

rockenrog

Well-known member
Apr 7, 2018
70
8
Hi John,
Thanks for your hard work, and very useable script!
This post is a continuation of the work started in my Post #31 on P2 of this thread.
I did a little study by comparing Image Integration via NSG script, vs the standard way of Image Integration with Noise Evaluation for weighting.
Attached is a file with several charts.
This data is based on high and varying background level data (659mm focal length) of M101. This where the NSG script should shine.
I wanted to see the correlation between the NWEIGHT and other factors when integrating without using NSG script.
Weight in normal integration (no NSG) was from the process console. At this time I have extracted only 40 frames of data from the process console.
I used Subframe Selector to get MAD, stars detected, and median from the frames used in normal integration.
I used same reference image for NSG script, integration after NSG script, and for normal integration.

In the end I asked myself:
1. Do registered individual images that look bad correlate better to NWEIGHT or to Weight from normal integration?
My answer; it varies:
-- All images that look bad have low NWEIGHT, but also some of these bad images also have low Weight from normal image integration.
-- No images with that look bad have high NWEIGHT. So NWEIGHT does not let me down.

2. Is the image after image integration looked better with NWEIGHT vs normal integration weights (no NSG)?
My answers:
-- The background gradient is better (simpler gradient to fix in DBE) using NSG script.
-- The normally integrated galaxy color is different (more blue after unlinked stretch). The NSG integrated galaxy looks brighter, but the readout data does not show any difference (out to 3 decimal places).
It is hard for me to say if the image quality is significantly different.
-- The edge goes to NWEIGHT integration.
-- Perhaps the final image quality is not significantly different if the weights are not exactly correct.

Any comments/questions/next steps I might do?

Thanks,
Roger
 

Attachments

midnightlightning

Active member
Nov 26, 2018
40
3
I agree with you, it makes sense to normalize all the images to the same reference frame
Can I just double check this with you, it surprised me a bit although it may be down to my lack of understanding.

I have 5 sessions on C31 which include LRGB and Ha. My best reference frame is in Ha, unsurprisingly, but I'm just wondering how using this for, say the Green stack, would work out. The green is poor by comparison and I'm wondering whether I will end up with not enough green in the mix after weighting?
 

jmurphy

PTeam Member
Jun 13, 2010
234
112
Basingstoke, England
Can I just double check this with you, it surprised me a bit although it may be down to my lack of understanding.

I have 5 sessions on C31 which include LRGB and Ha. My best reference frame is in Ha, unsurprisingly, but I'm just wondering how using this for, say the Green stack, would work out. The green is poor by comparison and I'm wondering whether I will end up with not enough green in the mix after weighting?
The reference frame must be from the same filter as the target files. I didn't make this clear enough in my reply, so thanks for giving me the opportunity to clarify!
 
  • Like
Reactions: midnightlightning

midnightlightning

Active member
Nov 26, 2018
40
3
The reference frame must be from the same filter as the target files. I didn't make this clear enough in my reply, so thanks for giving me the opportunity to clarify!
Thanks for the clarification.

Can I just ask which version I should be using. I have been watching the Adam Block videos and have seen two versions, one with Image Integration included and one without. I just installed the Beta 4 and it doesn't include Integration which I expected to be there?
 

jmurphy

PTeam Member
Jun 13, 2010
234
112
Basingstoke, England
Thanks for the clarification.

Can I just ask which version I should be using. I have been watching the Adam Block videos and have seen two versions, one with Image Integration included and one without. I just installed the Beta 4 and it doesn't include Integration which I expected to be there?
The dialog shows the version at the top. The latest version should include a new section for Image integration, where you can also specify the minimum image weight for enabled images. Beta4 is the latest.
 
  • Like
Reactions: midnightlightning

midnightlightning

Active member
Nov 26, 2018
40
3
The dialog shows the version at the top. The latest version should include a new section for Image integration, where you can also specify the minimum image weight for enabled images. Beta4 is the latest.
My mistake, I was looking for a tick box at the bottom of the screen.
 

KevinB

New member
Apr 11, 2020
3
0
ImageIntegration should open after you exit from NSG, provided that some target images have been normalized.
Are there any errors shown in the console?
An image of the console showing the summary section might be useful. Also, what OS are you using, and PixInsight version?
Hi John,

No errors that I can see...just warnings on overwriting processed files...and the fact that there are no real corrections going on for normalization as background pretty clean but still can use the weights and those seem good. I knew ImageIntegration only opened up after exiting but no luck on that so far.

Log and info on OS, PI release, PC specs attached.

Thanks
Kevin
 

Attachments