Understanding My Linear Fit Clipping Result

[GaryI]

I have been having a problem for several days with amp glow appearing in some, but not all, of my final integrated images.  I think I have isolated the problem and I would like your help understanding what I am doing wrong. 

I am running the batchpreprocessing script for a narrowband 3 channel image of NGC7380 (Wizard Nebula).  I am using 50 lights, applying cosmetic correction, and using masters for bias, dark and flat.  I am using defaults for all of the parameters.  If I use Winsorized Sigma Clipping for my rejection algorithm, the final image looks great.  If I use Linear Fit Clipping, which I usually do when the number of lights is over 20, it looks like the final image does not account for the master dark.  There is amp glow along the edges.  This is a repeatable result. 

I have attached 2 images which compare the final OIII result for each of the above rejection algorithms.  Any help you could provide would be appreciated.

Thanks,
Gary

[GaryI]

The jpg images above don't shown the amp glow very well.  When viewed in PI in its native form, it is obvious.

[GaryI]

Also, to help, the amp glow is mainly in the upper right and lower right of the 2nd image.

[pfile]

are you scaling (optimizing) the darks?

in the presence of certain chip artifacts (like RBI preflush and presumably amp glow) if you scale the dark you won't completely remove the artifact.

what do the calibrated subs look like?

rob

[GaryI]

Rob,

Thanks so much for your reply. 
Two answers:
1.  The calibrated subs are exactly the same in each case (blinking between them)
2.  You are right - optimizing the darks has a significant effect!  My darks are the same length as my lights, so I didn't think it would matter.  When I unchecked the box to optimize the darks, the resulting integrated image looks much better.  The amp glow is almost gone.  I don't understand it, but I am happy with the result.

Thanks,
Gary

[pfile]

cool!

the reason is that the amp glow isn't necessarily a function of exposure time, as the dark current itself is. PI optimizes darks by varying a linear scaling factor and finding the scaling factor for which the noise in the calibrated frame is minimized... it does not care about exposure times or temperatures, but instead takes this purely numeric approach.

scaling the dark works great if your darks contain only the dark signal (the bias signal is not a function of time, but that's why you first subtract a master bias from the master dark before scaling it). the amp glow in the dark is attenuated by the scaling and then when you subtract the dark from the light, the amp glow in the light is not fully removed.

rob

[ChoJin]

Does it mean we shouldn't use optimize if we have amp glows in the master dark? (e.g. with a dslr)

[GaryI]

ChoJin,

I suspect that amp glows are most often due to mismatched settings (exposure time, gain, temp, etc.) between the calibration frames (darks, bias, flats) and the light frames.
I think that my case may have been a bit unusual, and may have been specifically associated with my use of linear fit clipping.  But, in my case, it DID help reduce the glow to NOT optimize darks, especially since my dark exposure time was matched exactly to my lights time.

Gary

[ChoJin]

I was asking because I'm currently processing M31 (my first-light of my own equipment) using a canon 6D.

I carefully matched my darks using BatchStatistics (using temp, BWMV, MAD, Qn and Sn mostly) and I do see an amp glows on my resulting master dark. I also did use linear fit for my integration and I let imageintegration optimize my darks. That's why I was worrying when reading this thread.

But I don't think I'm seeing the amp glow in my resulting integrated image though (but I'm still worrying because I lack experiences, I therefore don't trust myself too much, even though I'm trying to be as throughout as possible). So maybe I'm ok on that aspect, you guys will tell me I guess when I'm done and I'll post my resulting image in the gallery :-)

[jkmorse]

Gary,

When you calibrated your lights with the Optimize button checked did you get any "No Correlation" errors reporting during the process.  I have had an issue that using the "Optimize" switch is resulting in me getting the "no correlation" warning with most images and when that happens, the darks are not subtracted for those lights.  I have reported the issue and am working it with Vicent to resolve , but any third party data would be helpful.  To check, try rerunning your calibration and just watching the results coming out of the process container window.

Thanks,

Jim   

[GaryI]

Jim,

Your post had me very excited, as that would well explain why I am sometimes seeing amp glow.  Alas, I reran the script and didn't see any "no correlation" messages, and I don't remember seeing any from the previous runs.  Oh well.  I believe that my issue may be due to the CMOS-style sensor (ZWO ASI1600mm camera) that I am using, due to the amp glow possibly not being consistent from day to day.  I see the glow in my final image when I run the Batchpreprocessing script using the Linear Fit Clipping rejection algorithm and settings of 5.0/3.5 for Linear Fit Low/High.   If I run it with a different algorithm, or with LFC settings of  5.0/2.5, the glow is barely noticeable.

By the way, thanks for the information that you have provided to me indirectly through your informative posts on other subjects.  I am on the verge of moving from LRGB to RGB imaging.  I have read your conversation with Juan on that subject, but I'll confess to not understanding everything yet.  Have you now moved in that direction?

I really enjoyed reading through your web site.  It is excellent and well done.  If you don't mind sharing, what hosting service are you using?  I would like to utilize something similar some day.

Also, I enjoyed a previous post where you alluded to your occupation.  We may have crossed paths - I spend 20 years working in West Houston in the same business, but as an engineer.  I am now retired and observing from the shores of Lake Livingston.  Much better skies for observing than West Houston, but not as nice as New Mexico of course.

All the best,

Gary Imm
Onalaska, TX

[ChoJin]

Since you are mentioning it, could you link the topic about the LRGB vs RGB flow you are referring to?

[GaryI]

Chojin - here is the thread - http://pixinsight.com/forum/index.php?topic=6042.15.

[GaryI]

And another - http://www.cloudynights.com/topic/492471-lrgb-vs-rgb-synth-l/

[GaryI]

Sorry to flood you with links, but this is another good one by Juan - http://pixinsight.com/forum/index.php?topic=1636.0