Image Calibration Problem

[KenR]

I am having an issue calibrating my light frames. I attached a screen shot of 4 different methods I tried. As you can see, none are perfect. First, the camera is a QHY-178C OSC which has significant "amp" glow. Normal for this camera. The 4 images are the same light frame just calibrated differently in each case. For calibration I used 100 bias frames and 6 dark frames. The lights and darks were all exposed for 300s with the same camera gain (0) and offset (30). Also, all frames were taken with the camera cooled and stable at -15C.

So what is the problem? Well normally, I would calibrate the individual dark frames with the master bias an then integrate the master dark. I did that in the lower 2 photos in the screen shot. Both were calibrated with the dark calibrate option disabled (since I already calibrated the dark frames). The one on the left has dark "optimization" turned on with default settings and the one on the right has optimization OFF. In my case dark optimization is MUCH worse for some reason. The temperature, gain, exposure are all equal. I'm not sure why optimization is doing this. Regardless, even with dark optimization off the calibration is poor. You can still see some of the glow in the upper right.

The top 2 photos were done differently. I did not calibrate the dark frames with the bias, simply integrated the dark master using the frames as is. Then when I calibrated the light frames I used the calibrate option there. Again, with optimization ON (much worse) and OFF. As you can see, dark optimization is not my friend.

So why do I get such bad results when I calibrate the dark frames before integration? Also, why does dark optimization work so poorly with my images? After this test I've settled on uncalibrated dark frames and using the calibrate option in image calibration with optimization off. I get the best results but still not perfect.

BTW, I used STF stretch on all for images in the screen shot. I can post the actual images if it helps.

[sharkmelley]

For CMOS cameras having amp glow (especially the "starburst" type), it is important to switch off dark optimisation.  This is because successful amp glow removal requires a 100% scaling factor for the dark but dark optimisation might calculated a scaling different to 100% leading to undercorrection or overcorrection.  With set-point cooling, dark optimisation is not required in any case.

Adressing your other problem, calibration of darks should also be avoided because subtracting the master bias from the dark can lead to negative numbers (and hence numerical truncation) with CMOS sensors.  The resulting master dark will then not be properly matched.

In summary, don't calibrate your darks and don't use dark optimisation.

Mark

[KenR]

Thanks so much Mark. Your advice agrees with my test results. I just could not understand the reason why.

For the dark calibration, what if I used a small pedestal value? Is there any advantage to calibrating individual dark frames over calibrating the master dark when the lights are calibrated?

[bulrichl]

No, when you are not using dark frame optimization there is no need to calibrate dark frames or calibrate the MasterDark at all - and if you don't calibrate dark frames or the MasterDark you must not use a MasterBias with your light frames either.

See https://pixinsight.com/forum/index.php?topic=11968.0 for an explanation in more detail.

Bernd

[AstroScience]

Guys, correct me if i'm wrong.
Let's say you have 200 Bias frames and 20 Dark frames. The Bias signal will be more precisely represented in a stack of 200, than in a stack of 20 Darks.
Stack of 200 will inject less noise to lights than a stack of 20.
So what I usually do is, I'm pre-calibrating 20 Darks with high quality master Bias (200 frames) using Pedestal which is a must to prevent truncation.
Therefore I more precisely remove Bias signal from Darks. Then stack and create Master Dark from those 20 frames.
During light calibration I disable Dark Optimization and Dark calibration. Doing so, I remove more precisely Bias signal from Lights, then, Dark signal and pedestal got removed also.
I believe this way I inject less noise and create better calibrated Lights, than if I just had calibrated them all with a stack of 20 Darks.

[sharkmelley]

Guys, correct me if i'm wrong.
Let's say you have 200 Bias frames and 20 Dark frames. The Bias signal will be more precisely represented in a stack of 200, than in a stack of 20 Darks.
Stack of 200 will inject less noise to lights than a stack of 20.
So what I usually do is, I'm pre-calibrating 20 Darks with high quality master Bias (200 frames) using Pedestal which is a must to prevent truncation.
Therefore I more precisely remove Bias signal from Darks. Then stack and create Master Dark from those 20 frames.
During light calibration I disable Dark Optimization and Dark calibration. Doing so, I remove more precisely Bias signal from Lights, then, Dark signal and pedestal got removed also.
I believe this way I inject less noise and create better calibrated Lights, than if I just had calibrated them all with a stack of 20 Darks.

I cannot see why calibrating the darks with a 200 frame master bias before creating the (calibrated) master dark is any different to creating a master dark from uncalibrated darks then calibrating the master dark with the 200 frame master bias.  You should get the same final result in both cases.

Maybe I simply misunderstood what you are trying to say.

Mark

[AstroScience]

Hi Mark,
yes, no problem with that, either way will work fine. Subtracting Master Bias from Master Dark during Light Calibration will not cause truncation.

My point was to address Bernd post " if you don't calibrate dark frames or the MasterDark you must not use a MasterBias with your light frames either."

I still believe that using high quality Master Bias of 200 frames will be better than calibrate your lights with just Master Dark made of 20 frames.
Of course, if I could get 200 frames of darks, then the Bias signal and Dark signal in the stack will be better represented and I can forget about Master Bias.

But I just brought an example, if you have Master Bias with 200 frames and Master Dark with 20 frames, will you skip the Master Bias correction and use only Master Dark made of 20 frames?
Just saying that I would not.

[bulrichl]

Sergio,

Mark is right: no difference whether you calibrate each dark frame or the Master Dark. But that's not the whole story:  if you don't use dark frame optimization, this operation (the calibration of dark frames or the calibration of the MasterDark) is useless. The math that is applied during calibration (no dark frame optimization) is:

(LF - MB) - (MD - MB) = LF - MD

LF: Light frame
MB: MasterBias
MD: MasterDark

Since the MasterBias is subtracted both from the light frame and from the MasterDark it is not needed at all: it is canceled out. So you are wasting your time when capturing and integrating the 200 bias frames. That's not a matter of belief - it's simple arithmetic.

Bernd

[AstroScience]

Brend,

if I was wasting my time with 200 frames Master Bias, then my guess it would apply to maybe 95% of astroimaging community, that they are also wasting their time...

The math is right , but it oversimplified when we don't know what MB and MD actually is. If I simply refer to the (LF - MB) - (MD - MB) = LF - MD
then it may make me believe, that I can simply use only one single dark frame and calibrate with it all the Lights and get good results.

While it is true, using only one single dark frame that can efficiently remove unwanted signal, why then we strive to create a good masters of many frames?
The quality of the Light will be totally depend on the quality of MB and MD which is subtracted from it.

Once again , if we just refer to the math (LF - MB) - (MD - MB) = LF - MD, does it mean that if I use MD made of 20 frames and MD made of 200 frames, the result of the calibrated LF will be the same?
We all can agree that it is not. The noise that will be injected from Master with 20 frames will be higher than from a master of 200 frames. Master of the 200 frames will represent the actual noise and FPN way better
than a one single frame or stack of 10 -20 frames.

So my point is, if you have a Master Dark made of 20 frames and Master Bias made of 200 frames, you would still benefit from subtracting them both instead of only subtracting a Master Dark of 20 frames.

[bulrichl]

Sergio,

in light frame calibration, a MasterBias is only needed when you use dark frame optimization. When dark frame optimization is not used, there is no need for a MasterBias. Of course it makes a substantial difference in light frame calibration whether you use only one dark frame or a MasterBias made from 20 dark frames.

The bottom line is: if you don't use dark frame optimization, take more dark frames, make a better MasterDark and skip the bias frames.

Bernd

[freed]

Bernd, when not using a Master Bias as you suggesting when not using dark frame optimisation, what is the recommended part for your master flats. Would you calibrate these with Master Bias or not ?

[bulrichl]

Flat frames have to be calibrated for two reasons: 1) the bias offset has to be subtracted, 2) the fixed pattern noise has to be eliminated.

Generally a MasterFlat-Dark (made of flat-darks with the same exposure time as the flat frames) should be better suited for the calibration of flat frames than a MasterBias. The difference between the two depends on the dark current (= exposure time x dark current rate) that built up in your flat frames. With low exposure times of the flat frames and/or with cameras that have low thermal noise, a MasterBias is virtually the same as a MasterFlat-Dark.

When using dark frame optimization, a MasterBias is already available. Use it for the flat frame calibration as well if these conditions (low exposure of flat frames, low thermal noise) are met, make a MasterFlat-Dark if not.

When you don't use dark frame optimization, make a MasterFlat-Dark for the calibration of the flat frames and forget about bias frames. (The effort for the preparation of a MasterBias is the same as for the preparation of a MasterFlat-Dark.)

Bernd

[freed]

Thank you.