Image Calibration Woes..Whats Going Wrong

[GeneralT001]

I have a number of Ha frames that I am trying to calibrate. I have tried the usual methods that have worked fine on other data sets in the past - but this time I get these results?

I even painstakingly went through the Adam Block series on preparing the Bias/Darks/Flats for Image Calibration - with the same results? What would account for this?

The uncalibrated (Raw Ha) shows a mean of 993.7 in the Statistics Process module and the calibrated Ha frames show a mean of 101.611?

[pfile]

can you post some subs together with your master bias and master dark? or are they in your CN thread?

rob

[GeneralT001]

Hi here are 10 Ha files a Master Bias and a Master Dark.

https://1drv.ms/u/s!AqJuORoJIH4jnWFcPcr1GA7J1w7e?e=j8JpUm

[pfile]

couple of things i noticed -

is the readout rate for the dark subs also 2MHz?

were the dark subs acquired with voyager also?

the dark is -25C and the light is at -30C. the levels in the dark are indeed higher than the light. because they both have the same exposure length, the levels should be similar... but the difference in temperature could be making the levels in the dark higher. it could be that you just need darks also taken at -30C.

i have on occasion seen situations where the background levels of a narrowband light are very similar to the levels in the dark, leading to a small number of pixels being clamped to 0 during calibration. this can be handled by adding an output pedestal to the calibration process... but clearly you're in a different territory as the dark median is almost 30DN above the light.

rob


rob

[bulrichl]

This is an interesting case.

Because the dark frames were captured at -25 °C and the light frames at -30 °C, I performed the calibration of the light frames with dark frame optimization, i.e ImageCalibration with MasterDark and MasterBias (in the Master Dark section: both options 'Calibrate' and 'Optimize' enabled). Nevertheless the calibrated light frames were severely clipped. So I applied an output pedestal (100, 150 and 200). An output pedestal of 150 does not last to avoid the clipping of the calibrated light frames.

Whereas the individual calibrated files look passable when calibrated with a pedestal of 200, this doesn't seem to solve the underlying problem. After registration and integration, the result is not correct with ImageIntegration's default setting 'Subtract Pedestals' enabled. However, when this setting is disabled, strongly varying results regarding pixel rejection are obtained, depending on the used value of the output pedestal. This doesn't seem reasonable at all.

Particularly strange is that the mean or median value of the uncalibrated light frames are quite different: it looks like the bias offset (which seems reasonable in image 1) is decreasing from image to image. This effect is especially pronounced from image 1 to 2, see appended plot. I guess this is the real problem. I tried to circumvent this effect by adding the difference median(light frame 1) - median(current light frame) to each uncalibrated light frame in PixelMath. Then these "corrected" light frames were calibrated as above with an output pedestal of 100. The result is shown in the appended screen section.

For me it looks as if the bias offset of this camera (a FLI MicroLine ML16200) is not stable. However, there could be a different cause, what about RBI (anti-ghosting) technology? I don't know a lot about this topic, perhaps other users of a similar camera can chime in.

Rob's notes are to be considered as well, the most important question probably is whether dark and light frames were captured with the same acquisition software (and the same settings except the differing temperature).


Bernd

[GeneralT001]

Hi,

Thanks for looking at this.

Both the Bias frames (25 x .01sec - maybe they need to be at "0"??) and the Darks were obtained with SGP. All settings the same.

[pfile]

try re-taking the calibration frames with voyager and see how it goes with calibration... in theory both programs should get the same data from the driver, but in practice things can and do differ. i think the readout rate is a candidate difference between the two programs at the very least.

SGP has built-in support for some number of cameras, but i don't know if FLI is among them. my understanding is that Voyager requires ASCOM drivers to run a particular camera, so there's at least a chance that SGP is using a different version of the driver than Voyager is.

i have sometimes seen bias frames come out marked with very short shutter times... in theory with a cooled camera a very short dark should be indistinguishable from a bias frame, so you are probably OK. i think the main problem is the two different capture programs.

rob

[GeneralT001]

I went ahead and added an output pedestal of 1000 which seems to make the calibrated frames look normal. Don't know why though. I guess I've never looked at one of the calibrated frames before and instead just went right thorough with CosmeticCorrection/StarAlign and Image Integration and things just seemed to work out. I will try the same thing using Voyager and see if it does make a difference.


Thanks

[pfile]

adding the pedestal prevents the negative values from getting clamped to 0. the process of calibration is simple math - if the calibration frame (dark, bias) has a higher value at a given pixel than the light, then the calibrated result will have a negative value at that pixel. those negative values get clamped to 0 when the file is written, and that's why your calibrated image looks destroyed. adding the pedestal raises the value of all the pixels, and any pixel that had a negative value will now have a positive value of pedestal-original_value.

it's certainly not normal for there to be more signal in the dark than there is in the light, and i assume you'll find that SGP and Voyager write out different files for a given exposure time and temperature.

the bottom line is that mixing capture programs across lights and calibration frames is fraught with problems like this...

rob

[GeneralT001]

Thanks pfile,

I did do the light frames with Voyager and the Darks/Bias with SGP. I'll redo the Bias/darks with Voyager and see if that helps things

[GeneralT001]

Here is a comparison between the master bias SGP made and the master bias that Voyager made. SGP is on the left and Voyager is on the right:

The mean for the SGP master bias is 1008 and the mean for the Voyager master bias is 858. Bit of a difference...not sure what it means exactly??

I am redoing the Darks as well and will compare those.

[pfile]

well, there must be some difference in how the hardware is configured between the two programs. i guess if you are really interested you could ask jared/ken and leo to review your settings or maybe explain any settings that are not visible to the user with respect to hardware/driver config.

rob

[wannaberocker]

Had a similar occurrence when trying to calibrate SII subs, pixels were basically rendered black. I chose to have Pedestal set to a literal value of "0" which seemed to solve the problem. I noticed PI would no longer subtract the value of 100 from each light frame. Not sure if that's the answer though. This SBIG STT8300 camera seems to have a Pedestal value of 100 written in the FITS header, I noticed PI was subtracting this pedestal value of 100. Not sure if I should be doing it that way but it made my SII subs appear properly calibrated instead of massively clipped in the blacks. HA and OII subs didn't seem to suffer from this issue, only the SII.

[pfile]

i think the PEDESTAL keyword might come from the capture software - i use SGP with my STT8300M and don't see the PEDESTAL keyword in my fits files. what capture software do you use?

rob