Calibration - I've gone stupid!

[kor]

For some reason, I can no longer get my O3 frames to calibrate.  I've only been using PixInsight for a couple of years, but I really thought I had the hang of calibration.  But now I can't seem to make it work and I'm out of ideas.  I've checked and double checked and am pretty sure that I'm using the same process as I have for other, successful, images.  I suspect it's something simple that I'm just not seeing and hope someone here can help!

My O3 raw images have always had rather significant sensor artifacts, but, until now, they have always calibrated out.  Other frames have much less obvious artifacts and seem to calibrate fine.

Here at the components used in my calibration:
 - Master Bias: <http://www.kor-astro.net/PI_Support/004/Master_BIAS_-25C.xsif>
 - Master Dark:<http://www.kor-astro.net/PI_Support/004/Master-DARK_-25C.xsif>
 - Master Flat-Dark: <http://www.kor-astro.net/PI_Support/004/FSQ_Master-FLAT-DARK-O3_-25C.xisf>
 - Process Icon: <http://www.kor-astro.net/PI_Support/004/Integrate_Dark_Bias.xpsm>

 - Sample Flat: <http://www.kor-astro.net/PI_Support/004/Flat_O3_0005.fit>
 - Master Flat: <http://www.kor-astro.net/PI_Support/004/Master-Flat-O3.xisf>
 - Flat Calibration Process Icon: <http://www.kor-astro.net/PI_Support/004/Calibrate_Flat.xpsm>
 - Flat Integration Process Icon: <http://www.kor-astro.net/PI_Support/004/Integrate_Flat.xpsm>

 - Sample Raw Light: <http://www.kor-astro.net/PI_Support/004/Light_O3_0005.fit>
 - Sample Calibrated Light: <http://www.kor-astro.net/PI_Support/004/C_Light_O3_0005.xisf>
 - Light Calibration Process Icon: <http://www.kor-astro.net/PI_Support/004/Calibrate_O3.xpsm>

Integration times are:

• Dark: 600 sec x  50 frames
• Bias: 200 frames
• Flat and Flat-Dark: 42.94 sec x 50 frames
• O3 Light: 30 min

I copied a previous (successfully calibrated) project to start this one.  I don't think I have changed the process icons other than file locations and I have checked and double checked them.  I'm using the same equipment to capture the darks, flats, and lights.

I've redone my flats twice with the same results.  I've tried doing everything in FITS instead of XSIF and, as expected, got the same result.  I've run out of stuff to try.  I really think I'm just overlooking something stupid.  I would really appreciate any help!

- Shane

[PeterLive]

I have been unable to batch process Raw files since the New update of PI.  Image Calibration does not work either.  Results of calibration indicate insufficient data to star align.
Calibration does complete but there is virtually no date in the image despite the light image being of high quality.
I have been able to stack these images without image calibration so what is wrong?  Any suggestions welcome.
This has always work seamlessly in the past.

Anyone else have this problem

[Juan Conejero]

Hi Shane,

Calibration works without problems with these files and the ImageCalibration tool in version 1.8.5. Since the master dark includes the bias, you must calibrate it by removing the master bias. You are already doing this correctly in the icon included in "Calibrate_O3.xpsm". The dark scaling factor is 2.603. I don't know why this is not working for you.

[kor]

Thanks, Juan.  I'll try it again.  I'll look closely at the dark scaling factors and see if they match yours.  As I said, I'm think I'm doing it the same way I have for the last couple of years ...

- Shane

[kor]

Juan, Sorry this took so long.  I've been out of town and wanted to try with 1.8.5 before I responded.

I still cannot seem to calibrate with a good result.  Everything runs fine.  I get the same dark scaling factor you posted (2.603), but the artifacts in the flat are still very evident on the calibrated light frame.

I started with a new, empty project, opened and reset the ImageIntegration tool and set the parameters to do only a single test file using the expected parameters and same BIAS, DARK, and FLAT from last time.  I ensured that "Calibrate" and "Optimize" were checked under Master Dark and nowhere else.  Here is the process icon for the ImageIntegration tool: https://www.kor-astro.net/PI_Support/004/New_Integration.xpsm

Running the tool results in the following Console output:

Code: [Select]

ImageCalibration: Global context

Loading master calibration frames:
Loading calibration frame image:
B:/2017/CBT1/PI_Support/Master_BIAS_-25C.xisf
Loading image: w=3326 h=2504 n=1 Gray Float32
1822 FITS keyword(s) extracted.
Loading calibration frame image:
B:/2017/CBT1/PI_Support/Master-DARK_-25C.xisf
Loading image: w=3326 h=2504 n=1 Gray Float32
422 FITS keyword(s) extracted.
Loading calibration frame image:
B:/2017/CBT1/PI_Support/Master-Flat-O3.xisf
Loading image: w=3326 h=2504 n=1 Gray Float32
422 FITS keyword(s) extracted.

Applying bias correction: master dark frame ...

Dark frame optimization thresholds:
Td0 = 0.00036410 (46805 px = 0.562%)
Computing master flat scaling factors ...
s0 = 0.370289

Calibration of 1 target frames:
* Using 1 worker threads

Calibrating target frame 1 of 1
Loading target frame:
B:/2017/CBT1/PI_Support/Light_O3_0005.fit
Reading FITS image: 16-bit integers, 1 channel(s), 3326x2504 pixels: done
55 FITS keywords extracted.

Writing output file: B:/2017/CBT1/PI_Support/Light_O3_0005_c.xisf
Dark scaling factors:
k0 = 2.603
Gaussian noise estimates:
s0 = 4.645e-04, n0 = 0.440 (MRS)
Writing image: w=3326 h=2504 n=1 Gray Float32
76 FITS keyword(s) embedded.

===== ImageCalibration: 1 succeeded, 0 failed, 0 skipped =====
4.573 s

And the resulting image using the default STF still shows the dark margin along the right edge and the large bright area right of center.
https://www.kor-astro.net/PI_Support/004/Light_O3_0004_c.xisf

Since this frame calibrated successfully when you did it, I must have something set up wrong.  Do you have any ideas on where to look?  Or what other techniques I could try to remove these artifacts?  Is there a way to somehow strengthen the effect of the flat application?

Thanks - Shane