Author Topic: Image Calibration Error with Darks  (Read 230 times)

Offline andyschlei

  • PixInsight Addict
  • ***
  • Posts: 157
    • View Profile
    • http://www.obsballona.org
Image Calibration Error with Darks
« on: 2019 August 31 12:19:49 »
This may be a user bug, but I believe I am doing all the steps correctly.  I have created master bias and darks for my images.  When I calibrate the image, a bunch of detail is lost.

Attachments are as follows:
  • Before and after calibration for one frame.  All images are auto stretched with STF.
Other files too big.
  • Calibration settings
  • Integration settings used for darks and bias (20 darks, 40 biases)
  • The same data processed in MaximDL (to prove I wasn't imagining things)

I've tried all different data settings for the master darks and biases.  The problem only occurs when I calibrate with darks.  It appears to be over-subtracting the dark data.

Thanks in advance for any help.
« Last Edit: 2019 August 31 17:45:24 by andyschlei »
Observatorio de la Ballona
CDK 12.5, NP-101, C-11
AP-1200, AP-900
ST-10 XME, CFW-8, Astrodon v2 filters
Pyxis Rotator, TCF Focuser

Offline andyschlei

  • PixInsight Addict
  • ***
  • Posts: 157
    • View Profile
    • http://www.obsballona.org
Re: Image Calibration Over Subtracting
« Reply #1 on: 2019 August 31 12:21:41 »
Here are my calibration and integration settings
Observatorio de la Ballona
CDK 12.5, NP-101, C-11
AP-1200, AP-900
ST-10 XME, CFW-8, Astrodon v2 filters
Pyxis Rotator, TCF Focuser

Offline andyschlei

  • PixInsight Addict
  • ***
  • Posts: 157
    • View Profile
    • http://www.obsballona.org
Re: Image Calibration Over Subtracting
« Reply #2 on: 2019 August 31 12:23:36 »
And here is the MaximDL calibrated version, shown in PixInsight, default STF
Observatorio de la Ballona
CDK 12.5, NP-101, C-11
AP-1200, AP-900
ST-10 XME, CFW-8, Astrodon v2 filters
Pyxis Rotator, TCF Focuser

Online pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4531
    • View Profile
Re: Image Calibration Error with Darks
« Reply #3 on: 2019 August 31 19:07:48 »
the dark subs are not bias-subtracted, right?

if the master dark contains the bias signal, then the way you have it configured will end up with a double bias subtraction, since the bias signal is in the bias and in the dark. the easiest way to fix this is to tick "calibrate" in the dark section.

if you don't want to scale the dark, just untick 'optimize' in the dark section and remove the master bias. that will work too.

rob

Offline andyschlei

  • PixInsight Addict
  • ***
  • Posts: 157
    • View Profile
    • http://www.obsballona.org
Re: Image Calibration Error with Darks
« Reply #4 on: 2019 August 31 20:58:29 »
That's great. I removed the bias from the calibration, checked calibrate on the dark, and unchecked optimize and it worked perfectly.

So what do I need a bias for?  (question 1)

But what would I do if I wanted to scale my darks?  (question 2)  How would I integrate the darks to remove the bias and then allow PI to scale the dark?  A link to an article/tutorial would be great.  I'm getting back into things after several years off, so there a many features I don't know and many things I have forgotten.

Thank you for your help!
Observatorio de la Ballona
CDK 12.5, NP-101, C-11
AP-1200, AP-900
ST-10 XME, CFW-8, Astrodon v2 filters
Pyxis Rotator, TCF Focuser

Online pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4531
    • View Profile
Re: Image Calibration Error with Darks
« Reply #5 on: 2019 August 31 23:10:55 »
no problem -

you really only need a master bias if you want to scale the dark.

you can either make a master bias and subtract it from each dark sub to create a calibrated master dark (in other words, run ImageCalibration on the dark subs and just load the master bias as the only calibration frame, then integrate the calibrated dark subs to make a calibrated master dark), then use the calibrated master dark and the master bias when calibrating the lights, with only "optimize" checked in the dark section, or,

you can just integrate the master dark as you have done, and then load the bias and the dark while calibrating the lights but tick both "calibrate" and "optimize" in the dark section.

method 1 (bias subtracting the dark subs) is sometimes a bad idea when you are dealing with cameras with extremely low dark signal, or short subexposures even in high-dark current sensors. since the dark signal is small, you're effectively subtracting a master bias from a bias frame, and due to the normal uncertainty in any given subframe, you may end up with negative valued pixels in the calibrated dark subs (which get clamped to 0). it's safer to build uncalibrated master darks and subtract a master bias from them at light calibration time, since the masters should have higher SNR and overall the master dark's pixels should all be greater than the master bias's pixels. this is what the BatchPreProcessing script does.

i don't know what tutorials to recommend offhand, there are a lot of them. but warren's book is pretty comprehensive and i'm sure it goes over this stuff in detail.

rob

Offline andyschlei

  • PixInsight Addict
  • ***
  • Posts: 157
    • View Profile
    • http://www.obsballona.org
Re: Image Calibration Error with Darks
« Reply #6 on: 2019 September 01 07:32:46 »
Rob,

Thank you for the comprehensive reply.   Very much appreciated.

I've been using PixInsight since it was open source, but when work took me way for a few years it's remarkable the details I've forgotten.

Clear skies,

Andy
Observatorio de la Ballona
CDK 12.5, NP-101, C-11
AP-1200, AP-900
ST-10 XME, CFW-8, Astrodon v2 filters
Pyxis Rotator, TCF Focuser