calibration using scaled darks

[troypiggo]

Instead of using darks that match the exposure time of the lights I am taking, I thought I'd try using a dark library and scaling the darks.  I had read somewhere that the ideal exposure time for the dark library in this situation was 3 to 5 times the maximum exposure length of your lights, so I created a dark library of 20x30 min darks for 1x1 binning.  Due to light pollution and lack of courage, by longest lights are usually around 5 mins, maybe 8 mins.

So as I understand it, when I'm actually imaging, I capture the lights, and for calibration frames I should only need to capture flats and bias on the night?

I came across the excellent tutorial for DSLR workflow by astropixel and Vicent: http://pixinsight.com/forum/index.php?topic=2570.msg19019#msg19019 to use as a basis, but it's not specific regarding the usage of scaled darks.

So my biases are pretty straight forward, they're not calibrated, no normalisation and don't care about weights, pixel rejection I used winsorized clipping without normalisation and 4 high/3 low, and saved as master bias.

My (main) question is about calibration of the calibration files.

I'm confused about whether or not I should calibrate my darks before I create the master dark.  I don't think I do, since the biases are taken on the night.  So I just have an uncalibrated integrated master dark from the library, and just tick the box "calibrate" for it when it's used to calibrate the lights.  Correct?  And PI takes into account the fact that the darks are 30 mins exposure while the lights are only, say, 5 mins?

And similarly for the flats.  Should they be calibrated as individual frames before they're integrated, using the above master bias and dark?  Or just tick the "calibrate" option for them at light calibration time?  And PI hanles again the different exposure times between darks, flats, and lights here?

[pfile]

i don't know how long a set of bias files are good (or darks for that matter), but generally i just have a master bias sitting around and different master darks. i don't take bias frames each time i image. i have a master dark for 240s, 480s, 1800s, but the temperatures vary a little bit.

when calibrating, i just load the uncalibrated master dark and the master bias, and tick "calibrate" and "optimize" for the dark, even if the dark duration matches the light duration. i figure if PI can evaluate the actual noise in the image i might as well let it scale the dark. i usually see scaling factors from about 0.8 to 1.2 when using a matching dark. i have however occasionally had overcorrection of hot pixels due to dark scaling.

for the flats, i sometimes shoot at a different ISO (which is technically wrong). in that situation it does not make sense to calibrate with the masters for the lights. even at the same ISO the flat durations are so short that i'll make separate flat darks and calibrate the flat images separately, so that the integrated flat master is already calibrated. with flat durations of a couple of seconds vs 100s of seconds for the master dark, it seems better to use darks which match the flats. so one way or another, i pretty much never check the flat calibration box.

[troypiggo]

Thanks for the reply.  Still not certain about what I should be doing.

Reason I'm shooting the biases and flats "on the night" is that they're relatively quick and easy to do.  The darks, as you know, are more time-consuming, so a library is more convenient.

I think what is clouding my thinking is not wanting to double up on bias subtraction, because it's subtracted from darks, and bias and darks are subtracted from flats, and then bias and darks are subtracted from lights and divided by flats.  Guess confirmation of what bits of the above PI does automatically and what needs to be done manually.

[pfile]

well... i guess you need not worry about double bias subtraction unless you come in to imageintegration with a pre-calibrated dark. if that's the case, just supply the master bias file and uncheck 'calibrate' on the dark.

same goes for the master flat, if it's already calibrated, just don't tick 'calibrate' and you won't subtract the bias or the dark from the master flat given.

basically, the 'calibrate' checkbox for the master dark will subtract the master bias given in the section above.
the 'calibrate' checkbox for the master flat will do the right thing as well - it won't subtract the bias twice from the flat. the only time that would happen is if you supply it with a calibrated flat but you ask to calibrate it again.

here's how i use it:

1) integrate master bias: BatchFormatConversion of .CR2 subs (Input Hint: "raw") -> FITS, ImageIntegration of bias FITS files -> master_bias

2) integrate master dark: BatchFormatConversion of .CR2 subs (Input Hint: "raw") -> FITS, ImageIntegration of dark FITS files -> master_dark

3) integrate master flat dark: BatchFormatConversion of .CR2 subs (Input Hint: "raw") -> FITS, ImageIntegration of flat dark FITS files -> master_flat_dark

4) calibrate flat subs with matching flat-dark of same duration: ImageCalibration - no master bias, master_flat_dark from step 3 (calibrate off, optimize off), no master flat.

5) integrate master flat: ImageIntegration of calibrated flat subs from step 4 -> master_flat

6) calibration of lights: ImageCalibration - master_bias from step 1, (calibrate off); master_dark from step 2, (calibrate on, optimize on), master_flat from step 5 (calibrate off).

then debayer, ImageAlignment and ImageIntegration of lights, and then all the rest of the stuff...

[troypiggo]

Thanks again mate.  Also been reading some tutorials/docs on image calibration and integration that I hadn't previously found.  This one agrees with you: http://www.pixinsight.com/tutorials/master-frames/en.html in terms of don't bias subtract the individual dark subs, just create the master dark uncalibrated, then tick the calibration box at image integration of lights stage.

Think I'm good to go now.