Proper calibration with Darks, Flats and FlatDarks

[MicroAstro]

My camera (QHY163M) has some amp glow, so I don't optimize my Darks and I use FlatDarks instead of Bias frames. What is the proper way to calibrate my lights and can I make any use of the easier workflow of the BPP script?

This is the way I think I need to do it:
1. Integrate Darks and FlatDarks (both carefully matched) to make MasterDark and MasterFlatDarks.
2. Calibrate Flats for ea filter using the matched MasterFlatDark
3. Integrate calibrated Flats to make a MasterFlat for ea filter
4. Calibrate Lights using MasterDark (contains the bias) and MasterFlats (bias removed by the MasterFlatDarks)

If this is correct, I'm wondering if I can use the Batch Pre-processing Script after step 3? It would be convenient to load my MasterDark, my MasterFlats, and my Lights and since I already have my MasterDark I can use it in the CosmeticCorrection tool. The BPP would then calibrate my Lights by filter and put them into the correct folders. This would make it easy to do the registration and integration.

However, how does BPP deal with the lack of Bias frames (it does give the error message). Since I don't need to subtract the Bias (it's in the MasterDark) and I don't optimize, does BPP just ignore the Bias and subtract the MasterDark and divide by the appropriate MasterFlat for ea filter?

Or am I heading down the wrong path altogether (a likely scenario!)?

thanks,
Keith

[M Covington]

Quick answer:  Put in your flat darks as bias frames.  (I assume they have exposures under 1/10 second.  That means they *are* bias frames.)

Longer answer:  See my thread, https://pixinsight.com/forum/index.php?topic=11984.0

And thanks for raising the issue!

[MicroAstro]

Interesting thread Michael, and interesting hack of the BPP. Sounds like what I want to do, except I don't scale/optimize my darks. But my FlatDarks are currently 1.5s so not so close to being Bias frames. I may try doing them with a brighter light source though.
...Keith

[pfile]

you can use BPP for all steps.

for each filter -
  1) run BPP with flat dark subs, bias frames, flats and a dummy light (maybe 3 dummy lights, as the integration of the dummy master light may fail with only 1 dummy light)
  2) make sure dark optimization is turned off (global options). this should cause BPP to ignore the bias frames and simply subtract the master dark it creates.
  3) discard the dummy master light, save the master flat for the current filter

now, in a new instance of BPP, load your bias frames, your darks that match the lights, the lights, and the 3 master flats. you may need to use the Add Custom functionality to assign the filter name to each master flat. the master flat was calibrated in the first run of BPP and will not be calibrated in the 2nd run - BPP does not calibrate master flats. make sure to tick Use Master Flat (global options)...

this time you can optimize the darks or not depending on what you want to do.

1/10s exposures are not technically bias frames. while the dark signal might be negligible, a true bias frame has a duration of exactly 0 seconds. covington is focused on DSLRs and they do behave somewhat differently which may warrant modifications to the BPP script, but for a cooled astro camera, BPP as it stands should be fine.

rob

[M Covington]

Yes -- there are no 0-second exposures on DSLRs, but 1/200-second flat darks are indistinguishable from 1/8000-second darks (the shortest available) on my cameras.

Am I right in thinking that if dark frame optimization is turned off, BPP does not need bias frames?  I haven't tried it.  Of course, in that situation the flats would have to be calibrated separately, by subtracting the flat darks.

[MicroAstro]

Actually the shortest exposure I can do with my camera is 0.3s, anything shorter and the frames will have a dark band at the top due to the Optic Black Calibration Circuit in the sensor according to Qiu. So I can't do a proper bias anyway.

Thanks for the discussion. Looks like I'm on the right track for processing with Darks and FlatDarks without any Bias frames.

...Keith

[M Covington]

Here is another way I tried out last night that works well:

Put the dark flats in as darks, and the regular darks also as darks.  They will form 2 groups of darks, grouped by exposure time.
Put the flats in as flats, the lights in as lights.

Turn "Optimize dark frames" OFF.  Then no bias frames will be required.

And let 'er rip.  This is basically the classic way of using calibration frames, without PixInsight's optimizer (scaler).

[M Covington]

OK, I hacked BatchPreprocessing a little to handle flat darks:  https://pixinsight.com/forum/index.php?topic=11992.0

[MicroAstro]

I've experimented with using a brighter light source for my flats, and I have now worked out the brightness I need to make my FlatDark exposures 0.3s, which is the minimum exposure time recommended for my camera to avoid dark banding across the top of the frame due to the Optical Black Calibration Circuit in the sensor. So my FlatDarks now correspond to the closest I can get to Bias frames with this camera.

So I think I can use the BPP script with my FlatDarks as Bias frames, matched Darks (without Optimization), and Flats. If my understanding is correct, if the darks are not optimized then the FlatDark/Bias master will only be subtracted from the Flats and the Dark master (containing the Bias) will be subtracted from the Lights.

Correct?
...Keith

[M Covington]

With my modified script, regardless of whether you turn optimization on, the bias frames are used not only as bias frames but also as darks for the flats.  The regular darks are used as darks for the lights.

Optimization should work very well (better than with the original script) but is not necessary.

[MicroAstro]

Thanks Michael

...Keith