PixInsight Forum

PixInsight => General => Topic started by: bill on 2019 April 11 16:05:09

Title: Flat Darks (F/D) vs. Using Flats, Darks and Bias Frames(F/D/B)
Post by: bill on 2019 April 11 16:05:09
Is there an advantage in using F/D vs. F/D/B frames?
The discussion I was in with a colleague and it centered around creating F/D vs F/D/B frames and the 'double subtraction' of Dark frames. I went through the standard workflow for P/I, which I use, and can find no instance where Darks are subtracted 2x using P/I. My colleague claims that by using F/D's   Since your flats are shot at different exposure times than your lights, you’ll need to shoot separate darks for your flats, and subtract them out for your master flats, just as you do your regular darks from your lights. If you don’t, you’re double processing the dark info.you don't double subtract vs using the standard processing workflow which he claims subtracts the darks 2x.
Does one process work better for CMOS vs CCD produced frames?
Thanks,
Bill
Title: Re: Flat Darks (F/D) vs. Using Flats, Darks and Bias Frames(F/D/B)
Post by: bulrichl on 2019 April 12 02:27:51
Hi Bill,

I don't know what is "... the standard workflow for P/I, which I use ...". Are you preparing your MasterFlat according to https://www.pixinsight.com/tutorials/master-frames/ (https://www.pixinsight.com/tutorials/master-frames/)?

Whether the usage of a MasterFlat-Dark (and not using dark frame optimization in the calibration of the flat frames) is advantageous will depend on the camera and on the exposure time of the flat frames.

Bernd
Title: Re: Flat Darks (F/D) vs. Using Flats, Darks and Bias Frames(F/D/B)
Post by: bill on 2019 April 12 05:37:51
Bernd,
My standard workflow was gleaned (by me) from reading 'Inside PixInsight'.  I have followed the tutorials. I prepare my master flats accordingly. I have never had issues with preparation of my masters, nor calibration of my flats.  I always take more bias frames than darks. I also just saw this line in the tutorial; 'Don't waste time acquiring dark-flat frames: IC will rescale the master dark to match the dark noise in your flat frames.' Thank you.
Do you know where would I find info about the dependency issue regarding camera/ and flat frame exposure times?
I am fairly new at working with P/I and the more I am able to use it, the more I like and respect just how powerful it is. I migrated over from using Nebulosity/Photoshop. That being said, any guidance is always appreciated as it assists in my ability to understand the nuances of the procedure as a whole.
Thank you for your help!
Bill
Title: Re: Flat Darks (F/D) vs. Using Flats, Darks and Bias Frames(F/D/B)
Post by: Niall Saunders on 2019 April 12 08:25:45
Quote
I also just saw this line in the tutorial; 'Don't waste time acquiring dark-flat frames: IC will rescale the master dark to match the dark noise in your flat frames.'

Hi Bill,

 The data in a Dark contains no Flat signal of any kind - aftter all, you exposed with the (equivalent of) the lens cap fitted. What you do have in the exposed Dark is an additive Bias signal.

This Bias signal is presentin every image that you expose and, to all intents, is statisticaly identical - irrespective of the type of exposure, and indeed irrespective of the temperature of the imager.

Depending on your imager, it is therefore possible to statistically evaluate a Master Dark by arithmetically averaging the exposed Darks. Because Bias frames are statistically identicaal for all exposure types your Master Dark will contain a predictable signal due to the Bias data.

The same argument leads to the statement that all Light frams also contain this same Bias signal. If the Master Dark is than arithmetically subtracted from each Light frame every interim Lightframe would now only have the Light signal present.

So far, so good - you have dealt with the thermal imperfections in your imager. However, you may still need to deal with imperfections in your optical train. Now, I know and accept that there are different approachs, but I am a firm believer in processes that can be broken down into individual stages - which can be more clearly underdtood (leaving users free to choose the most complex method that suitd them).

If you expose Flats then you also need to expose Flat Darks, and then statistically average these to give you a Master Flat. if your imager produces Dark data that does not have significant variation irrespective of exposure time and temperature, then one set of Master Flat Darks may well be all that you need for all Flats. (personally though, it takes me so little time to acquire Flat Dark data that I go through the process anyway).

If you then arithmetically subtract the Master Flat Dark from your Flats these can then be statistically aveaged giving calibrated Flats that have no Dark or Bias signal components left in them.

Don't waste time acquiring Flat Biases- these will have been 'processed out' by now anyway.

All that is niw needed is to apply the MasterFlat (which is free of both Dark and Bias data) to each Light ftame (which, having already been partially calibrated, is also free of Dark and Bias data).This is done be using a statistically 'multiplicative' process - namely, division.

This then creates a set of CalibratedLights which can be further processed by PixInsight.

Now, I know I could explan the prcess better (and, with better eyesught I might make fewer typos), but I hope that my efforts might help a little (or at least not confuse things further  ;) )
Title: Re: Flat Darks (F/D) vs. Using Flats, Darks and Bias Frames(F/D/B)
Post by: bulrichl on 2019 April 12 11:36:19
My standard workflow was gleaned (by me) from reading 'Inside PixInsight'.

Warren Keller wrote (1st edition, p. 15, 16):
"Just as the light frames should be bias and dark subtracted, so should the flats. Due to their short exposures of a bright light source, flats are relatively noise-free. A master bias can therefore be used as what's called a flat-dark. Some imagers take specific flat-darks that match the exposure time of the flats themselves. As we'll see, PixInsight recommencds a different method than either of these." He then describes the calibration of the flats with the normal MasterDark and MasterBias using dark frame optimization.

So he describes three different methods of calibrating the flat frames:
a) with MasterBias, not using dark frame optimization,
b) with MasterFlat-Dark, not using dark frame optimization (both 'Calibrate' and 'Optimize' unchecked),
c) with MasterBias and normal MasterDark, using dark frame optimization (both 'Calibrate' and 'Optimize' checked).
The MasterDark or the MasterFlat-Dark shall not be pre-calibrated in these examples (i.e. they still contain the the bias offset).

I also just saw this line in the tutorial; 'Don't waste time acquiring dark-flat frames: IC will rescale the master dark to match the dark noise in your flat frames.'

If your camera exhibits strong "amp-glow", it is not a good idea to take approach c) and scale the MasterDark (= use dark frame optimization). In this case it is also not advisable to use dark frame optimization for the calibration of the light frames. When you don't use dark frame optimization at all, you will not need bias frames and MasterBias. So the capturing of dark-flat frames is no wasting of time at all, you capture flat-darks instead of bias frames. Also note that some CMOS cameras give rise to issues with bias frames / MasterBias.

Do you know where would I find info about the dependency issue regarding camera/ and flat frame exposure times?

So what is the optimal approach? It depends, it is reasonable to differentiate 3 cases:
1) If your camera exhibits strong "amp glow" (e.g. some CMOS cameras like ASI183 or ASI294),
I suggest not to use dark frame optimization at all. This means: you will not need a Master Bias. Take flat frames, dark frames and flat-darks. Integrate the dark frames to the MasterDark and the flat-darks to the MasterFlat-Dark. Calibrate the flat frames with the MasterFlat-Dark and integrate the calibrated flat frames to MasterFlat. Calibrate the light frames with the MasterDark (both 'Calibrate' and 'Optimize' not checked) and the MasterFlat ('Calibrate' not checked).

2) If your camera doesn't exhibit "amp glow" and is not cooled (e.g. most modern DSLR cameras),
Take approach a) (calibrate the flats with the MasterBias).
You can try whether the application of dark frame optimization for light frame calibration yields in a better integration result. DSLR cameras that are not cooled may benefit the most with this approach, cooled CCD cameras probably will not.

3) If your camera doesn't exhibit "amp glow" and is cooled (e.g. most CCD cameras),
If you capture LRGB with a monochrome camera and broad band filters or color images with an OSC camera, the flat frames usually are taken with rather short exposure time. In this case the difference between a flat-dark and a bias frame may be negligible. This might be different when you capture narrow band, due to longer exposure time.
As the case may be, I suggest to try either approach a) (calibrate the flats with the MasterBias) or c) (bias frame, dark frames, scaling the MasterDark when calibrating the flat frames).
For the light frame calibration, dark frame optimization is not expected to yield better results.

Bernd

Title: Re: Flat Darks (F/D) vs. Using Flats, Darks and Bias Frames(F/D/B)
Post by: bill on 2019 April 12 15:23:20
Thank you Niall and Bernd for you thoughtful replies! I do use a CCD (SBIG) 8300M and have had no problems with my L/R/G/B processing so far. Will experiment with your suggestions. If you don't fail once in a while, you'll never learn.
Regards,
Bill
Title: Re: Flat Darks (F/D) vs. Using Flats, Darks and Bias Frames(F/D/B)
Post by: Niall Saunders on 2019 April 13 06:42:14
Hi Bill,

Above all, have fun
Title: Re: Flat Darks (F/D) vs. Using Flats, Darks and Bias Frames(F/D/B)
Post by: bemo47 on 2019 May 07 10:47:49
Hi,

i come back on this extract in the above posts :

"1) If your camera exhibits strong "amp glow" (e.g. some CMOS cameras like ASI183 or ASI294),
I suggest not to use dark frame optimization at all. This means: you will not need a Master Bias. Take flat frames, dark frames and flat-darks. Integrate the dark frames to the MasterDark and the flat-darks to the MasterFlat-Dark. Calibrate the flat frames with the MasterFlat-Dark and integrate the calibrated flat frames to MasterFlat. Calibrate the light frames with the MasterDark (both 'Calibrate' and 'Optimize' not checked) and the MasterFlat ."

I am using an ASI1600MC, CMOS and Color, and i us Pix for treatments. So i think i am in this case where i have a strong ampglow.

I just want to check if my understanding of Pix processes is correct.

No need of Master Bias : ok, no need to take bias images
Flat-dark : i read above that they are dark frames with same exposure than flats, so i must take these in place of normal bias frames, how much ? high number like for darks ? 50 ?
Dark frames : as usual, same temp and exposure time than lights
Flat frames : as usual, same temp and an ADU between 35000 and 50000 ? but may be better not taking them on the sky as the exposure time will be more easy to match for flat-darks ?

Masterdark : process image integration
Masterflat-dark : same process image integration (replacing masterbias)
Masterflat : process image calibration with masterflat-dark replacing the masterbias ? , then image integration wich will produce the masterflat
Light frames : process image calibration with masterdark (both 'Calibrate' and 'Optimize' not checked) and masterflat ('Calibrate' not checked) and no masterfalt-dark ?

Sorry to bother u may be with basic questions, but i have read many posts about this, also the book "inside Pix", and i am still not totally clear....
Thanks for ur advices,
Bernard