ImageIntegration and master dark

[enzosantin]

Hi all,
I want to use the powerfull ImageIntegration tool for creating master dark, master bias and master flat. Any suggestion? Wich parameter I must select? I suppose that the rejection alghorythm must be "CCD noise model", but the other parameters? Thank you very much!

Enzo Santin

[Juan Conejero]

Hi Enzo,

I can answer your question for flats. For darks and biases, I prefer to let Vicent Peris answer your question, since he knows much more than me about the practical implementation of these procedures. I'll ask Vicent to reply to this post.

To integrate a set of flat frames, you must select the following options in ImageIntegration:

Combination: Average

Normalization: Multiplicative

Weights: Don't care

Evaluate noise: disabled (in general, you aren't interested in evaluating the noise of master flats, I think)

Rejection algorithm: For sky flats, you should use a rejection algorithm good for rejection of stars. We normally use Percentile Clipping, since the number of flat frames is usually not very large (say less than 10).

Rejection Normalization (if you use rejection): Equalize fluxes.

Hope this helps.

[enzosantin]

Hi Juan,
thanks for your answer! I have forgotten to specify that usually I take 25 flats in artificial light (flat box, not sky flats) and that I work with 50 darks and 50 bias. Thanks again for your suggestions. Best regards

Enzo

[Jack Harvey]

I recently had the opportunity to discuss this with Vicent and here was his take on the issue.  Pretty much what Juan says with one difference

                                   Flats                                       BIAS/DARK
Combination               Average                                     Average
Normalization            Mulltiplicative                             No Normalization
Weights                      Noise evaluation                          Noise Evaluaiton

Rejection Clip               Percentile                                Percentile
Normailzation              Equalize Fluxes                      No Normalization

On both all three boxes in Pixel rejection are checked and you have to work with the Percentile High in FLats to remove stars

SO this is what I learned and hope I did not screw up the translation<G>.  If you have questions you will have to direct them to Juan or Vicent as this is above my pay grade.

[Niall Saunders]

Hi all,

That seems to be useful information Jack, although I will need to get back in front of PI to fully get my head around it !

In the meantime, I am looking for clarification on the use of Bias frames, For ocassions where an imager intends only acquiring Lights and Darks (let's ignore Flats and FlatDarks at this time)

In my mind - providing Lights and Darks share common Exposure times and CCD Temperatures - then there is no need for BiasOffsets at all. The contribution of BiasOffset 'noise' to any image is, statistically, identical once ExpTime and CCDTemp have been 'fixed'. In other words, given these particular conditions, there is no benefit available by also acquiring BiasOffsets as well. Is this an incorrect assumption on my part?

If I am correct however, then - providing enough Light and Darks have been acquired to satisfy statistical analysis requirements - all that is needed to calibrate the dataset is the 'subtraction' of a MasterDark from each Light.

In which case, all that is then required is the definition of an appropriate integration method for the creation of the MasterDark. To date, I have always used 'Median Combine' for this, and have (whenever possible) tried to acquire a minimum of 11 Darks (or, at least, the same number of Darks as Lights) in order to ensure that my Darks were 'statistically viable'.

Can anyone explain why I might also need to include BiasOffsets as well? And, if they should be acquired/included, then how should they be captured, and how should they be included in the calibration procedure.

Finally, whilst on the subject of BiasOffsets, what are folk's feelings about using a 'simulated' MasterBiasOffset frame (ie an image created to have ALL pixel ADU values set to a fixed value, the value being chosen to represent, for example, the MEDIAN of a 'real' MasterBiasOffset frame)? There certainly appears to be some argument in favour of this (I seem to remember reading about this in either HAIP, or perhaps one of Ron Wodaski's books, maybe even online somwhere).
 
Cheers,

[Simon Hicks]

Hi Niall,

The Deepskystacker site has some quite descriptive info on this. See the link below.

http://deepskystacker.free.fr/english/theory.htm#CalibrationProcess

BiasOffset noise can usually be quite low, because its very easy to fire off 100 or so 1/4000th of a second frames in the comfort of your living room....much easier than those darned Dark frames. 

Simulated BiasOffsets sound like a bad idea to me. You want to capture the detailed BiasOffset signal. I don't think its a flat single level that all pixels have, its more of a streaked pattern....well it seems to be with the Canon cameras anyway. So you want to capture a nice sharp (no smoothing functions) noise free image of this signal or pattern or image or whatever the best term for it is. And the way to do that is to take many of them.

In my experience, the BiasOffset is quite a small factor to deal with, i.e. the signal level is quite a bit smaller that the Dark level (I have an uncooled DSLR) and much smaller variation across the FOV than the effect of flats or sky gradients. So one side of the arguement is...well why bother with them. But the other side is that a full calibration should have them, and they are very easy to acquire in great numbers. So the discussion goes on.

I'm not an expert so others might want to chip in here with more info.

Cheers
         Simon

[enzosantin]

Jack, I would remember to you that I take 25 flats in artificial light (flat box, not sky flats) and that I work with 50 darks and 50 bias. Maybe some of the parameters you suggest must to be different because I have not stars to remove from my flats and because I take a lot of flats (25) and not only 10.

Niall, about bias frames: I take bias frames only for subtracting them from my flats. Cheers,

Enzo
 

[Jack Harvey]

ahh yes a light box and no stars.  SO no problems trying to remove them.  What is the issue with the 50 darks.  Image integration should handle that many?  I guess I am wondering why so many calibration frames.  For flats most big Observatories use 3 (Gemini South, etc) and the majority of amateurs use 3-5.   With darks using amateur setups with cooling of -15 to -30 below ambient usually 20 or so is plenty and really often 10-12 do a great job?  I suspect there is some issue I am missing

[Harry page]

Hi Jack

I use about 30 flats to get a low noise one !

Then I always miss something


Regards Harry

P.S.  Whats a dark   , got to love a Starlight xpress camera

[enzosantin]

Hi Jack,
you ask me why I take 25 flats and not only 3/4. I have applyed a very interesting theory by Richard Crisp
on flat fielding, here the link: http://www.narrowbandimaging.com/images/Flat%20Fielding.pdf
and another link: http://tech.groups.yahoo.com/group/narrowbandimaging/message/6335

Substantially Crisp says that the function of a flat is not
to only remove the optical system non-uniform light intensity
distribution, including dust motes and vignetting etc, but primarily
to remove the "fixed pattern noise" from the image. The fixed
pattern noise (pixel to pixel sensitivity variation ) is typical (and particularly evident)
for the Kodak KAF ccd sensors like the KAF 16803 in my camera.
But the noise of the flat will affect the noise of the calibrated image. So it
is essential that the flats be low noise or they will add noise to the
image and that will defeat the purpose of applying the flats. In its
equations, Crisp demonstrates that when the number of electrons contained
in the data set used for building the master flat approaches 1 million
electrons (that for my camera it is equivalent to take 20/25 flats),
the corrected image's noise approaches the "shot noise
limited" ideal response. I am not a mathematician and probably I have
not totally understood all the equations in the theory of Crisp, but I
have empirically experimented that, when I apply 25 flats with a total of 1 million
electrons collected, my images are really better.

As for the darks: the cooling power of my camera permits me to have only two darks library:
one at -25 and the other at -30C, therefore it is not so difficult to take some fresh dark (3/4)
during the day and substitute older darks with the newer in the library and then rebuild and refresh my master dark.
So, why not to use more darks if I have them?
Sorry for my English, I hope I have written in a understandable language! Regards

Enzo 

[Simon Hicks]

With darks using amateur setups with cooling of -15 to -30 below ambient usually 20 or so is plenty and really often 10-12 do a great job?  I suspect there is some issue I am missing

Hi Jack,

If my DSLR was at -30oC below ambient (which usually means at -20 to -40oC actual) then I would not bother with darks at all. The dark signal is less than the bias signal on my DSLR when its below about -15oC.

Unfortunately the dark signal goes up quite quickly at the usual 0 to 15oC range that I am typically imaging over.

So whether you will significantly benefit from using darks or bias frames will depend to a large extent over what imaging camera you have, whether it is cooled, and to some extent the ISO (or gain) and the exposure time. These vary wildly between different setups and conditions, which I think is why these questions come up all the time and people have so many opinions as to what is best. Its actually quite a complex balance of quite a few different factors.

The best thing is to measure the bias signal and measure the dark signal under your conditions and gauge their relative size and noise levels compared to the signal in the image you are capturing. Of course this is easier said than done! 

Cheers
         Simon

[Jack Harvey]

ANother variable Harry pointed out is the difference in dark noise in a Sony vs Kodak chip<G>

[mmirot]

Flats mainly effect the the high S/N areas.
I am not sure you gain that much by making sure the flats become read noise limited.

Max

[mmirot]

Here is nice eval of a camera by richard.
http://tech.groups.yahoo.com/group/ccd-imaging-technology/message/8439

Look at the PTC chart see the link to his web page.
Note the convergence of the number of flats and the point they converge in DN( ADU) on the curve.
Most of the impact is is bright areas and there is diminishing returns to the number of flats.
You camera is different. I will bet you won't be able to measure much of difference between 6 flats and 25

Max

[enzosantin]

Here is nice eval of a camera by richard.
http://tech.groups.yahoo.com/group/ccd-imaging-technology/message/8439

Look at the PTC chart see the link to his web page.
Note the convergence of the number of flats and the point they converge in DN( ADU) on the curve.
Most of the impact is is bright areas and there is diminishing returns to the number of flats.
You camera is different. I will bet you won't be able to measure much of difference between 6 flats and 25

Max

Hi Max,
thank you... yes, I know this PTC chart of the FLI MicroLine4022. Here the PTC chart of my camera: http://www.narrowbandimaging.com/images/pl16803_ptc_dn_units.jpg 
Regards

Enzo