Author Topic: Flats, Darks, etc..  (Read 1912 times)

Offline tgervais

  • Newcomer
  • Posts: 30
    • View Profile
Flats, Darks, etc..
« on: 2016 December 20 08:13:57 »
Just trying to align and stack images here and one of the things I have tried to get right is the use of Flat, Dark, Bias, Light images.

Last night I took 10 light images at 3 mins each.  And than when I went to take the Flat images at 10 times each - I was told only ONE flat shot (image) was needed.  You don't need to go and take 10 images all over again like you did with the Light images.  Just ONE shot for the FLAT image is all that is needed.  Same with the Dark and Bias images.  Just one shot is all that is needed.  Anymore than that and you are wasting eveyone's time.

That is what I was told.  So I came home with no flat, dark ,  or Bias images.  I was sure I was previouly told that you take the same number of flat, dark, and bias images as you did with the light images.  Same ISO in all cases.  etc..

Is this true?  Have I been wasting my time etc..


Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4549
    • View Profile
Re: Flats, Darks, etc..
« Reply #1 on: 2016 December 20 08:29:27 »
not true, you need to stack flats as well. a single flat has too much photon shot noise and will inject that noise into your lights when you calibrate the light with the single flat.

i always do at least 20-30 flats at 30k ADU (half well depth for my camera)

more importantly you need many more darks and bias frames. i never use less than 50 darks if i can get them, and never less than 100 bias frames.

anyone who tells you you need just one of each calibration frame does not understand the fundamentals of astrophotography.

rob

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
    • View Profile
Re: Flats, Darks, etc..
« Reply #2 on: 2016 December 20 09:08:47 »
Quote
Just ONE shot for the FLAT image is all that is needed.  Same with the Dark and Bias images.  Just one shot is all that is needed

What you have to understand here is the difference between a 'standard' ex[posure and an image that has been created from several exposures. When people talk about just using one Flat (or Dark or Bias) what they mean is to use what is more correctly referred to as a MASTER frame (i.e. a MasterFlat, a MasterDark, or a MasterBias).

These 'Master' frames have to be created from a number of otherwise 'standard' individual frames - and the number that you choose to acquire has to be 'statistically significant', and has to be within your available frame captiring session.

Now, by 'statistically significant', what I mean is that you need to capture enough of these 'raw' frames to be able to eliminate some of the noise that you will always acquire - simply by actually capturing the image in the first place. Typically, a good 'rule of thumb' would be to capture as many of these raw 'calibration' frames as you hope to acquire by way of raw Light frames (i.e. those that contain the image data that you are hoping to acquire in the first place).

So - sounds like a whole load of effort - making life almost impossible for an average imager? Not really. First, you don't need to reacquire all of these calibration frames every night (for example 30-each for Biases, Flats and Darks - plus a further 30 for your Lights or, worse, 30-each for L+R+G+B+Ha !!!)

Biases are very short exposures - in fact the shortest exposures you can possibly set out acquire, and so don't really take up too much time. But, careful analysis of your Bias frames may leave you realising that you might not actually need these at all. Depending on your circumstances (camera used, local environment, etc.) you can get away with either a 'Pseudo-MasterBias' (where you use PI to create a MasterBias comprising pixel values all set to be the same value as the average value you usually encounter when acquiring raw Bias data), ir you can simpy ignore the bothersome issue of Bias frames altogether (they are only really ever used if you did not have, or did not acquire, Dark Frames at the same exposure length  as your Light Frames - but the penalty is that you really need the CCD temperature to be 'the same' for the acquisition of both the Darks and the Lights - and that usually means some form of TEC cooling system on your imager).

A quick word on TEC cooling might also help here - if you can run your imager night-after-night at a fixed, repeatable, temperature, the you definitely do not need Bias Frames. All you need is a good collection of MasterDark frames, one each for the various exposure times that you feel you are most likely to encounter. Best yet - the raw Darks needed to create these MasterDarks can be collected 'anywhere' - you don't need to be connected to your telescope, you only need to be connected to your TEC-cooled CCD imager. And, I lied, even better - you can gather copius numbers of DarkFrames whilst you are asleep, at work, out, etc., etc. - just make sure that no stray light can get anywhere near the imager. (And, you  may be ahead of me already here, if you 'really' wanted to collect Bias frames, these can also be collected when you are not actually imaging - I do this, occasionally, just to double-check that the Bias levels on my imager have not changed over time - and, since buying the QHY10 many, many, years ago, they haven't changed enough for me to be able to detect the change using any of the tools available in the PI arsenal).

In summary then, depending on your imager - you don't need Biases, and you can create MasterDarks when not actually in the middle of an imaging session.

That just leaves your Flats and your Lights. I work from an observatory - nothing changes in my optical train for months at a time. Sometimes I don't even bother with focus because environmental conditions remain so stable. So, I take Flats 'every so often', using a large piece (A3-size) of white foam-board pinned to the observatory wall about 2m (6 feet) from the aperture of the OTA, and I 'point' the OTA (very roughly) 'at' the board. This is illuminated by a 60W (eq.) CFL lamp about 4m (12 feet) away (and slightly to one side). Iis the illuminated field 'even'? Again, I haven't found any means within PI to tell that it isn't. Is the illumination 'perfectly white' - no, it isn't, it's anything but white - but I have a very simple routine in PI that sorts all that out before the MasterFlat actually gets used to calibrate the raw Lights.

Great! You can now acquire your Flats before your imaging session even starts (using the overhead sky, perhaps through a white, cotton tee-shirt; perhaps, like me, through another sheet of white foam-board, with the OTA pointed 'straight up').

And that then leaves your entire night-time session free for imaging. Or, like me, watching for clouds, tripping over things in the dark, falling asleep at the computer (or, worse, on the floor - waking up to find spider's webs strung from my nose to my ears), drinking hot chcolate by the pint or just waiting for the gentle 'bong-chime' of PHD to tell me that the sun is now well above the horizon!

Hope this helps.
« Last Edit: 2018 November 06 10:14:24 by Niall Saunders »
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chr├ętien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline mschuster

  • PTeam Member
  • PixInsight Jedi
  • *****
  • Posts: 1064
    • View Profile
Re: Flats, Darks, etc..
« Reply #3 on: 2016 December 20 09:14:55 »
One thing that might not be recognized is the advantage of dithering. No matter how many bias, dark, and flats you use for your masters, dithering your lights will dramatically reduce the bad effects of residual noise in the masters. With dithering master noise across light frames will add in quadrature rather than in superposition.

Thanks,
Mike

Here is an example, one of my blog images. 32 light frames, narrowband, not sky limited, noisy camera, I really wanted to limit master noise as much as possible. My goal was 5% increase in detector noise due to the dark-subtraction.

Had I not dithered, a 60 dark master would increase noise by ~25% (a scale of 1.25 in the graph below). Had I dithered optimally 10 darks would have sufficed. But actually, my dithering was not optimal (dither spacing too small, I discovered this by measurement), and 22 darks were needed.

« Last Edit: 2016 December 20 09:57:58 by mschuster »

Offline jkmorse

  • PixInsight Padawan
  • ****
  • Posts: 916
  • Two questions, Mitch . .
    • View Profile
    • Jim Morse Astronomy
Re: Flats, Darks, etc..
« Reply #4 on: 2016 December 20 14:26:39 »
So long as you collect sufficient ADU, you don't need to go crazy with Flats.  I target 150k total ADU which I get from 5 sky flats shot at around 30k each (if my set up had the option of a light panel like the one Rob describes, I would shoot more, but you can only cram in so many when using the sky).  Like Rob says, 30k ADU per flat is a great target since its a little less than half of the full well depth of your camera/CCD and should be well within the linear range of your camera/CCD.  But, since you don't need to shoot them nearly as often, don't scimp on the darks and bias frames. 

One other point worth mentioning, Niall is right that you don't need to shoot flats every day BUT ONLY if you have no dust donuts.  But if you have dust donuts anywhere in the image chain and do any changes to the chain, such as change the rotation of your camera or switch between filters, then the dounts do not stay put and you might end up creating more problems than you solve if you don't shoot new flats for each change you make.  In my case, where my system sits in a remote observatory that inevitably gets a bit of dust on the primary mirror, I shoot different flats for each position angle and different filter, though things stay the same for the most part that I only need one set for each image run, even it is takes several weeks to capture all the light subs. 

Flats are the most confusing part of the imaging process but getting the process right deserves your attention and study since they really help when done properly.

Jim
Really, are clear skies, low wind and no moon that much to ask for? 

New Mexico Skies Observatory
FLI Microline 16803
Planewave CDK17 - Paramount MEII
Planewave IFR90 - Astrodon LRGB & NB filters
SkyX - MaximDL - ACP

http://www.jimmorse-astronomy.com
http://www.astrobin.com/users/JimMors

Offline RickS

  • PTeam Member
  • PixInsight Jedi
  • *****
  • Posts: 1296
    • View Profile
Re: Flats, Darks, etc..
« Reply #5 on: 2016 December 21 01:04:48 »
One thing that might not be recognized is the advantage of dithering. No matter how many bias, dark, and flats you use for your masters, dithering your lights will dramatically reduce the bad effects of residual noise in the masters. With dithering master noise across light frames will add in quadrature rather than in superposition.

It's interesting to see graphically how much benefit you get from dithering.  Thanks, Mike.  Did you do the graph using R?

So long as you collect sufficient ADU, you don't need to go crazy with Flats.  I target 150k total ADU which I get from 5 sky flats shot at around 30k each

My target is half a million e-.  Sounds like a lot but it is only about a dozen flats with my KAF-16803 camera.  I got the number from a Flat Field Photon Transfer Curve.  500Ke- is about where diminishing returns kick in for my camera.

Cheers,
Rick.

Offline mschuster

  • PTeam Member
  • PixInsight Jedi
  • *****
  • Posts: 1064
    • View Profile
Re: Flats, Darks, etc..
« Reply #6 on: 2016 December 21 07:54:17 »
It's interesting to see graphically how much benefit you get from dithering.  Thanks, Mike.  Did you do the graph using R?

Mathematica. Yes, of course dithering helps address hot pixels, but it also helps reduce bias or dark-subtration noise. With dithering noise from different pixels in the calibration master are being combined, with a quadrature sum advantage.

Thanks,
Mike