Author Topic: best time to debayer?  (Read 780 times)

Offline jamesRC

  • PixInsight Addict
  • ***
  • Posts: 102
best time to debayer?
« on: 2019 July 10 08:39:57 »
Hello,
I just got a nice new one-shot color camera. I'd like to ask when is the best or the correct time, during processing, to debayer my images?
Thank you

James

Offline wvanreeven

  • PixInsight Enthusiast
  • **
  • Posts: 82
Re: best time to debayer?
« Reply #1 on: 2019 July 10 09:06:58 »
Bias, dark amd flat frames should never be debayered. Simply integrate the bias frames to a master bias, calibrate the darks with the master bias, integrate them to a master dark, calibrate the flats with the master bias and dark and integrate them to a master flat.

Then calibrate the lights with the three masters, perform cosmic correction, debayer, star align and integrate. Between star alignment you may opt for local normalization. After integration you may opt for drizzle integration and in order to do that you need to have Generate Drizzle Data enabled in both Star Alignment and Image Integration.


HTH, Wouter

Offline bulrichl

  • PixInsight Guru
  • ****
  • Posts: 524
Re: best time to debayer?
« Reply #2 on: 2019 July 10 10:36:09 »
Darks should NOT be calibrated with the MasterBias, this approach can cause severe clipping of the MasterDark. See https://pixinsight.com/forum/index.php?topic=11968 .

Bernd

Offline wvanreeven

  • PixInsight Enthusiast
  • **
  • Posts: 82
Re: best time to debayer?
« Reply #3 on: 2019 July 10 10:39:16 »
Yes I have seen that topic but frankly I never had any problems calibrating darks with the master bias. Anyway, thanks for the reminder!

Offline bulrichl

  • PixInsight Guru
  • ****
  • Posts: 524
Re: best time to debayer?
« Reply #4 on: 2019 July 10 10:53:45 »
This problem occurs with DSLR cameras and astro cameras with CMOS sensors (my experience with Canon EOS 600D, ASI294MC Pro and ASI 071MC Pro). The approach not to pre-calibrate the darks (or the MasterDark) should work independently from the used sensor. Maybe that cameras with a CCD sensor do not suffer from this issue.

Bernd

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Re: best time to debayer?
« Reply #5 on: 2019 July 10 11:01:33 »
Hi James,

You need to start by understanding your camera, especially taking the time to understand how you determine what values of Bias and Gain you should be using (hint: you are not playing a 'guessing game' here)

Have a look at Craig Stark's excellent series of articles at
https://www.cloudynights.com/articles/cat/column/fishing-for-photons/signal-to-noise-understanding-it-measuring-it-and-improving-it-part-1-r1895
and
https://www.cloudynights.com/articles/cat/column/fishing-for-photons/signal-to-noise-understanding-it-measuring-it-and-improving-it-part-2-understanding-one-pixel-r1902
and
https://www.cloudynights.com/articles/cat/column/fishing-for-photons/signal-to-noise-part-3-measuring-your-camera-r1929

You may want to also look at some of Craig's other excellent articles at
http://www.stark-labs.com/craig/articles/articles.html

Hope this helps - I'll try to follow-up after a rest
« Last Edit: 2019 July 10 12:06:17 by Niall Saunders »
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline wvanreeven

  • PixInsight Enthusiast
  • **
  • Posts: 82
Re: best time to debayer?
« Reply #6 on: 2019 July 10 11:09:21 »
This problem occurs with DSLR cameras and astro cameras with CMOS sensors (my experience with Canon EOS 600D, ASI294MC Pro and ASI 071MC Pro). The approach not to pre-calibrate the darks (or the MasterDark) should work independently from the used sensor. Maybe that cameras with a CCD sensor do not suffer from this issue.

Bernd

Perhaps indeed that CCD cameras don’t suffer from this but I have only imaged with Canon cameras and never had the issue. Or perhaps I did and I never realised it! Anyway, thanks once more for the reminder. I’ll review my processing workflow.


Wouter

Offline jamesRC

  • PixInsight Addict
  • ***
  • Posts: 102
Re: best time to debayer?
« Reply #7 on: 2019 July 10 11:47:17 »
Ahem fellas,

My new one-shot is a CMOS not a dslr. Perhaps there's no difference, sorry if that is so. I was asking, at what point in processing do I debayer the subs. I never even thought of debayering the darks and flats  :surprised: though I would have likely asked about it next. I was assuming one does the preprocessing with darks and flats and bias, as usual with the bayered subs and  debayer them at some special point in the process maybe when preprocessing is done. Now I don't know. The answers here have confused me, I am kinda a newbie at this. I never even realized that darks etc are also bayered. I have to read these replies some more and hope for more enlightenment. A step-by-step maybe  ;D

Thank you very much for the replies.

James

Offline jamesRC

  • PixInsight Addict
  • ***
  • Posts: 102
Re: best time to debayer?
« Reply #8 on: 2019 July 10 11:49:12 »
btw, wvanreeven pretty much gives me a step by step so I have that to go with. Thanks, man!

James

Offline jamesRC

  • PixInsight Addict
  • ***
  • Posts: 102
Re: best time to debayer?
« Reply #9 on: 2019 July 10 11:58:24 »
. . .
and also, I don't plan to use bias because there as a lot of discussion that bias is not necessary, being included in  the dark frames. Any comments?

James

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: best time to debayer?
« Reply #10 on: 2019 July 10 14:07:34 »
the reason to use bias frames is if you want to scale (or "optimize") your master dark.

the idea is this: the dark current in any sensor is a function of temperature and time. since we usually use cooled cameras, we can hold the temperature constant between the darks and the lights. it turns out the dark current is a linear function of time, so it is possible to linearly scale the dark before subtraction from the light, where the dark and light durations do not match. so for instance you might sometimes do 20 minute lights for narrowband and sometimes 10min lights for LRGB. you can make a dark out of 20 minute subs and then scale them to 10 minutes by multiplying them by 0.5 before subtraction, instead of maintaining two separate master darks.

but there is a problem - the bias signal is not a function of time. so if you want to scale your dark, first you have to subtract the bias signal, and then scale the dark. if you don't do this, the calibration will be completely wrong. so when optimizing darks you either have to load a master bias and tell PI to both calibrate and optimize the dark, or you need to make a master dark which has the bias signal pre-subtracted. in that case you *always* need to load the master bias (whether or not you optimize the dark) since the bias signal is missing from the master dark. in this case you need to be sure to never check "calibrate dark" as that will amount to a double bias subtraction, which is also bad.

as an aside, PI doesn't actually care about the dark or light durations. what it does is iteratively scale the dark by calibrating a small portion of the light with different dark scaling factors and choosing the scaling factor that minimizes the noise in the calibrated result.

as a 2nd aside, this generally works for dedicated astro cameras. DSLRs play all sorts of dark current suppression tricks in the camera firmware and these tricks can not be turned off. so sometimes dark optimization does not work well for DSLRs.

rob

Offline wvanreeven

  • PixInsight Enthusiast
  • **
  • Posts: 82
Re: best time to debayer?
« Reply #11 on: 2019 July 11 01:41:18 »
On top of Rob's excellent explanation, some cameras suffer from amp glow and this doesn't seem to be linear as well. Properly taken darks can eliminate this but this again requires different darks for different exposure times. Again, DSLR cameras don't suffer from this but I wanted to add this comment in case owners of other cameras read this thread.


Wouter

Offline bulrichl

  • PixInsight Guru
  • ****
  • Posts: 524
Re: best time to debayer?
« Reply #12 on: 2019 July 11 02:58:34 »
My new one-shot is a CMOS not a dslr. Perhaps there's no difference, sorry if that is so. I was asking, at what point in processing do I debayer the subs. I never even thought of debayering the darks and flats  :surprised: though I would have likely asked about it next. I was assuming one does the preprocessing with darks and flats and bias, as usual with the bayered subs and  debayer them at some special point in the process maybe when preprocessing is done. Now I don't know. The answers here have confused me, I am kinda a newbie at this. I never even realized that darks etc are also bayered. I have to read these replies some more and hope for more enlightenment. A step-by-step maybe  ;D

I am sorry, I really didn't intend to confuse you. Wouter has outlined the order of steps in the workflow correctly. My only concern was that pre-calibrating the darks (or the MasterDark) is not working properly with many cameras.

Bernd

Offline wvanreeven

  • PixInsight Enthusiast
  • **
  • Posts: 82
Re: best time to debayer?
« Reply #13 on: 2019 July 11 04:45:55 »
I was assuming one does the preprocessing with darks and flats and bias, as usual with the bayered subs and  debayer them at some special point in the process maybe when preprocessing is done. Now I don't know. The answers here have confused me, I am kinda a newbie at this. I never even realized that darks etc are also bayered. I have to read these replies some more and hope for more enlightenment. A step-by-step maybe  ;D

James, sorry for the confusion. Any image taken with a color sensor will have the Bayer pattern encoded so that includes darks, bias and even day-time images. DSLR cameras normally debayer the day-time images for you so that's why nobody notices that the RAW files actually are composed of Red, Green and Blue pixels.

CMOS astronomical cameras do not have the functionality of auto-debayering and that's why this needs to be done at some point in the image processing chain. As a matter of fact, when using a DSLR the RAW images should be processed. And in both cases is recommended to do this as late as possible in the processing chain because debayering involves assigning a colour to each pixel based on the surrounding pixel values for the different colours Red, Green and Blue and that value depends on the debayering algorithm used though the differences are small.

So, please have a look at the workflow that I outlined in my first replay and take Bernd's remarks of his first reply into account and you should be fine.


Wouter