Author Topic: alternate path toward recovering color in a bayer image?  (Read 3249 times)

Offline johnpane

  • PixInsight Enthusiast
  • **
  • Posts: 93
I am curious if anyone has discussed or tried stacking Bayer-pattern images without deBayering? Naively, it seems like it may be possible to avoid the interpolation and gain some resolution.

Without deBayering, each pixel represents not only a brightness, but a specific color. Let's assume that we can retain that information through the registration process. Let's also assume there is sufficient dithering (natural or deliberate) that any particular pixel stack will be composed of pixels of all colors after registration.

Then, the integration step has to accumulate brightness and color from each pixel in the pixel stack. At this point, the bayer pattern would be consulted to obtain color information, and the integrated pixel would become a full-color pixel.

I'm guessing this may have already been discussed but did not find such discussion in a quick search of the forums.

Thanks,
John



Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Re: alternate path toward recovering color in a bayer image?
« Reply #1 on: 2016 October 09 07:39:13 »
John,

As I see things (yet will be happy to be corrected) you have two (basic) choices:-

1.) You DeBayer first, then you Register and Integrate

2.) You Register and Integrate first, then you DeBayer

In (1) the resultant 'noise' that is generated by the DeBayer process is acquired many, many time - as many times as you have raw images.

In (2) the DeBayer 'noise' only gets generated once - when you process the final image and so, hopefully, is far less than in (1).

However, the very act of DeBayering requires that the colour information is extracted from within a 'group' of pixels - and it is the actual location of a pixel within a raw image that determines how the colour information should be extracted. This is why, I believe, that the preferred method is (1).

If you use (2), then each 'raw' pixel in the aligned image will effectively have 'lost' its original position within the coulour matrix - and so cannot be Debayered. However, given that the 'Reference' image used for the alignment process *does* still have its DeBayer reference, then it could be assumed that any image subsequently aligned with that reference still has a 'reference' for DeBayering (one caveat being that the Registered image must retain exactly the same pixel dimensions as the Reference image). In other wordseach Registered image must be 'cropped' (or 'padded', where necessary) to remain the same size as the Reference.

Of course, this has not considered processes involving Dark or Flat frames. With Darks, we don't need to worry about anything, as these are applied as Raw data to your RawLights - DeBayering has no involvement here.

However, this is not the case for Flats. To me, it seems (this time) that there are three choices:-

3.) You perform the FlatFrame division to each and every Raw image (before step (1) above)

4.) You perform the FlatFrame division to the final image (after step (2) above)

5.) You perform the FlatFrame division to each and every DeBayered image (during step (1) above)

In step (3), there will be cumulative 'noise' effects from each of the (many) division processes that will be implemented. However, there will be no noise contribution due to the DeBayering of the FlatFrame (which remains as a Raw image).

In step (4) the 'noise' from the division process is not cumulative, as the operation is only performed once.

In step (5), not only is there cumulative noise from the many division operations, but there is also the cumulative noise from the many DeBayer operations as well.

Of course - you have to also consider the Flats themselves. There are choices here as well (after all, a Flat is really just a Light!). Choices (1) and (2) apply - with the exception of the Registration process (as this is not required for Flats). In other words:-

6.) DeBayer each Flat first, then Integrate the resultant images

7.) Integrate all of the Flats, then DeBayer the final image

Again, in (6) there is cumulative 'noise' from the multiple DeBayer processes - whereas, in (7), there is only noise from one single DeBayer step.

Further, depending on the 'colour' of the light used to illuminate the CCD sensor during FlatFrame acquisition, you may well end up with a significant 'colour cast' after the division process - which will need to be eliminated later, during post-processing. So, do you try and 'fix' the colour cast before the division process, or do you treat it after the process?

I worked on a method that extracted each of the four DeBayer arrays from the MasterFlat (Integrated, but still in Raw format). Each of these was then 'normalised' (to result in the maximum value of any pixel being 1.0), before then recombining the four processed sub-images back into a NormalisedMasterFlat. This definitely resulted in a colour cast that was less noticeable, and easier to eliminate during post-processing (I don't use 'natural', or 6500K colour-temperature illumination for my flats, so I had noticeable colour cast that needed to be dealt with).

Note that a simple 'normalisation' of the image - without first splitting into the four CFA sub-images - does *not* help. In fact it makes things far worse, stretching the range of colour cast even further.

However, all of these thoughts are mine, and have not been discussed with others until now. I may be wrong (I often am  :laugh:) - and I am happy to have others rip my arguments to pieces (providing they can justify why their solution has more merit  :P).

In any case - I hope it helps you to start formulating your own thoughts.

Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline johnpane

  • PixInsight Enthusiast
  • **
  • Posts: 93
Re: alternate path toward recovering color in a bayer image?
« Reply #2 on: 2016 October 09 10:08:38 »
Thank you Niall for helping think through this. I have a couple of comments.

First, where true noise has to be introduced, it seems preferable for it to occur prior to integration because then it is suppressed by averaging in the integration process. The implications of this method on image calibration is something I have not fully thought through.

Second, I would like to be clear on what we call the regaining of color information in this alternative method. In the normal deBayer process, each pixel has actual data for one color and data for the other colors is imputed from adjacent pixels. In the proposed method, only real signal would be used. This is the reason why I was wondering if it might be advantageous. But maybe it should not be called deBayering to avoid confusion?

For the sake of a simplified illustration, lets imagine a scenario with four subframes, such that after registration each image has been shifted so that the Bayer pattern for the first two pixels of each subframe are:
subframe 1: RG
subframe 2: GR
subframe 3: BG
subframe 4: GB

We then determine the first pixel in the integrated image by integrating the (real, not imputed) R contribution from subframe 1, G from subframes 2 and 4, and B from subframe 3; for the second pixel, R from subframe 2, G from subframes 1 and 3, and B from subframe 4; proceeding accordingly for the whole image.

Of course, in real life, the subframes will not line up exactly. However, during registration of a particular subframe the necessary geometric transformation to rotate and shift the subframe into registration also determines the percentage contribution for four input pixels into each output pixel. Presumably, ImageRegistration already does this. So, after registration, if subframe 1 is the reference frame and we now examine only at the first pixel in each image, we have:
subframe 1: 100%R, 0%G, 0%B
subframe 2: Xa%R, Ya%G, Za%B
subframe 3: Xb%R, Yb%G, Zb%B
subframe 4: Xc%R, Yc%G, Zc%B

These are color images and ImageIntegration can proceed as normal.

What this means, I think, is that this idea could be easily implemented by converting each RAW subframe into a color image without interpolation. That is, assign each pixel's data 100% to the actual Bayer sensor color, without any interpolation from adjacent pixels. (Well, have to make sure green doesn't get overweighted since there are twice as many green pixels as the others.) Then, registration and integration can proceed as normal. (Again, I have not fully thought through implications for calibration).

John
« Last Edit: 2016 October 09 11:21:39 by johnpane »

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: alternate path toward recovering color in a bayer image?
« Reply #3 on: 2016 October 09 11:26:15 »
i don't have a good handle on the inner workings of this stuff, but this sounds like perhaps what Bayer Drizzle already does?

rob

Offline johnpane

  • PixInsight Enthusiast
  • **
  • Posts: 93
Re: alternate path toward recovering color in a bayer image?
« Reply #4 on: 2016 October 09 14:01:55 »
i don't have a good handle on the inner workings of this stuff, but this sounds like perhaps what Bayer Drizzle already does?

Indeed! It does look like this is what I was thinking of. Thank you!

Offline IanL

  • PixInsight Addict
  • ***
  • Posts: 116
    • The Imaging Toolbox
Re: alternate path toward recovering color in a bayer image?
« Reply #5 on: 2016 October 10 00:31:45 »
i don't have a good handle on the inner workings of this stuff, but this sounds like perhaps what Bayer Drizzle already does?

Yes exactly that.  Open up "Script > Batch Processing > BayerDrizzlePrep" and read the documentation for the full process.  Use the DrizzleIntegration tool with a Scale of 2 and a Drop Shrink of 1 and you're there.