New Version of BatchPreprocessing Script with Bayer Drizzle Support

[whwang]

Juan, Collin,

Thank you.  I like the new way it works!

Cheers,
Wei-Hao

[jtrezzo]

I can't seem to get this to work. I keep getting "Missing CFA source file path when I try to run DrizzleIntegration" with CFA images box checked.
 
I calibrated lights, ran StarAlignment with drizzle data box checked, did Image Integration, then tried the DrizzleIntegration. I also tried doing Debayer before StarAlignment. No luck.

[Warhen]

Hi JT, If taking a manual track (no BPP), you'd definitely have to debayer before alignment, this is an always whether working with drizzle or not. When you mentioned ImgInt afterwards, you didn't confirm that you also had Generate Drizzle Data there as well, since ImgInt needs to update the xdrz files produced by SA. That may not be your entire answer, but they are significant points. Good luck!

[Alfredo Beltran]

Warren

Could you please elaborate why does the debayering needs to be done prior to the drizzle integration? I’m confused, since Bayer Drizzle is supposed to not require an interpolation process.

Thanks,

Alfredo

[dld]

Hello PI fans,

I have exactly the same question with Alfredo. Maybe due to my noobness I may have missed a post/tutorial answering my question.

Thanks!

[Alfredo Beltran]

I can't seem to get this to work. I keep getting "Missing CFA source file path when I try to run DrizzleIntegration" with CFA images box checked.
 
I calibrated lights, ran StarAlignment with drizzle data box checked, did Image Integration, then tried the DrizzleIntegration. I also tried doing Debayer before StarAlignment. No luck.

I’m having exactly the same issue when using the BPP. To the Gurus: Any ideas why is this happening?

Thanks in advance.

Alfredo

[sharkmelley]

Could you please elaborate why does the debayering needs to be done prior to the drizzle integration? I’m confused, since Bayer Drizzle is supposed to not require an interpolation process.

You still need to perform the StarAlignment to generate registered images and StarAlignment requires debayered files.

Mark

[sharkmelley]

To perform Bayer Drizzle:

If you are using BPP then make sure you select "Generate drizzle data" in the Image Registration section and "CFA images" in the Global Options section.

If you are not using BPP then:

• Do all the steps up to and including debayering as usual.
• Run StarAlignment on the debayered files with "Generate drizzle data" selected

In either case you now have a set of registered files and drizzle files:

• In ImageIntegration add the registered files and add the drizzle files (.xdrz) and make sure "Generate drizzle data" is selected before you run.  This is important because ImageIntegration must update the drizzle files with additional information.
  
• Run DrizzleIntegration on the drizzle files (.xdrz) making sure that "Enable CFA drizzle" is selected.

The error "Missing CFA source file path" in DrizzleIntegration is because you forgot to select the "CFA images" in the Global Options section of BPP so the debayering step in BPP did not happen - at least that is how I managed to reproduce the error.  If you're not using BPP then it's because you performed StarAlignment on non-debayered files.

The error "Missing required LocationEstimates element" in DrizzleIntegration is because you forgot to update the drizzle files using ImageIntegration.

An error similar to "calibrated/light/DSC00429_c.xisf: No such file" in DrizzleIntegration is because you have moved or deleted the calibrated lights.

Mark

[Warhen]

Mark, Thank you for ArcsinhStretch! It will be detailed in my book's second edition. Afredo/DLD, Mark explains the error well. But, in general, debayering must occur (not only before Drizzle, but) before alignment. The matrix's info has to be decoded before the transformation that occurs during registration- got it?! As Mark explains Drizzle, it's a bit mysterious to those of us 'not under the hood.'  Debayer collects the matrix offset info to pass on to StarAlignment, which generates the Drizzle files. ImgInt then updates these xdrz files for DrizzleIntegration. This took me a bit to get too! Yes, the Debayer process is used in the proper sequence, but as you say, it's DI that produces the final, uninterpolated color image. The good news is, OSC/CFA Drizzle is now easier than ever with either BPP or the manual track.

[Alfredo Beltran]

Thanks for your detailed process Mark, and Warren for your explanation.

I figured out that my problem was that I had set the CFA Pattern to Auto in the BPP. When I switched it to RGGB, the issue was solved and the script worked with no issues at all.

What I did see is that if I set Drizzle Scale to 1 and and drop shrink also as 1.0 (as suggested by Juan in post #1) then I don't see  difference between the regular debayered image and the one with Bayer Drizzle. But when I set Scale to 2, then I can definitely see an increase in resolution, as you can see in the comparison made in the attached image. So, the question is: should I see an increase in resolution if I set scale to 1.0 in Bayer Drizzle?

Also, I can see some patterns of the CFA matrix on the 2x Bayer drizzle image. Should this be present?

Regards,

Alfredo

[sharkmelley]

What I did see is that if I set Drizzle Scale to 1 and and drop shrink also as 1.0 (as suggested by Juan in post #1) then I don't see  difference between the regular debayered image and the one with Bayer Drizzle. But when I set Scale to 2, then I can definitely see an increase in resolution, as you can see in the comparison made in the attached image. So, the question is: should I see an increase in resolution if I set scale to 1.0 in Bayer Drizzle?

Also, I can see some patterns of the CFA matrix on the 2x Bayer drizzle image. Should this by present?

I would usually expect to see a slight increase in detail in a x1 Bayer Drizzle.  The pattern in the x2 Bayer Drizzle is possibly because you don't have sufficiently even coverage - take a look at the weights image produced by DrizzleIntegration and zoom in at full scale to see if this is the case.

Mark

[jtrezzo]

Thank you Mark and Warren for your responses. Unfortunately I am still having the issue for some reason. I am debayering right after calibrating the light frames (with RGGB set for my camera, VNG demosaicing), then I run subframe selector, then star alignment (with generate drizzle data checked). After this I run image integration with the drizzle data files added so they're updated, and then try drizzle integration. Then I get Error: Missing CFA source file path. I am completely stumped as to what I am doing wrong, as I have followed all the steps written previously. It seems like somewhere in the mix it is not recognizing that they are CFA files? I don't know.

[sharkmelley]

Thank you Mark and Warren for your responses. Unfortunately I am still having the issue for some reason. I am debayering right after calibrating the light frames (with RGGB set for my camera, VNG demosaicing), then I run subframe selector, then star alignment (with generate drizzle data checked). After this I run image integration with the drizzle data files added so they're updated, and then try drizzle integration. Then I get Error: Missing CFA source file path. I am completely stumped as to what I am doing wrong, as I have followed all the steps written previously. It seems like somewhere in the mix it is not recognizing that they are CFA files? I don't know.

I notice from another thread that you are starting with raw FITS files from QHY247C.  Is this still the case?  The only thing I can think of is maybe the information about Bayer pattern "RGGB" is not being found by DrizzleIntegration for some reason or there is some other reason that DI is upset by working with raw FITS files.  For instance, I wonder if some header keyword needs to set within those files?

Mark

[Alfredo Beltran]

Then I get Error: Missing CFA source file path.

Just select your CFA pattern instead of Auto. It should work.

[jtrezzo]

Thank you Mark and Warren for your responses. Unfortunately I am still having the issue for some reason. I am debayering right after calibrating the light frames (with RGGB set for my camera, VNG demosaicing), then I run subframe selector, then star alignment (with generate drizzle data checked). After this I run image integration with the drizzle data files added so they're updated, and then try drizzle integration. Then I get Error: Missing CFA source file path. I am completely stumped as to what I am doing wrong, as I have followed all the steps written previously. It seems like somewhere in the mix it is not recognizing that they are CFA files? I don't know.

I notice from another thread that you are starting with raw FITS files from QHY247C.  Is this still the case?  The only thing I can think of is maybe the information about Bayer pattern "RGGB" is not being found by DrizzleIntegration for some reason or there is some other reason that DI is upset by working with raw FITS files.  For instance, I wonder if some header keyword needs to set within those files?

Mark

Yes I am using raw FITS. A keyword header is was what I was thinking too, but I have no idea what to add or how to add it. I have been selecting the pattern manually instead of using Auto too, both in Debayer and when I try DrizzleIntegration. Both RGGB.