Author Topic: New Version of BatchPreprocessing Script with Bayer Drizzle Support  (Read 31597 times)

Offline whwang

  • Member
  • *
  • Posts: 61
    • View Profile
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #45 on: 2017 September 19 20:35:58 »
Juan, Collin,

Thank you.  I like the new way it works!

Cheers,
Wei-Hao

Offline jtrezzo

  • Newcomer
  • Posts: 8
    • View Profile
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #46 on: 2017 November 28 23:17:06 »
I can't seem to get this to work. I keep getting "Missing CFA source file path when I try to run DrizzleIntegration" with CFA images box checked.
 
I calibrated lights, ran StarAlignment with drizzle data box checked, did Image Integration, then tried the DrizzleIntegration. I also tried doing Debayer before StarAlignment. No luck.

Offline Warhen

  • PTeam Member
  • PixInsight Old Hand
  • ****
  • Posts: 481
    • View Profile
    • Billions and Billions
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #47 on: 2017 December 05 11:42:12 »
Hi JT, If taking a manual track (no BPP), you'd definitely have to debayer before alignment, this is an always whether working with drizzle or not. When you mentioned ImgInt afterwards, you didn't confirm that you also had Generate Drizzle Data there as well, since ImgInt needs to update the xdrz files produced by SA. That may not be your entire answer, but they are significant points. Good luck!
Best always, Warren

Warren A. Keller
www.ip4ap.com

Offline Alfredo Beltran

  • PixInsight Addict
  • ***
  • Posts: 114
    • View Profile
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #48 on: 2017 December 05 12:47:22 »
Warren

Could you please elaborate why does the debayering needs to be done prior to the drizzle integration? I’m confused, since Bayer Drizzle is supposed to not require an interpolation process.

Thanks,

Alfredo
iOptron CEM60
Celestron 925 EdgeHD Hyperstar
MicroTouch Focuser
ZWO ASI071 MC Pro
Bogota, Colombia
http://www.flickr.com/photos/alfredo_beltran/

Offline dld

  • PixInsight Addict
  • ***
  • Posts: 116
    • View Profile
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #49 on: 2017 December 06 04:13:00 »
Hello PI fans,

I have exactly the same question with Alfredo. Maybe due to my noobness :o I may have missed a post/tutorial answering my question.

Thanks!

Offline Alfredo Beltran

  • PixInsight Addict
  • ***
  • Posts: 114
    • View Profile
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #50 on: 2017 December 08 19:20:39 »
I can't seem to get this to work. I keep getting "Missing CFA source file path when I try to run DrizzleIntegration" with CFA images box checked.
 
I calibrated lights, ran StarAlignment with drizzle data box checked, did Image Integration, then tried the DrizzleIntegration. I also tried doing Debayer before StarAlignment. No luck.

I’m having exactly the same issue when using the BPP. To the Gurus: Any ideas why is this happening?

Thanks in advance.

Alfredo
iOptron CEM60
Celestron 925 EdgeHD Hyperstar
MicroTouch Focuser
ZWO ASI071 MC Pro
Bogota, Colombia
http://www.flickr.com/photos/alfredo_beltran/

Offline sharkmelley

  • PTeam Member
  • PixInsight Addict
  • ***
  • Posts: 234
    • View Profile
    • Mark Shelley Astrophotography
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #51 on: 2017 December 09 11:13:49 »
Quote from: Alfredo Beltran
Could you please elaborate why does the debayering needs to be done prior to the drizzle integration? I’m confused, since Bayer Drizzle is supposed to not require an interpolation process.

You still need to perform the StarAlignment to generate registered images and StarAlignment requires debayered files.

Mark
Takahashi Epsilon 180ED
H-alpha modified Sony A7S
http://www.markshelley.co.uk/Astronomy/

Offline sharkmelley

  • PTeam Member
  • PixInsight Addict
  • ***
  • Posts: 234
    • View Profile
    • Mark Shelley Astrophotography
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #52 on: 2017 December 09 11:23:18 »
To perform Bayer Drizzle:

If you are using BPP then make sure you select "Generate drizzle data" in the Image Registration section and "CFA images" in the Global Options section.

If you are not using BPP then:
  • Do all the steps up to and including debayering as usual.
  • Run StarAlignment on the debayered files with "Generate drizzle data" selected

In either case you now have a set of registered files and drizzle files:
  • In ImageIntegration add the registered files and add the drizzle files (.xdrz) and make sure "Generate drizzle data" is selected before you run.  This is important because ImageIntegration must update the drizzle files with additional information.
  • Run DrizzleIntegration on the drizzle files (.xdrz) making sure that "Enable CFA drizzle" is selected.

The error "Missing CFA source file path" in DrizzleIntegration is because you forgot to select the "CFA images" in the Global Options section of BPP so the debayering step in BPP did not happen - at least that is how I managed to reproduce the error.  If you're not using BPP then it's because you performed StarAlignment on non-debayered files.

The error "Missing required LocationEstimates element" in DrizzleIntegration is because you forgot to update the drizzle files using ImageIntegration.

An error similar to "calibrated/light/DSC00429_c.xisf: No such file" in DrizzleIntegration is because you have moved or deleted the calibrated lights.

Mark
« Last Edit: 2017 December 09 12:16:29 by sharkmelley »
Takahashi Epsilon 180ED
H-alpha modified Sony A7S
http://www.markshelley.co.uk/Astronomy/

Offline Warhen

  • PTeam Member
  • PixInsight Old Hand
  • ****
  • Posts: 481
    • View Profile
    • Billions and Billions
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #53 on: 2017 December 09 16:12:47 »
Mark, Thank you for ArcsinhStretch! It will be detailed in my book's second edition. Afredo/DLD, Mark explains the error well. But, in general, debayering must occur (not only before Drizzle, but) before alignment. The matrix's info has to be decoded before the transformation that occurs during registration- got it?! As Mark explains Drizzle, it's a bit mysterious to those of us 'not under the hood.'  Debayer collects the matrix offset info to pass on to StarAlignment, which generates the Drizzle files. ImgInt then updates these xdrz files for DrizzleIntegration. This took me a bit to get too! Yes, the Debayer process is used in the proper sequence, but as you say, it's DI that produces the final, uninterpolated color image. The good news is, OSC/CFA Drizzle is now easier than ever with either BPP or the manual track.
Best always, Warren

Warren A. Keller
www.ip4ap.com

Offline Alfredo Beltran

  • PixInsight Addict
  • ***
  • Posts: 114
    • View Profile
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #54 on: 2017 December 10 17:30:34 »
Thanks for your detailed process Mark, and Warren for your explanation.

I figured out that my problem was that I had set the CFA Pattern to Auto in the BPP. When I switched it to RGGB, the issue was solved and the script worked with no issues at all.

What I did see is that if I set Drizzle Scale to 1 and and drop shrink also as 1.0 (as suggested by Juan in post #1) then I don't see  difference between the regular debayered image and the one with Bayer Drizzle. But when I set Scale to 2, then I can definitely see an increase in resolution, as you can see in the comparison made in the attached image. So, the question is: should I see an increase in resolution if I set scale to 1.0 in Bayer Drizzle?

Also, I can see some patterns of the CFA matrix on the 2x Bayer drizzle image. Should this be present?

Regards,

Alfredo
« Last Edit: 2017 December 13 04:19:41 by Alfredo Beltran »
iOptron CEM60
Celestron 925 EdgeHD Hyperstar
MicroTouch Focuser
ZWO ASI071 MC Pro
Bogota, Colombia
http://www.flickr.com/photos/alfredo_beltran/

Offline sharkmelley

  • PTeam Member
  • PixInsight Addict
  • ***
  • Posts: 234
    • View Profile
    • Mark Shelley Astrophotography
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #55 on: 2017 December 12 01:52:03 »
Quote from: Alfredo Beltran
What I did see is that if I set Drizzle Scale to 1 and and drop shrink also as 1.0 (as suggested by Juan in post #1) then I don't see  difference between the regular debayered image and the one with Bayer Drizzle. But when I set Scale to 2, then I can definitely see an increase in resolution, as you can see in the comparison made in the attached image. So, the question is: should I see an increase in resolution if I set scale to 1.0 in Bayer Drizzle?

Also, I can see some patterns of the CFA matrix on the 2x Bayer drizzle image. Should this by present?

I would usually expect to see a slight increase in detail in a x1 Bayer Drizzle.  The pattern in the x2 Bayer Drizzle is possibly because you don't have sufficiently even coverage - take a look at the weights image produced by DrizzleIntegration and zoom in at full scale to see if this is the case.

Mark
Takahashi Epsilon 180ED
H-alpha modified Sony A7S
http://www.markshelley.co.uk/Astronomy/

Offline jtrezzo

  • Newcomer
  • Posts: 8
    • View Profile
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #56 on: 2017 December 12 19:47:06 »
Thank you Mark and Warren for your responses. Unfortunately I am still having the issue for some reason. I am debayering right after calibrating the light frames (with RGGB set for my camera, VNG demosaicing), then I run subframe selector, then star alignment (with generate drizzle data checked). After this I run image integration with the drizzle data files added so they're updated, and then try drizzle integration. Then I get Error: Missing CFA source file path. I am completely stumped as to what I am doing wrong, as I have followed all the steps written previously. It seems like somewhere in the mix it is not recognizing that they are CFA files? I don't know.

Offline sharkmelley

  • PTeam Member
  • PixInsight Addict
  • ***
  • Posts: 234
    • View Profile
    • Mark Shelley Astrophotography
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #57 on: 2017 December 12 22:33:49 »
Quote from: jtrezzo
Thank you Mark and Warren for your responses. Unfortunately I am still having the issue for some reason. I am debayering right after calibrating the light frames (with RGGB set for my camera, VNG demosaicing), then I run subframe selector, then star alignment (with generate drizzle data checked). After this I run image integration with the drizzle data files added so they're updated, and then try drizzle integration. Then I get Error: Missing CFA source file path. I am completely stumped as to what I am doing wrong, as I have followed all the steps written previously. It seems like somewhere in the mix it is not recognizing that they are CFA files? I don't know.

I notice from another thread that you are starting with raw FITS files from QHY247C.  Is this still the case?  The only thing I can think of is maybe the information about Bayer pattern "RGGB" is not being found by DrizzleIntegration for some reason or there is some other reason that DI is upset by working with raw FITS files.  For instance, I wonder if some header keyword needs to set within those files?

Mark
Takahashi Epsilon 180ED
H-alpha modified Sony A7S
http://www.markshelley.co.uk/Astronomy/

Offline Alfredo Beltran

  • PixInsight Addict
  • ***
  • Posts: 114
    • View Profile
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #58 on: 2017 December 13 04:09:58 »

Quote from: jtrezzo
Then I get Error: Missing CFA source file path.

Just select your CFA pattern instead of Auto. It should work.
iOptron CEM60
Celestron 925 EdgeHD Hyperstar
MicroTouch Focuser
ZWO ASI071 MC Pro
Bogota, Colombia
http://www.flickr.com/photos/alfredo_beltran/

Offline jtrezzo

  • Newcomer
  • Posts: 8
    • View Profile
Re: New Version of BatchPreprocessing Script with Bayer Drizzle Support
« Reply #59 on: 2017 December 15 20:06:36 »
Quote from: jtrezzo
Thank you Mark and Warren for your responses. Unfortunately I am still having the issue for some reason. I am debayering right after calibrating the light frames (with RGGB set for my camera, VNG demosaicing), then I run subframe selector, then star alignment (with generate drizzle data checked). After this I run image integration with the drizzle data files added so they're updated, and then try drizzle integration. Then I get Error: Missing CFA source file path. I am completely stumped as to what I am doing wrong, as I have followed all the steps written previously. It seems like somewhere in the mix it is not recognizing that they are CFA files? I don't know.

I notice from another thread that you are starting with raw FITS files from QHY247C.  Is this still the case?  The only thing I can think of is maybe the information about Bayer pattern "RGGB" is not being found by DrizzleIntegration for some reason or there is some other reason that DI is upset by working with raw FITS files.  For instance, I wonder if some header keyword needs to set within those files?

Mark

Yes I am using raw FITS. A keyword header is was what I was thinking too, but I have no idea what to add or how to add it. I have been selecting the pattern manually instead of using Auto too, both in Debayer and when I try DrizzleIntegration. Both RGGB.