New Version of BatchPreprocessing Script with Bayer Drizzle Support

[sharkmelley]

I notice from another thread that you are starting with raw FITS files from QHY247C.  Is this still the case?  The only thing I can think of is maybe the information about Bayer pattern "RGGB" is not being found by DrizzleIntegration for some reason or there is some other reason that DI is upset by working with raw FITS files.  For instance, I wonder if some header keyword needs to set within those files?

Mark

Yes I am using raw FITS. A keyword header is was what I was thinking too, but I have no idea what to add or how to add it. I have been selecting the pattern manually instead of using Auto too, both in Debayer and when I try DrizzleIntegration. Both RGGB.

How frustrating.  There is also no keyword solution.  Here (https://pixinsight.com/forum/index.php?topic=11466.0) Juan says the following about FITS files:

"PixInsight 1.8.5 implements a new automatic CFA pattern detection mode in the Debayer tool and the BatchPreprocessing script (which uses Debayer internally). This mode works perfectly for DSLR raw files. However, FITS files cannot store information on CFA patterns, so you have to select the Bayer pattern of your camera manually in the BPP script. The same is true for the Debayer tool if you use it directly."

You say you are setting the CFA pattern manually at each step, so I can't see what else you can do.  I assume you have already tried BPP for your Bayer Drizzle pipeline? I ask because earlier in this very thread (https://pixinsight.com/forum/index.php?topic=7184.msg58989#msg58989) Juan says:

Right now the Bayer drizzle method can only be implemented as an automatic process using the BatchPreprocessing script. An optimal implementation of Bayer drizzle requires a tight integration with a preprocessing pipeline, such as BPP.

Has anyone successfully Bayer Drizzled FITS files with PI 1.8.5.  If so, what did you do?

Mark

[Juan Conejero]

Hi all,

Sorry for having been without attending this forum for so long. Unfortunately, I have other problems that have been occupying most of my time during these weeks. I chime in here briefly because I see that something very important is being overlooked:

Has anyone successfully Bayer Drizzled FITS files with PI 1.8.5.  If so, what did you do?

When I announced version 1.8.5 of PixInsight I included the following among the list of new features:

DrizzleIntegration can now work with monochrome CFA frames directly. That is, Bayer drizzle is now supported directly without needing to create RGB Bayer images anymore. The BatchPreprocessing script has also been updated accordingly.

So yes, Bayer drizzle (or, more generically and correctly, CFA drizzle) can now be performed directly with the DrizzleIntegration tool. There is no need to generate RGB Bayer images as in previous versions. Just check the "Enable CFA drizzle" option and read the corresponding tool tip.

If you use BPP to calibrate and align your data, then ImageIntegration manually to optimize integration and rejection parameters, as usual, DI will perform a Bayer drizzle procedure by just activating its "Enable CFA drizzle" option.

As for the Bayer pattern / FITS problem, this is just what FITS is: an endless source of interoperability problems and deficient implementations. The FITS format does not define a standard way to include information about CFA patterns in FITS files. We can't do anything to fix this. So you have to select your Bayer pattern manually.

[jtrezzo]

Hi all,

Sorry for having been without attending this forum for so long. Unfortunately, I have other problems that have been occupying most of my time during these weeks. I chime in here briefly because I see that something very important is being overlooked:

Has anyone successfully Bayer Drizzled FITS files with PI 1.8.5.  If so, what did you do?

When I announced version 1.8.5 of PixInsight I included the following among the list of new features:

DrizzleIntegration can now work with monochrome CFA frames directly. That is, Bayer drizzle is now supported directly without needing to create RGB Bayer images anymore. The BatchPreprocessing script has also been updated accordingly.

So yes, Bayer drizzle (or, more generically and correctly, CFA drizzle) can now be performed directly with the DrizzleIntegration tool. There is no need to generate RGB Bayer images as in previous versions. Just check the "Enable CFA drizzle" option and read the corresponding tool tip.

If you use BPP to calibrate and align your data, then ImageIntegration manually to optimize integration and rejection parameters, as usual, DI will perform a Bayer drizzle procedure by just activating its "Enable CFA drizzle" option.

As for the Bayer pattern / FITS problem, this is just what FITS is: an endless source of interoperability problems and deficient implementations. The FITS format does not define a standard way to include information about CFA patterns in FITS files. We can't do anything to fix this. So you have to select your Bayer pattern manually.

Juan, I have specified the Bayer pattern manually throughout all steps and my OSC FITS files still will not work with Bayer/CFA Drizzle. I've tried it with and without debayering, neither work.

[Don]

I was having the same problem as JT.  I found the step in my processing that caused Drizzle Integration to fail with the "Missing CFA source file path" error when "Enable CFA drizzle" is selected.

If the SubFrameSelector script is used between debayering and star alignment to copy approved files to a folder, the .XDRZ files created by Star Alignment in the next step, and later updated by Image Integration, will not contain the <CFASourceImage> tag.  You cannot run Drizzle Integration and select the "Enable CFA drizzle" option with these files.  Drizzle Integration works, but only if you do not select the "Enable CFA drizzle" option.

At least that is what I have experienced with my CFA images, running PixInsight 01.08.05.1352.

Don

[acmalko]

I was having the same problem as JT.  I found the step in my processing that caused Drizzle Integration to fail with the "Missing CFA source file path" error when "Enable CFA drizzle" is selected.

If the SubFrameSelector script is used between debayering and star alignment to copy approved files to a folder, the .XDRZ files created by Star Alignment in the next step, and later updated by Image Integration, will not contain the <CFASourceImage> tag.  You cannot run Drizzle Integration and select the "Enable CFA drizzle" option with these files.  Drizzle Integration works, but only if you do not select the "Enable CFA drizzle" option.

At least that is what I have experienced with my CFA images, running PixInsight 01.08.05.1352.

Don

Exactly the same for me. I often use Subframe Selector for weighting (with FITS keyword) and can't "enable CFA Drizzle" .
It's OK without Subframe Selector step

Thanks, it helped me

[johnpane]

I reported this as a bug last November and did not receive any reply. However, I discovered a SubframeSelector process that will apparently replace the script, and the process does preserve the PCL:CFASourceFilePath property. See https://pixinsight.com/forum/index.php?topic=11820.0

[GregW]

Is there a fix yet for the subframe selector script? Mine crashes every time we try it. However the subframe selector process works okay but it is not nearly as useful

[GregW]

I actually got a bunch of new stuff today under script with an update to 1457.
So long,
And thanks for all the scripts