New Version of BatchPreprocessing Script with Bayer Drizzle Support
- Thread starter Juan Conejero
- Start date
Note that there is now a new BayerDrizzlePrep script available for testing here:
http://pixinsight.com/forum/index.php?topic=9218.0
This should make it possible to perform stand-alone Bayer drizzling more easily, without the need for the BPP script.
http://pixinsight.com/forum/index.php?topic=9218.0
This should make it possible to perform stand-alone Bayer drizzling more easily, without the need for the BPP script.
whwang said:
Dear Wei-Hao,
In the release post for 1.8.5, there is this comment:
* DrizzleIntegration can now work with monochrome CFA frames directly. That is, Bayer drizzle is now supported directly without needing to create RGB Bayer images anymore. The BatchPreprocessing script has also been updated accordingly.
I'm guessing that this is the answer - however, I think you need to create the RGB bayer images still so you can carry out the standard integration to allow pixel rejection BEFORE you do the drizzle integration.
I hope Juan can give us the definitive answer, since the above is just a guess - it would be useful to find out the complete workflow. I think it is:
1) BPP for most of the work, with the drizzle option selected.
2) LN to set up the LN files.
3) Image integration with the LN files and drizzle files (taken from the registered images folder, rather than the registered debayer folder?)
4) Drizzle integration with the LN files and the drizzle files again.
Can anyone confirm this is correct?
Thank you,
Colin
Colin
Hi Wei-Hao / Colin,
As Colin has pointed out, drizzle works now for CFA frames just as it has always worked for monochrome CCD frames. Enable generation of drizzle files in BPP (it is now enabled by default), then proceed with calibration, debayering, and registration in BPP, as usual. Update the .drz files with ImageIntegration, also as usual. Then use DrizzleIntegration with the CFA option enabled. Since version 1.8.5 of PixInsight, Bayer drizzle is applied in a completely automatic way directly on the monochrome CFA frames, so RGB Bayer images are no longer needed.
LocalNormalization *must not* be used 'blindly' on a regular basis, and definitely *must not* be used without a correct understanding of the LN algorithm, its limitations and potential issues. As most inverse problems, local normalization is powerful but inherently ill-posed, so it must be applied along with a critical analysis of its performance for each particular data set. Please wait until I complete a new tutorial on the LN tool.
As Colin has pointed out, drizzle works now for CFA frames just as it has always worked for monochrome CCD frames. Enable generation of drizzle files in BPP (it is now enabled by default), then proceed with calibration, debayering, and registration in BPP, as usual. Update the .drz files with ImageIntegration, also as usual. Then use DrizzleIntegration with the CFA option enabled. Since version 1.8.5 of PixInsight, Bayer drizzle is applied in a completely automatic way directly on the monochrome CFA frames, so RGB Bayer images are no longer needed.
LocalNormalization *must not* be used 'blindly' on a regular basis, and definitely *must not* be used without a correct understanding of the LN algorithm, its limitations and potential issues. As most inverse problems, local normalization is powerful but inherently ill-posed, so it must be applied along with a critical analysis of its performance for each particular data set. Please wait until I complete a new tutorial on the LN tool.
I can't seem to get this to work. I keep getting "Missing CFA source file path when I try to run DrizzleIntegration" with CFA images box checked.
I calibrated lights, ran StarAlignment with drizzle data box checked, did Image Integration, then tried the DrizzleIntegration. I also tried doing Debayer before StarAlignment. No luck.
I calibrated lights, ran StarAlignment with drizzle data box checked, did Image Integration, then tried the DrizzleIntegration. I also tried doing Debayer before StarAlignment. No luck.
Hi JT, If taking a manual track (no BPP), you'd definitely have to debayer before alignment, this is an always whether working with drizzle or not. When you mentioned ImgInt afterwards, you didn't confirm that you also had Generate Drizzle Data there as well, since ImgInt needs to update the xdrz files produced by SA. That may not be your entire answer, but they are significant points. Good luck!
Alfredo Beltran
Well-known member
Warren
Could you please elaborate why does the debayering needs to be done prior to the drizzle integration? I?m confused, since Bayer Drizzle is supposed to not require an interpolation process.
Thanks,
Alfredo
Could you please elaborate why does the debayering needs to be done prior to the drizzle integration? I?m confused, since Bayer Drizzle is supposed to not require an interpolation process.
Thanks,
Alfredo
I may have missed a post/tutorial answering my question.Alfredo Beltran
Well-known member
jtrezzo said:
I?m having exactly the same issue when using the BPP. To the Gurus: Any ideas why is this happening?
Thanks in advance.
Alfredo
sharkmelley
Well-known member
[quote author=Alfredo Beltran]
Could you please elaborate why does the debayering needs to be done prior to the drizzle integration? I?m confused, since Bayer Drizzle is supposed to not require an interpolation process.
[/quote]
You still need to perform the StarAlignment to generate registered images and StarAlignment requires debayered files.
Mark
Could you please elaborate why does the debayering needs to be done prior to the drizzle integration? I?m confused, since Bayer Drizzle is supposed to not require an interpolation process.
[/quote]
You still need to perform the StarAlignment to generate registered images and StarAlignment requires debayered files.
Mark
sharkmelley
Well-known member
To perform Bayer Drizzle:
If you are using BPP then make sure you select "Generate drizzle data" in the Image Registration section and "CFA images" in the Global Options section.
If you are not using BPP then:
In either case you now have a set of registered files and drizzle files:
The error "Missing CFA source file path" in DrizzleIntegration is because you forgot to select the "CFA images" in the Global Options section of BPP so the debayering step in BPP did not happen - at least that is how I managed to reproduce the error. If you're not using BPP then it's because you performed StarAlignment on non-debayered files.
The error "Missing required LocationEstimates element" in DrizzleIntegration is because you forgot to update the drizzle files using ImageIntegration.
An error similar to "calibrated/light/DSC00429_c.xisf: No such file" in DrizzleIntegration is because you have moved or deleted the calibrated lights.
Mark
If you are using BPP then make sure you select "Generate drizzle data" in the Image Registration section and "CFA images" in the Global Options section.
If you are not using BPP then:
- Do all the steps up to and including debayering as usual.
- Run StarAlignment on the debayered files with "Generate drizzle data" selected
In either case you now have a set of registered files and drizzle files:
- In ImageIntegration add the registered files and add the drizzle files (.xdrz) and make sure "Generate drizzle data" is selected before you run. This is important because ImageIntegration must update the drizzle files with additional information.
- Run DrizzleIntegration on the drizzle files (.xdrz) making sure that "Enable CFA drizzle" is selected.
The error "Missing CFA source file path" in DrizzleIntegration is because you forgot to select the "CFA images" in the Global Options section of BPP so the debayering step in BPP did not happen - at least that is how I managed to reproduce the error. If you're not using BPP then it's because you performed StarAlignment on non-debayered files.
The error "Missing required LocationEstimates element" in DrizzleIntegration is because you forgot to update the drizzle files using ImageIntegration.
An error similar to "calibrated/light/DSC00429_c.xisf: No such file" in DrizzleIntegration is because you have moved or deleted the calibrated lights.
Mark
Mark, Thank you for ArcsinhStretch! It will be detailed in my book's second edition. Afredo/DLD, Mark explains the error well. But, in general, debayering must occur (not only before Drizzle, but) before alignment. The matrix's info has to be decoded before the transformation that occurs during registration- got it?! As Mark explains Drizzle, it's a bit mysterious to those of us 'not under the hood.' Debayer collects the matrix offset info to pass on to StarAlignment, which generates the Drizzle files. ImgInt then updates these xdrz files for DrizzleIntegration. This took me a bit to get too! Yes, the Debayer process is used in the proper sequence, but as you say, it's DI that produces the final, uninterpolated color image. The good news is, OSC/CFA Drizzle is now easier than ever with either BPP or the manual track.
Alfredo Beltran
Well-known member
Thanks for your detailed process Mark, and Warren for your explanation.
I figured out that my problem was that I had set the CFA Pattern to Auto in the BPP. When I switched it to RGGB, the issue was solved and the script worked with no issues at all.
What I did see is that if I set Drizzle Scale to 1 and and drop shrink also as 1.0 (as suggested by Juan in post #1) then I don't see difference between the regular debayered image and the one with Bayer Drizzle. But when I set Scale to 2, then I can definitely see an increase in resolution, as you can see in the comparison made in the attached image. So, the question is: should I see an increase in resolution if I set scale to 1.0 in Bayer Drizzle?
Also, I can see some patterns of the CFA matrix on the 2x Bayer drizzle image. Should this be present?
Regards,
Alfredo
I figured out that my problem was that I had set the CFA Pattern to Auto in the BPP. When I switched it to RGGB, the issue was solved and the script worked with no issues at all.
What I did see is that if I set Drizzle Scale to 1 and and drop shrink also as 1.0 (as suggested by Juan in post #1) then I don't see difference between the regular debayered image and the one with Bayer Drizzle. But when I set Scale to 2, then I can definitely see an increase in resolution, as you can see in the comparison made in the attached image. So, the question is: should I see an increase in resolution if I set scale to 1.0 in Bayer Drizzle?
Also, I can see some patterns of the CFA matrix on the 2x Bayer drizzle image. Should this be present?
Regards,
Alfredo
Attachments
sharkmelley
Well-known member
[quote author=Alfredo Beltran]
What I did see is that if I set Drizzle Scale to 1 and and drop shrink also as 1.0 (as suggested by Juan in post #1) then I don't see difference between the regular debayered image and the one with Bayer Drizzle. But when I set Scale to 2, then I can definitely see an increase in resolution, as you can see in the comparison made in the attached image. So, the question is: should I see an increase in resolution if I set scale to 1.0 in Bayer Drizzle?
Also, I can see some patterns of the CFA matrix on the 2x Bayer drizzle image. Should this by present?
[/quote]
I would usually expect to see a slight increase in detail in a x1 Bayer Drizzle. The pattern in the x2 Bayer Drizzle is possibly because you don't have sufficiently even coverage - take a look at the weights image produced by DrizzleIntegration and zoom in at full scale to see if this is the case.
Mark
What I did see is that if I set Drizzle Scale to 1 and and drop shrink also as 1.0 (as suggested by Juan in post #1) then I don't see difference between the regular debayered image and the one with Bayer Drizzle. But when I set Scale to 2, then I can definitely see an increase in resolution, as you can see in the comparison made in the attached image. So, the question is: should I see an increase in resolution if I set scale to 1.0 in Bayer Drizzle?
Also, I can see some patterns of the CFA matrix on the 2x Bayer drizzle image. Should this by present?
[/quote]
I would usually expect to see a slight increase in detail in a x1 Bayer Drizzle. The pattern in the x2 Bayer Drizzle is possibly because you don't have sufficiently even coverage - take a look at the weights image produced by DrizzleIntegration and zoom in at full scale to see if this is the case.
Mark
Thank you Mark and Warren for your responses. Unfortunately I am still having the issue for some reason. I am debayering right after calibrating the light frames (with RGGB set for my camera, VNG demosaicing), then I run subframe selector, then star alignment (with generate drizzle data checked). After this I run image integration with the drizzle data files added so they're updated, and then try drizzle integration. Then I get Error: Missing CFA source file path. I am completely stumped as to what I am doing wrong, as I have followed all the steps written previously. It seems like somewhere in the mix it is not recognizing that they are CFA files? I don't know.
sharkmelley
Well-known member
[quote author=jtrezzo]
Thank you Mark and Warren for your responses. Unfortunately I am still having the issue for some reason. I am debayering right after calibrating the light frames (with RGGB set for my camera, VNG demosaicing), then I run subframe selector, then star alignment (with generate drizzle data checked). After this I run image integration with the drizzle data files added so they're updated, and then try drizzle integration. Then I get Error: Missing CFA source file path. I am completely stumped as to what I am doing wrong, as I have followed all the steps written previously. It seems like somewhere in the mix it is not recognizing that they are CFA files? I don't know.
[/quote]
I notice from another thread that you are starting with raw FITS files from QHY247C. Is this still the case? The only thing I can think of is maybe the information about Bayer pattern "RGGB" is not being found by DrizzleIntegration for some reason or there is some other reason that DI is upset by working with raw FITS files. For instance, I wonder if some header keyword needs to set within those files?
Mark
Thank you Mark and Warren for your responses. Unfortunately I am still having the issue for some reason. I am debayering right after calibrating the light frames (with RGGB set for my camera, VNG demosaicing), then I run subframe selector, then star alignment (with generate drizzle data checked). After this I run image integration with the drizzle data files added so they're updated, and then try drizzle integration. Then I get Error: Missing CFA source file path. I am completely stumped as to what I am doing wrong, as I have followed all the steps written previously. It seems like somewhere in the mix it is not recognizing that they are CFA files? I don't know.
[/quote]
I notice from another thread that you are starting with raw FITS files from QHY247C. Is this still the case? The only thing I can think of is maybe the information about Bayer pattern "RGGB" is not being found by DrizzleIntegration for some reason or there is some other reason that DI is upset by working with raw FITS files. For instance, I wonder if some header keyword needs to set within those files?
Mark
Alfredo Beltran
Well-known member
[quote author=jtrezzo]
Then I get Error: Missing CFA source file path.
[/quote]
Just select your CFA pattern instead of Auto. It should work.
Then I get Error: Missing CFA source file path.
[/quote]
Just select your CFA pattern instead of Auto. It should work.
sharkmelley said:
I notice from another thread that you are starting with raw FITS files from QHY247C. Is this still the case? The only thing I can think of is maybe the information about Bayer pattern "RGGB" is not being found by DrizzleIntegration for some reason or there is some other reason that DI is upset by working with raw FITS files. For instance, I wonder if some header keyword needs to set within those files?
Mark
[/quote]
Yes I am using raw FITS. A keyword header is was what I was thinking too, but I have no idea what to add or how to add it. I have been selecting the pattern manually instead of using Auto too, both in Debayer and when I try DrizzleIntegration. Both RGGB.