New Version of BatchPreprocessing Script with Bayer Drizzle Support

idiaz

Well-known member
I did this comparison of fully processed data from my recent OmegaCent session (22 dithered frames), using bayer drizzle 1x vs drizzle 2x integration (reduced back to original resolution at the end). Not very scientific as both processes are not exactly the same but similar (both done on a best effort basis).

Based on this I can say that deconvolution behaved better with the bayer drizzled data (easier to parametrize and less color bleeding in stars). The final result shows sharper and better colored stars, but less of them in the background. This last thing may be the result of the somewhat different processing of background noise. I speculate that the luminance PSF used during deconvolution is more consistent with the PSF of each individual color channel, in the bayer drizzle case.

FWIW, I also tried bayer drizzle on a different data set (30 dihtered frames), and got color pixelation in some parts of the frame (??).

Ignacio

 

Attachments

whwang

Well-known member
Nov 6, 2012
78
3
Hi Ignacio,

From your examples, I see that the Bayer-drizzled image is better than the normal-drizzled image
in all aspects. (The difference in background faint stars should be some processing artifact.) This confirms my thinking that Bayer CFA images should NOT be drizzled.  Instead, Bayer-drizzle is the right way to go for Bayer CFA images.

Cheers,
Wei-Hao
 

fulatoro

Active member
Dec 31, 2013
36
0
Having issues getting DrizzleIntegration working. I ran the BatchProcessing script according the instructions, but when I run DrizzleIntegration and add the drz files, I get the following error:

* Parsing drizzle data file 22 of 22:
D:/GDRIVE/Astrophotography/Lights/M51-Baader/ppDrizzle2/registered/bayer/Whirlpool Galaxy (M51)_300sec_1x1__frame9_c_d_r_b.drz
*** Error: No location vector definition.
*** Error: No image could be integrated.
<* failed *>
 

rockyraccoon

Well-known member
Jun 2, 2013
63
0
Berkeley, CA
www.astrodan.com
The script errors out, if no darks/bias/flats are selected.
I don't understand why any of those would be necessary to drizzle an image (I would expect warnings, but no errors)
but the script just quits with the following messages in the console:
Code:
run --execute-mode=auto "/Applications/PixInsight.app/Contents/src/scripts/BatchPreprocessing/BatchPreprocessing.js"

Processing script file: /Applications/PixInsight.app/Contents/src/scripts/BatchPreprocessing/BatchPreprocessing.js


run --execute-mode=auto "/Applications/PixInsight.app/Contents/src/scripts/BatchPreprocessing/BatchPreprocessing.js"

Processing script file: /Applications/PixInsight.app/Contents/src/scripts/BatchPreprocessing/BatchPreprocessing.js

*********************************************************************
* Begin calibration of light frames
*********************************************************************

* Searching for a master dark frame with exposure time = 120s -- best match is a master dark frame of unknown exposure time.

*********************************************************************
* Begin calibration of light frames
*********************************************************************

* Searching for a master dark frame with exposure time = 120s -- best match is a master dark frame of unknown exposure time.
Interestingly the script claims best match for a dark frame that doesn't really exist.
 

gvanhau

Well-known member
Jun 21, 2010
346
0
67
Santiago, Chile
Hello
I could finally test it.

The results are very good in my  modest opini?n.
I attached a zoom 3:1 of a crop of a normal integrated image (left) and the same but Bayer drizzle integration on the right of a set of 24 pictures I took using a 24mm Lens on an canon 450D camera of the galactic center.
The stars in the Bayer drizzle image are rounder, showing an increase in resolution.
I dont see a increment in noise

Thanks Juan (and team) for this excelent tool.

Geert
 

Attachments

james7

Well-known member
Jan 17, 2012
67
2
Am I correct in assuming that the only way you can currently use the Bayer Drizzle is with the BatchPreprocessing script? That is, there is no tool specifically to produce a set of files that have been reconstructed into RGB images using the Bayer Drizzle technique.
 

Juan Conejero

PixInsight Staff
Sep 2, 2004
8,282
408
57
Valencia, Spain
pixinsight.com
james7 said:
Am I correct in assuming that the only way you can currently use the Bayer Drizzle is with the BatchPreprocessing script? That is, there is no tool specifically to produce a set of files that have been reconstructed into RGB images using the Bayer Drizzle technique.
Yes, you're correct. We plan on releasing new tool versions to support Bayer drizzle outside BatchPreprocessing. They will be released as updates to the upcoming version 1.8.3 of PixInsight.
 

NKV

PTeam Member
Dec 3, 2008
677
0
Juan Conejero said:
We plan on releasing new tool versions to support Bayer drizzle outside BatchPreprocessing. They will be released as updates to the upcoming version 1.8.3 of PixInsight.
Juan, please do it faster.
Because BatchPreprocessing don't provide full of power manual calibration and do it wrong. For example: BatchPreprocessing calibrate MasterFlat which already calibrated . Impossible to disable calibrate it.
 

bitli

PTeam Member
Feb 1, 2009
525
6
Switzerland
I am afraid that this is an old known issue:
http://pixinsight.com/forum/index.php?topic=6936.msg47052#msg47052
But I never got an answer - the problem with BatchPreprocessing is that it is useless to use complex algorithms and 32 bit  images if we cannot be sure that the basic math is correct (or understand why it would do things differently).
-- bitli
 

vicent_peris

PTeam Member
Sep 3, 2004
995
12
Valencia (Spain)
www.astrofoto.es
NKV said:
Juan Conejero said:
We plan on releasing new tool versions to support Bayer drizzle outside BatchPreprocessing. They will be released as updates to the upcoming version 1.8.3 of PixInsight.
Juan, please do it faster.
Because BatchPreprocessing don't provide full of power manual calibration and do it wrong. For example: BatchPreprocessing calibrate MasterFlat which already calibrated . Impossible to disable calibrate it.
Hi Nikolay,

You mean that, when using a master flat frame (checking the "Use master flat" option), the master flat frame is again bias and dark subtracted?


Best regards,
Vicent.
 

NKV

PTeam Member
Dec 3, 2008
677
0
Juan Conejero said:
We haven't seen this problem, and I am unable to reproduce it with any of our test data sets. Can you put a repeatable example of this problem?
OK, I prepare set of files.

vicent_peris said:
You mean that, when using a master flat frame (checking the "Use master flat" option), the master flat frame is again bias and dark subtracted?
Yes.

Juan Conejero said:
We plan on releasing new tool versions to support Bayer drizzle outside BatchPreprocessing. They will be released as updates to the upcoming version 1.8.3 of PixInsight.
Hi, Juan.
Independent Bayer drizzle possible right now.
Please check my new module CFA2RGB
 

Rod771

Member
Jul 11, 2013
22
0
Guys, I keep coming back to this when process my under sampled data.

I must have missed something? When generating a bayer drizzle image, when do you use the files in <output-dir>/calibrated/light/bayer. ? It says the Drizzle integration tool will use thm , however Step 7 in the OP says "After ImageIntegration, open the DrizzleIntegration tool. Select the same .drz files on the <output-dir>/registered/bayer directory" These are the registered file that Image Intergration used. Just confused a bit.

Cheers

Rod
 

jkmorse

Well-known member
Rod,

Yes, they are the same .drz files that you added in the Image Integration step.  Image Integration updates the ones that were produced as a part of Star Alignment so after Image Integration you should have an entirely new set that are in the same folder that you used to populate the Image Integration tool.  Load them into Drizzle and watch the magic happen  :D

Best,

Jim
 

Rod771

Member
Jul 11, 2013
22
0
Thanks Jim

So to confirm , the files exported to <output-dir>/calibrated/light/bayer  during BPP are never selected in Image Integration or Drizzle Integration. Only use files in <output-dir>/calibrated/registered/bayer.
 

whwang

Well-known member
Nov 6, 2012
78
3
Hi Juan,

In this thread:
http://pixinsight.com/forum/index.php?topic=7151.0
you instructed that we should select the "generate drizzle data" option in the ImageIntegration tool.  On the other hand, for Bayer Drizzle here, you did not instruct to do so.  May I ask why is there such a difference?  I guess what I am trying to understand is what this "generate drizzle data" actually does.

Cheers,
Wei-Hao
 

Juan Conejero

PixInsight Staff
Sep 2, 2004
8,282
408
57
Valencia, Spain
pixinsight.com
Hi Wei-Hao,

On the other hand, for Bayer Drizzle here, you did not instruct to do so.
I forgot to mention it. Yes, the ImageIntegration tool must always update the .drz files with rejection and normalization data, so the generate drizzle data option has to be enabled also in the case of Bayer drizzle. Note that ImageIntegration enables this option automatically when you select drizzle files (by clicking the Add Drizzle Files button), so in general one doesn't need to care about this.

what this "generate drizzle data" actually does
A drizzle data file (with the .drz suffix) stores all the data required to perform a drizzle integration process:

1. The full file path of the source drizzle image (calibrated, de-Bayered, but not registered). This is the image that will be integrated with DrizzleIntegration.

2. The full file path of the registration target image (calibrated, de-Bayered and registered). This is the image that should be integrated with ImageIntegration to compute normalization parameters, image weights, and pixel rejection data. This image will not be used in the final drizzle integration process.

3. The geometry of the reference registration image. This defines the geometry of the final drizzle-integrated image (multiplied by the drizzle output scale).

4. The geometrical transformations necessary to register the image. This may include a projective transformation matrix and/or surface spline control points and parameters, if local distortion correction is being used. These transformations are applied by DrizzleIntegration to compute the geometry of each 'drop' that 'rains' over the drizzle-integrated image

5. The coordinates of each rejected pixel. In the next version of the ImageIntegration module this will be more complex, since we'll implement a fuzzy-logic rejection feature. For now, rejection is a purely binary process (rejected/not_rejected).

6. Image normalization parameters, including estimates of statistical location and scale.

7. Image weighting parameters. These are normally based on noise estimates computed during the image calibration phase (and stored as XISF properties and FITS keywords in the image), but can also be generated from statistical parameters or image properties/keywords by ImageIntegration.

Drizzle data files are created by the StarAlignment process with items 1, 2, 3 and 4. The ImageIntegration process updates drizzle files with items 5, 6 and 7.

In the case of Bayer drizzle, the trick consists of replacing item 1 with the path to the calibrated Bayer image, that is, the image after calibration but *before* applying the de-Bayering process. However, drizzle cannot use the monochrome CFA data directly (mainly because color planes would get intermixed without sense), so the CFA must be split into three separate R, G and B planes, with black pixels replacing the lacking data in each plane, and conveniently packed as an RGB image. This is what we call a separate RGB Bayer image. Then the drizzle process can be applied unmodified, as if it was working with a normal RGB image. The BatchPreprocessing script does this automatically, including generation of separate RGB Bayer data under the hood. The same task can also be implemented manually, with the help of Nikolay's SplitCFA tool to automate part of the process, but it is somewhat tedious.

Right now the Bayer drizzle method can only be implemented as an automatic process using the BatchPreprocessing script. An optimal implementation of Bayer drizzle requires a tight integration with a preprocessing pipeline, such as BPP. This will also be implemented outside BPP, probably as a standalone tool, but we have other priorities at present.

Hope this clarifies what this drizze data stuff is all about.