New DrizzleIntegration Tool Released

[whwang]

Hi Wei-Hao,

Let me try to understand this.  For R and B in a Bayer matrix, Bayer drizzle conceptually is just like normal 2x drizzle.

Bayer drizzle is just the same drizzle algorithm applied to CFA data. You normally perform a 1x Bayer drizzle, not 2x, because filling the Bayer pattern holes requires a *lot* of images. Filling Bayer holes plus the extra grid spacing in a 2x Bayer drizzle requires a really huge amount of data. For the same reason, Bayer drizzle is usually applied without any drop shrink (drop shrink factor = 1).

A 1x Bayer drizzle process can be used as a nice deBayering algorithm that works with the actual raw data without any interpolation. In this case there is no resolution improvement, but provided that you have enough frames to cover the image, the results should be better than a normal deBayering with interpolation such as VNG, AHD, etc.

Hi Juan,

I guess we used different terminology.

Use R as an example. In an N-mega-pixel Bayer CFA, only N/4 MP are in R.  But a normal 1x Bayer-drizzled final image contains N MP of R data.  Therefore, this is just like a 2x normal-drizzle (not Bayer-drizzle) of a N/4 MP of normal gray-scale image.

For R, and B, 1x Bayer-drizzle = 2x normal-drizzle (or you can call it standard drizzle).  In terms of algorithm, 1x Bayer-drizzle and 2x normal-drizzle are essentially identical.  This is simple because for R or B, the distribution of data on one CFA image is:
R E
E E
(E means empty, no data)
or
E E
E B
This is exactly the configuration of the drop in a normal 2x drizzle.

So, I was not talking about making an N-MP CFA image become 4N-MP after stacking.

...out of every four pixels, only 1 is real R, or real B.  On the other hand, G is somewhat different.  Out of every 4 pixels, two are G, instead of one.  So to make Bayer drizzle works like a normal 2x drizzle for G, there must be two G channels, instead of one (in my naive thinking).  In the above 7 steps you wrote, which one has this G issue taken into account?

This is the idiosyncrasy of Bayer filters. There is two times more green data than red and blue in a Bayer CFA frame, and Bayer drizzle doesn't change this. Your final image will have roughly two times more signal in the green channel than on the red and blue channels. We could think on a variation of Bayer drizzle that attempts to compensate for this relative lack of signal, but this implies interpolation, which is just what we are trying to avoid with Bayer drizzle.

The above statement of mine (1x Bayer-drizzle = 2x normal-drizzle) is not true for G.  On the CFA image, the configuration for G is:
E G
G E
This is not the drop configuration.  This is why I asked in the previous post how this was taken into account in the procedures you outlined earlier.

Cheers,
Wei-Hao

[MikeOates]

I have just tried drizzle on two sets of data and I have noticed that the rejection is not as good as I am getting a satellite trail remaining in the drizzle image in both cases, even though it is not in the image produced with the ImageIntegration tool.

Rejection is exactly the same on the ImageIntegration and DrizzleIntegration tools. They are identical by construction, so this "cannot fail" geometrically, modulo bugs.

However, if you are integrating few frames, the relative lack of SNR over rejected areas combined with some particular subpixel alignments may make rejected pixels visible as a result of different noise patterns or distributions after drizzle. This shouldn't happen if you integrate a sufficient number of images. How many frames are you using with drizzle?

Juan,

I will ignor the first set I tried because the dither was not set during image capture, but the next set was 34 x 600s subs which should have been ok. I used scale 2 and drop shrink 0.9, I also tried with a drop shrink of 0.7 with the same result.

I have uploaded the following files which show what I mean. Sorry it took a while to upload these (warning big files!).

Standard Integration
Drizzle Integration
Drizzle_weights

Mike

[Alejandro Tombolini]

Hi, I am facing the se same rejection issue and cannot solve it completly. Also have doubts with the interpretation of the drizzle_weight obtained. Does it means bad dietering? Is it ok the pattern shown? (I have stretched it conveniently to show it)



This set has 84 images, and in drizzle integration remain pixels that are rejected in standard integration using winsorized sigma clippling.
I have tried with more aggressive rejection values but do not help so much and even make worst other issue with rejection in stars.

Then I change from winsorized to Linear fit and was much better, only a minor issue with rejection and some yellow lines appear in the standard integration, not in the drizzle.



My worry is how it looks in corners mainly, and also in some stars. Could it be related to flats used?
 


I would appreciate if you could check the images I have uploaded to ForumSharedFiles>AlejandroTombolini>Trifida in endor. I have used the image "2014_06_28 LT 00_17_59 ISO 1600050005_0033" as reference for registration and in II.

Saludos, Alejandro.

Ps1: using the last update.
Ps2: in drizzle module, "from preview" takes the same time for integration that the big images.

[Juan Conejero]

Hi Alejandro,

I can't reproduce any of the problems you are reporting In fact, I've got an extremely nice Bayer-drizzled image with linear fit clipping rejection, without any rejection issue. I have uploaded it as "bayer_drizzle_integration.fit" to your Trifida folder on Endor.

I have also made a try with 2x Bayer drizzle and 0.85 drop shrink. It works well, but there are a few issues on regions where the number of rejected frames is relatively high, mainly around some bright stars with many rejected pixels due to small tracking issues and focus shifts. Other than that, with 84 frames a 2x Bayer drizzle is feasible with your data, although it actually makes no sense because your image is not undersampled.

Your master flat overcorrects the upper left corners of your light frames. Also you have some light frames that are rotated by 180 degrees due to a meridian flip. These frames require a specific flat and are obviously not being flattened well. Your master flat was created with PixInsight Core 1.8.0.1087, which was released in February, while your images have been acquired in June. Despite that, the result is very nice but you should crop the central area of the image IMO.

How are you applying drizzle? I have used the BatchPreprocessing script with both the drizzle and Bayer drizzle options enabled. Then I have used ImageIntegration to update the Bayer drizzle .drz files, and the DrizzleIntegration tool with scale=1 and drop_shrink=1. The result is great:

Click for full-size image

I can post more screenshots to show comparisons and more processing steps if you wish.

[Alejandro Tombolini]

Hi Juan, sorry for the late response, I was without computer.

and thanks you for your help!

Your master flat overcorrects the upper left corners of your light frames. Also you have some light frames that are rotated by 180 degrees due to a meridian flip. These frames require a specific flat and are obviously not being flattened well. Your master flat was created with PixInsight Core 1.8.0.1087, which was released in February, while your images have been acquired in June. Despite that, the result is very nice but you should crop the central area of the image IMO.

Yes, you are right about the flats, I couldn't do it during the sesion and used a set that do not fit so well, besides I have some street lights  that realy bother the image after the flip. I believe that influence considerably calibration and rejection issues'.

I've got an extremely nice Bayer-drizzled image with linear fit clipping rejection, without any rejection issue.

   I have downloaded your Bayer-drizzled image and it has a better rejection that my approach but still there are hot pixels that I would like to reject.
See the screen shot where I made previews for you to check in the bayer_drizzle_integration.

How are you applying drizzle? I have used the BatchPreprocessing script with both the drizzle and Bayer drizzle options enabled. Then I have used ImageIntegration to update the Bayer drizzle .drz files, and the DrizzleIntegration tool with scale=1 and drop_shrink=1.

Yes, I did drizzle as you describe, but 2x Bayer drizzle and 0.85 drop shrink ( with 2x the stars look much better... )

My worry is how it looks

What I meant is that background does not look uniforme, if you extract V and auto STF it, in Bayer-drizzled image can be seen differences (smooth at the center and kind of lines upper and down the image). Can it be due to bad dithering?

Ps2: in drizzle module, "from preview" takes the same time for integration that the big images.

I did not check again, but did you check this, or is it the normal behavior?

Thanks again!!!
Saludos, Alejandro

[Leandro VARGAS]

Hello,

The option Drizzler in BPP version 1.36 we need check the option Generate Drizzle Data in Image Registrator and check Bayer Drizzle in DeBayer OR only check option Generate Drizzle Data in Image Registrator ?

Thank you for your help.

Leandro

[Alejandro Tombolini]

The option Drizzler in BPP version 1.36 we need check the option Generate Drizzle Data in Image Registrator and check Bayer Drizzle in DeBayer OR only check option Generate Drizzle Data in Image Registrator ?

Hi Leandro, both and then use ImageIntegration to update the Bayer drizzle .drz files, and finally DrizzleIntegration.
Nota that last version of BPP is 1.37

Saludos, Alejandro.

[Leandro VARGAS]

Hello Alejandro,

Where Can I download the BPP 1.37?, my PI not Update the script .

Saludos,

Leandro

Thank You for the information in your reply.

[Alejandro Tombolini]

That is extrange. Please check if your version of PixInsight is 01.08.02.1098

Saludos, Alejandro.

[Leandro VARGAS]

Alejandro,

I have this versión, but when open PI not Update nothing.

Leandro

[Alejandro Tombolini]

Have you tried this?

[Leandro VARGAS]

Alejandro,

Yes, but I have the next text:

https://pixinsight.com/update/: Network error: schannel: next InitializeSecurityContext failed: Unknown error (0x80092013) - La función de revocación no puede comprobar la revocación debido a que el servidor de revocación está sin conexión.
https://pixinsight.com/update-doc/: Network error: schannel: next InitializeSecurityContext failed: Unknown error (0x80092013) - La función de revocación no puede comprobar la revocación debido a que el servidor de revocación está sin conexión.

But when PI send information all was normal.

My Internet access was fine.

Leandro

[Leandro VARGAS]

Hello Alejandro,

I have the solution when update PI--> firewall problem, now my PI fine.

When finish the Drizzle generates two images -> the result of the drizzle reconstruction and a drizzle weights image, to continue the others process with PI to arrive the good photo I need save the result of the drizzle reconstruction ?

Thank you

Leandro

[jkmorse]

Juan,

You state in the introduction to the tool re Output Scale that:

Output scale, or subsampling ratio. This is the factor that multiplies input image dimensions (width, height) to compute the dimensions in pixels of the output integrated image. For example, to perform a 'drizzle x2' integration, the corresponding drizzle scale is 2 and the output image will have four times the area of the input reference image in square pixels.

Can you describe the circumstances where it may be beneficial to make the output scale greater than 2, recognizing that the image will get huge?  Is it dependent on anything else such as number of subframes or the extent of the undersampling?

Took a series of 30 RGBs each of a number of summer targets this past weekend (North America Nebula, Antares Region, and M8 & M20 together) using fast Canon lenses and an STF8300 and can't wait to get processing now that this is part of the PI toolkit. 

Thanks again for such a superb toolkit.

Best,

Jim

[Juan Conejero]

Hi Jim,

Can you describe the circumstances where it may be beneficial to make the output scale greater than 2, recognizing that the image will get huge?  Is it dependent on anything else such as number of subframes or the extent of the undersampling?

To perform a drizzle x3 integration you need a *huge* amount of data. For a drizzle x4 you need a *monster* amount of data. Et cetera. If you analyze how the drizzle algorithm works, you'll see that in order to cover all output pixels with a sufficient number of drops to get no dry pixels, you need many frames. Even more if you use reduced drops, and you really want to reduce drops to something in the range 0.7 - 0.9, or otherwise your output image will be convolved with a too large PSF (that is, blurred).

The DrizzleIntegration tool writes input and output data fractions on the console for each integrated frame. These fractions give you an idea of the data that you "lose" with drizzle, with respect to a normal integration with the same data set. The output data fraction is the sum of the areas of intersection of all drops with output pixels, divided by the total area of the output image in square pixels. In other words, the output data fraction tells you the total area of the output image that "gets wet" after drizzling one frame. The input data fraction tells you the total "water" (where 1 is the total "light" available on the frame) that has been used to dampen the output image. Both data fractions take into account the drop shrink factor and all rejected pixels (which are "dry spots"). Of course, in a normal integration we always have output data = input data = 1, modulo rejected pixels.

In practical terms, a drizzle x2 integration is normally sufficient to model the PSF reasonably well in most cases.