Drizzle integration with WBPP gives different outputs than with DrizzleIntegration ??

[fruciak]

Hi,
I am a newbie (18 months learning curve) and I am starting to use drizzle integration with my undersampled setup (90mm aperture 540mm FL with OSC 3.75µ pixel).
I am using WBPP in an interactive way (for LN reference).
I am using it with separated RGB channels with Bias, Darks, Flats (one set per imaging session) and asking for drizzling with x2 and 0.9 for drop shrink.
Everything works well and I get my 3 sets of master lights R G and B and the same for drizzle integration.
Having read somewhere on this forum that integration from WBPP was just to have an idea (not really understanding why) I reran drizzle integration from the process menu with exactly the same set of .xrdz and .xnml files produced by WBPP.
And I was really estonished to see a significant difference in the results.
Is it normal? if not what I am doing wrong in WBPP to produce what seems to be a poor drizzle integration?
Included a zoom on the side of M5 showing the WBPP DI of the R channel and the standard DI of the same R channel.
I have all the xisf files it can helps to diagnose my question.
For the time being I am using DrizzleIntegration process in addition of WBPP. PI is a fantastic tool
Thanks for your support, Frédéric

 

[nico1038]

Hi,
I am a newbie (18 months learning curve) and I am starting to use drizzle integration with my undersampled setup (90mm aperture 540mm FL with OSC 3.75µ pixel).
I am using WBPP in an interactive way (for LN reference).
I am using it with separated RGB channels with Bias, Darks, Flats (one set per imaging session) and asking for drizzling with x2 and 0.9 for drop shrink.
Everything works well and I get my 3 sets of master lights R G and B and the same for drizzle integration.
Having read somewhere on this forum that integration from WBPP was just to have an idea (not really understanding why) I reran drizzle integration from the process menu with exactly the same set of .xrdz and .xnml files produced by WBPP.
And I was really estonished to see a significant difference in the results.
Is it normal? if not what I am doing wrong in WBPP to produce what seems to be a poor drizzle integration?
Included a zoom on the side of M5 showing the WBPP DI of the R channel and the standard DI of the same R channel.
I have all the xisf files it can helps to diagnose my question.
For the time being I am using DrizzleIntegration process in addition of WBPP. PI is a fantastic tool
Thanks for your support, Frédéric

Hello @fruciak

WBPP simply runs the DrizzleIntegration process, so there is no reason for it to give a different result unless some settings are different.
To find the differences can your share your WBPP log (the complete one in the log folder) and a screenshot of the drizzle integration process as you run it?


And the idea that WBPP is just for testing is an old one. WBPP has evolved over the years and is now considered as a fully capable preprocessing tool.

Nico

 

[fruciak]

Hi Nico, it seems logical, I include the WBPP log (to be downloaded) and screenshots of DI process and WBPP used to get the results. Thank you for you time, Frédéric

 

[nico1038]

Ok, the difference is that WBPP enable the CFA setting for the drizzle integrations.
At first glance this seems curious since the data is monochrome at this point but I've never completely understood how things work in WBPP when you separate the RGB channels of an OSC camera and drizzle in top of that.

Here are some discussions about this situation:

WBPP, Separate RGB Channels + CFA Drizzle

Thank you @robyx. Am I right to interpret this as "CFA drizzle has never worked correctly for separated RGB channels"? yes it was not designed to handle the separated RGB channels. That feature came later and Drizzle Integration needs to be update to handle it. I see that WBPP was updated to...

pixinsight.com

fixed - WBPP v2.5.9: Bug when drizzling CFA separated channels

Below I describe short test scenario specifically setup to confirm this bug. I have 16 CFA images taken with L-eXtreme filter and setup WBPP to produce masters cropped, drizzled x2 with separated channels. The pipeline correctly shows everything that should happen during the process. Execution...

pixinsight.com

I'm not sure what the situation is at the moment in WBPP with this combination (maybe @robyx or someone else can help)?

 

[fredvanner]

I've never completely understood how things work in WBPP when you separate the RGB channels of an OSC camera and drizzle in top of that.

I've been meaning to ask about this for some time. I too simply don't know what happens in this case. In my mind the problem is roughly this:

• Drizzle for CFA images takes the drizzle data from the undebayered CFA image - so no bayer interpolation, leaving "holes" between the single channel samples (which must be filled in by dithering).
• The separated channels when you run CFA processing with channel separation are separated debayered channels (with debayer interpolation "filling in the holes").
• So do the drizzle files generated in this case refer back to the debayered separated channel files, or do they still "reach back" to the original CFA files, and drizzle only uninterpolated data (in this case drizzling only the single appropriate bayer filter).

I'd be interested to know the answer. It may be needed to explain the OP observation.

 

[fruciak]

Well thank you to point this out. As you understand, when I am running drizzle integration after WBPP, I am using the same exact files produced during WBPP. As far as I understand there is no way for PI to know where these files come from and treat them in another way as pure monochrome image files. And doing this produces better results at least from my point of view as the crop samples show. If this is the general observation WBPP should simply replicate the drizzle integration the same way for separate channels in WBPP. At least I kwow what to do to get around.

 

[nico1038]

Well thank you to point this out. As you understand, when I am running drizzle integration after WBPP, I am using the same exact files produced during WBPP. As far as I understand there is no way for PI to know where these files come from and treat them in another way as pure monochrome image files. And doing this produces better results at least from my point of view as the crop samples show. If this is the general observation WBPP should simply replicate the drizzle integration the same way for separate channels in WBPP. At least I kwow what to do to get around.

Keep in mind that by doing what you do you are probably losing one of the advantage of drizzling CFA images which is to avoid any interpolation when creating the RGB image (because, as Fred said, you are working with debayered channels already interpolated).
I think it would be interesting to compare your results with a WBPP run with drizzle but without "separate RGB".

 

[fruciak]

Hi Nico, I will do it and share the results here for comparison.
Soon to come. Frédéric

 

[fruciak]

Hi Nico,
here is the link to the two master light files. The file named masterLight_RGB_CFA_drizzle_2x is the one produced directly within WBPP with CFA processing (drizzle x2 0.9) (see below). The one named masterlight_RGBcomb_drizzlex2 is the one produced with a separated drizzle integration starting from the 3 masterlights of R, G and B channels (with drizzle x2 0.9 enabled) produced by WBPP with separated channels. They are all based on the same set of frames with the exact same parameters but as I have to choose the reference frame for local_normalization, and M5 was low on the horizon there my be some impact on the different color channels for the CFA treatment
These files are big because of x2 from a ASI2600MC camera. They are not perfectly aligned most probably as WBPP didn't choose the same reference frame for registration (CFA combined versus CFA separated). I let you see at the files without any "spoiling" of my own impression. See you soon.
At the end I am wondering if there is any difference in WBPP if I choose separate Channels PLUS RGB recombination in the posprocessing setup. I will try and share here.
Frédéric

 

[nico1038]

Hi Frédéric and thanks for sharing. This is interesting.
Did you do any kind of gradient correction on the individuals R, G and B channels before combining them? I don't understand why the gradient is so different between the two files (and this makes comparison a bit difficult).

At the end I am wondering if there is any difference in WBPP if I choose separate Channels PLUS RGB recombination in the posprocessing setup. I will try and share here.

As far as I understand the "Recombination RGB" is just an automatic combination of the three channels at the end of the run. It shouldn't make any differences unless you process the individual channels before recombining them.

 

[fruciak]

Hi Nico, yes I ran gradientcorrection with default parameters and automatic convergence on R, G and B separately before combining them. I am running WBPP with full options (RGB and R, G, B as separate channels with RGB recombine at the end, both with drizzle integration x2 and 0.9) and will share the results, to have outputs without any bias.
As fas as I have seen up to now the outcomes from drizzle integration with CFA and very similar to the RGB combination of drizzle integration of R, G and B channels separetely. We will see if the GC ran on R, G, and B before combination is the origin of the difference between the two images shared in my previous post.
stay tuned, Frédéric

 

[fredvanner]

I ran gradientcorrection with default parameters and automatic convergence on R, G and B separately before combining them.

This is almost certain to degrade the performance of SPCC after recombination. You should do as little processing as possible on the separated channels.

 

[cloudbait]

This is almost certain to degrade the performance of SPCC after recombination. You should do as little processing as possible on the separated channels.

The developers of the tool also recommend against using it before the channels are combined.

 

[fredvanner]

If you are capturing at low elevation, with a possible risk of atmospheric dispersion, splitting and aligning the colour channels separately can mitigate the resulting effects. Otherwise, there is little or no benefit in separating the colour channels.

 

[fruciak]

Hi all,
you are right I forgot this good advice to perform SPCC as the very first process on RGB so obviously true not on separated channels.
At the end I ran WBPP with everything produced in a single run ie RBG + separate R G and B + recombination + drizzle x2 and 0.9 for every stream. Here are the two drizzle integration masters at the end one coming from RGB and the other coming from R G and B treated separetely and recombined at the end. These masters look incredibly better than the non drizzled one (I dithered each 300s frame).They look almost exactly the same except that the recombined master shows a different background and gradient that the RGB one. I imagine that this is coming from the local normalization done channel per channel that is giving exactly the same result as the one done on a single RGB image. M5 is indeed captured at low elevation that is why I thought it would give me a better image without atmospheric dispersion without introducing a dedicated equipement in my optical path.
At least I know that when separated channels will be worth it I can just ask WBPP to perform drizzle integration on them and recombine at the end with the exact same result
Thanks for the discussion and learnings, Frédéric

 

[nico1038]

I can see a very slight amelioration in the separate RGB+recombination over the "normal" drizzle x2 master. The lateral chromatism is corrected on some stars and they are a little bit more symmetrical. But both masters are indeed very similar. And if you correct the gradient and color calibrate them they look very good.

The only thing I can see (in both files) is a very slight pinkish artefact at the center of some stars.

I wonder if a Drop Shrink of 1 would correct this?

Both these masters are better in my opinion than the one where you did the drizzle integration yourself for the individual channels. With this master, once calibrated by SPCC, the colors are not the same and don't look right. I believe that's because of the CFA drizzle which is not done in this case.

I think that example shows that we can trust WBPP to do separate RGB + drizzle. That doesn't really explain what you noticed in the first message but I'm not sure the visualization of the individual R,G,B files is meaningful in this situation. They are intermediate files and conclusions must only be drawn from the final master, once the channels are recombined.

 

[fruciak]

Hi Nico, thanks for taking the time to download this huge files and look at them. I agree with you regarding the pinkish artefact at the center of some stars, fortunately one can not see them at normal zoom and I can use my 10h on M5. I was wondering if this is not a consequence of the 300s subframe exposures with some saturated pixels (see histogram of a subframe below) I read that saturated stars could create artifact issues with some algorithm when it comes to value exceeding 1 and the way they manage it in various channels and so on. There is an old post on PI in PI forum identifying a bug (since corrected) with dark hole in the center of saturated stars ;-)

 

[fredvanner]

I read that saturated stars could create artifact issues with some algorithm when it comes to value exceeding 1 and the way they manage it in various channels and so on.

I believe that you are correct. It should really be possible to fix this (I regard it as a bug).

 

[cloudbait]

I believe that you are correct. It should really be possible to fix this (I regard it as a bug).

Given tools to identify stars and fit their PSFs to mathematical models (tools that appear to be well developed in PI) there's no reason not to have a correction process that rebuilds saturated stars. I was doing this 30 years ago with my own tools.

 

[fredvanner]

I'm sure that this is not caused by saturated stars. I'm sure I have seen these artefacts in stars much lower than the maximum brightness in the image (I suspect the artefacts in the post #16 image fall into this category). These artefacts are already visible in stars of very low brightness in the uploaded images. I would like to compare the image to the non-drizzled image from the same data.