Drizzle Star Core Artefacts

M

mAnKiNd

New member
Hi, I just started using drizzle integration with my new lens and osc and I'm getting these weird colored holes punched into the cores of a bunch of stars and some hard green pixels in some brighter stars (see screenshot)

Briefly, 200 guided 30s exposures ~25% dithered (perhaps not frequent enough?) in Sharpcap . Calibrated in WBPP with drizzle, then image integration to update drizzle and then drizzle integration with cfa drizzle.

Since it takes ages to go through this process with 200 frames, instead of trying every setting permutation and landing on the right parameters after forever, I thought someone here might be able to help point out the error in my ways - much appreciated.

So far I've tried checking/unchecking clip high range in image integration, choosing scale 1 and 2 and drop shrink 0.9 and 1 in drizzle integration, all with the same artefacts.

Kind regards,
Minos

drizzle artefact.jpg
 
Last edited:
Just to answer my own question, after ~a day of experimenting - I found the culprit. When I reran everything the same way but without cosmetic correction (auto, default, clip high) when calibrating in wbpp, imageintegration and drizzleintegration worked as expected with a good, clean result that was artefact free. Hope this helps anyone who might confront a similar issue. Thanks and clear skies :)

1591719005468.png
 
rugbyrene said:
I've had the same issue and couldn't figure out how to fix it. Thanks heaps for that.
Rene
Click to expand...

Thanks, I do have this behavour too.
Preprocessing of my OSC image was made with WBPP Script. From left to right in my screenshoot:
Normal image Integration, Bayer Drizzle, Bayer Drizzle with 2x showing dark holes, Bayer Drizzle with 2x WITHOUT CosmeticCorrection


Anmerkung 2020-06-15 195745.jpg


Question to Pixinsight experts, is it possible to run CosmeticCorrection without this effect.

Thomas
 
I still haven't gotten my hands on this kind of data yet.
But there are 2 methods of CC. Method one is AUTO...it works on standard deviation which applies to any pixel in the image.
Method #2 uses a master dark frame to identify warm/hot pixels... and these will be not correlated with the centers of stars.
So... I think itis possible to run CC without the effect. I am not entirely convinced it is CC..but I haven't gotten my hands on this kind of data yet.
The above experiment is diagnostic. If you only use the master dark for Hot pixel removal and still get the holes... it is *not* CC.
Would you agree?

-adam
 
Thanks for your reply. I do use AUTO to eliminate cosmics and other stuff. Hotpixels were removed mostly through BIAS and DARKs. I got good MASTERBIAS and MASTERDARKs of the same temperature, length and gain.
Yesterday I gave up with BayerDrizzle2x, because resulting images are too big (12000px width), also BayerDrizzle1x does not improve my images in any case.
Thomas
 
ngc1535 said:
I still haven't gotten my hands on this kind of data yet.
But there are 2 methods of CC. Method one is AUTO...it works on standard deviation which applies to any pixel in the image.
Method #2 uses a master dark frame to identify warm/hot pixels... and these will be not correlated with the centers of stars.
So... I think itis possible to run CC without the effect. I am not entirely convinced it is CC..but I haven't gotten my hands on this kind of data yet.
The above experiment is diagnostic. If you only use the master dark for Hot pixel removal and still get the holes... it is *not* CC.
Would you agree?

-adam
Click to expand...



UPDATE: well i can say it greatly reduces it, but there are still a few stars where the core is blown out


Hi Adam

I can confirm that switching CC from using auto to only using dark master works to eliminate problems at the star cores. If you're interested to see the data and/or results, happy to share.

Brian
 
Last edited:
Hi,

I tried not using CC but I still get the artefacts in the middle of my stars. Any ideas on how I can prevent this from happening?

Rene
 
Just to confirm the pattern. It appears that rejection of any kind (be it by CC or regular rejection) causes this issue.
I do not know the answer- this is one where I think we need Juan to jump in (it is one of the mysteries I have been pinging PI experts about for many months now).

My experiment above was diagnostic in demonstrating that it really is a rejection that is correlated with star centers- but it is rejection in general- CC is only one way. Regular rejection of values, likely due to saturation or near saturation (think ABG response) in the PSF seems to trigger it.

This is something I am looking forward to know the answer to !

So...you can do the brute force thing and turn of rejection...but then you don't get the benefits from it.

-Adam
 
Unfortunately, I cannot reproduce this problem with any of our data sets. Please provide a data set, including raw frames and calibration frames (not master frames) so that we can try to reproduce the problem and hence understand and fix a possible bug. We cannot help without data.
 
Juan,

I believe I can have a quality set sent your way. I am not the owner..but I know the person who is...
-adam
 
 
If I re-drizzle the data as downloaded I get the artefacts. However, if I then re-integrate the data (with drizzle generation) they disappear. I've tried integrating with Winsorised clipping and linear fit rejection with fairly stringent upper clipping limits, but still no artefacts. I wonder what the original integration parameters were?
 
fredvanner said:
If I re-drizzle the data as downloaded I get the artefacts. However, if I then re-integrate the data (with drizzle generation) they disappear. I've tried integrating with Winsorised clipping and linear fit rejection with fairly stringent upper clipping limits, but still no artefacts. I wonder what the original integration parameters were?
Click to expand...
Did you use Auto CC on the data? Either CC some kind of rejection (likely whatever is chosen automatically based on number of frames, could be LinearFItClipping) is the trigger I think. Also Brian included the LogFile which will show what happened.
-adam
 
Last edited:
Here is another user complaining of the same issue today. This is an issue that comes up a few times a week (which for this hobby is quite frequent).
 

Attachments

  • Capture3.JPG
    Capture3.JPG
    180.1 KB · Views: 83
Thank you for uploading this data set. I know what happens:

clipLowRange.png


The WBPP script sets the rangeClipHigh parameter of ImageIntegration to true, as can be seen in line 3704 on the above screenshot. This is the Clip high range parameter on ImageIntegration's interface (in the Pixel Rejection (1) section). This will reject all pixels with values greater than or equal to the rangeHigh parameter, which has a default value of 0.98 (equivalent to about 64224 in the 16-bit unsigned integer range).

With regular integrations this is rarely a problem because in these cases the ImageIntegration process works with interpolated data (interpolated for image registration), where extreme pixel values correlate with close neighbors, and hence tend to be smoothed acquiring lower values. Besides this, ImageIntegration replaces completely rejected pixel stacks with median values, so no 'black holes' can happen.

On the other hand, DrizzleIntegration performs no interpolation of the calibrated raw data and, in order to preserve raw data in a completely 'pure' way, it does no replacement of completely rejected pixels either. So all saturated pixels are always rejected when rangeClipHigh=true in DI. Since bright star cores are saturated in all of these frames, all of them are rejected and replaced with zeros in the final drizzle integrated image.

The inclusion of rangeClipHigh = true in the WBPP script can be interpreted in two opposite ways, and the funny part is that both interpretations are equally valid in my opinion:

(1) As a bug, since this parameter defaults to false and should only be set to true in special cases, mainly for verification purposes.

(2) As a feature, since by setting rangeClipHigh = true you can clearly identify saturated pixels in the generated integration rejection maps.

The integrated image generated by WBPP is just a convenience preview of the achievable image, but it should always be deleted/ignored, and the integration should always be done manually with the registered frames. This is the only way to obtain an optimal integrated image with full control over the normalization, pixel rejection and noise reduction tasks.

Let's repeat this n+1 times, where n already tends to infinity:

The integrated master light frame generated by WBPP should not be used for production purposes. It is just a preview of the image that can be achieved with the calibrated data set, but it is not, in general, the optimal result. Image integration should be performed manually to optimize pixel rejection and signal-to-noise ratio.

Given the way people tend to prefer "click here" automatisms everywhere (which is an extremely wrong attitude towards image processing and towards astrophotography in general—but this is a different topic), our best option is undoubtedly to follow interpretation (1), so we'll change the current behavior by setting rangeClipHigh to its default value of false in WBPP. We'll release this fix in the next update with WBPP 2.0.3.
 
Juan Conejero said:
Given the way people tend to prefer "click here" automatisms everywhere (which is an extremely wrong attitude towards image processing and towards astrophotography in general—but this is a different topic), our best option is undoubtedly to follow interpretation (1), so we'll change the current behavior by setting rangeClipHigh to its default value of false in WBPP. We'll release this fix in the next update with WBPP 2.0.3.
Click to expand...
The problem with people using these “click here” solutions is that many of them don’t understand what is happening. Hence the plethora of problems that are being posted in this forum. I have never seen the need to go beyond the method Vicent describes here https://pixinsight.com/tutorials/master-frames/index.html for generating calibration frames, followed by image integration of the calibrated light frames. Each step in the process can be checked for correctness and it is me rather than the black box that is in charge.
 
Juan's explanation is completely in line with my results.
One myth that should be killed off quickly is that this has anything to do with CosmeticCorrection; the cause is fairly, squarely and completely in ImageIntegration with "Clip high range" enabled. This will always produce black holes in saturated stars following DI with "Enable pixel rejection" selected (or coloured holes in CFA images where not all channels saturate). Many perfectly respectable images include saturated cores for very bright stars, so this is clearly not a sensible default setting, even for a "preview" integration. I would regard it a specialised diagnostic setting - one I've never needed to use yet.
 
fredvanner said:
this is clearly not a sensible default setting, even for a "preview" integration.
Click to expand...
Perhaps I should justify this further.
I believe pixel rejection in ImageIntegration provides a powerful mechanism for statistical enhancement of the image. If a small number of frames in the stack have statistical outliers at a particular pixel, the "majority vote" can overrule them. The underlying assumption is that the outlier data is unreliable (wrong).
At a saturated pixel, all (or most) frames agree that it is saturated - this is the correct value for that pixel ("as right as you can get"). It is, in my opinion absurd to replace this with zero ("as wrong as you can get"), for anything other than diagnostic purposes - and I personally would not chose ImageIntegration as my tool for analysing saturation in my images (it is too complex, and so is open to all sorts of confusion).
If anyone really really wants this diagnostic capability in II, I suggest that you offer an optional "saturated pixel" map output.
 
Last edited:
You must log in or register to reply here.
Back
Top