2 strange artefacts introduced by WBPP

[rfrancis]

I recently got a camera with a much larger sensor (IMX461) than anything I've used before and I'm seeing strange artefacts in the frames processed by WBPP (which are not in the originals). This is WBPP v. 2.7.0 and I'm still on Build 1597 of 1.8.9-2.

I've attached a JPEG of a screenshot of one subframe from Blink (I haven't attached the file itself as it's 460Mb).


The first artefact, the black horizontal line about 20 pixels wide, appears in the calibrated subs. It affects about 10% of them, sometimes with two lines.

The second, the grid-like change in intensity, appears in all of the registered subs.

These artefacts are not immediately apparent in the integrated images but presumably do degrade them. But I don't know what's causing them nor how to get rid of them.

 

[bulrichl]

The grid-like artifact might be a moiree effect that occurs when a registered image is viewed when zoomed-in. Please take a look whether this effect is also present in zoom 1:1.

The black horizontal line artifact cannot be caused by correct image calibration. In order to analyze the issue, one of the effected calibrated light frames, the corresponding raw light frame and the master calibration files used for image calibration are needed (upload the files to a file hoster and post the shareable link).

Bernd

 

[rfrancis]

The grid-like artifact might be a moiree effect that occurs when a registered image is viewed when zoomed-in. Please take a look whether this effect is also present in zoom 1:1.

I think you're probably right. It's difficult to say at 1:1 (because the image is so big compared to my screen), but I could not see any evidence of the pattern.

For the cal effects, I have uploaded the file of the same example in my first post, together with the relevant cal frames. I also added the registered image for good measure. Here's the link: https://www.dropbox.com/scl/fo/prgs...OLIcrjP4?rlkey=ry3zitrgepdszrgisatzeet2d&dl=0

Bear in mind that the 5 files in this link come to over 3.1 Gb, so it might be a while before they're all uploaded from my computer as we live in a bit of an internet backwater (it goes with dark skies).

Thanks,
Richard

 

[pfile]

sometimes the moire pattern is real, and it is caused by zero-valued pixels (actually negative pixels, see below) in the calibrated subs, which are then re-aligned and this pattern appears when the aligned frames are stacked. the zero-valued pixels are most likely to occur when you are shooting narrowband from a dark place - in this case there's very little signal in the background of the image, and so the background is basically like a dark frame. just due to the natural variation in dark current from frame to frame, the dark master can oversubtract some pixels in each light.

the solution is to use a pedestal when calibrating, so that these pixels never become negative - since negative pixel values never make sense, any negative pixels are truncated to 0 by ImageCalibration. adding the pedestal guarantees that all pixels in the calibrated frame are nonzero.

rob

 

[bulrichl]

Hi Richard,

you uploaded one raw SII frame, the corresponding registered frame, MasterBias, MasterDark and MasterFlat.

The registered frame was generated in WBPP by application of ImageCalibration, LinearDefectCorrection (LDC) and then StarAlignment. Whereas LDC can be helpful for certain CCD sensors that show column defects, it is generally not applicable for modern CMOS cameras. Probably LDC in your case is creating the black horizontal line artifact.

The registered frame was clipped in the low range. With the present data I cannot determine whether a sufficient output pedestal was applied. One thing is for sure: you'll need a pedestal for your narrowband filtered images.

My recommendations:
In WBPP, disable 'Linear Defects Correction' in the 'Lights' panel for all light frame groups. In the 'Calibration' panel, section 'Output pedestal', set Mode to 'Literal value' and Value (DN) to 100.

---

If these changes do not solve your issue, you should take a look at your MasterFlats. The uploaded SII MasterFlat shows two peculiarities:
1) The top row is very dark (median: 199 DN, compared to 18346 DN for the complete image).
Since the MasterFlat was created with WBPP, I suppose that ALL raw SII flat frames contain this dark top row. However, neither the raw light frame nor MasterBias or MasterDark show this artifact. This suggests that the flat frames were captured with a different camera driver or image acquisition software. If this is true, I strongly recommend not to do this in the future.

2) The 'Rejection low' map of the SII MasterFlat shows four bright horizontal lines at:
y = 607 - 628 (height: 22 pixels)
y = 2967 - 2988 (height: 22 pixels)
y = 4585 - 4607 (height: 23 pixels)
y = 8698 - 8719 (height: 22 pixels)
I cannot account for this observation.

Bernd

 

[rfrancis]

Hi Bernd,

thanks for this detailed analysis and way forward, which I am implementing now.

You are completely correct that my cal frames were made with another software (TheSkyX), while I am now using KStars/Ekos. I am, at this moment, collecting another set of cal data with KStars/Ekos.

I activated LinearDefectCorrection (not knowing its background) as I have some weak horizontal structure in the noise of (calibrated) images. I thought it might help. I will disable it.

Thanks again. I am very appreciative of the time you have spent on this.

Richard

 

[bulrichl]

I activated LinearDefectCorrection (not knowing its background) as I have some weak horizontal structure in the noise of (calibrated) images. I thought it might help. I will disable it.

I also noticed some horizontal structures in the result of ImageCalibration when only the MasterDark was applied.

You are completely correct that my cal frames were made with another software (TheSkyX), while I am now using KStars/Ekos. I am, at this moment, collecting another set of cal data with KStars/Ekos.

I am really interested whether the actions that you currently take (all frames, light and calibration frames, captured with the same image acquisition application) will also solve the issue with horizontal structures in individual calibrated frames. I would like to hear about the result.

Bernd