Leftover "band" after Cosmetic Correction of column defect

[SeabirdNZ]

Hi all.

I recently got some images on a 1m RC with a FLI ProLine 16803, and I'm having troubles with a "band" of pixels next to a column defect with values above the noise floor.
Attached are 3 snapshots at 100% showing a portion of raw luminance, the same after calibration and cosmetic correction (including defect list with white or black columns), and finally a stack of 8 such frames with auto stretch (all in bin 1:1).

I'm no expert and do not know if I can use any other trick to remove or significantly reduce that "band". Any suggestions/ideas ?
By the way, I see that on the colour frames (in bin 2:2) the problem is either very faint if there at all ... 

Thanks in advance,

Ben

[RickS]

A defect covering multiple columns (the single column that's very bad and the adjacent columns that are affected as well) is difficult to fix. Is your data well dithered?  You might be able to write ones or zeros over all the bad columns so they reject out if they don't coincide in all or most of the subs.  Alternatively, you may be able to do some gentle manipulation with PixelMath to get the slightly warm columns to match better (scale the data values to get them close to the median value for the whole image?)

Cheers,
Rick.

[SeabirdNZ]

Thanks Rick, it's a start to know that something is doable. And yes the subs are dithered, not sure if it'll be enough though.

Now, as I said I'm no expert and I need a bit more help to figure out how to implement what you suggest : which tool/script do I use to write values in columns ? Or a formula in PixelMath for that ?
In the meantime I will have a look on the forum.

Lastly, has anyone got any idea why this problem appears on bin1 images but not on the bin2 ?

Cheers,

Ben

[RickS]

Hi Ben,

If you can upload a few subs to Dropbox or similar I'd be happy to see if I can come up with a PixelMath expression to clean them up.

Cheers,
Rick.

[ngc1535]

Hi,

Sorry to barge in on the discussion. I used this chip (and recognized the column weirdness from the image!).
My solution was to make the column for the defect map 4 pixels in width.
The dithering was sufficient to fill the pixels and give a reasonable result. I had to stretch the image to a point I wouldn't be displaying it at to see the left over "glow."


https://www.adamblockstudios.com/articles/DemoPage

The final image was
https://www.adamblockphotos.com/ngc-2997.html

I ended up cropping not because of the glow... but I wanted to minimize the number of blurred diffraction spikes I was looking at.

-adam

[RickS]

We're you using an especially large dither, Adam?

Cheers,
Rick.

[SeabirdNZ]

@Rick : I found a couple of possible interesting solutions on the forum, and will try them before I post any subs for hands-on help 

@Adam : cheers mate, Sergey mentioned that some advanced users of the Chilescope 1m might be able to help indeed.   
I initially did try adding more columns to the defect list in Cosmetic Correction, but the result - at the time - didn't seem satisfying.
I need to sit through the bin1 luminance subs again, try your 4px wide approach, and possibly combine it with PixelMath to slowly smooth out the remaining warm pixels.

Re dithering, I had it set on 7px but more might be required to get around this issue a tad more easily ?

[ngc1535]

We're you using an especially large dither, Adam?

Cheers,
Rick.

No, I do not believe I used a large dither. This is ChileScope (we were both using the same system). I suspect my dither was 8 pixels.

-adam

[SeabirdNZ]

I tried Adam's 4 pixel idea, but tweaked it a bit after some trials.
I also ran the images through a PixelMath formula found here, after integration : pixinsight.com/forum/index.php?topic=6846.0

Attached are snapshots of the results, first one is the Cosmetic Correction with more than just the black column, and the second one is the same after PixelMath.
I manually stretched on STF to get the background and surrounding galaxies at a half decent level, rather than autostretch.

Rick, I will see about making the subs available. How many do you need, and pre-calibration, post, etc ... ?

[ngc1535]

I tried Adam's 4 pixel idea, but tweaked it a bit after some trials.
I also ran the images through a PixelMath formula found here, after integration : pixinsight.com/forum/index.php?topic=6846.0

Attached are snapshots of the results, first one is the Cosmetic Correction with more than just the black column, and the second one is the same after PixelMath.
I manually stretched on STF to get the background and surrounding galaxies at a half decent level, rather than autostretch.

Rick, I will see about making the subs available. How many do you need, and pre-calibration, post, etc ... ?

Hmmm... We really should be getting roughly the same result...it is the same chip!
You apply Defect Map immediately following calibration and CC but before registration correct?

-adam

[SeabirdNZ]

Sorry Adam, there might be a small confusion here : I do not use Defect Map, I use the defect list in Cosmetic Correction.
I searched how to use the DM, but since both bias and dark do not show this "band" of warm pixels (only the bin1 luminance) I don't see the point ... the DM won't correct for it, right ?

[ngc1535]

Sorry Adam, there might be a small confusion here : I do not use Defect Map, I use the defect list in Cosmetic Correction.
I searched how to use the DM, but since both bias and dark do not show this "band" of warm pixels (only the bin1 luminance) I don't see the point ... the DM won't correct for it, right ?

There is some confusion. A couple of things.

1. Both the bias and dark frames show the primary column issue strongly. I can see evidence of "warm" pixels to the right of the primary column in the dark master frame I produced. It just isn't obvious because of the noise.

2. I am not using DefectMap on the master dark or bias. I am applying it (with this single 4 pixel wide column) to the individually calibrated frames (before registration). I use the Auto method of CosmeticCorrection to take care of hot pixels beforehand (as part of BPP) and DefectMap afterwards using an image container for the column.

3. I do not know (nor could I find) the method of pixel substitution that CC uses if you use its column map. I would have guessed you would have gotten the same answer I did... but apparently this isn't true. In DefectMap I used median as my replacement value with a square kernel. I don't know the method that CC uses.

So, you might want to give my method a try- in a rational world with deterministic machines..you should get the same answer I did.

In my result- looking at just a single frame- the 4 pixel wide  column has a perfect replacement... even zooming in I can't easily see it. The band of warm pixels though exists even in a single frame. This band subtends around 100 pixels to the right of the column. No initial replacement will take care of this issue- and I doubt you can model it well enough from single frames. Only by creating a combined image can this signal really be seen (and it isn't too bad). From that point- there are some simple pixel math expressions that can be employed to help minimize it further. I didn't because in my case it didn't really affect the result.

-adam

[SeabirdNZ]

Thanks again Adam for your help. I'm running out of spare time to dive deeper into this issue, but I'll definitely come back to it and try to find a way to correct T1's images. It would a shame not to use such a beautiful scope