Fix debayered saturated stars?

[TinySpeck]

I shoot astrophotos with a 14-bit DSLR.  My raw images are therefore constrained to 0 - 0.25 in PixInsight data space.  My exposures and ISO are set to bring my histogram humps up about 10% from the left side, so my photo noise is separated from my camera noise.

When I do this, there are usually 20 or so bright stars in the image whose cores are saturated.  Right after debayering I see magenta cores with R, G, or B pixel values well above 0.25.  So saturation and wonky data are happening way early in the process.

Because the saturated cores are artificially well over 0.25 due to debayering, they limit the dynamic range available during stretching.  I want to limit the saturated cores while still linear so stretching can work with its optimum dynamic range.

I've tried the Repaired HSV Separation script, and it helps but does not work very well.  It still leaves saturated cores with artificial data.  It also puts a break in the image history since it creates a new image rather than updating the original.

I've also done HDRComposition using subs with a wide range of exposures from 100 s down to 3 s.  This works better, although there are still saturated cores at 3 s.  It's also a big pain.  The results are nice though: stars which simply show their color almost to the center of their core.

What I want is to merge the linear pixel data from just outside the saturated cores inward to fill the saturated core.  This would result in just about the same thing HDRComposition does, but would be a lot easier and less time consuming.  It would also work with images where HDR short-exposure subs weren't taken.  I can't find any PI processes or scripts to do this, though, and can't think of a way to coax it out of the ones I find.

Any ideas?  Thanks for any help.

[pfile]

i stopped processing OSC regularly images a long time ago, but i kind of hit this problem when working on DSLR images from the 2017 eclipse.

i wonder if all of this gets easier if you use Rescale to 'expand' the image so that the things that are saturated at 0.25 get remapped to 1.0. that way you don't have this funky "saturated but not saturated" data in your linear master. doing this may cause the repairHSV script to work a little better.

as to your actual question though, i think it is somewhat difficult. morphological transformation sort of does what you are asking about based on the mode, but i think you'd have to do a lot of masking and/or working on a duplicate image and then only copy back the pixels that correspond to the saturated pixels in the original image. not super helpful, sorry.

rob

[TinySpeck]

Thanks for the reply, Rob.  I think I really do want to limit that out-of-range data instead of rescaling to it.

I've had vague notions along the lines of your MT suggestion too, but haven't gone there.  It may not be too bad and I'll give it a try.

Oh, another thing I have tried is using DefectMap, which also doesn't work.  For some reason it replaces the saturated cores with something completely wrong.  I'm not sure why.

[ngc1535]

I demonstrate a technique in my lessons to take care of problems like this.
The basic idea is to go ahead and do a linear stretch on the image to make stars look nice in the center (like you mention... this is the data you will be filling with).
Now black clip this image so that only the brightest stars are visible.
Finally blend your working color image with this star image using the SCREEN blending mode. There is both a pixel math formula for this as well as the "Blend" script that Hartmut Bornemann made.
The screen blending will ignore black pixels- so star brightness will fill in just as you want.
As an option, you can slightly blur the star image for a better blend.

Let me know if you attempt this.
-adam

[TinySpeck]

Thanks, Adam.  You've mentioned some things I haven't tried, but I'm not sure this will do it.  First, when I stretch with MaskedStretch or ArcSinhStretch my cores DON'T come out right, that's not the color I want to fill with.  Other stretches might avoid this; I don't know.  But I want to do the saturation false data fix before any stretching so stretching will have the proper dynamic range to work with.

I'm not sure what you mean by "blend your working color image with this star image".  Is the working color image the original linear image?

There may be a way to get what I want using blending, though.  I'll play around with that -- thanks.

[ngc1535]

Thanks, Adam.  You've mentioned some things I haven't tried, but I'm not sure this will do it.  First, when I stretch with MaskedStretch or ArcSinhStretch my cores DON'T come out right, that's not the color I want to fill with. "

Correct. I usually just simply stretch (lower the white point) of an RGB image. This will create correctly colored and bright "cores" that you will use to fill in your working color image (the one with the problems).


 Other stretches might avoid this; I don't know.  But I want to do the saturation false data fix before any stretching so stretching will have the proper dynamic range to work with.

My recommendation is to fix your stretched working image. It is perhaps possible to do this in the linear domain- but the problems stars are expressing a non-linear issue fundamentally. So I suspect you will not find good solutions.

I'm not sure what you mean by "blend your working color image with this star image".  Is the working color image the original linear image?

See above. The working color image is the one with the bad cores. I am recommending making an image with only "good" cores from a linearly stretched version of your image (before any kind of other stretches). This is prepared by making visible only the threshold of star cores that are required (the brightest stars) and black clipping everything else. Then you blend this thing with your working color image. viola (at least in instances I have this issue).

Even if it does not work for you in this case... it is a very good tool to know/have.
 
-adam

There may be a way to get what I want using blending, though.  I'll play around with that -- thanks.

[TinySpeck]

Correct. I usually just simply stretch (lower the white point) of an RGB image. This will create correctly colored and bright "cores" that you will use to fill in your working color image (the one with the problems).

I just tried this a few different ways, using HistogramTransformation on a preview which included just a saturated star.  No matter what I did with the midtone or white point the saturated core was pushed toward (1.0, 1.0, 1.0), not the color immediately surrounding the saturated core.  By the time the "magenta" of the saturated core disappeared I was at (1.0, 1.0, 1.0), full white.  Am I doing something wrong?

Thanks for the explanation on the method, it's clear now.  If I can get the proper color for my saturated cores that does sound like a good technique, although I still think that limiting the linear data properly would allow a better stretch.  I'm also trying to write a script to do this in the linear domain but finding it impossible to get useful information on the PixInsight scripting classes.

[TinySpeck]

I just posted a script to do the fix the way I've been trying.  It does a decent job and results in more natural-looking stars and the full dynamic range for stretching.  You can find it here: https://github.com/CarpeCimex/ColorClip .