Color Calibration

[dmcclain]

Not sure what you are referring to RE: flat normalization. The usual way of taking flats is likely okay after all.

What I'm zeroing in on now is the Midtones Transfer Function (MTF) used in the Histogram tool, and presumably in the STF(?). After all, when you copy an STF over to the Histogram, it produces a black clipping point along with a midtones transfer function.

Reading the XISF doc tells us that this MTF is a rational function of order (1,1). My coming along after a stretch with a simple Curves transform of raising or dropping the level at 0.5 by 1/8, turns the combined MTF into a (2,2)  = ratio of quadratics instead of ratio of linears. I can see no apriori arguments either for nor against such an MTF, nor just about any other that you could choose that is smooth and monotonic.

So this is getting back to my query about human perception. There is no rule that states that our vision would follow a (1,1) MTF, as far as I know. So perhaps relying too heavily on the (1,1) MTF is the source of the problem with too much red?

I can either follow the normal STF with a simple 1 point Curve correction that drops Red by 1/8 at x = 1/2, or by boosting the Blue by 1/8 at x = 1/2. The only difference visually seems to be overall brightness of the result. But the increased presence of Blue is seen either way.

And the comment about H-beta being 20% of H-alpha was very helpful. My images show less than 10%, sometimes much less of the blue channel. Still too much red. But that 20% gives me a good target.

[dmcclain]

I tripped over an image color transform yesterday that is easy to apply, but begs for a widget GUI to show previews. The gist of the method is to exponentially scale the color channels. Exponents above 1 produce a sharp rolloff of color with fading intensity in the image, while exponents below 1 produce a boost with falling intensity.

Doing this exponential preserves the dark level and the white level, but allows for considerable variation at the lower intensity levels. An example is shown below. The left frame is the original, color calibrated, image. The middle one is what I was hoping to achieve initially. The rightmost image is really a bi-color version where green was synthesized as the mean of scaled red and blue channels. While the color is way off, it provides better viewing contrast of the dark clouds (IMHO). An artifact of our sensory system...

[pfile]

have you compared with ExponentialTransformation? seems like its stated purpose is to increase contrast in the shadows

http://www.deepskycolors.com/PixInsight/IntensityTransformations.html
http://www.astro-imaging.com/Tutorial/PixInsight/EXP/en.htm

second link is ridiculously old but describes a lot

rob

[Carlos Milovic]

So this is getting back to my query about human perception. There is no rule that states that our vision would follow a (1,1) MTF, as far as I know. So perhaps relying too heavily on the (1,1) MTF is the source of the problem with too much red?

Human vision roughly follows the gamma curve (~ x^n). In PI the gamma curve is not used because the MTF yields a more contrasted look, and hence faint features are "better" represented (in terms of visual impact). It do not tries to mimic the human perception, but to enhance the representation of the data. And, in that sense, the MTF works quite well. An intermediate curve is the logarithmic function (log(x+delta)/(1+delta)).
Despite the function used, I believe that all of them have almost a linear response for low values. So, I would inspect the data while linear to find any differences in the color balance. Also, if you have a neutral gray in any part of the image, after the nonlinear adjustment (MTF, gamma, whatever) it should remain, if you use the same parameters to all the channels. If there is a red cast, then the MTF will probably amplify that due to it's greater contrast (the ration between the channels should remain more or less the same, but with greater saturation due to the higher contrast).
If you want to not change the hue of the image, then here are some possibilities:
- Process the image in any luminance/crominance space that lets you preserv the hue information (HSL, Lch, for example).
- Extract the hue of the image before stretching, and then insert it again after that step.
- Stretch the Value channel (HSV model). It seems that this procedure yields better colors, at least in the stars.


Regarding your equations for stretching, this just seems like a gamma stretch, with a modified green channel. If gch = 0, you are completelly replacing the green information with the one from the red and blue channels. If you are worried about keeping a "documentary" approach, I would not destroy such data in that way.

[dmcclain]

Yes, for a documentary presentation the rightmost image is complete fiction, just better visual contrast for the deep dark clouds.

I will have a try with some of your recommendations. The exponential applied separately to each color channel is really just the same as using the CurvesTransformation tool. I just tripped over it yesterday as I was looking for some modification to an already stretched image that would both preserve the dark level and the white level. The exponential idea came to me upon thinking about gamma, and the likely possibility that our gammas vary with color.

The notion of a variation on MTF came out after finding that at least one other group uses Bezier curves in their midtones transfer function. I like the MTF in PI, especially because of its mathematical properties on inversion. But I realized that the MTF does not particularly respect any manner of our visual system transfer function, i.e., gamma. Once you get the black level and the white level through color calibration, it seems like everybody does whatever looks good from an artistic viewpoint. I was hoping for something a bit more rigorous, but there doesn't seem to be enough knowledge about our visual system do go that way.

What prompted this for me was that left most image, where there is just too much red in the image, and not enough variation to tease out dust scattering from ionized Hydrogen emission, even though it is accurately color calibrated from M31.

[dmcclain]

Your comment about gray levels was very interesting. The deep red image on the left came from using the heart of that big dark blob as my dark reference in BackgroundNeutralization and ColorCalibration. When I use the mid-level neutral patch near the middle of the image as my "gray" reference, the result becomes more like the middle image after color calibration and normal MTF stretching.

So perhaps all that is needed for BackgroundNeutralization is to choose a neutral region, not necessarily the darkest region?

[Carlos Milovic]

I don't think that the lack of knowledge about human vision is the problem. The issue is that human vision is not enough to represent an astronomical image to it's "full potential". We may use just a gamma curve (the same one for every channel), or even try more advanced techniques like Retinex... but at the end this is not about mimiking what our eye/brain does. For most of the people, it is about making the prettier image possible. And for that one have to get into messy things like other non-linear and spatially variant functions. At the end, the focus isn't anymore on our visual system, but on how to represent the data better in terms of visual impact (and this is highly subjective!).

The exponential applied separately to each color channel is really just the same as using the CurvesTransformation tool

Yes. With the Curves you have more control and flexibility over the curve, but you may also introduce some artifacts. But, as far as you don't move the first and last points, you should preserve black and white points. This does not mean that color balance is preserved for the entire image, of course, and the same is true for any other function working with different parameters at each channel.
On a side note, even if you are using the same parameters, it is not quaranteed that a particular function (specially if non-linear) will preserve hue information if applied in the RGB space.

Oh, and as another possible suggestion, you may try the process I wrote called "ImportL" (http://www.astrophoto.cl/Research.html). You may use an stretched luminance with this process to try to get another color rendition. It is based more on the HVS. I used the formulation described by Fattal et al in their HDRC algorithm.

[Carlos Milovic]

Your comment about gray levels was very interesting. The deep red image on the left came from using the heart of that big dark blob as my dark reference in BackgroundNeutralization and ColorCalibration. When I use the mid-level neutral patch near the middle of the image as my "gray" reference, the result becomes more like the middle image after color calibration and normal MTF stretching.
So perhaps all that is needed for BackgroundNeutralization is to choose a neutral region, not necessarily the darkest region?

Yes, BN should be used to achieve a neutral gray in the desired area. Not necessarily the darkest one (a dark nebula may have in reality a strong red cast). BN forces this region to become gray (in the sense of equal mean values, after some pixel rejections).
ColorCalibration, on the other hand, doesnt' force the Background Reference to be neutral gray. It preserves its values.

[dmcclain]

Wow! I finally got significant separation between H-alpha and dust scattering.

The way I achieved it was to take the raw image (left pane below), duplicate it and take the MMT of that duplicate, leaving only the lowest spatial frequencies (in my case I excluded all but the residual layer after a 5 layer separation). I then previewed the mid background neutral region in that MMT image (middle pane), and used that preview as my reference region for BackgroundNeutralization and ColorCalibration. The result after channel-locked STF stretch is the right panel below.

I believe (maybe I'm crazy?) that this is a more accurate representation of the nebula than all those deeply saturated red images. Red is H-alpha (plus a bit of H-beta for a pinkish variant of red), orange is dust scattering.
 

[cs_pixinsight]

Yowza, that's a HUGE difference. 

Just so I'm not missing something, this is an RGB integration and all you did was Background Neutralize and Color Calibrate the left image using the middle image as the reference to get the resulting right image.  No HA was added in this right hand image after the fact.