Auto clip for histograms doesn't work right for green channel

[Nocturnal]

Hi,

I was experimenting with auto clips for the histogram module and the colors went way out of whack. The green channel was shifted way to the left, clipping the peak. So I tried the individual channels and found that while R and B work fine, G does not. It puts the black point way too far to the right.

This was with a 32 bit float RGB FITS file.

Could you please take a look?

[Juan Conejero]

Hi Sander,

I have verified this and it seems to work well. I've revised also the corresponding source code (HistogramTransformationInterface.cpp in the IntensityTransformations module; it's included in the PCL distribution) and everything looks ok.

The problem may be that histogram autoclipping has surprising results sometimes. For example, if the image has a lot of pixels in the midtones and highlights, it may happen that a 1% clipping clips part of the main histogram peak. And if there are significant differences between channels, a single automatic clipping percentage may not work well for the three channels. Note that this is irrespective of the fact that the three histogram peaks are aligned or not.

These are completely normal results, which just demonstrate that automatic histogram clipping based on percentages of pixels isn't a good general solution. This leads to a new discussion: can we implement a different (i.e. more "intelligent") histogram clipping strategy? Yes, of course: consider this PixelMath expression:

Code: [Select]

k = 3;
iif( $target < Med( $target ) - k*StdDev( $target ), 0, $target )

This is a simple kappa-sigma clipping strategy. It cuts the histogram at a point located k times the standard deviation of the image to the left of the main histogram peak (assuming that the histogram is single-modal). To apply this expression with PixelMath, you must define k as a variable and ensure that the "Rescale result" option is enabled if you want to redistribute linearly the histogram after clipping, disabled otherwise. Of course, nothing stops you from defining different values of k for each channel (disable the "Use a single expression" option and copy/edit the expression on each channel slot).

Hope this helps

[Nocturnal]

Hi Juan,

you know me, I won't give up so easily What I saw was wrong or at least completely unintuitive. When using the 'auto black point' the green channel was treated very differently from the other two. I will reproduce and upload the image.

[Nocturnal]

Hi,

I don't think I described the problem accurately. The problem appears to be with 'auto zero shadows'. The way I expect this to work is that it throws away all pixel values on the left (dark) that are not being used at all. That doesn't appear to be happening with the green channel. I've uploaded a small screen capture video:

http://cid-a93625fef5ca95fb.skydrive.live.com/self.aspx/PixInsight/pic|_001.wmv

and the file that I used:

http://cid-a93625fef5ca95fb.skydrive.live.com/self.aspx/PixInsight/Horse-crop.FTS

Note that I show the zero clip first and then the auto clip. The Auto clip seems to move the black point very far to the right (I would not expect it in the peak) but that's harder to determine without knowing what percentage of pixels fit into that bucket (see my enhancement request posted earlier where I ask for that feature).

I hope this clarifies what I saw

[Nocturnal]

Juan,

please have a look at this and comment. I took the time to create a video so it should be easy to see if I'm doing something wrong or if there's a problem with the software.

[Juan Conejero]

Hi Sander,

I have looked at your video (excellent way of showing these things by the way) and downloaded the image. There's nothing wrong with the HistogramTransformation interface; both the AutoZero and AutoClip functions are working as expected.

What happens is that your image has a tail of marginal values at the left side of the main histogram peak. This is quite frequent with linear raw images, and is just noise at pixel values with very low probabilities. These tails are not seen at 8-bit plotting resolution, due to rounding of values, but they are clearly shown at 16-bit.

I have prepared a set of screenshots where you'll see this more clearly:

The histograms of the image shown at 16-bit plot resolution:


After AutoZero clipping:


After AutoZero clipping, red histogram - the left tail is particularly relevant here:


After AutoZero clipping, green histogram - no tail, so the histogram is clipped just at the starting point of the main peak:


After AutoZero clipping, blue histogram - a short tail is also present:


Note that AutoZero has clipped just at the first nonzero histogram count on each channel, as expected. The AutoClip feature also works well. With the default 1% clipping value it indeed clips a small fraction of each main histogram peak. This can be fine tuned with the Auto Clip Setup dialog.

Note also that histograms are always calculated with 16-bit resolution in PixInsight (65536 discrete values). The different plot resolutions available on the HistogramTransformation interface allow you to *graphically* represent the histograms with reduced resolutions by interpolation; however all functions on this interface work with the original 16-bit histogram data, including AutoZero, AutoClip and all pixel counts.

Thank you for your video and cropped image. Let me know if this helps in clarifying things with respect to HistogramTransformation.

[Nocturnal]

Hi Juan,

something strange is still going on. I have a raw image, switch to the green channel and click 'auto zero'. The black point gets moved underneath the peak. I don't see how that can possibly be correct. It works fine for 16 bit but not 256. Please attempt to reproduce exactly what I'm doing without switching to different sample sizes. Then tell me what I'm misinterpreting.

- load the image referenced above
- start histogram process
- set to 8-bit (256)
- switch to green channel
- click 'auto zero shadows'
- black point moves under the peak

I'm having a really hard time understanding that, sorry. True, in 16 bit mode this doesn't happen but that's not what I'm asking about