PixInsight 1.8.0 Ripley: 24-bit STF LUTs

Juan Conejero

PixInsight Staff
Sep 2, 2004
8,380
467
57
Valencia, Spain
pixinsight.com
Hi all,

One of the most important new features introduced by the new version 1.8.0 of PixInsight is 24-bit screen transfer function lookup tables. Before I put several examples to show how 24-bit STFs work in practice, let me explain briefly what this stuff is all about.

A lookup table (LUT) is a list of precomputed function values at specific points of interest. For example, suppose that you need to evaluate the same function y = f(x) repeated times to perform a given task, but you know in advance that an 80% of function evaluations (for example) will be done, on average, for a reduced set of 1000 particular values of the independent variable x. If you precompute those 1000 function values and store them in a fast-access table (such as a random-access array in memory), then you can transform your function as follows:

Code:
   if x belongs to the set of 1000 precomputed values, then
      look for the corresponding table element, which is the value of f(x).
   else
      compute f(x).
 

Transformed this way, your task can be executed much faster because you replace an 80% of function evaluations with fast table lookups, which can be done in constant time. In fact, if your f(x) is relatively expensive, your task will run approximately an 80% faster thanks to this trick. This scheme is a special form of hash table, and is the basic concept behind LUTs.

In its simplest and most efficient form, a LUT is used to discretize a function for a finite set of integer values of x. This is just what PixInsight does to represent some types of images on the screen. Since the screen (as seen by a software application) is an 8-bit device, each screen pixel can only take 28=256 different values for each color, from 0 (black) to 255 (white). For RGB color images, this makes up a set of 23*8=16,777,216 representable colors. For example, to represent a 16-bit integer image on the screen, we have a precomputed LUT of 65536 (=216) 8-bit values, where each LUT entry is equal to its table position divided by 257 (=65535/255) and rounded to the nearest integer:

round(0/257)=0, round(1/257)=0, round(2/257)=0, ..., round(65535/257)=255.

This allows us to render a 16-bit image much faster because we can replace a floating-point multiplication (followed by a rounding operation) by a simple table lookup operation. For other pixel sample formats a LUT is impractical in this context; for example, for 32-bit integer images the same scheme would require a LUT of length 232, which would occupy 4 gigabytes.

A screen transfer function (STF) is just a histogram transformation that PixInsight applies to each pixel of an image to represent it on the screen. This allows us to see how a linear image would look like after a nonlinear intensity transformation, but without changing the actual pixel values. STFs are an essential component of PixInsight's graphical interface because they make it possible to work directly with linear images, which has important advantages in many cases (background modelling, noise reduction) and is absolutely necessary in others (color calibration, deconvolution). The most computationally expensive part of the STF is a midtones transfer function (MTF). The MTF isn't too complicated: if properly implemented, it requires two multiplications, one division, one sum, and two subtractions for each pixel, all of them floating point operations. This may seem quite cheap, but if we have to repeat it several millions of times to perform a real-time task such as rendering an image on the screen, then we have a problem. As you probably have figured out, to perform this task in real time we use a precomputed LUT to discretize the STF of each image.

In previous versions of PixInsight, STF LUTs were always generated with 16-bit precision, that is, each STF LUT had only 65536 elements. While this is more than sufficient for most well-exposed linear images, it is clearly insufficient in some important cases. When the number of precomputed LUT elements is not sufficient to represent the STF accurately, the resulting screen rendition shows posterization. Posterization leads to complete loss of image detail on certain areas of an image's screen representation as a result of excessive simplification. This happens frequently with high dynamic range (HDR) images, which usually require extremely aggressive midtones transfer functions.

The following screenshot shows an HDR image of the M42 region by Vicent Peris and Jos? Luis Lamadrid:


The same image has been used a number of times on this forum, especially to document new HDR features and tools, For example, I used it to describe the HDRComposition tool when we first released it, back in 2010.

On the screenshot above we have a good example of how linear images look like: black. Of course, the histogram explains why: most of the data are concentrated in a narrow peak close to the beginning of the numeric range. As a result of this distribution of the data, we simply cannot see the image. In the case of linear HDR images, the problem becomes severe: the histogram peak is much narrower, and it is much closer to zero. On the screenshot, note that the histogram is being represented with 999x magnification on the horizontal axis (the maximum zoom factor available on the HistogramTransformation tool, actually).

What happens if we apply a 16-bit STF to this image? Since the required midtones transfer curve is very steep, it requires much more than what 16-bit integers can offer to represent its initial---almost vertical---slope accurately. The result is severe posterization:


Posterization is much worse than ugly screen representations: it simply makes it very difficult (and in some cases completely impossible) to work with these images in the linear stage.

The new version 1.8.0 of PixInsight can generate and use 24-bit LUTs to discretize STFs. With 24 bits we have 16,777,216 discretization points, or 256 times more resolution than 16-bit LUTs. Of course, this also means 256 times more computational work to generate each LUT, and 256 times more memory space requirements (16 megabytes for each color channel instead of 64 kilobytes). This explains why 16-bit LUTs are still used by default, since they are sufficient for most linear images.

With a 24-bit STF, the same M42 HDR image shows no posterization at all:


Do you think that this M42 image is a good example of posterization? I was thinking the same, and was proud of the new 20-bit STFs that were working so nicely, when Vicent uploaded the following "thing":


Look at the histogram to understand why I call this image a "thing". With a 16-bit STF---and exactly the same with a 20-bit STF---, this image simply redefines the concept of posterization:


Fortunately, a 24-bit STF provides a much better screen rendition:


With a 24-bit STF this image still shows a bit of posterization on its darkest areas, but nothing problematic. To be completely free from posterization, this image would require a 29-bit LUT, which is impractical for obvious reasons (it would occupy 1.5 GB for a color image). Here are enlarged views for better comparison:

With 16-bit STF:

With 24-bit STF:

Finally, I want to point out the fact that 24-bit STFs are not only useful for HDR images. They provide much more accurate renditions of weak raw images, where the histogram peaks are comparatively narrow and close to zero, and also for images with extremely smooth gradients, such as background models. I hope you find this new feature useful.
 

mschuster

PTeam Member
Nov 27, 2011
1,137
6
San Francisco
Hi Juan,

Has the midtones transfer function used by STF and HT changed in this new release?

I notice the default STF Shadows clipping value is changed to -2.8.

This is a bit of a concern for me because STF and HT may now give different default results.

Thanks,
Mike
 

mschuster

PTeam Member
Nov 27, 2011
1,137
6
San Francisco
Hi Juan,

Maybe I am mistaken, but STF works very differently now on Win7.

Applying a default STF to an integration in 1053 gives a significantly different screen visualization than that produced by 1023.

Transferring the STF to HT, I see different shadows and midtones values. And of course when I apply the HT I get different results.

When I copy the actual shadows and midtones from 1023 and paste them into 1053 and then apply, I get almost identical results (maybe slight rounding differences?).

Trying to "emulate" the 1023 STF in 1053 is possible but difficult, via a binary search on the shadows clipping value.

I have old projects that sometimes apply STFs to integrations with HT. Those projects can not be rerun easily now.

Also, just having STF give different results on old subs and integrations is a problem IMO.

Thanks,
Mike
 

Juan Conejero

PixInsight Staff
Sep 2, 2004
8,380
467
57
Valencia, Spain
pixinsight.com
papaf said:
Is there a reason why 24bit isn't selected/performed by default? It doesn't seem to matter to "normal" images...
There are two main reasons:

- Most images don't need 24-bit STFs.

- 24-bit STFs require 256 times more memory and computation time.

If you want 24-bit STFs enabled by default, go to Global Preferences > Miscellaneous Image Window Settings, and check the option "Use 24-bit...". Apply the process globally (F6 or click the circle button), and from now on 24-bit STFs will be enabled by default for newly created views.
 

Juan Conejero

PixInsight Staff
Sep 2, 2004
8,380
467
57
Valencia, Spain
pixinsight.com
Has the midtones transfer function used by STF and HT changed in this new release?
There are two changes in the STF AutoStretch function:

- Black and white pixels are now rejected for statistics calculation.

- The median absolute deviation from the median (MAD) is used instead of average deviation as a scale estimator.

I notice the default STF Shadows clipping value is changed to -2.8.
This change is precisely to minimize differences between the new AutoStretch function and the old one. The use of MAD requires a slightly wider clipping range. I have determined this constant by experimentation.

Also, just having STF give different results on old subs and integrations is a problem IMO.
For images without relatively large black and/or saturated areas, the differences should be small, or at least this is the intent. My intention has not been to change the AutoStretch behavior in general, but just to make this feature much more robust. For example, AutoStretch is now immune to black/white pixels, which was a serious limitation in previous versions. The use of MAD also provides more robustness to characterize the true dispersion of the image.

If you really need the old behavior, I can provide a version of the STF AutoStretch routine to emulate it.
 

mschuster

PTeam Member
Nov 27, 2011
1,137
6
San Francisco
Thank you Juan this is helpful information.

The median absolute deviation from the median (MAD) is used
I guessed the scale estimator changed, but guessed wrong: II's default.

This change is precisely to minimize differences between the new AutoStretch function and the old one.
This integration has few zero or saturated areas, yet old/new STFs produce fairly different results IMO. I included both HT's and a notes listing HT values.

https://dl.dropboxusercontent.com/u/109232477/PixInsight/STF.zip

If you really need the old behavior, I can provide a version of the STF AutoStretch routine to emulate it.
Please wait for input from other users before doing this. Knowing exactly what you are doing is enough for me now, as I can emulate the old way myself when necessary.

Also, I agree with Harry. The green STF indicator is very nice to have.

Thank you,
Mike

 

Juan Conejero

PixInsight Staff
Sep 2, 2004
8,380
467
57
Valencia, Spain
pixinsight.com
Hi Mike,

https://dl.dropboxusercontent.com/u/109232477/PixInsight/STF.zip
I have just checked these images in detail. thank you for this example, which is certainly interesting for a number of reasons.

IMO, the STF AutoStretch routine in the new version performs better for this image. In previous versions the result was too contrasted. In the latest version the background at the top left corner is being less darkened, and hence you can visually detect more structures. The same happens on dark nebulae around the center of the image.

Keep in mind that AutoStretch STF is not intended to provide a necessarily 'nice' rendition, but a useful rendition in the sense of revealing the true image contents, to give you an approximate idea of what can be achieved and the problems you'll have to face during post-processing. As an aside, the image you've used in this example is wonderful, by the way.

All in all, this example shows the kind of results that I was trying to achieve with the new implementation. This is an interesting case because the distribution of brightness in this image presents a difficult problem for histogram characterization. MAD is working better IMO because it is applying more stress to the background and other dark regions, precisely where these images are more challenging post-processing wise.
 

mschuster

PTeam Member
Nov 27, 2011
1,137
6
San Francisco
Thanks Juan. My other integrations also look less contrasty by default. This is OK.

Also, IMO, the decrease in contrast is due, at least in part and maybe mostly for my integrations, to the decrease in default Shadow clipping value. avgDev > MAD, so a switch to MAD tends to increase contrast for a fixed Shadow clipping.

Mike
 

dgbarar

Well-known member
Nov 28, 2012
91
0
65
Phoenix, AZ
Hi Juan,

I am delighted that we are able to try out the new 24 bit STF.  Attached is screen shot of Bubble Nebula image that I am working on.  The image is the result of HDRComposition and is 64 bit.  The image on the left is linear and has only dynamic background extraction performed.  The 24 bit STF looks good for processing to this point.  The image on the right is the next step in the process.  In addition to DBE, I have also performed a color correction operation.  The STF on this image is badly posterized.  So far it looks as though the 24 bit STF LUT has not helped.  Your thoughts would be beneficial.

Cheers,

Don
 

Attachments

Juan Conejero

PixInsight Staff
Sep 2, 2004
8,380
467
57
Valencia, Spain
pixinsight.com
Hi Don,

That happens because you have the 24-bit LUT option disabled for the active image (look at the STF tool bar). By default, 24-bit LUTs are disabled for newly created images. You can enable them by default through preferences: Edit > Global Preferences > Miscellaneous Image Window Settings > Use 24-bit ..., although I don't recommend doing this in general, since 16-bit LUTs are more than sufficient for most images.