BlurXTerminator AI2 beta

[rcroman]

A beta version of AI2 for BlurXTerminator is available for testing. This update is primarily intended to address the shift in star color occasionally observed with AI1. It should also eliminate the subtle vertical banding artifacts observed in very bright areas (e.g., galaxy cores) on some images. No other intended changes have been made, but the network has received much more training, so other differences may be observed, and in general the results should be more refined.

The main test I've been using to judge this update is to see how much the dispersion of the SPCC R/G and B/G curves changes after applying BXT. Note that this isn't a recommendation to use SPCC after BXT... as noted in another thread, it's not recommended to use SPCC after decon of any kind. This beta version, however, seems to result in much less increase in dispersion than AI1 or classical LR decon.

I'd be interested in feedback on 1) have the star color shift and vertical banding been successfully addressed, and 2) are there regressions (worse performance) in any other areas. Again, no other intentional changes have been made, so this version probably won't handle excessive coma, guiding errors, etc., any better than AI1 does.

To beta test, download the network definition file corresponding to your OS:

• Windows/Linux
• MacOS

You can place the file anywhere, but the most convenient location is PixInsight's library directory. Select the file using the "Select AI" button in the BXT module. If you place it in PI's library directory, BXT will default to it if you click the reset icon, but you can still select AI1 manually if needed.

 

[fmeschia]

Thanks Russ!
I was just processing an image, and I immediately tested AI2 beta, comparing AI1 with AI2.
With the same settings, AI2 seems to do a better job at shrinking stars and preserving colors. The non-stellar details, though, seem to be less sharpened than with AI1. It’s subtle but it’s there. I am attaching a screenshot that shows this. The left portion is processed with AI1, the right portion with AI2.
Francesco

 

[herman]

Thanks Russ for the early look at this. I am observing similar results to what fmeschia posted. Improved color preservation but slightly softer non-stellar. The attached examples were executed right after SPCC.

 

[nico1038]

Thank you Russell,

I did a quick test with the new AI regarding the color shift.
Here is what I get on my test samples (taken with an ASI2600MC camera):

SPCC > Stretch > Saturation

SPCC > BlurX 1 (default settings) > Stretch > Saturation

SPCC > BlurX 2beta (default settings) > Stretch > Saturation

There is a very clear progress with the new AI but it's still not perfect and there is still a slight red shift on some of the stars.

I will continue my test with other images.

 

[kring]

Thanks, I gave it a whirl with good results. stars looked much better than v1 AI.

side-benefit, a noticeable speed improvement, v1 processing went from 17seconds to v2 in 11 seconds, ~60% performance improvement on GPU.

v1 - 17 seconds

v2 - 11 seconds

Nice work!

 

[limburg11]

Hi Russ and thanks for the update!

Here the results on my MacBook Pro M1 Max (MacOS Monterey 12.6.1), where BXT shows severe color posterization in the cores of bright stars.

Top left: Linear image (unstretched, no STF as well)
Top right: After BXT AI1
Lower right: After BXT AI2

After an arcsinh stretch it looks as follows:

The ugly star cores remain visible. The halos are not affected by posterization, but there is still a color shift (I think that's the same observation that others reported).

So, it's different, but the posterization problem (Mac specific) still exists.

I didn't test the other performance criteria. As of speed, here AI2 took 10 % longer than AI1.

Bernd

PS: Don't get me wrong if I mentioned only my (minor) issues with BXT, this is really a great tool, and together with Russ' other XTerminators Noise and Star it's a game changer IMO...

 

[fmeschia]

Yes the macOS posterization effect is due to the 16-bit width of the model data, as Russ wrote in the documentation. No amount of training will fix that.

 

[rcroman]

I'm currently looking into enabling float32 precision on MacOS. Looks do-able, but it will most likely require MacOS 12 or later. I'll need to provide both options so users that are hardware-limited to older MacOS versions aren't left completely in the cold.

 

[AstroShed]

superb tool Russ
I have noticed that the star shape correction in the corners of my images is not as good, I know this is not primarily the intended purpose, but it was a real good feature, also it corrects the stars better in the correct only mode, than it does on the default mode, is this intended…?
and it does not work well on small stacks of images, 20 x 5 min, as it does on much larger stacks like 60 x 5 mins….but that was the same on the previous version….

 

[rcroman]

superb tool Russ
I have noticed that the star shape correction in the corners of my images is not as good, I know this is not primarily the intended purpose, but it was a real good feature, also it corrects the stars better in the correct only mode, than it does on the default mode, is this intended…?
and it does not work well on small stacks of images, 20 x 5 min, as it does on much larger stacks like 60 x 5 mins….but that was the same on the previous version….

If the aberrations in the corners get larger than the limits I used during training (about 9x star radius), correction won't work. Correcting first definitely works better on some images.

 

[fmeschia]

The improvement in colors seems to be somewhat dependent on the image. Here’s an example for which AI1 and AI2 both seem to give the same alteration of colors:

 

[fmeschia]

On the other hand, I found that extracting the H component before applying BXT, and then restoring it, somewhat restores the colors while retaining the BXT improvements. Maybe it would be worth thinking of adding the ability to apply BXT to luminance only, as available in the Deconvolution process? Or automatically restore H?

 

[rcroman]

Thanks Russ!
I was just processing an image, and I immediately tested AI2 beta, comparing AI1 with AI2.
With the same settings, AI2 seems to do a better job at shrinking stars and preserving colors. The non-stellar details, though, seem to be less sharpened than with AI1. It’s subtle but it’s there. I am attaching a screenshot that shows this. The left portion is processed with AI1, the right portion with AI2.
Francesco

I am thinking that this is the "more refined" bit due to more network training.

When training this network, it starts out trying to enhance detail in a fashion not unlike an unsharp mask operation. As training continues, it gets more sophisticated and starts approximating a true deconvolution better and better. More training of this AI2 beta version has probably increased the deconvolution accuracy over AI1. The results posted so far seem to be consistent with this theory.

 

[rcroman]

On the other hand, I found that extracting the H component before applying BXT, and then restoring it, somewhat restores the colors while retaining the BXT improvements:

The network is probably trying to correct for uneven FWHM in the color channels. This is a tricky proposition since it can be difficult to distinguish between this and actual star color differences, especially with saturated (clipped) stars. This is the main reason I'm adding a luminance-only mode in 1.1.0: this will result in zero change to color information.

Note that the luminance-only mode won't be able to make any "cross-channel" corrections, i.e., corrections that depend on the differences between color channels. Chromatic aberration is a prime example.

 

[fmeschia]

I am thinking that this is the "more refined" bit due to more network training.

When training this network, it starts out trying to enhance detail in a fashion not unlike an unsharp mask operation. As training continues, it gets more sophisticated and starts approximating a true deconvolution better and better. More training of this AI2 beta version has probably increased the deconvolution accuracy over AI1. The results posted so far seem to be consistent with this theory.

Agreed. In the picture I posted today with AI1 vs. AI2, the AI2 rendition looks more “natural” (for lack of a better word), consistent with what you’re saying about more training.

 

[rcroman]

Version 1.1.0 of the BlurXTerminator process module is ready for beta testing. This is a significant update with the following changes:

• Added a Luminance Only mode to ignore color information during deconvolution
• Improved consistency of aberration correction near the edges and corners of an image
• Enabled 32-bit neural network computation precision on MacOS. Requires MacOS 12 (Monterey) or later. "Float16" versions of the AI files will continue to be distributed with official releases for compatibility back to MacOS 10.15 (Catalina).
• Updated the documentation to reflect the above changes.

To join in the testing, edit the repository information in PixInsight to point to the test repo: https://www.rc-astro.com/BlurXTerminator/PixInsight-test

NOTE: This test repo will go away once the official release is made, and you'll need to edit away the "-test" bit. The test repo may also get updated at random times, and make break briefly while signatures and checksums are being edited.

For MacOS users, the repo now includes "float16" and "float32" versions of AI1 and the beta2 version of AI2. MacOS 12 or later will automatically select the float32 "mlpackage" versions when using the Select AI button or clicking the reset icon. Earlier MacOS versions will automatically choose the float16 "mlmodel" versions.

 

[fmeschia]

Thanks Russ! I’ll download and test it immediately. I’ve just got a new M1 Max MBP, I’m curious to see if there’s any difference in performance going from 16-bit to 32-bit floats (I understand the Neural Engine won’t work on 32-bit floats, and the GPU will take over).

 

[rcroman]

Thanks Russ! I’ll download and test it immediately. I’ve just got a new M1 Max MBP, I’m curious to see if there’s any difference in performance going from 16-bit to 32-bit floats (I understand the Neural Engine won’t work on 32-bit floats, and the GPU will take over).

I tested on a Mac Pro with an AMD GPU and on an M1-based MBP. Oddly enough, I saw very little difference in speed between running the float16 and float32 versions on either machine. Perhaps even a bit faster in float32 mode. This may come down to the "mlpackage" network definition vs. the older "mlmodel" method.

 

[fmeschia]

Indeed the 32-bit float model is ~20% faster on my MBP. Using Instruments I can see the CoreML load shifted entirely on the GPU, and probably it ends up being faster for me because I have 32 GPU cores vs. 16 ANE cores.

As for the quality of the result, I don't see any quantization problem anymore! Now I'm looking forward to see SXT go float32

And the "Luminance Only" check box works perfectly, too!
Thank you so much for these enhancements, they're really useful.

 

[rcroman]

Indeed the 32-bit float model is ~20% faster on my MBP. Using Instruments I can see the CoreML load shifted entirely on the GPU, and probably it ends up being faster for me because I have 32 GPU cores vs. 16 ANE cores.

As for the quality of the result, I don't see any quantization problem anymore! Now I'm looking forward to see SXT go float32

And the "Luminance Only" check box works perfectly, too!
Thank you so much for these enhancements, they're really useful.

Great! Yep... SXT and NXT are next in line for updates. The BXT project resulted in a number of techniques that can be applied to the other tools.