StarXTerminator 2.0.0 pre-release available for beta testing

Thanks all for the beta testing. I've retired the test repo and released version 2.0.0 and AI11 at the regular repo location. Please re-edit your repository list to point to:

https://www.rc-astro.com/StarXTerminator/PixInsight/

The official version of AI11 has some small improvements with additional training on user-provided data (thanks!), mainly better handling of bright stars with very wide-spread scattered light, such as is seen with certain astrographs (e.g., Tak Epsilon).

One point of interest that came up during testing pertains to images stretched using alternate methods such as GHS. These methods often produce star profiles that are virtually indistinguishable from small elliptical galaxies. The network could be trained to remove them, of course, but then it would zap the small ellipticals. We wouldn't want that – it is STARXTerminator, after all. ?

There are two recommendations if you intend to stretch using GHS or similar methods. The first (and best) is to remove the stars prior to stretching. This approach has the added flexibility of being able to perform different stretches on the stars and starless images. If you don't want to back up that far in an ongoing workflow, the second possibility is to run StarXTerminator twice. In testing it did a reasonable job of picking up any GHS-stretched stars it missed on the first run. Not as good as removing stars in the linear image, but it's worth a try.

If you use the second method and end up with two stars images, you can combine them using PixelMath to 1) add them together if you didn't use the "unscreen" option, or 2) screen blend them if you did.
 
DangerZone2486 said:
Have you tried replacing the tensorflow file?

That's what I do whenever a new version of StarXTerminator comes out.

You can get the tensorflow.dll file from DarkArchon's post about using the GPU for star removal. I have seen some people managing to get the newer versions of tensorflow.dll using the latest versions of CUDA and whatnot, but I'd rather stick with the one I have got, it's still fast and I did manage to get it working because I had already replaced the original tensorflow.dll file whenever a new version of PixInsight comes out.

All you need to do is to just copy the GPU ready tensorflow.dll file into PixInsight's bin folder. It will ask you if you want to replace it, just click yes. You might also get the administration warning too. If you do, just click yes. Then run a test with StarXTerminator AI version 11 and see how that goes. My results were 17.715 seconds using AI version 11.

Let me know if you run into any troubles, and I'm more than happy to help out!

Zak :)
Click to expand...
Hi Zak,

Yes, I had the GPU enabled tensorflow.dll file (V2.6) and it didn't work with AI V11, but ran fine with AI V10. When I switched to the tensorflow.dll that came with Pixinsight everything worked OK, but ran slower.

Later I'll try with newer versions of NVIDIA software and tensorflow files. I'll report my findings.

Thanks a lot for your help.

Francisco
 
fbitto said:
Yes, I had the GPU enabled tensorflow.dll file (V2.6) and it didn't work with AI V11, but ran fine with AI V10. When I switched to the tensorflow.dll that came with Pixinsight everything worked OK, but ran slower.
Click to expand...

AI11 uses some more advanced operations in its architecture than earlier versions. This may be the reason for it not working with previous GPU DLL/driver versions. Updating drivers and DLLs is worth a shot.
 
Just FYI, I did some testing, and here are my results:

RTX3050 mobile

libtensorflow V2.7.0
CUDA V11.7.1
cudnn V8.2.4.15

ProcessResult
StarXTerminator with AI V11Fails
StarXTerminator with AI V10OK
StarnetOK
Starnet V2OK

Then I updated cudnn to the latest version

libtensorflow V2.7.0
CUDA V11.7.1
cudnn V8.5.0.96

And got exactly the same results.

In both cases I'm using the latest GeForce Game Ready Drivers, V516.94

When using the tensorflow.dll file that is installed with Pixinsight, StarXTerminator with AI V11 works fine.

As a side note, NoiseXTerminator works fine.

On my desktop PC, StarXTerminator with AI V11 works fine, but I have a 1660 Super and older tensorflow, CUDA and cudnn versions.

I use my desktop PC for WBPP, etc, as I can leave it unattended thru the night. But my laptop PC is used for postprocessing the stacked images, hence my interest in having GPU acceleration in StarXTerminator.

Thanks a lot for your support,

Francisco
 
Hi,
I changed the folder of the repository and Pixinsight shows me that 2.0.0 an A11 have been installed, but when restart pxinsight nothing change; still the version with the linear data checkbox. I think I do something wrong, any ideas?
 
tomtom2509 said:
Hi,
I changed the folder of the repository and Pixinsight shows me that 2.0.0 an A11 have been installed, but when restart pxinsight nothing change; still the version with the linear data checkbox. I think I do something wrong, any ideas?
Click to expand...
Not sure why this would be. You might try deleting the updates.xri file to stimulate a fresh install of all updates.
 
Just got it working on an RTX A6000 after a bit of troubleshooting and online sleuthing. The key was to download, of all things, the zlib data compression library and place the zlibwapi.dll file in the CUDA bin directory (where you copy the CUDNN DLLs).

This is actually covered by NVIDIA in their installation instructions: https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

Download zlib here: ZLIB DLL

From the extracted download, find the dll_x64 folder and the zlibwapi.dll file within. This needs to be placed in a location that is in your system PATH environment variable so it can be found when needed. Since the PATH variable already needs to be set to find the CUDA and CUDNN libraries, I placed it in the same directory. I'm using CUDA 11.7.1, so the path is:

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin

Before this, SXT would crash PI every time. Afterward, hey presto... blazing fast SXT. ?

For reference, I'm using the 2.9.1 tensorflow.dll, and CUDNN 8.5.0.
 
Last edited:
rcroman said:
Just got it working on an RTX A6000 after a bit of troubleshooting and online sleuthing. The key was to download, of all things, the zlib data compression library and place the zlibwapi.dll file in the CUDA bin directory (where you copy the CUDNN DLLs).

This is actually covered by NVIDIA in their installation instructions: https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

Download zlib here: ZLIB DLL

From the extracted download, find the dll_x64 folder and the zlibwapi.dll file within. This needs to be placed in a location that is in your system PATH environment variable so it can be found when needed. Since the PATH variable already needs to be set to find the CUDNN libraries, I placed it in the same directory. I'm using CUDNN 8.5.0, so the path is:

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin

Before this, SXT would crash PI every time. Afterward, hey presto... blazing fast SXT. ?

For reference, I'm using the 2.9.1 tensorflow.dll, and 11.7.1 CUDA toolkit.
Click to expand...
I'll try that ASAP. I also noticed that there is a newer version, V2.10.0 but I´ll install V2.9.1 and give you some feedback.

Thanks a lot
 
rcroman said:
Just got it working on an RTX A6000 after a bit of troubleshooting and online sleuthing. The key was to download, of all things, the zlib data compression library and place the zlibwapi.dll file in the CUDA bin directory (where you copy the CUDNN DLLs).

This is actually covered by NVIDIA in their installation instructions: https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

Download zlib here: ZLIB DLL

From the extracted download, find the dll_x64 folder and the zlibwapi.dll file within. This needs to be placed in a location that is in your system PATH environment variable so it can be found when needed. Since the PATH variable already needs to be set to find the CUDA and CUDNN libraries, I placed it in the same directory. I'm using CUDA 11.7.1, so the path is:

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin

Before this, SXT would crash PI every time. Afterward, hey presto... blazing fast SXT. ?

For reference, I'm using the 2.9.1 tensorflow.dll, and CUDNN 8.5.0.
Click to expand...
OK, I installed CUDA 11.7.1, cudnn 8.5.0, copied zlibwapi.dll to the CUDA bin folder and tensorflow.dll 2.9.1 to PI bin folder.

No more crashes to desktop. If I delete the zlibwapi.dll file I get CTD every time. So that part is solved.

But I still get the OOM error:

Code: 
StarXTerminator: Processing view: blend
Writing swap files...
1805.486 MiB/s
RC-Astro StarXTerminator version 2.0.0, AI version 11
Tile overlap = 20%
Removing stars:   0%
*** Error: ERROR in ProcessTile(). Error 8: 2 root error(s) found.
  (0) RESOURCE_EXHAUSTED: OOM when allocating tensor with shape[1,48,1024,1024] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
     [[{{node starX/dd1_mconv/Conv2D}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

     [[Identity_1/_5]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

  (1) RESOURCE_EXHAUSTED: OOM when allocating tensor with shape[1,48,1024,1024] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
     [[{{node starX/dd1_mconv/Conv2D}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

0 successful operations.
0 derived errors ignored.
26.762 s

My GPU has 4GB of VRAM, ¿Could that be the cause? I don't see a lot of VRAM getting used when running AI V10 (about 2.4GB)

Thanks a lot.
 
fbitto said:
My GPU has 4GB of VRAM, ¿Could that be the cause? I don't see a lot of VRAM getting used when running AI V10 (about 2.4GB)
Click to expand...
That could be. ? AI11 processes images in 1024x1024-pixel tiles, while earlier versions used 512x512-pixel tiles. The AI11 network is actually smaller than AI10, so it shouldn't be as bad as 4x higher memory usage, but it could push a 4GB GPU over the edge.

I'll look into making a version of AI11 that runs 512x512 tiles. It would be an alternate download, and performance on very large structures (long diffraction spikes, big star glow, etc.) would suffer, but it would probably work well on many images.
 
rcroman said:
That could be. ? AI11 processes images in 1024x1024-pixel tiles, while earlier versions used 512x512-pixel tiles. The AI11 network is actually smaller than AI10, so it shouldn't be as bad as 4x higher memory usage, but it could push a 4GB GPU over the edge.

I'll look into making a version of AI11 that runs 512x512 tiles. It would be an alternate download, and performance on very large structures (long diffraction spikes, big star glow, etc.) would suffer, but it would probably work well on many images.
Click to expand...
My desktop PC has a GTX1660 Super that has 6GB of VRAM and AI11 runs fine, using 5.7GB of VRAM. I don't know if all the memory is being used by SXT or there are other processes using a part of it.

Thanks a lot for your help.
 
EDIT: I've released StarXTerminator 2.0.1 with AI 11 in two flavors: the original full neural network, and a "lite" version for users with older or less performant hardware such as laptops. Run Resources->Updates->Check for Updates to get the new version.

Give this a try: <link removed>

Unzip and place the .pb file in PixInsight's library directory. Then select it from the SXT module. I named it version "1" as an expedient due to the way SXT searches for AI files.

I haven't tested it thoroughly. A quick look showed the expected degradation on very large structures, but otherwise it looked pretty good.

Edit: some older Macs also struggle with the memory requirements. Here is a link to the MacOS version of the "lite" AI11 network:

<link removed>
 
Last edited:
rcroman said:
Not sure why this would be. You might try deleting the updates.xri file to stimulate a fresh install of all updates.
Click to expand...
Have done so and all updates were download again. But it is still the the same behavior. Your process is still the previous one.
 
rcroman said:
Give this a try: http://www.rc-astro.com/StarXTerminator/PixInsight/StarXTerminator.1.pb.zip

Unzip and place the .pb file in PixInsight's library directory. Then select it from the SXT module. I named it version "1" as an expedient due to the way SXT searches for AI files.

I haven't tested it thoroughly, but a quick look showed the expected degradation on very large structures. Otherwise it looked pretty good.
Click to expand...
I gave it a try and works perfectly!

So far the results are excellent with M31, the Lobster Nebula, the Lion Nebula, etc. M45 is harder, the biggest three stars were not removed, but that is an extreme case. In those few cases I can revert to the non GPU accelerated DLL.

Thank you very much for the fast solution.
 
rcroman said:
Send me screen shot of your repo list, PM if you prefer.
Click to expand...

Hi Russel,

I hope this what you need.
screen1.jpg

screen2.jpg


BR Thomas
 
The repo entry looks correct. I'm mystified as to how the old version could remain after doing an update. I would manually delete the StarXTerminator-pxm.dll file from PI's bin directory, then run a fresh update by deleting the updates.xri file again. If it still remains after that, there is another copy of the file floating around somewhere.
 
Hi, is it possible to use Unscreen on Linear images? If I create a star image from Linear, how can I apply the Unscreen method to it?
 
You must log in or register to reply here.
Back
Top