GPU acceleration for PixInsight with Linux (Kubuntu or Ubuntu) using RC Astro Tools (eg StarXterminator) or Starnet ++

here is what is in my /usr/local/cuda/targets/x86_64-linux/lib/

cmake libcufftw_static.a libcusolver_static.a libnppif.so.11.8.0.86 libnpps_static.a
libaccinj64.so libcufile_rdma.so libcusparse.so libnppif_static.a libnvblas.so
libaccinj64.so.11.8 libcufile_rdma.so.1 libcusparse.so.11 libnppig.so libnvblas.so.11
libaccinj64.so.11.8.87 libcufile_rdma.so.1.4.0 libcusparse.so.11.7.5.86 libnppig.so.11 libnvblas.so.11.11.3.6
libcublasLt.so libcufile_rdma_static.a libcusparse_static.a libnppig.so.11.8.0.86 libnvjpeg.so
libcublasLt.so.11 libcufile.so liblapack_static.a libnppig_static.a libnvjpeg.so.11
libcublasLt.so.11.11.3.6 libcufile.so.0 libmetis_static.a libnppim.so libnvjpeg.so.11.9.0.86
libcublasLt_static.a libcufile.so.1.4.0 libnppc.so libnppim.so.11 libnvjpeg_static.a
libcublas.so libcufile_static.a libnppc.so.11 libnppim.so.11.8.0.86 libnvptxcompiler_static.a
libcublas.so.11 libcufilt.a libnppc.so.11.8.0.86 libnppim_static.a libnvrtc-builtins.so
libcublas.so.11.11.3.6 libcuinj64.so libnppc_static.a libnppist.so libnvrtc-builtins.so.11.8
libcublas_static.a libcuinj64.so.11.8 libnppial.so libnppist.so.11 libnvrtc-builtins.so.11.8.89
libcudadevrt.a libcuinj64.so.11.8.87 libnppial.so.11 libnppist.so.11.8.0.86 libnvrtc-builtins_static.a
libcudart.so libculibos.a libnppial.so.11.8.0.86 libnppist_static.a libnvrtc.so
libcudart.so.11.0 libcurand.so libnppial_static.a libnppisu.so libnvrtc.so.11.2
libcudart.so.11.8.89 libcurand.so.10 libnppicc.so libnppisu.so.11 libnvrtc.so.11.8.89
libcudart_static.a libcurand.so.10.3.0.86 libnppicc.so.11 libnppisu.so.11.8.0.86 libnvrtc_static.a
libcufft.so libcurand_static.a libnppicc.so.11.8.0.86 libnppisu_static.a libnvToolsExt.so
libcufft.so.10 libcusolver_lapack_static.a libnppicc_static.a libnppitc.so libnvToolsExt.so.1
libcufft.so.10.9.0.58 libcusolverMg.so libnppidei.so libnppitc.so.11 libnvToolsExt.so.1.0.0
libcufft_static.a libcusolverMg.so.11 libnppidei.so.11 libnppitc.so.11.8.0.86 libOpenCL.so
libcufft_static_nocallback.a libcusolverMg.so.11.4.1.48 libnppidei.so.11.8.0.86 libnppitc_static.a libOpenCL.so.1
libcufftw.so libcusolver.so libnppidei_static.a libnpps.so libOpenCL.so.1.0
libcufftw.so.10 libcusolver.so.11 libnppif.so libnpps.so.11 libOpenCL.so.1.0.0
libcufftw.so.10.9.0.58 libcusolver.so.11.4.1.48 libnppif.so.11 libnpps.so.11.8.0.86 stubs

note there are no libcudnn files located there. They are supposed to install to this directory: /usr/lib/x86_64-linux-gnu/

if I change to the /usr/lib/x86_64-linux-gnu/ directory here is what I find:
cd /usr/lib/x86_64-linux-gnu/
dir libcudnn*
libcudnn_adv_infer.so libcudnn_adv_train_static_v8.a libcudnn_cnn_train_static.a libcudnn_ops_train.so.8.8.0
libcudnn_adv_infer.so.8 libcudnn_cnn_infer.so libcudnn_cnn_train_static_v8.a libcudnn_ops_train_static.a
libcudnn_adv_infer.so.8.8.0 libcudnn_cnn_infer.so.8 libcudnn_ops_infer.so libcudnn_ops_train_static_v8.a
libcudnn_adv_infer_static.a libcudnn_cnn_infer.so.8.8.0 libcudnn_ops_infer.so.8 libcudnn.so
libcudnn_adv_infer_static_v8.a libcudnn_cnn_infer_static.a libcudnn_ops_infer.so.8.8.0 libcudnn.so.8
libcudnn_adv_train.so libcudnn_cnn_infer_static_v8.a libcudnn_ops_infer_static.a libcudnn.so.8.8.0
libcudnn_adv_train.so.8 libcudnn_cnn_train.so libcudnn_ops_infer_static_v8.a
libcudnn_adv_train.so.8.8.0 libcudnn_cnn_train.so.8 libcudnn_ops_train.so
libcudnn_adv_train_static.a libcudnn_cnn_train.so.8.8.0 libcudnn_ops_train.so.8

it looks like you may have installed the libcudnn files to the wrong directory. if you find these files in both directories, delete the libcudnn files from the /usr/local/cuda/targets/x86_64-linux/lib/ directory
 
lblock said:
here is what is in my /usr/local/cuda/targets/x86_64-linux/lib/

cmake libcufftw_static.a libcusolver_static.a libnppif.so.11.8.0.86 libnpps_static.a
libaccinj64.so libcufile_rdma.so libcusparse.so libnppif_static.a libnvblas.so
libaccinj64.so.11.8 libcufile_rdma.so.1 libcusparse.so.11 libnppig.so libnvblas.so.11
libaccinj64.so.11.8.87 libcufile_rdma.so.1.4.0 libcusparse.so.11.7.5.86 libnppig.so.11 libnvblas.so.11.11.3.6
libcublasLt.so libcufile_rdma_static.a libcusparse_static.a libnppig.so.11.8.0.86 libnvjpeg.so
libcublasLt.so.11 libcufile.so liblapack_static.a libnppig_static.a libnvjpeg.so.11
libcublasLt.so.11.11.3.6 libcufile.so.0 libmetis_static.a libnppim.so libnvjpeg.so.11.9.0.86
libcublasLt_static.a libcufile.so.1.4.0 libnppc.so libnppim.so.11 libnvjpeg_static.a
libcublas.so libcufile_static.a libnppc.so.11 libnppim.so.11.8.0.86 libnvptxcompiler_static.a
libcublas.so.11 libcufilt.a libnppc.so.11.8.0.86 libnppim_static.a libnvrtc-builtins.so
libcublas.so.11.11.3.6 libcuinj64.so libnppc_static.a libnppist.so libnvrtc-builtins.so.11.8
libcublas_static.a libcuinj64.so.11.8 libnppial.so libnppist.so.11 libnvrtc-builtins.so.11.8.89
libcudadevrt.a libcuinj64.so.11.8.87 libnppial.so.11 libnppist.so.11.8.0.86 libnvrtc-builtins_static.a
libcudart.so libculibos.a libnppial.so.11.8.0.86 libnppist_static.a libnvrtc.so
libcudart.so.11.0 libcurand.so libnppial_static.a libnppisu.so libnvrtc.so.11.2
libcudart.so.11.8.89 libcurand.so.10 libnppicc.so libnppisu.so.11 libnvrtc.so.11.8.89
libcudart_static.a libcurand.so.10.3.0.86 libnppicc.so.11 libnppisu.so.11.8.0.86 libnvrtc_static.a
libcufft.so libcurand_static.a libnppicc.so.11.8.0.86 libnppisu_static.a libnvToolsExt.so
libcufft.so.10 libcusolver_lapack_static.a libnppicc_static.a libnppitc.so libnvToolsExt.so.1
libcufft.so.10.9.0.58 libcusolverMg.so libnppidei.so libnppitc.so.11 libnvToolsExt.so.1.0.0
libcufft_static.a libcusolverMg.so.11 libnppidei.so.11 libnppitc.so.11.8.0.86 libOpenCL.so
libcufft_static_nocallback.a libcusolverMg.so.11.4.1.48 libnppidei.so.11.8.0.86 libnppitc_static.a libOpenCL.so.1
libcufftw.so libcusolver.so libnppidei_static.a libnpps.so libOpenCL.so.1.0
libcufftw.so.10 libcusolver.so.11 libnppif.so libnpps.so.11 libOpenCL.so.1.0.0
libcufftw.so.10.9.0.58 libcusolver.so.11.4.1.48 libnppif.so.11 libnpps.so.11.8.0.86 stubs

note there are no libcudnn files located there. They are supposed to install to this directory: /usr/lib/x86_64-linux-gnu/

if I change to the /usr/lib/x86_64-linux-gnu/ directory here is what I find:
cd /usr/lib/x86_64-linux-gnu/
dir libcudnn*
libcudnn_adv_infer.so libcudnn_adv_train_static_v8.a libcudnn_cnn_train_static.a libcudnn_ops_train.so.8.8.0
libcudnn_adv_infer.so.8 libcudnn_cnn_infer.so libcudnn_cnn_train_static_v8.a libcudnn_ops_train_static.a
libcudnn_adv_infer.so.8.8.0 libcudnn_cnn_infer.so.8 libcudnn_ops_infer.so libcudnn_ops_train_static_v8.a
libcudnn_adv_infer_static.a libcudnn_cnn_infer.so.8.8.0 libcudnn_ops_infer.so.8 libcudnn.so
libcudnn_adv_infer_static_v8.a libcudnn_cnn_infer_static.a libcudnn_ops_infer.so.8.8.0 libcudnn.so.8
libcudnn_adv_train.so libcudnn_cnn_infer_static_v8.a libcudnn_ops_infer_static.a libcudnn.so.8.8.0
libcudnn_adv_train.so.8 libcudnn_cnn_train.so libcudnn_ops_infer_static_v8.a
libcudnn_adv_train.so.8.8.0 libcudnn_cnn_train.so.8 libcudnn_ops_train.so
libcudnn_adv_train_static.a libcudnn_cnn_train.so.8.8.0 libcudnn_ops_train.so.8

it looks like you may have installed the libcudnn files to the wrong directory. if you find these files in both directories, delete the libcudnn files from the /usr/local/cuda/targets/x86_64-linux/lib/ directory
Click to expand...
thanks
after I have deleted the libcudnn files in the /usr/local/cuda/targets/x86_64-linux/lib/ directory I can now run sudo ldconfig /usr/local/lib with out error, but I always have this error and I don't think that pixinsight is using the GPU because starX takes 2 minutes
Code: 
username@Inspiron-16-Plus-7620:~$ sudo ldconfig /usr/local/lib
username@Inspiron-16-Plus-7620:~$ PixInsight

PixInsight Core 1.8.9-2 Ripley (x64)
Copyright (c) 2003-2024 Pleiades Astrophoto

2024-02-18 19:51:22.067655: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-02-18 19:51:41.050114: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-02-18 19:51:42.026186: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:357] MLIR V1 optimization pass is not enabled
 
you may need to delete and reinstall pixinsight with the proper environment parameters. Pixinsight needs the libcudnn files (cuda version 11.8 to use the 2.14.0 tensorflow libraries. It appears pixinsight is trying to use the tensorflow libraries it created instead of the 2.14.0 version.
If this happens, uninstall PixInsight. Reinstall the latest version. Do not restart PixInsight until you enter the following:
sudo rm /opt/PixInsight/bin/lib/tensorflow*.
Now PixInsight should be GPU accelerated.
(thanks to @airscottdenning for this)
 
Last edited:
Hello,

I just did this today on KUbuntu 22.04.4 LTS and it went flawlessly. It only took about 30 minutes total. I installed the latest build of PI 20240301 and it works perfectly. On this same machine I dual boot with Windows 11. The same image under Windows removing stars took 32.5 seconds, on KUbuntu 24.16 seconds. Very Nice!

I did find a typo in the OP, this line here has an underscore (_) instead of a dash (-).

sudo tar -C /usr/local -xzf libtensorflow-gpu-linux-x86_64_2.14.0.tar.gz

it should be

sudo tar -C /usr/local -xzf libtensorflow-gpu-linux-x86_64-2.14.0.tar.gz

Thanks to @lblock this now works on Linux and thanks to @rcroman it works under Windows too!

Great stuff guys!!

My PC
Operating System: Ubuntu 22.04.4 LTS
Kernel: Linux 5.15.0-94-generic
Architecture: x86-64
Hardware Vendor: Gigabyte Technology Co., Ltd.
Hardware Model: Z390 AORUS MASTER, LGA1171, 64Gb RAM
Intel i9-9900KF 8 cores, 16 logical, OC to 4600MHz
Graphics: Nvidia GeForce RTX 4070, 12 Gb
Western Digital SN850X SSD
 
I followed @lblock's instructions a month ago and got everything working again in kubuntu 22.04.4 LTS.

Yesterday I updated to the newest the PI version and everything still works (fast, on the GPU). All I had to do after install was to
sudo rm /opt/PixInsight/bin/lib/tensorflow*
Click to expand...

The previous steps install the libtesorflow stuff with the GPU versions. The only thing to be careful of is that the CPU versions don't get found at runtime by PI.
 
Thank you for these precise directions, @lblock ! I struggled with other instructions elsewhere, then went through these and things worked immediately.

FWIW I upgraded to an RTX4060Ti from a GTX1050Ti because of this success, and got the following numbers for everything from RCAstro...


blurXt 439" (no CUDA)
141" (1050Ti, 4GB)
46" (4060Ti, 16GB)

noiseXT 70" (no CUDA)
39" (1050Ti, 4GB)
17" (4060Ti, 16GB)

starXT 312" (no CUDA)
93" (1050Ti, 4GB)
40" (4060Ti, 16GB)

Some complaints about memory issues which showed with the 1050 disappeared with the new card (I saw references to 13GB structures being created by tensorflow instead). The 1050Ti card has something like 768 cores, while the 4060Ti has 4352.

All the above numbers were first runs. If I repeated the process as I often do (messing with settings), the times decreased by 5-12 seconds, indicating a chunk of timing lost to initializations. Use of different AIs (e.g. AI 11 lite instead of 11 in starXT) also changed things--as expected.

The images were full frame, not APS-C or other. My cpu is a Ryzen 7 3700X; this will likely get upgraded if I find a good Linux program to control liquid cooling.
 
dackerman1 said:
Hello,

I just did this today on KUbuntu 22.04.4 LTS and it went flawlessly. It only took about 30 minutes total. I installed the latest build of PI 20240301 and it works perfectly. On this same machine I dual boot with Windows 11. The same image under Windows removing stars took 32.5 seconds, on KUbuntu 24.16 seconds. Very Nice!

I did find a typo in the OP, this line here has an underscore (_) instead of a dash (-).

sudo tar -C /usr/local -xzf libtensorflow-gpu-linux-x86_64_2.14.0.tar.gz

it should be

sudo tar -C /usr/local -xzf libtensorflow-gpu-linux-x86_64-2.14.0.tar.gz

Thanks to @lblock this now works on Linux and thanks to @rcroman it works under Windows too!

Great stuff guys!!

My PC
Operating System: Ubuntu 22.04.4 LTS
Kernel: Linux 5.15.0-94-generic
Architecture: x86-64
Hardware Vendor: Gigabyte Technology Co., Ltd.
Hardware Model: Z390 AORUS MASTER, LGA1171, 64Gb RAM
Intel i9-9900KF 8 cores, 16 logical, OC to 4600MHz
Graphics: Nvidia GeForce RTX 4070, 12 Gb
Western Digital SN850X SSD
Click to expand...
I corrrected the typo. Thank you.
 
airscottdenning said:
I followed @lblock's instructions a month ago and got everything working again in kubuntu 22.04.4 LTS.

Yesterday I updated to the newest the PI version and everything still works (fast, on the GPU). All I had to do after install was to


The previous steps install the libtesorflow stuff with the GPU versions. The only thing to be careful of is that the CPU versions don't get found at runtime by PI.
Click to expand...
I added this to the guide. Thanks !
 
eporte said:
thanks
after I have deleted the libcudnn files in the /usr/local/cuda/targets/x86_64-linux/lib/ directory I can now run sudo ldconfig /usr/local/lib with out error, but I always have this error and I don't think that pixinsight is using the GPU because starX takes 2 minutes
Code: 
username@Inspiron-16-Plus-7620:~$ sudo ldconfig /usr/local/lib
username@Inspiron-16-Plus-7620:~$ PixInsight

PixInsight Core 1.8.9-2 Ripley (x64)
Copyright (c) 2003-2024 Pleiades Astrophoto

2024-02-18 19:51:22.067655: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-02-18 19:51:41.050114: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-02-18 19:51:42.026186: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:357] MLIR V1 optimization pass is not enabled
Click to expand...

eporte said:
thanks
after I have deleted the libcudnn files in the /usr/local/cuda/targets/x86_64-linux/lib/ directory I can now run sudo ldconfig /usr/local/lib with out error, but I always have this error and I don't think that pixinsight is using the GPU because starX takes 2 minutes
Code: 
username@Inspiron-16-Plus-7620:~$ sudo ldconfig /usr/local/lib
username@Inspiron-16-Plus-7620:~$ PixInsight

PixInsight Core 1.8.9-2 Ripley (x64)
Copyright (c) 2003-2024 Pleiades Astrophoto

2024-02-18 19:51:22.067655: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-02-18 19:51:41.050114: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-02-18 19:51:42.026186: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:357] MLIR V1 optimization pass is not enabled
Click to expand...
It turns out af
eporte said:
thanks
after I have deleted the libcudnn files in the /usr/local/cuda/targets/x86_64-linux/lib/ directory I can now run sudo ldconfig /usr/local/lib with out error, but I always have this error and I don't think that pixinsight is using the GPU because starX takes 2 minutes
Code: 
username@Inspiron-16-Plus-7620:~$ sudo ldconfig /usr/local/lib
username@Inspiron-16-Plus-7620:~$ PixInsight

PixInsight Core 1.8.9-2 Ripley (x64)
Copyright (c) 2003-2024 Pleiades Astrophoto

2024-02-18 19:51:22.067655: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-02-18 19:51:41.050114: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-02-18 19:51:42.026186: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:357] MLIR V1 optimization pass is not enabled
Click to expand...
Thanks to @airscottdenning :
I would suggest uninstalling PixInsight and reinstalling with the proper parameters from the guide.
then enter
sudo rm /opt/PixInsight/bin/lib/tensorflow*
This makes sure you use the GPU acceleration 2.14.0 tensorflow libraries. Do this BEFORE you restart PixInsight.
 
Last edited:
lblock said:
It turns out af

Thanks to @airscottdenning :
I would suggest uninstalling PixInsight and reinstalling with the proper parameters from the guide.
then enter
sudo rm /opt/PixInsight/bin/lib/tensorflow*
This makes sure you use the GPU acceleration 2.14.0 tensorflow libraries. Do this BEFORE you restart PixInsight.
Click to expand...
thank you very much
now it works using the GPU, it's much faster :love:
but sudo rm /opt/PixInsight/bin/lib/tensorflow* does not work, it's sudo rm /opt/PixInsight/bin/lib/libtensorflow*
Best Regards
 
Last edited:
eporte said:
thank you very much
now it works using the GPU, it's much faster :love:
but sudo rm /opt/PixInsight/bin/lib/tensorflow* does not work, it's sudo rm /opt/PixInsight/bin/lib/libtensorflow*
Best Regards
Click to expand...
Sorry about that. I meant to say sudo rm /opt/PixInsight/bin/lib/libtensorflow*.
This info has been added to the end of the guide for future installs of PixInsight
I am glad it is now working for you !
 
Last edited:
Thanks so much for this! I literally spent a month trying to get this to work and gave up. I got through this with no errors and it finally works. I am thrilled.

Paul
 
I can confirm that the NVIDIA 535 driver works and seems to be quicker (about 20%) than the originally suggested 525 driver, much to my surprise. The CUDA 11.8, cuDNN 8.8.0 and the libtensorflow files have remained untouched.
I am using Kubuntu 22.04. My GPU is a NVIDIA GeForce GTX 1660.
 
From the Kubuntu command line I just ran the "sudo apt update and then sudo apt upgrade" commands and it installed the new driver.
 
I upgraded but there was a driver/library version mismatch. What solved it was this:

remove completely the existing Nvidia Drivers and reinstall a proper version

first purge the Nvidia drivers

sudo apt-get purge '^nvidia.*'
sudo rm -rf /etc/X11/xorg.conf
sudo rm -rf /etc/modprobe.d/nvidia*
sudo rm -rf /usr/local/nvidia*
sudo rm -rf /usr/lib/xorg/modules/drivers/nvidia*

then add the proper directories

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update

then install the good drivers sudo apt-get install nvidia-driver-535 then disable the nouveau drivers

sudo bash -c "echo 'blacklist nouveau' > /etc/modprobe.d/blacklist-nouveau.conf"
sudo bash -c "echo 'options nouveau modeset=0' >> /etc/modprobe.d/blacklist-nouveau.conf"

then update the initial ramdisk sudo update-initramfs -u

then reboot

sudo reboot


if there is still error check sudo apt-get install nvidia-settings to be sure that all Components are upgraded correctly
 
Last edited:
I changed the Nvidia driver version in the guide to 535. My processing time dropped to 13.9 seconds (from about 15 seconds)
 
Last edited:
I have a very cook book approach to the command line in Ubuntu but you obviously understand it. thank you very much for pointing out what to do if it all goes pear-shaped.
 
You must log in or register to reply here.
Back
Top