GPU acceleration under WSL - Ubuntu 24.04 and 1.8.9-3

rbotero

Well-known member
Following posts here: https://pixinsight.com/forum/index....nder-wsl-ubuntu-22-04-and-1-8-9-2-1597.22711/ and links therein I have PxI running nicely in WSL within Windows 11. PxI is much faster running in Linux than in Windows (11 or else) and although this is a fact in native form, I always wanted to check its speed within the WSL container. I can confirm that it runs at least 20% faster than my native Windows installation (sharing same SSD and hardware). Moreover as per below instructions, GPU acceleration with an NVDIA (I have a GeForce RTX 4070) can also be ported onto WSL.

I47q_wRfGrrB_16536x0_PBe27Uwd.jpg


I am no Linux expert but I set out below the commands I followed to get this running on my system.

Roberto
 
Follow Microsoft's guide to install WSL but make sure you choose the 24.04 Ubuntu LTS distribution: https://learn.microsoft.com/en-us/windows/wsl/install#install

Or this guide specifically to install 24.04 and get Linux GUIs running natively from Windows: https://kskroyal.com/run-ubuntu-24-04-on-windows-11-with-gui-using-wsl/

Once you have Ubuntu 24.04 up and running under WSL,

sudo apt update && sudo apt upgrade
sudo apt install build-essential

sudo apt install nvidia-driver-550
At this point you should see Ubuntu as one of your apps within the Windows start menu:
1720348524231.png
 
Update your Ubuntu sources:

sudo nano /etc/apt/sources.list.d/ubuntu.sources

within that file edit:

Types: deb
URIs: http://cz.archive.ubuntu.com/ubuntu
Suites: jammy-updates
Components: main universe

Now install libtinfo5 just in case:
sudo apt-get update
sudo apt-get install libtinfo5
Create a folder within your home directory to hold all your PxI, CUDA, Tensorflow files for ease of installation/deployment:
cd /home/YOUR USER NAME/
mkdir PxI_Installation
cd PxI_Installation
Update the Cuda sources:

wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin

sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
Download the latest Cuda and install:

wget https://developer.download.nvidia.c...repo-wsl-ubuntu-12-5-local_12.5.0-1_amd64.deb

sudo dpkg -i cuda-repo-wsl-ubuntu-12-5-local_12.5.0-1_amd64.deb

sudo cp /var/cuda-repo-wsl-ubuntu-12-5-local/cuda-*-keyring.gpg /usr/share/keyrings/

sudo apt-get update
sudo apt-get -y install cuda-toolkit-12-5
Download cdnn as explained in other threads - you will need a user name and login - go to: https://developer.nvidia.com/rdp/cudnn-archive and download version 8.9.7.29 which works with Cuda 12 and tensorflow. Choose the Ubuntu 22.04 x86_64(deb) installation:
1720349120990.png
 
The above archive most likely will download to your Windows/Download folder. You can now move it to the PxI_Installation folder in Linux:

mv /WHEREVER IT SAVED//cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb
Install it:
sudo dpkg -i cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb

sudo cp /var/cudnn-local-repo-ubuntu2204-8.9.7.29/cudnn-*-keyring.gpg /usr/share/keyrings/

sudo apt-get update

sudo apt-get -y install libcudnn8

Now get Tensorflow for the GPU acceleration to work:


Make a folder for the Tensorflow libraries to live and for future easy updates:

cd /home
mkdir tensorflow
cd tensorflow

tar -xvf /WHERE YOU SAVED THE TENSORFLOW DOWNLOAD//libtensorflow-gpu-linux-x86_64-2.15.0.tar.gz
 
Last edited:
Go back to your PxI_Installation and download the latest (1.8.9-3 / build 1612) installation file for PixInsight and move it to your PxI_Installation folder and install it:

sudo tar -xf PI-linux-x64-1.8.9-3-20240619-c.tar.xz

sudo ./installer

Move out the tensorflow libraries shipped with PixInsight out and link to the 2.15 version you downloaded and installed above:

sudo mkdir /opt/temp

cd /opt/PixInsight/bin/lib

sudo mv libtensorflow* /opt/temp

Update your bashrc at the bottom of the file:

sudo nano ~/.bashrc
add to bottom of file:
export TF_FORCE_GPU_ALLOW_GROWTH="true"
 
Edit PixInsight.sh to choose the right tensorflow libraries:

cd /opt/PixInsight/bin/

sudo nano PixInsight.sh

And edit that file to point to the folder where you uncompressed Tensorflow v2.15:

#!/bin/bash
appname=`basename $0 | sed s,\.sh$,,`
dirname=`dirname $0`
if [ "${dirname:0:1}" != "/" ]; then
dirname=$PWD/$dirname
fi
LD_LIBRARY_PATH=$HOME/tensorflow/lib:$dirname/lib:$dirname <--- new line
#LD_LIBRARY_PATH=$dirname/lib:$dirname <--- old line now commented out
 
You can now run PixInsight from /opt/PixInsight/bin/

./PixInsight.sh

Or from the Windows Start Menu with full GPU acceleration:

1720349998234.png


Hopefully future updates don't break this as it happened when Qt6 was introduced under Ubuntu 22.04.
Good luck!

Roberto
 
Last edited:
@rbotero thanks a lot again for the guide.
I used your guide some month ago when I installed PI on Linux.
In the end I was able to make it work and it was really a boost in performance with my 2x 3090 (I think only one was really working)
Now, after some busy times, I've realized that there are 2 versions more of PI and by chance I saw your new post.
I've also abandoned the idea to use a separate boot for Linux, as it has some issues with Bluetooth and it's really not noob friendly in the long run.
With WSL2 I was even getting better performance than on pure Linux...


So today for one full day I tried to replicate it but it doesn't fully work, but I am not sure if it's because there is anything wrong with your procedure, rather with RC tools (and just because PI keeps changing)

I have a few questions:
did you make a fresh install of WSL or did you updated previous versions?
I've noticed that installing Ubuntu 24.04 directly without going through 22.04 is giving me errors.

Do you think that the following lines are not needed anymore in the bashrc file?
export PATH="/usr/local/cuda/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda/lib64:$LD_LIBRARY_PATH"

Now, regardless of the above, since I've tried every combination, I reach the point where i get the following messages from RC suite:
/opt/PixInsight/bin/NoiseXTerminator-pxm.so
<* failed *>
*** PixInsight API Error: Module: /opt/PixInsight/bin/NoiseXTerminator-pxm.so
Cannot load library /opt/PixInsight/bin/NoiseXTerminator-pxm.so: (libtensorflow_framework.so.2: cannot open shared object file: No such file or directory): Module load error
/opt/PixInsight/bin/StarXTerminator-pxm.so
<* failed *>
*** PixInsight API Error: Module: /opt/PixInsight/bin/StarXTerminator-pxm.so
Cannot load library /opt/PixInsight/bin/StarXTerminator-pxm.so: (libtensorflow_framework.so.2: cannot open shared object file: No such file or directory): Module load error
/opt/PixInsight/bin/BlurXTerminator-pxm.so
<* failed *>
*** PixInsight API Error: Module: /opt/PixInsight/bin/BlurXTerminator-pxm.so
Cannot load library /opt/PixInsight/bin/BlurXTerminator-pxm.so: (libtensorflow.so.2: cannot open shared object file: No such file or directory): Module load error
* Scripting resources updated.
55 of 58 module(s) installed.
I've sent a ticket to RC, to see if by any chance he can help me with this.
I've noted down every step I've taken, eventually with slight changes from your procedure (I've done also the one step to step identical to yours, but i think that for fresh installation it can't be fully followed. If and when the problem is solved, I will post this list.

Anyway, thanks again.
It's really a pity that Linux/WSL are such a pain when they are the platforms with best performance.
I really wish we could have something more solid for CUDA acceleration, nowadays everyone would benefit from it, I think.
 
@rbotero thanks a lot again for the guide.
I used your guide some month ago when I installed PI on Linux.
In the end I was able to make it work and it was really a boost in performance with my 2x 3090 (I think only one was really working)
Now, after some busy times, I've realized that there are 2 versions more of PI and by chance I saw your new post.
I've also abandoned the idea to use a separate boot for Linux, as it has some issues with Bluetooth and it's really not noob friendly in the long run.
With WSL2 I was even getting better performance than on pure Linux...


So today for one full day I tried to replicate it but it doesn't fully work, but I am not sure if it's because there is anything wrong with your procedure, rather with RC tools (and just because PI keeps changing)

I have a few questions:
did you make a fresh install of WSL or did you updated previous versions?
I've noticed that installing Ubuntu 24.04 directly without going through 22.04 is giving me errors.

Do you think that the following lines are not needed anymore in the bashrc file?
export PATH="/usr/local/cuda/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda/lib64:$LD_LIBRARY_PATH"

Now, regardless of the above, since I've tried every combination, I reach the point where i get the following messages from RC suite:

I've sent a ticket to RC, to see if by any chance he can help me with this.
I've noted down every step I've taken, eventually with slight changes from your procedure (I've done also the one step to step identical to yours, but i think that for fresh installation it can't be fully followed. If and when the problem is solved, I will post this list.

Anyway, thanks again.
It's really a pity that Linux/WSL are such a pain when they are the platforms with best performance.
I really wish we could have something more solid for CUDA acceleration, nowadays everyone would benefit from it, I think.
Hi @lbarcaro

I am no Linux or WSL expert but I started from a clean/new WSL installation with Ubuntu 24.04.

I only added the line for the GPU Growth to the bashrc file. I have not checked if mine has the lines you quote but why would you want to remove them?

I think what you are seeing with XTerminator process is to do with the location of the tensorflow libraries. Did you make sure that your PixInsight.sh file now points to where you installed the tensorflow libraries?

Roberto
 
Hi @lbarcaro

I am no Linux or WSL expert but I started from a clean/new WSL installation with Ubuntu 24.04.

I only added the line for the GPU Growth to the bashrc file. I have not checked if mine has the lines you quote but why would you want to remove them?

I think what you are seeing with XTerminator process is to do with the location of the tensorflow libraries. Did you make sure that your PixInsight.sh file now points to where you installed the tensorflow libraries?

Roberto
Hello, I think so...
That would be this part right?

#!/bin/bash
appname=`basename $0 | sed s,\.sh$,,`
dirname=`dirname $0`
if [ "${dirname:0:1}" != "/" ]; then
dirname=$PWD/$dirname
fi
LD_LIBRARY_PATH=$HOME/tensorflow/lib:$dirname
#LD_LIBRARY_PATH=$dirname/lib:$dirname <--- old line now commented out

Basically i added the green line like in your tutorial and commented with the # the existing one.
Nothing else was changed.

It really looks like RC package is looking for the lib and doesn't find them, like you said. So, I also tried playing with the green line, since i saw two times "/lib:$dirname" but that was correct because nothing was being detected with the line different from what it is in your posts.

Tonight I will try again without going via Ubuntu 22.04.
 
If you installed the tensorflow libraries at the /home/yourusername/ level, then $HOME alone in the line above will work. If an another folder, then you need to specify that path.

What does
echo "$HOME"
give you?

That will help. Maybe you need an additional directory direction for XTerminator to find the libraries.

Roberto
 
If you installed the tensorflow libraries at the /home/yourusername/ level, then $HOME alone in the line above will work. If an another folder, then you need to specify that path.

What does

give you?

That will help. Maybe you need an additional directory direction for XTerminator to find the libraries.

Roberto
Tonight i will check this.
I have created the folders exactly like in your tutorial in principle, but i will check with this command and see what happens.

Are you using Ubuntu 22.04? That release did NOT work for me as per the link at the very top of this thread.
No, i am using 24.04
In the beginning i installed wsl directly with the 24.04 but it wasn't working in some step (12.5 cuda toolkit step)
Then i installed the WSL with default ubuntu which was the 22.04, then upgraded to the 24.04. But still it wasn't working.
I was looking at the notes of the step I took (once it works, i want to be able to repeat just in case :D) and probably i took some missteps when dealing with the keyring part.
So i want to try again go directly for the 24.04 since the beginning.
 
@rbotero You were right about the folder mistake
My tensorflow folder was not in /home/USERNAME/ but in /home/
Moved it there, and now it works!

However, I was not able to install the cuda toolkit from direct installation of Ubuntu 24.04, but I had to go through the 22.04 again and then upgrade it to 24.04.
Otherwise the following error was happening
The following packages have unmet dependencies: cuda-nsight-12-5 : Depends: default-jre (>= 1.7) but it is not installable or openjdk-7-jre but it is not installable cuda-nvvp-12-5 : Depends: default-jre (>= 1.7) but it is not installable or openjdk-7-jre but it is not installable nsight-systems-2024.2.3 : Depends: libxcb-cursor0 but it is not installableE: Unable to correct problems, you have held broken packages.
Tomorrow I will make a detailed recap of the modified procedure, just in case someone has the same issue.
Thanks again for everything :)
 
@rbotero You were right about the folder mistake
My tensorflow folder was not in /home/USERNAME/ but in /home/
Moved it there, and now it works!

However, I was not able to install the cuda toolkit from direct installation of Ubuntu 24.04, but I had to go through the 22.04 again and then upgrade it to 24.04.
Otherwise the following error was happening

Tomorrow I will make a detailed recap of the modified procedure, just in case someone has the same issue.
Thanks again for everything :)
Glad you sorted it out. For cuda, did you install the WSL version I pointed to? Or the native Ubuntu one? I would not expect the latter to be fully compatible with WSL.
 
Here I am to make a recap, just in case someone encounter my same problems.
I've initially tried to follow the initial post to the letter, but I suppose that some updates happened somewhere and therefore something went wrong.
It could be also my ignorance, since my Linux/IT knowledge is limited.
I made deep use of chat GPT to try to troubleshoot some of the issues I encountered. For many things it was giving decent information. I am pasting here some of the results.
I was not able to use any GUI functions for the moment, so I followed the link provided in the first post only partially.

Therefore, with this guide you will only get the "base Ubuntu version". I am not sure I am using proper wording.

Clean WSL Install
First of all I recommend to clean the WSL installation that you could have had before.
To do so you can go in the Terminal as administrator

Step 1: Uninstall WSL Distributions​

Open PowerShell as Administrator:

  • Right-click on the Start button and select "Windows Terminal (Admin)" or "PowerShell (Admin)"
wsl --list --verbose
  • Unregister/Uninstall Each Distribution:For each distribution listed, use the following command to unregister it:
wsl --unregister <DistroName>

Replace <DistroName> with the name of the distribution you want to uninstall.

Step 2: Uninstall Windows Subsystem for Linux​

  1. Open Settings:
    • Press Win + I to open Settings.
  2. Navigate to Apps:
    • Go to "Apps" > "Optional features".
  3. Find and Uninstall WSL:
    • Scroll down and find "Windows Subsystem for Linux".
    • Click on it and select "Uninstall".

Step 3: Restart Your Computer​

Restart your computer to ensure all changes take effect.

Step 4: Follow the initial part of the second link from first post​

https://kskroyal.com/run-ubuntu-24-04-on-windows-11-with-gui-using-wsl/

Follow until the end of Step 4 of the link above.

Step 5: Install WSL2​

Open the power shell
wsl --install
Follow the base instructions, select your username and your passwords
At this point you should have installed the Ubuntu 22.04.

Step 6: Install stuff that are seemingly needed to avoid problems with the Cuda Toolkit installation​

While in the WSL terminal
sudo apt update
sudo apt upgrade
sudo add-apt-repository universe
sudo add-apt-repository multiverse
sudo apt update
sudo apt install default-jre libnuma1 libxcb-xinerama0 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-render-util0 libxcb-shape0 libxkbcommon-x11-0 libxcb-xinput0 libxcb-cursor0 libopengl0
sudo apt install openjdk-8-jre
sudo apt update
sudo apt upgrade
sudo apt install openjdk-11-jre

Step 7: Start the update from Ubuntu 22.04 to Ubuntu 24.04​

sudo nano /etc/apt/sources.list

put mantic instead of jammy everywhere in the file that is open.
What does it mean? You are opening a file and there are several lines where the word jammy is. It's important to change it with mantic in the lines that are not commented (the ones that don't start with "#").
Basically from what chatGPT says jammy is referring to 22.04, mantic is referring to 24.04.
Of course save your file with CTRL+0.
Confirm the name as is.
Control+X to close the file.
sudo apt update
sudo apt upgrade -y
sudo apt dist-upgrade -y
sudo apt autoremove -y
sudo do-release-upgrade
During the steps above there will be several request to confirm to proceed and install stuff. I have accepted all blindly.
sudo reboot

Step 8: Follow again the initial posts from rbotero​

Here basically we are in a situation where we have Ubuntu 24.04 installed, which seems to have all the libraries or whatever they are that are prerequisites for rbotero's steps.
sudo apt install build-essential
sudo apt install nvidia-driver-550
sudo nano /etc/apt/sources.list.d/ubuntu.sources
within that file edit:
Types: deb
URIs: http://cz.archive.ubuntu.com/ubuntu
Suites: jammy-updates
Components: main universe
sudo apt-get update
Now install libtinfo5 just in case:
sudo apt-get update
sudo apt-get install libtinfo5
Create a folder within your home directory to hold all your PxI, CUDA, Tensorflow files for ease of installation/deployment:
cd /home/YOUR USER NAME/
mkdir PxI_Installation
cd PxI_Installation

Step 9: Install the cuda toolkit 12.5 (slightly different approach)​

You are now in your /PxI_Installation/ folder
Here I am using the points directly available on the Nvidia website.
Looking at the original post #3, there are the two lines under "Update the Cuda sources" that works properly, however the two down the "Download the latest Cuda and Install", are not. Most likely NVidia changed the link, as you could get some authentication error.
Therefore what i recommend to do is the visit the following link: https://developer.nvidia.com/cuda-downloads
1721166198036.png


Your selection should be: Linux > x86_64 > WSL-Ubuntu > 2.0 > deb(local)
A small list of commands will be open, basically similar to what Rbotero postes in his post #3.
wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.c...repo-wsl-ubuntu-12-5-local_12.5.1-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-12-5-local_12.5.1-1_amd64.deb</span><span>sudo cp /var/cuda-repo-wsl-ubuntu-12-5-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda-toolkit-12-5
I want to highlight one thing. The red line has a "*" at some point. You shouldn't copy this line as is.
When you enter the blue line, a feedback will be given in the terminal window. Basically it's going to give you the red line but with actual numbers instead of the asterisk. Copy that and use that instead of the red line. Then go directly to the "sudo apt-get update".

Step 10: Install cuDNN v8.9.7 (December 5th, 2023)​

Here basically I've followed to the letter rbotero's post.
You can't download the file via "wget", so you will be downloading it most likely in your "Download" folder.
Make sure you download the 8.9.7 from the 5th of December, when you read this it might be that you will see a more recent version. It may work, it may not.
The link is here.
Install the Local Installer for Ubuntu22.04 x86_64(Deb), as rbotero said.

Now you need to move that from Download to your /PiX_Installation/ folder.
Here the example for my case:
mv /mnt/c/Users/USERNAMEWINDOWS/Downloads/cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb ~/PxI_Installation/
Of course put your Windows11 username instead of USERNAMEWINDOWS
sudo dpkg -i cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb

sudo cp /var/cudnn-local-repo-ubuntu2204-8.9.7.29/cudnn-*-keyring.gpg /usr/share/keyrings/

sudo apt-get update

sudo apt-get -y install libcudnn8

I want to highlight again one thing. The red line has a "*" at some point. You shouldn't copy this line as is.
When you enter the blue line, a feedback will be given in the terminal window. Basically it's going to give you the red line but with actual numbers instead of the asterisk. Copy that and use that instead of the red line. Then go directly to the "sudo apt-get update".

Step 11: Installing tensor flow in the right folder​

Here please make sure that you install your tensorflow folder within /home/USERNAMEUBUNTU/ and not only /home/.
Otherwise you will get some issue when you install some of the libraries (e.g. RusselClow Xsterminator suite).

cd /home/USERNAMEUBUNTU/

sudo mkdir tensorflow

cd tensorflow

Here again you must download the tensorflow libraries in your Donwload Windows11 folder and move it into the just created folder.
You can install the version 2.15.0. Mind that at the time I am writing this the 2.15.1 is already available. I didn't try it yet.

You can download it here.

Otherwise visit the Nvidia website and find from the list of version this one. You can try a newer one but you might need to start all from beginning.

mv /mnt/c/Users/USERNAMEWINDOWS/Downloads/libtensorflow-gpu-linux-x86_64-2.15.0.tar .
sudo tar -xvf /home/tensorflow/libtensorflow-gpu-linux-x86_64-2.15.0.tar.gz

Step 12: Install Pixinsight​

At the time that I am writing this, a newer version of 1.8.9-3 is available from the website. It's working. Hopefully it will work also whatever version you will be able to download.

From wherever you are you move back to the PxI_installation folder
For example, from where you should be:

cd USERNAMEUBUNTU
cd PxI_Installation

Move Pixinsight from the Download folder to your PxI_Installation folder:

mv /mnt/c/Users/USERNAMEWIN11/Downloads/PI-linux-x64-1.8.9-3-20240625-c.tar.xz ~/PxI_Installation
sudo tar -xf PI-linux-x64-1.8.9-3-20240625-c.tar.xz
Please, note that the red part depends on the version you are downloading. It may be different when you do it.
Of course, put your Windows username to select the right source folder of the Pixinsight file you downloaded.

sudo ./installer

Step 13: Move the native libraries and set up the new ones​

This is exactly the same as rbotero wrote. I explicited some passage.

sudo mkdir /opt/temp

cd /opt/PixInsight/bin/lib

sudo mv libtensorflow* /opt/temp

Update your bashrc at the bottom of the file:


sudo nano ~/.bashrc
add to bottom of file:
export TF_FORCE_GPU_ALLOW_GROWTH="true"
CTRL + O
Accept the file name as is
CTRL + X

Edit PixInsight.sh to choose the right tensorflow libraries:


cd /opt/PixInsight/bin/

sudo nano PixInsight.sh

And edit that file to point to the folder where you uncompressed Tensorflow v2.15:


#!/bin/bash
appname=`basename $0 | sed s,\.sh$,,`
dirname=`dirname $0`
if [ "${dirname:0:1}" != "/" ]; then
dirname=$PWD/$dirname
fi

LD_LIBRARY_PATH=$HOME/tensorflow/lib:$dirname/lib:$dirname <--- new line
#LD_LIBRARY_PATH=$dirname/lib:$dirname <--- old line now commented out with the addition of #
The green is already existing,
you add the blue without the yellow part
you add an "#" to the orange line that is also already existing.

Step 14: Install your suite making use of cuda and enjoy​

I've installed RC suite, tested it on one image.
If you open the "Task manager" you can check what's ramping up. If it's the CPU, not good, it's not working.
If it's the GPU... good for you.


Again I want to thank rbotero and also the other people mentioned/authors of the link in his post.

I have 2 x 3090 installed. Only one is ramping up.
I hope that I can find the way to use all of them.
 

Attachments

  • 1721166168497.png
    1721166168497.png
    84.7 KB · Views: 127
Last edited:
Back
Top