No luck with StarNet++ on RTX 3070.

M

mxcoppell

Member
Folks,

Anybody got experience how to make RTX 3070 Cuda working with StarNet++?

I've followed darkarchon's tutorial: https://darkskies.space/pixinsight-starnet-cuda/. When I execute StarNet, it stuck at:

"Restoring neural network checkpoint: C:/Program Files/PixInsight/library/mono_starnet_weights.pb"

Tried to figure if the Cuda libraries and Cudnn libraries are too old for RTX 3070. Tried to deploy the following packages:
  1. cuda_11.1.1_win10_network
  2. cudnn-11.1-windows-x64-v8.0.5.39
  3. libtensorflow-gpu-windows-x86_64-2.4.0-rc2

Now the "Restoring neural network checkpoint:..." hanging is gone. But it doesn't look like StarNet++ is using Cuda. It's still running on the CPU.
** I've also tried libtensorflow-gpu-windows-x86_64-2.3.1 (built on 10/02/2020). Still only on CPU.

My graphics card is ASUS Dual NVIDIA GeForce RTX 3070 OC Edition Gaming Graphics Card.

Thanks in advance for any suggestions!

-Min
 
I think you still should use Cuda 10.0 or 10.1, because this is a pretty old release.

I am about to get PC with RTX3090 and then I can try and figure out if there are specific issues with 30xx cards and maybe create a new release!

Meanwhile maybe you could try to use Python version? That one will definitely work well!
 
itsSticky said:
Hey @mxcoppell and any others on ampere - I managed to get pixinsight working with my Nvidia RTX 3080 today!
Exactly the same steps as the usual guide except I updated the cuDNN v8.0.5 (November 9th, 2020), for CUDA 10.1

https://developer.nvidia.com/rdp/cudnn-download#a-collapse805-101 for the architecture category

For windows directly: https://developer.nvidia.com/comput...201106/cudnn-10.1-windows10-x64-v8.0.5.39.zip
Click to expand...

I tried this and it did not work, Starnet did not freeze but it ran using the CPU only. Can you verify that using cuDNN v8.0.5 allowed starnet to run using your GPU CUDA cores and not just run on the CPU?

According to this link: https://www.tensorflow.org/install/gpu
"On systems with NVIDIA® Ampere GPUs (CUDA architecture 8.0) or newer, kernels are JIT-compiled from PTX and TensorFlow can take over 30 minutes to start up. This overhead can be limited to the first start up by increasing the default JIT cache size with: 'export CUDA_CACHE_MAXSIZE=2147483648' (see JIT Caching for details). "

This seems to confirm what I see on my RTX 3080. If I use the standard cuDNN v7.6.5 from DarkArchon's guide, PI seems to freeze up for 15-30 minutes. Eventually, the starnet process runs quickly and uses the CUDA cores, but the resulting image is just all black.

I tried applying the environment variable mentioned above, which allowed subsequent starnet runs to start without the 15 minute delay, however the results were still an all-black image.
 

Attachments

  • pi-starnet.jpg
    pi-starnet.jpg
    783.2 KB · Views: 117
itsSticky said:
Hey @mxcoppell and any others on ampere - I managed to get pixinsight working with my Nvidia RTX 3080 today!
Exactly the same steps as the usual guide except I updated the cuDNN v8.0.5 (November 9th, 2020), for CUDA 10.1

https://developer.nvidia.com/rdp/cudnn-download#a-collapse805-101 for the architecture category

For windows directly: https://developer.nvidia.com/comput...201106/cudnn-10.1-windows10-x64-v8.0.5.39.zip
Click to expand...

Sorry about the late reply. I also attempted the method suggested by @itsSticky but there is no luck on my end as well.
There was no hanging or delay when StarNet++ restoring the neural network checkpoint. But StarNet++ was using CPU only.

+ @NightBear
 
Bubkis so far. We need someone who knows how to program CUDA to work out what the problem is exactly. It seems that 3000 series RTX GPU's work differently but they are still so new and hard to find I guess the right people don't have them yet...
 
I also cannot execute starnet with my RTX 3060ti... So sad.
I have sent a private message to Nikita, the developer of starnet++. Hope he can give us a possible solution!
 
Woohoo it works!

Code: 
StarNet: Processing view: Mosaic_L_DC_MMT_nl
Writing swap files...
4614.686 MiB/s
Restoring neural network checkpoint: C:/Program Files/PixInsight/library/mono_starnet_weights.pb
Processing 2993 image tiles: done
04:13.45

Code: 
StarNet: Processing view: Mosaic_L_DC_MMT_nl
Writing swap files...
4660.575 MiB/s
Restoring neural network checkpoint: C:/Program Files/PixInsight/library/mono_starnet_weights.pb
Processing 2993 image tiles: done
28.517 s

Almost 9x as fast on a 48 megapixel monochrome image using a 3080 RTX, compared to CPU-only on an AMD 5900X.
 
Thanks a lot guys. Changed from a 20 Series card to a 3070TI

Worked with:
libtensorflow-gpu 2.4.0
CUDA 11.0
cudnn v8.1.1.33

Fantastic!
 
You must log in or register to reply here.
Back
Top