StarNet - new module for star removal

[mmisiura1]

Hi Rob,

Yes I did as per attached

You should copy files into PI's bin directory.

[Jkulin]

Thanks,

Just did that and still no joy.

0 Modules found either with the recursive checked or unchecked and all the files are in the c:\program files\pixinsight\bin directory

[mmisiura1]

Thanks,

Just did that and still no joy.

0 Modules found either with the recursive checked or unchecked and all the files are in the c:\program files\pixinsight\bin directory

These are not PI module files, those are files for standalone version. Here is what you need: https://sourceforge.net/projects/starnet/files/PixInsight_module/

[Jkulin]

Thanks for that, I didn't check the download as sourceforge came up with its automatic download in 5 secs, is now I have gone and I think downloaded the correct module and will check.

Many Thanks.

NB. Update, Many Thanks, it is now working, appreciate your patience.

[mmisiura1]

Thanks for that, I didn't check the download as sourceforge came up with its automatic download in 5 secs, is now I have gone and I think downloaded the correct module and will check.

Many Thanks.

NB. Update, Many Thanks, it is now working, appreciate your patience.

Yes, that automatic download is pretty confusing, I agree, but don't know how to change that...

[photonhunter1]

I had the same issue, my cpu is to old. reference the readme text file that came with Starnet.
"Second, the tensorflow.dll was compiled to support AVX instructions
and will only work if your CPU supports these AVX instructions.
Which means that CPUs made before 2011 will not work."

I'm running an AMD Ryzen 1700X that is only 1.5 years old and I receive the same error message. Can it be that my relatively new AMD is not supported? Does this mean that I am unable to use the application at the command line as well? Thanks in advance for the guidance.

Mike

[photonhunter1]

I had the same issue, my cpu is to old. reference the readme text file that came with Starnet.
"Second, the tensorflow.dll was compiled to support AVX instructions
and will only work if your CPU supports these AVX instructions.
Which means that CPUs made before 2011 will not work."

I'm running an AMD Ryzen 1700X that is only 1.5 years old and I receive the same error message. Can it be that my relatively new AMD is not supported? Does this mean that I am unable to use the application at the command line as well? Thanks in advance for the guidance.

I may have solved my problem - I didn't realize the auto download was not the PI file. I'm downloading the PI file and we'll try again.
Mike

[photonhunter1]

I had the same issue, my cpu is to old. reference the readme text file that came with Starnet.
"Second, the tensorflow.dll was compiled to support AVX instructions
and will only work if your CPU supports these AVX instructions.
Which means that CPUs made before 2011 will not work."

I'm running an AMD Ryzen 1700X that is only 1.5 years old and I receive the same error message. Can it be that my relatively new AMD is not supported? Does this mean that I am unable to use the application at the command line as well? Thanks in advance for the guidance.

I may have solved my problem - I didn't realize the auto download was not the PI file. I'm downloading the PI file and we'll try again.
Mike

New file, same results. Does anyone know if the Ryzen 1700X does not support AVX?

[dave_galera]

I also get this message, but I just ignore it and everything works just fine, does it fail on your installation or is it working?
I think it may be just a general warning message.

[photonhunter1]

Guess I either read the README file too quickly or I didn't understand that all of the files in the Starnet folder, with the exception of the LICENSE and README files, needed to be in PixInsight\bin and not PixInsight\bin\Starnet_PI. We're up and running. Thank you.

Mike

[Juan Conejero]

To Nikita and all users interested in this module,

Sorry so much for not being able to invest the required time and resources to put this module in the standard tool set. Nikita already uploaded his source code to our GitLab repository, and I have been taking a look. The module requires substantial work, but this is not difficult, just time-consuming. Perfectly normal for a new PixInsight tool.

The real problem here is TensorFlow. This is an extremely complex and huge software library (undoubtedly one of the most complex ones I've ever seen). There are several problems. Once of them is building TensorFlow on all platforms supported by PixInsight, which is not a trivial task. It is a very complex task that requires time and care. Unfortunately, the prebuilt libraries provided by Google don't work, so the only way to get this working is rebuilding everything.

Another problem is TensorFlow's dimensions. If included in PixInsight's standard distribution, it would increase the size of a PixInsight installation by about 300 MB. A fixed footprint like that is questionable to support a single module. A good solution to this problem is creating a dedicated update repository for StarNet on our official software distribution server, so the StarNet module would be an optional component. For example, the same will happen with the new Gaia star databases that we are preparing for version 1.8.7 of PixInsight.

Please bear with me while I can find the required time to perform all of these tasks as required. Currently we have a lot of priorities that require all of my attention and concentration. The new version of PixInsight comes with very important improvements (A rewritten StarAlignment tool with a completely new distortion correction algorithm, important new features in ImageIntegration and DrizzleIntegration, just to name a few) on which I have been working very hard during July, August and September. I ask for patience...

[georg.viehoever]

...The real problem here is TensorFlow. This is an extremely complex and huge software library (undoubtedly one of the most complex ones I've ever seen). There are several problems. Once of them is building TensorFlow on all platforms supported by PixInsight, which is not a trivial task. It is a very complex task that requires time and care. Unfortunately, the prebuilt libraries provided by Google don't work, so the only way to get this working is rebuilding everything.

Another problem is TensorFlow's dimensions. If included in PixInsight's standard distribution, it would increase the size of a PixInsight installation by about 300 MB. A fixed footprint like that is questionable to support a single module. A good solution to this problem is creating a dedicated update repository for StarNet on our official software distribution server, so the StarNet module would be an optional component. For example, the same will happen with the new Gaia star databases that we are preparing for version 1.8.7 of PixInsight.
....

Juan,

Yes, this is a real problem, and I have encountered it elsewhere as well. If you try to do everything in-process (i.e. inside the PixInsight binary), then each and every library that you use needs to be binary compatible, i.e. has to be built with a compatible compiler, and the library itself and all the libraries that it depends on need to be compatible with PixInsight and its libraries. For Tensorflow (and Python, and numpy, and scipy, and astropy, and ....) the number of dependent libs is huge.

A minimal installer for Tensorflow with Python on Linux is something like 320MB, and if you add the usual scientific stuff, you are at 640MB (compressed). Installed you are at 3.2GB.

A possible solution: Run Tensorflow et. al as an external process, and communicate with them via some RPC mechanism (like gRPC, Python pickle or the mechanism you designed for communication between PI instances). Let users install Tensorflow as an addon (for instance via the Anaconda distribution https://www.anaconda.com/distribution/, very liberal license and rather complete), which also removes from you the burden to compile all this stuff with PI.

Georg

[Tertsi]

I just installed PI Linux module and if I use StarNet++ process to image I get an error message:


Restoring neural network checkpoint...
*** Error: Checkpoint file not found!


I got it working after executing following command in ProcessConsole first:

cd /opt/PixInsight/bin

Then Stanet++ process woks.

[mmisiura1]

A possible solution: Run Tensorflow et. al as an external process, and communicate with them via some RPC mechanism (like gRPC, Python pickle or the mechanism you designed for communication between PI instances). Let users install Tensorflow as an addon (for instance via the Anaconda distribution https://www.anaconda.com/distribution/, very liberal license and rather complete), which also removes from you the burden to compile all this stuff with PI.

This is a good idea, I personally like it, but unfortunately majority of users will hate it. Tensorflow does not come with Anaconda by default, so people will have to install it, which will be problematic for many. Plus we will have possible issues with a version control, which will cause problems for some people, I am sure.

I already tasted all this with my Python version of StarNet, many people just don't get all these things and don't wanna mess with it.

[mmisiura1]

To Nikita and all users interested in this module,

Sorry so much for not being able to invest the required time and resources to put this module in the standard tool set. Nikita already uploaded his source code to our GitLab repository, and I have been taking a look. The module requires substantial work, but this is not difficult, just time-consuming. Perfectly normal for a new PixInsight tool.

The real problem here is TensorFlow. This is an extremely complex and huge software library (undoubtedly one of the most complex ones I've ever seen). There are several problems. Once of them is building TensorFlow on all platforms supported by PixInsight, which is not a trivial task. It is a very complex task that requires time and care. Unfortunately, the prebuilt libraries provided by Google don't work, so the only way to get this working is rebuilding everything.

Another problem is TensorFlow's dimensions. If included in PixInsight's standard distribution, it would increase the size of a PixInsight installation by about 300 MB. A fixed footprint like that is questionable to support a single module. A good solution to this problem is creating a dedicated update repository for StarNet on our official software distribution server, so the StarNet module would be an optional component. For example, the same will happen with the new Gaia star databases that we are preparing for version 1.8.7 of PixInsight.

Please bear with me while I can find the required time to perform all of these tasks as required. Currently we have a lot of priorities that require all of my attention and concentration. The new version of PixInsight comes with very important improvements (A rewritten StarAlignment tool with a completely new distortion correction algorithm, important new features in ImageIntegration and DrizzleIntegration, just to name a few) on which I have been working very hard during July, August and September. I ask for patience...

Juan,

Thank you for looking into it! This is not a trivial task indeed!