Support for new Apple M1 "System-on-Chip" Processors

Any news or updates on this? Really looking forward to an M1 (M1X? M2?) iMac later this year, but I use StarNet quite often in my processing routines, so it would be a real drag to lose it.
 
The current situation is as follows:

- Our main codebase, including the entire PCL development framework and all PixInsight modules, is standard C++17 with just a few exceptions. This means that it can be ported to run on ARM processors quite easily.

- The PixInsight core application depends on the Qt framework. Currently we are using the latest Qt 5.15.3 LTS for the incoming version 1.8.8-8 of PixInsight, but Qt 5 does not support Apple Silicon processors. Full support of these processors on macOS will only be available in Qt 6. However, the current versions 6.0.x of Qt are just proofs of concept, completely useless for a large and complex application like PixInsight. Stable and complete support of Apple Silicon for all Qt modules (including QtWebEngine, which is a port of the Chromium browser) is supposed to arrive with Qt 6.2 by the end of this year. However, based on my experience of more than 15 years doing Qt-based development, I don't expect this to happen before Qt 6.5, to be realistic. This means from 1 to 1.5 years.

- Our JavaScript development platform (PJSR) uses Mozilla's SpiderMonkey engine. The SM version 24 that we are using in current versions of PixInsight is now obsolete and does not support ARM processors with the appropriate level of optimization. I am starting to port the entire PJSR to a current version of SpiderMonkey, but the amount of work required to complete this task is huge. It requires a complete rewrite of hundreds of thousands of lines of complex and extremely delicate source code. I need a lot of time to complete this task, especially considering that we have many other ongoing projects and priority tasks.

- PixInsight depends on many other open-source and proprietary libraries, including file format support libraries such as CFITSIO, LibTiff, LibRaw, etc, and several utility, numerical and hardware control libraries. All of these components must also be ported to run on Apple Silicon processors in a completely stable way. This ranges from very simple cases to quite complex ones, which will also require a lot of work.

To summarize: being realistic, don't expect a native Apple Silicon version of PixInsight to be released before one year, and probably more time, depending on how other third-party components evolve, especially Qt.

I use StarNet quite often in my processing routines

Then you have a strong dependency on a process that performs an extremely specialized task. This is not a good thing IMO. Image processing is much richer and more interesting than removing stars.
 
So far I'm quite satisfied with PI's performance under Rosetta2 (Mac mini M1/16GB), it's almost twice as fast than my old 2015 intel iMac.

Of course it would be great to have a native version soon or to have the Starnet module modified to run on ARM processors. But for now it works fine.
 
Then you have a strong dependency on a process that performs an extremely specialized task. This is not a good thing IMO. Image processing is much richer and more interesting than removing stars.

I was more concerned about the many script utilities I use (EZ Processing Suite, etc) that seem to rely on StarNet.

I appreciate the update, and your hard work is always appreciated. I used to do very basic coding a lifetime ago in high school and college, so I understand (on a broad brush strokes level) the amount of work something like this entails.

I think Marcelofig might be on to something. If Rosetta2 runs PI quite well on M1 processors (and I've seen more than enough reports to believe that it does) other than StarNet, might just the StarNet module be ported/modified as a stop-gap measure until all of PI with its dependencies are ported over? That would "kick the can down the road" quite a long distance, and provide a far more casual schedule for you to finish the overall process.
 
might just the StarNet module be ported/modified as a stop-gap measure until all of PI with its dependencies are ported over? That would "kick the can down the road" quite a long distance, and provide a far more casual schedule for you to finish the overall process.

starnet itself is OK, but it has a huge external dependence on tensorflow. and i'm not sure google has released an ARM binary for this either.

i was looking at this yesterday - to see what it would take to recompile tensorflow w/o AVX2/FMA instructions. it looks like a lot of work and is pretty opaque since google relies on specialized tools as part of the compile flow for tensorflow. it's not even clear to me that dylibs are an end product of their published flow. also there is a depenence on python and python is a serious world of hurt when it comes to getting the exact version of python that they expect you to have installed.

rob
 
I probably should have read this before buying an m1 macbook air the other day

I dont use starnet that much but a lot of stuff like ezprocessing suite I do use relies in starnet to generate starmasks. this will be problematic.

This little air seems very good otherwise and for $1600 is a good deal. I can still return it and maybe get a 16" intel MPB but that's a lot more $$ to get specd how I want and maybe not even faster.
 
i havent looked into rosetta2 at all... for the M1 tensorflow to work with the x86 version of PI, rosetta2 would have to support loading and calling ARM dynamic libraries from a translated x86-64 binary, and i don't know if this would work. in theory it should work since lots of dylibs that are part of the OS are ARM and they can be called fine from x86-64 binaries. but we'd have to see. i only have access to one M1 mac and it is pretty heavily used by my daughter so i don't get many chances to experiment with it.
 
Hi, Personally losing Starnet is not an issue, PI works well on Mac mini M1 with 16Gig of Ram... I was wondering if it would be relevant to change some settings in Preferences to further improve PI performance on M1? Like Number of threads, etc. Or leave it as is...? Of course, when fully compatible, it will be fantastic!!!
 

Attachments

  • Capture d’écran, le 2021-05-20 à 22.27.29.png
    Capture d’écran, le 2021-05-20 à 22.27.29.png
    489.4 KB · Views: 102
The current situation is as follows:

- Our main codebase, including the entire PCL development framework and all PixInsight modules, is standard C++17 with just a few exceptions. This means that it can be ported to run on ARM processors quite easily.

- The PixInsight core application depends on the Qt framework. Currently we are using the latest Qt 5.15.3 LTS for the incoming version 1.8.8-8 of PixInsight, but Qt 5 does not support Apple Silicon processors. Full support of these processors on macOS will only be available in Qt 6. However, the current versions 6.0.x of Qt are just proofs of concept, completely useless for a large and complex application like PixInsight. Stable and complete support of Apple Silicon for all Qt modules (including QtWebEngine, which is a port of the Chromium browser) is supposed to arrive with Qt 6.2 by the end of this year. However, based on my experience of more than 15 years doing Qt-based development, I don't expect this to happen before Qt 6.5, to be realistic. This means from 1 to 1.5 years.

- Our JavaScript development platform (PJSR) uses Mozilla's SpiderMonkey engine. The SM version 24 that we are using in current versions of PixInsight is now obsolete and does not support ARM processors with the appropriate level of optimization. I am starting to port the entire PJSR to a current version of SpiderMonkey, but the amount of work required to complete this task is huge. It requires a complete rewrite of hundreds of thousands of lines of complex and extremely delicate source code. I need a lot of time to complete this task, especially considering that we have many other ongoing projects and priority tasks.

- PixInsight depends on many other open-source and proprietary libraries, including file format support libraries such as CFITSIO, LibTiff, LibRaw, etc, and several utility, numerical and hardware control libraries. All of these components must also be ported to run on Apple Silicon processors in a completely stable way. This ranges from very simple cases to quite complex ones, which will also require a lot of work.

To summarize: being realistic, don't expect a native Apple Silicon version of PixInsight to be released before one year, and probably more time, depending on how other third-party components evolve, especially Qt.



Then you have a strong dependency on a process that performs an extremely specialized task. This is not a good thing IMO. Image processing is much richer and more interesting than removing stars.

Juan,

Meanwhile, PI works quite well with Rosetta2, I get total benchmark of about 52 seconds... I tried some tweaks in Global Preferences, do you have any suggestions so to get optimal performance of PI on macOS Big Sur and M1 chip... Example, I have disable CUDA... what would be the best threads numbers reading and writing, or any other setting to gain in performance... like, how to improve swap storage... you wrote about this subject but cannot get back to it ... Many thanks, AC
 
Last edited:
hello,
i am planning to buy PI, currently I have macbook pro 2019, probably I will "upgrade" to new m1 chip, my question is, even if PI is not built on m1 sdk, PI will run in mac m1?
regards
 
Hi, wondering if someone can help. I installed PI on a new iMac M1 but each time it launches and gets to the loading modules point it crashes. I tried re-installing and it just keeps happening. Anyone have any ideas?
I've attached a screen shot of the crash log. It seems to happen in regards to libtensorflow??
Thanks
Carlos
 
Any news on when we can expect a native PI version for M1/Pro/Max apple computer? Also a solution to allow starnet to function with M1?
 
starnet will actually work on an M1 processor if you've got the chops to replace the tensorflow libraries that ship with PI with the tensorflow libraries that shipped with the command line version of StarNet++ for OSX.

the only problem is that the PI installer deletes the StarNet module if the architecture is ARM, so you need to use Pacifist to dig out the dylib from the installer package and manually put it in the /Applications/PixInsight/bin folder.

rob
 
Hi all, I am still on Mojave on my M1, will there be any problem going to Monterey? ( PI works just fine (-) Starnet). Thanks.
 
starnet will actually work on an M1 processor if you've got the chops to replace the tensorflow libraries that ship with PI with the tensorflow libraries that shipped with the command line version of StarNet++ for OSX.

the only problem is that the PI installer deletes the StarNet module if the architecture is ARM, so you need to use Pacifist to dig out the dylib from the installer package and manually put it in the /Applications/PixInsight/bin folder.

rob

Can you provide more details? The libraries in the MacOS standalone StarNet contains libtensorflow.so and libtensorflow_framework.so. Simply put these in the PixInsight bin directory? Do they need to be renamed and replace the dylib versions? Or maybe not rename but just remove the dylib versions?

I plucked out the StarNet-pmx.dylib out of the PixInsight distribution and put it back in the bin directory, but when I restart PixInsight, the StarNet module is not present anywhere under the Process menu. If I open the Install Module dialog, performing a Search doesn't find it either. Was something else removed from the distribution in addition to the StarNet-pmx.dylib file?

Alex
 
no nothing else was removed - when PI searches for modules, any module it finds it attempts to run, and if it won't run, it's not presented to the user. so basically you don't see it because the tensorflow libraries are missing.

anyway, with PI shut down, first move all of the libtensorflow* files out of the way. you can move them to a subdirectory or just move them into your home directory or whatever. then, copy libtensorflow.so and libtensorflow_framework.so to the pixinsight bin directory, and then rename them to libtensorflow.2.dylib libtensorflow_framework.2.dylib, respectively

one thing to check is if the quarantine flag is set on StarNet-pxm.dylib to find out, using Terminal, do "ls -l StarNet*" without the quotes and see if there is an @ sign in the stuff on the left (would look something like this: -rwxr-xr-x@). if there is an @, you need to clear the quarantine bit with "sudo xattr -c StarNet-pxm.dylib" (again no quotes)

you'll need to type in your account password becuase of the sudo command. also, my permissions look like this:

-rwxr-xr-x 1 root wheel 741952 Sep 2 11:59 StarNet-pxm.dylib
-rw-r--r--@ 1 root wheel 148513576 Oct 29 13:29 libtensorflow.2.dylib
-rw-r--r--@ 1 root wheel 12942544 Oct 29 13:29 libtensorflow_framework.2.dylib

as you can see the quarantine bits are set on the libraries so apparently this does not matter there, but the quarantine bit needs to be cleared on StarNet-pxm.dylib. the permissions on the StarNet-pxm.dylib module need to be 755, so if they are not you can do "sudo chmod 755 StarNet-pxm.dylib"

after all this stuff if you start PI and search for modules it should find StarNet.

rob
 
Back
Top