PixInsight 1.8.8-5 Released - StarNet module installation

[pfile]

someone in my local club realized that if they disable SIP (system integrity protection) then the module will load OK. i guess this shouldn't come as a surprise, but i didn't think of it. although in a lot of cases after installing something with SIP turned off it still works with SIP turned back on, in this case when SIP is re-enabled PI fails to load the module on the next launch.

so that's not a solution unless people are comfortable with turning SIP off.

rob

[Juan Conejero]

Hi all,

The problems you are experiencing are being caused by the latest changes Apple has introduced in their software security policies.

All newly distributed macOS applications must be notarized by Apple. The notarization process requires all executable files to be digitally signed with a valid developer certificate issued by Apple, and all binaries, including dynamic libraries, must be executed with the hardened runtime enabled. Finally, a scriptable application like PixInsight, which includes a JavaScript engine, requires some special entitlements to be able to execute JIT-generated code. The hardened runtime and the entitlements can only be selected as part of the digital signature process, which in turn makes a digital signature absolutely necessary for all seriously distributed applications on macOS.

A signed/notarized application bundle cannot be modified. You can't change a single bit of its contents, or the digital signature will be invalidated and the application won't run. This means that if I have to change a small text file of, say, 1 KB inside PixInsight.app, I'll have to released an update of nearly 200 MB to replace the entire application bundle. Fortunately, current Apple policies allow us to preserve the PixInsight distribution, i.e. distributing all modules on the /Applications/PixInsight/bin folder. If this changes in the future, then alas, the macOS version of PixInsight will lose our automatic updates system. I hope this won't happen, and I can't see any valid reason for it to happen, but who knows, everything is basically marketing-guided to keep users 'feeling secure'.

Disabling SIP is a very bad idea. Please don't do that because your system will be vulnerable, which is extremely dangerous.

So the only way to keep this module running is to sign it digitally with a valid Apple developer ID. Besides this, you cannot put the required TensorFlow library inside the application bundle, for the reasons explained above, so you'll probably have to use the install_name_tool utility to change the location where the module wants to load it from. If you provide a link where I can download the module and all required files, I'll try to fix this problem quickly.

Please note that this module is not part of the official PixInsight distribution, or obviously these problems wouldn't exist. This module requires significant changes and refactoring,  and its inclusion in our development chain is very problematic, mainly because of its dependencies on third-party components. Currently I have no free time to invest in these tasks because I am working on other higher priority projects.

[pfile]

tensorflow can live in /usr/lib at least on older versions of os x and pi, so hopefully installation of the tensorflow dylibs into the pixinsight application bundle is not necessary.

the zip file for starnet is on sourceforge at

https://sourceforge.net/projects/starnet/files/PixInsight_module/

rob

[fmeschia]

Hi Juan, Rob –
Do you know if all the module dylibs need to be signed by a certificate belonging to the same team id? I’m asking because I tried signing the StarNet module with my self-signed certificate (which is obviously trusted on my local machine), but still PI doesn’t “see” it. Or maybe is an actual Apple-issued certificate required?
Francesco

[dave_galera]

Hi Juan, Rob –
Do you know if all the module dylibs need to be signed by a certificate belonging to the same team id? I’m asking because I tried signing the StarNet module with my self-signed certificate (which is obviously trusted on my local machine), but still PI doesn’t “see” it. Or maybe is an actual Apple-issued certificate required?
Francesco

It also has to be notarized by Apple otherwise Gateway will throw it out

[pfile]

Hi Juan, Rob %u2013
Do you know if all the module dylibs need to be signed by a certificate belonging to the same team id? I%u2019m asking because I tried signing the StarNet module with my self-signed certificate (which is obviously trusted on my local machine), but still PI doesn%u2019t %u201Csee%u201D it. Or maybe is an actual Apple-issued certificate required?
Francesco

It also has to be notarized by Apple otherwise Gateway will throw it out

i don't know for sure about the team thing, this is what prevented me from paying apple $100. i should have asked on sjaa-astroimaging if anyone already had a developer certificate.

here's how to find out what the exact error is though - quit PI, edit your ~/Library/PixInsight/PixInsight.plist to manually add the starnet module, and restart PI. here is what the lines look like but you'll have to increment the module number and perhaps add these lines at the end of the section rather than in any random place:

        <key>000.Modules.00000046</key>
   <string>@@pxi_bin_dir/StarNet-pxm.dylib</string>

when PI fails to load the module it should put up a dialog with the error message and the message will also be on the console. hopefully that will illuminate the situation further.

rob

edit: i didn't read your message carefully enough - thought you had a paid apple account - i already tried a self-signed certificate and that doesn't work. thought you had an apple cert. i guess i might as well renew my apple developer ID to find out, it's just $100.

[dave_galera]

In the apple documentation from version 10.15.x all modules have to be notarized by apple otherwise Gatekeeper will reject the request, this is what they say:

Beginning in macOS 10.15, all software built after June 1, 2019, and distributed with Developer ID must be notarized. However, you aren't required to notarize software that you distribute through the Mac App Store because the App Store submission process already includes equivalent security checks.

https://developer.apple.com/documentation/xcode/notarizing_macos_software_before_distribution

[fmeschia]

In the apple documentation from version 10.15.x all modules have to be notarized by apple otherwise Gatekeeper will reject the request, this is what they say:

Beginning in macOS 10.15, all software built after June 1, 2019, and distributed with Developer ID must be notarized. However, you aren't required to notarize software that you distribute through the Mac App Store because the App Store submission process already includes equivalent security checks.

https://developer.apple.com/documentation/xcode/notarizing_macos_software_before_distribution

Yes but that’s not the issue, because it is already broken in 10.14.6 which doesn’t enforce notarization yet.

[pfile]

from what i read, the problem is that there is a compile-time flag, which can be set in xcode at the time the application is built, that enforces this check (that all dylibs loaded by the application are signed). i think that explains why PI -4 and earlier have no problem, as ostensibly they were built without this flag set. i don't know what version(s) of OSX this flag would apply to, but i know on 10.14 the unsigned module won't load.

rob

[fmeschia]

from what i read, the problem is that there is a compile-time flag, which can be set in xcode at the time the application is built, that enforces this check (that all dylibs loaded by the application are signed). i think that explains why PI -4 and earlier have no problem, as ostensibly they were built without this flag set. i don't know what version(s) of OSX this flag would apply to, but i know on 10.14 the unsigned module won't load.

rob

That flag, if set, is already enforced in 10.14.