PixInsight Forum (historical)
PixInsight => General => Topic started by: tdgm on 2020 February 28 17:57:57
-
HI Juan
I d not know if this is my computers (Mac) but afterinstallation of 1.8.8.5 I can't install Stanet+ module. Have followed all instructions and allow the third party developer in security/Privacy. Please Help
Steve
-
I d not know if this is my computers (Mac) but afterinstallation of 1.8.8.5 I can't install Stanet+ module. Have followed all instructions and allow the third party developer in security/Privacy. Please Help
I experienced exactly the same problem. Reverted back to 1.8.8-4 for now.
-
ill check it out this weekend and rebuild the OSX version, assuming there is a new PCL available for -5. then nikita will have to repackage and distribute it.
rob
-
before i installed -5 i backed up the -4 installation. i then copied everything starnet related in the old bin/ folder to the new bin/ folder, and likewise with the lib/ folder (the weights.)
starnet runs with no problem this way under -5. if there is a problem with gatekeeper i wouldn't see it because ostensibly the starnet dylib is already blessed. also i am way back on high sierra, so if there is a problem with newer versions of OSX i won't see that. i did compile the module on high sierra and at least up till now no one running 10.14 or 10.15 had problems installing and running starnet...
rob
-
Hi Rob
Well, I tried your trick to insert backup files into a new instance of PxInsight 1.8.8.5. I am running Catalina on My NBP and it still does not recognize the StarNet + Module so I went back to 1.8.8.4.
How do you backup pixinsight?
I just moved the program to a different folder (not trash)and copied the correct files into the correct places but no joy on Catalina. My other machine is Mac OSX 10.14.6 and the same thing happens when I try to install the module.
Any other suggestions
Steve
-
before i installed -5 i backed up the -4 installation. i then copied everything starnet related in the old bin/ folder to the new bin/ folder, and likewise with the lib/ folder (the weights.)
starnet runs with no problem this way under -5. if there is a problem with gatekeeper i wouldn't see it because ostensibly the starnet dylib is already blessed. also i am way back on high sierra, so if there is a problem with newer versions of OSX i won't see that. i did compile the module on high sierra and at least up till now no one running 10.14 or 10.15 had problems installing and running starnet...
I followed the same procedure on Catalina 10.15.3 except did not save the "blessed" dylib. Instead, I installed a copy directly from the distribution. When attempting to install with Install Modules, Gatekeeper was triggered, but after approval StarNet was not installed, nor did Install Modules find it anymore in a recursive search. After observing this, I deleted everything and reproduced the same steps, achieving the same result.
Today, after seeing Rob's report, I tried for a third time using an already-blessed dylib. Everything acted the same as above, except this time the Install Modules did not find StarNet at all and no Gatekeeper notification was generated.
In all cases, PixInsight reports, "0 additional PixInsight module(s) were found on directory: /Applications/PixInsight/bin (Recursive search)". It is as if PixInsight thinks the module is installed, although it is not.
John
-
Hi Rob
I have the exact same problem with Starnet running Mojave.
Frank
-
i will have to try all this on a machine running 10.14. i don’t have one running 10.15 yet... but i have a laptop with 10.14.
-
with pixinsight not running, and from the terminal can you guys try:
spctl --add --label "StarNet" /Applications/PixInsight/bin/StarNet-pxm.dylib
and tell me what the output of the program is? you will need to enter your password when spctl is run - it pops up a dialog box.
also can you try then running PI and adding the module and see if it works after this? also check if the module is already installed, sometimes that happens.
on my system if i ask for the status on /Applications/PixInsight/bin/StarNet-pxm.dylib, i get "rejected" but of course it does load and is running.
rob
-
Hi Rob,
I am having this issue on Mojave. I ran the spctl command, provided my password. The dialog closed and no response was returned on the command line. But I still could not install the StarNet++ module nor was it already installed in the processes menu. I tried also using the copy from my previous -4 installation but there was no change in behavior.
Will StarNet be part of the Mac distribution in 1.8.9?
Andy
-
ok well i'm going to have to try doing installing everything from scratch on a 10.14 laptop (i dont have 10.15 yet anywhere) and see what happens. but since you have problems on 10.14 hopefully figuring it out there will make it work for 10.15 as well.
i should have mentioned that there would be no response on the command line to this program - all it is supposed to do is tell gatekeeper that the library is safe to execute. because PI is pretty silent about why a module doesn't load it was kind of a shot in the dark under the theory that gatekeeper is preventing it from running. but PI will also fail to display/load a module if it can not be started properly (like for instance if the tensorflow libraries are missing.)
which actually reminds me - since it could be failing due to the tensorflow libraries being blocked by gatekeeper, maybe you can also try:
cd /Applications/PixInsight/bin
spctl --add --label "StarNet" libtensorflow*
and then see if PI will load the module. however, in theory those libraries should have been signed by tensorflow.org and should work.
anyway, i'm not sure why copying everything out of -4 worked for me, but i have some ideas.
i think juan has declined to include starnet in the main PI distribution because as distributed, it requires processors with a minimum set of AVX instructions to run. plus the weights files are pretty huge and overall would increase the distribution size of PI by 500MB or something.
rob
-
whoops, actually i had forgotten where the tensorflow libraries are supposed to live. i had a copy in /Applications/PixInsight/bin but i dont think OSX will find them there.
the tensorflow libraries are supposed to be in /Applications/PixInsight/PixInsight.app/Contents/Frameworks/
on my system i also have them in /usr/local/lib/ so even though i had forgotten to copy those libraries into that directory after installing -5, it's still working for me.
so anyway the command i posted above should be:
cd /Applications/PixInsight/PixInsight.app/Contents/Frameworks/
spctl --add --label "StarNet" libtensorflow*
rob
-
Hi Rob
I tried the terminal suggestions with my Mojave computer and 1.8.8.5 and no joy still will not see module. I have also tried on my Catalina machine and still no joy. So now it is in your ballpark. I am not proficient past this point of evaluation. All is fine with1.8.8.4 and not 1.8.9.5.
Steve
-
Hi Rob,
I tried blessing the tensorflow libraries with spctl but I see no change.
I just noticed something in the processing console upon startup of PixInsight:
--
...
/Applications/PixInsight/bin/NoiseGeneration-pxm.dylib
* Scripting resources updated.
49 of 49 module(s) installed.
*** Error:
The PixInsight Core application has been launched with module installation disabled. Cannot proceed.
Processing script file: /Applications/PixInsight/etc/init.pjsr/pjsr-core-defs.js
* 1 initialization script(s) executed.
Processing script file: /Applications/PixInsight/etc/startup/startup.scp
* 1 startup script(s) executed.
...
--
So the error "The PixInsight Core application has been launched with module installation disabled. Cannot proceed." seems to be a bit part of my problem. I am going to start from a fresh install of PI 1.8.8-5 and see if I can determine at which point I start see this error.
Andy
-
thank you andrew, that is interesting info.
the spctl stuff was a shot in the dark... thought it was worth a try.
i am not familiar with that error and we'll have to get Juan to weigh in on that - never seen that before. i have been busy all morning and have not yet been able to do a clean installation to find out what's going wrong. clearly there is a problem as people are reporting this on cloudynights as well.
rob
-
I think the error was probably my doing. When I realized StarNet was not loading, I started modifying permissions of the StarNet libraries to match other modules.
When I reverted back to a clean install, I never saw that error again.
I also tried installing the Tensor flow modules in /usr/local/lib but saw no change.
When I choose browse under Module Installation, I can see StarNet in the PixInsight/bin dir, it's just greyed out and not installable.
For now I am reverting back to 1.8.8-4.
Thanks for your suggestions Rob.
-
ok - anyway at some point i'll be able to sit down and experiment with this.
rob
-
Hi Rob,
I have just tried an initial install of -5 and I do not get the error message that Andrew reported above, all seems ok on that front, however the StarNet Module does not show in the list as reported by others.
* Scripting resources updated.
49 of 49 module(s) installed.
Processing script file: /Applications/PixInsight/etc/init.pjsr/pjsr-core-defs.js
* 1 initialization script(s) executed.
Processing script file: /Applications/PixInsight/etc/startup/startup.scp
* 1 startup script(s) executed.
_____________ ________
-
I am checking it out right now on a 10.14.6 machine... going to create a new user and install everything from scratch
-
I guess I see the problem, however, I am not sure why this is happening now vs. before, since it seems like an OS-level (gatekeeper related) problem. however some people are reporting that it still works on -4 so perhaps it's some nuance of how -5 makes system calls to load modules.
like others, PI won't find the module on search under -5. so what I did was just add the module to the PixInsight.plist, which forces PI to try to load the module at startup time. I get this error:
Cannot load library /Applications/PixInsight/bin/StarNet-pxm.dylib: (dlopen(/Applications/PixInsight/bin/StarNet-pxm.dylib, 6): no suitable image found. Did find:
/Applications/PixInsight/bin/StarNet-pxm.dylib: code signature in (/Applications/PixInsight/bin/StarNet-pxm.dylib) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.): Module load error
which is a little weird because I thought this had been ad-hoc signed but maybe not. so maybe if I rebuild it (or just do an ad-hoc signing of the dylib, if that is separately possible) we'll be good. anyway I have to do some research.
rob
-
looking into this a little more, -5 may have been compiled in such a way that modules need to be signed by the same developer that built the application, that is, Juan. I am not 100% sure of this though. could be that a different paid apple developer account could sign the module in such a way that it will be loaded by PI.
rob
-
Thanks for looking into this Rob! I think you are on the right track.
-
Hi Rob
I too thank you for looking into this problem. i would have thought more people would have noticed this problem with the new release but either way you re on the Case Just like " Colombo" you will always get your man.
Steve
-
Just like " Colombo" you will always get your man.
Steve
Now you are showing your age LOL
-
Rob,
For your info when I try to do a search in add module I get a pop-up with following error message:
"
StarNet-pxm.dylib cannot be opened because the developer cannot be verified.
macOS cannot verify that this app is free from malware.
"
(I can't attach the screen shot because this forum craps out when I try to add it)
-
yes there is something wrong with the forum for sure.
i didn't see that particular error when i was testing this yesterday but it's certainly consistent. when i built the dylib, i didn't sign it. yesterday i tried signing it with an ad-hoc signature which didn't work, and then i made a self-signed certificate and signed it with that and that didn't work either.
it looks like there's a flag in XCode that would turn off this check and it's possible that older versions of PI were built with this flag turned off. i don't know if juan turned this on on purpose or if it is just a new default. what i have not been able to determine yet is if i were to pay the $100 to apple for a developer account and then sign the dylib with a "real" signature if it would load. there's a chance that the dylib will not load unless it is signed by the same "team" as the application was, and obviously i can't do that because juan has his own team and developer certificate with apple.
i feel like if applications were required to have all dylibs they might load signed by the same team or developer that everything would break, so perhaps just having my own real developer certificate would work.
rob
-
Hi
Presume this is a non windows problem as I just installed it no problem in windows :)
Harry
-
yes this has to do with OSX's malware protection stuff.
rob
-
Rob,
I have been playing with this:
xattr -d com.apple.quarantine /Applications/PixInsight/bin/StarNet-pxm.dylib
Does this give you any ideas? .......unfortunately I am a mainframe (NCR Criterian and IBM AS400) developer not Unix so you may make more of it than me.
-
at least on my desktop machine the tool reports the dylib has no such attribute. i think that attribute is for files that were directly downloaded from the web (and mine was built locally.) i'll have to try it on the laptop and see. but the error messages i received point straight to the lack of a valid code signature for the dylib...
rob
-
someone in my local club realized that if they disable SIP (system integrity protection) then the module will load OK. i guess this shouldn't come as a surprise, but i didn't think of it. although in a lot of cases after installing something with SIP turned off it still works with SIP turned back on, in this case when SIP is re-enabled PI fails to load the module on the next launch.
so that's not a solution unless people are comfortable with turning SIP off.
rob
-
Hi all,
The problems you are experiencing are being caused by the latest changes Apple has introduced in their software security policies.
All newly distributed macOS applications must be notarized (https://developer.apple.com/documentation/xcode/notarizing_macos_software_before_distribution) by Apple. The notarization process requires all executable files to be digitally signed with a valid developer certificate issued by Apple, and all binaries, including dynamic libraries, must be executed with the hardened runtime (https://developer.apple.com/documentation/security/hardened_runtime) enabled. Finally, a scriptable application like PixInsight, which includes a JavaScript engine, requires some special entitlements (https://developer.apple.com/documentation/bundleresources/entitlements) to be able to execute JIT-generated code. The hardened runtime and the entitlements can only be selected as part of the digital signature process, which in turn makes a digital signature absolutely necessary for all seriously distributed applications on macOS.
A signed/notarized application bundle cannot be modified. You can't change a single bit of its contents, or the digital signature will be invalidated and the application won't run. This means that if I have to change a small text file of, say, 1 KB inside PixInsight.app, I'll have to released an update of nearly 200 MB to replace the entire application bundle. Fortunately, current Apple policies allow us to preserve the PixInsight distribution, i.e. distributing all modules on the /Applications/PixInsight/bin folder. If this changes in the future, then alas, the macOS version of PixInsight will lose our automatic updates system. I hope this won't happen, and I can't see any valid reason for it to happen, but who knows, everything is basically marketing-guided to keep users 'feeling secure'.
Disabling SIP is a very bad idea. Please don't do that because your system will be vulnerable, which is extremely dangerous.
So the only way to keep this module running is to sign it digitally with a valid Apple developer ID. Besides this, you cannot put the required TensorFlow library inside the application bundle, for the reasons explained above, so you'll probably have to use the install_name_tool utility to change the location where the module wants to load it from. If you provide a link where I can download the module and all required files, I'll try to fix this problem quickly.
Please note that this module is not part of the official PixInsight distribution, or obviously these problems wouldn't exist. This module requires significant changes and refactoring, and its inclusion in our development chain is very problematic, mainly because of its dependencies on third-party components. Currently I have no free time to invest in these tasks because I am working on other higher priority projects.
-
tensorflow can live in /usr/lib at least on older versions of os x and pi, so hopefully installation of the tensorflow dylibs into the pixinsight application bundle is not necessary.
the zip file for starnet is on sourceforge at
https://sourceforge.net/projects/starnet/files/PixInsight_module/
rob
-
Hi Juan, Rob –
Do you know if all the module dylibs need to be signed by a certificate belonging to the same team id? I’m asking because I tried signing the StarNet module with my self-signed certificate (which is obviously trusted on my local machine), but still PI doesn’t “see” it. Or maybe is an actual Apple-issued certificate required?
Francesco
-
Hi Juan, Rob –
Do you know if all the module dylibs need to be signed by a certificate belonging to the same team id? I’m asking because I tried signing the StarNet module with my self-signed certificate (which is obviously trusted on my local machine), but still PI doesn’t “see” it. Or maybe is an actual Apple-issued certificate required?
Francesco
It also has to be notarized by Apple otherwise Gateway will throw it out
-
Hi Juan, Rob %u2013
Do you know if all the module dylibs need to be signed by a certificate belonging to the same team id? I%u2019m asking because I tried signing the StarNet module with my self-signed certificate (which is obviously trusted on my local machine), but still PI doesn%u2019t %u201Csee%u201D it. Or maybe is an actual Apple-issued certificate required?
Francesco
It also has to be notarized by Apple otherwise Gateway will throw it out
i don't know for sure about the team thing, this is what prevented me from paying apple $100. i should have asked on sjaa-astroimaging if anyone already had a developer certificate.
here's how to find out what the exact error is though - quit PI, edit your ~/Library/PixInsight/PixInsight.plist to manually add the starnet module, and restart PI. here is what the lines look like but you'll have to increment the module number and perhaps add these lines at the end of the section rather than in any random place:
<key>000.Modules.00000046</key>
<string>@@pxi_bin_dir/StarNet-pxm.dylib</string>
when PI fails to load the module it should put up a dialog with the error message and the message will also be on the console. hopefully that will illuminate the situation further.
rob
edit: i didn't read your message carefully enough - thought you had a paid apple account - i already tried a self-signed certificate and that doesn't work. thought you had an apple cert. i guess i might as well renew my apple developer ID to find out, it's just $100.
-
In the apple documentation from version 10.15.x all modules have to be notarized by apple otherwise Gatekeeper will reject the request, this is what they say:
Beginning in macOS 10.15, all software built after June 1, 2019, and distributed with Developer ID must be notarized. However, you aren't required to notarize software that you distribute through the Mac App Store because the App Store submission process already includes equivalent security checks.
https://developer.apple.com/documentation/xcode/notarizing_macos_software_before_distribution
-
In the apple documentation from version 10.15.x all modules have to be notarized by apple otherwise Gatekeeper will reject the request, this is what they say:
Beginning in macOS 10.15, all software built after June 1, 2019, and distributed with Developer ID must be notarized. However, you aren't required to notarize software that you distribute through the Mac App Store because the App Store submission process already includes equivalent security checks.
https://developer.apple.com/documentation/xcode/notarizing_macos_software_before_distribution
Yes but that’s not the issue, because it is already broken in 10.14.6 which doesn’t enforce notarization yet.
-
from what i read, the problem is that there is a compile-time flag, which can be set in xcode at the time the application is built, that enforces this check (that all dylibs loaded by the application are signed). i think that explains why PI -4 and earlier have no problem, as ostensibly they were built without this flag set. i don't know what version(s) of OSX this flag would apply to, but i know on 10.14 the unsigned module won't load.
rob
-
from what i read, the problem is that there is a compile-time flag, which can be set in xcode at the time the application is built, that enforces this check (that all dylibs loaded by the application are signed). i think that explains why PI -4 and earlier have no problem, as ostensibly they were built without this flag set. i don't know what version(s) of OSX this flag would apply to, but i know on 10.14 the unsigned module won't load.
rob
That flag, if set, is already enforced in 10.14.