PixInsight 1.6.9 - Update System

[Juan Conejero]

Hi all,

While I'm writing this post, the last installation archives for PixInsight 1.6.9 are being uploaded to our file server, so the public release of this new version is imminent for all supported platforms: FreeBSD, Linux, Mac OS X and Windows, both 32-bit and 64-bit versions (only 64-bit for the FreeBSD release).

This version is actually a milestone in the history of PixInsight due to a significant new feature: the PixInsight Update System. You probably won't understand how important this feature is until you start receiving updates regularly with bug fixes, new tools and features, new documentation, etc., and you see them applied to your PixInsight installation automatically. The update system will change the whole PixInsight project dramatically, probably more than any other feature that we have implemented so far. From now on PixInsight will be really the dynamic and evolving platform that we have had in mind since the beginning, back in 2003.

Although the update system is quite complex internally (I must confess that I underestimated its complexity, and this has been one of the reasons for the delay in publishing a new version), I have made efforts to keep its user interface as simple and easy-to-use as possible, and I honestly think it is actually one of the most easy-to-use PixInsight tools I've designed

Automatic Check for Updates

Each time you run the PixInsight Core application, it initiates a "check for updates" routine automatically. This feature can be disabled via global preferences, but it is enabled by default. During the automatic update check process a PixInsight icon appears in the system icon tray. If there are no updates available, the icon is hidden silently, but if there are one or more updates available for your current PixInsight installation, a message box is shown similar to the next screenshot.

You can either click on the message window to review the updates, or right-click the system tray icon (left-click on Mac OS X) to select two relevant options from a context menu:

The update check routine can also be launched at any time from the new Resources > Updates menu. On this menu you have access to all of the functions that control the update system.

PixInsight Repositories

PixInsight updates are provided by PixInsight repositories. From the end user perspective, a repository is just the URL of a website that provides update packages, which are the basic elements managed by the update system. An update package is a compressed archive (in various formats, from standard tar.gz UNIX archives to 7-zip archives) whose contents can replace existing elements in the current PixInsight installation, add new elements, or remove existing ones (for example, there can be 'cleanup packages' that remove obsoleted or superseded features).

The user is free to add new repositories to the update system. The official PixInsight repository is maintained by Pleiades Astrophoto and its URL is:

http://update.pixinsight.com/

This is the default PixInsight repository and obviously where most PixInsight updates will be available for bug fixes, new versions of the Core application, new and improved standard modules, reference documentation, etc.

Third-party developers can --and I hope they will-- create their own repositories very easily. I'll describe the procedure to create a PixInsight repository in a new post. When a developer on the PixInsight/PCL or PJSR platforms creates a repository, (s)he should announce it on this forum --we are also glad to issue massive emails to announce new repositories to all PI users, so developers can count on us in this regard.

By selecting the Resources > Updates > Manage Repositories main menu option you can add, remove and edit repository URLs at your discretion:

Checking for New Updates

By selecting the Resources > Updates > Check for Updates main menu option, PixInsight will check all selected repositories for new packages. New packages are those that haven't been installed yet and are applicable to your current version, platform and hardware architecture; PixInsight keeps track of all installed updates in an internal database that is maintained automatically by the update system.

Selecting and Downloading Updates

When the check for new updates is completed, there may be new updates available for you. When this happens, the Select and Download Updates dialog opens and allows you to select/unselect specific packages. This dialog provides extensive information on the repositories and all available packages. This information is provided by the package author(s) and should be complete enough as to keep you informed about the updates you are applying to your PixInsight installation.

Once you have selected the desired updates (typically, all of them should be selected, especially all the updates available on the official PixInsight repository), you can click the Apply button to start dowloading packages.

Update packages are downloaded to the location selected as the downloads directory on global preferences (Edit > Global Preferences > Directories and Network). By default, the downloads directory is the system temporary directory, but you can define it as any directory where you can create/remove/write files. If some of the packages that you have selected were already downloaded in a previous session (for example, because you had to abort the download process for any reason), they won't be downloaded again if the files are valid.

When you download a PixInsight Core application update, you must authenticate as a PixInsight user with your user name and password.

The user name and password that you must use here are not your user identifier and activation code that you used to activate your license. You must use the data of your account on our file servers, which are the same user name and password that you use to access our Software Distribution System.

Your authentication data and the core application packages travel strongly encrypted (double AES-256 encryption with entropy maximization, a custom protocol similar to SSH in some aspects) to protect your data transactions from possible listeners or other bad boys trying to redirect your connection to their own 'services'.

Once all packages have been downloaded, you get a report and a message telling that the updates will be installed when you exit PixInsight. You can also unschedule the update installation with the corresponding main menu item.

Update Installation

Installation of updates is performed when you exit the PixInsight Core application. A dialog box allows you to confirm or cancel the installation, along with selecting whether to relaunch the PixInsight Core application at the end of the process. If you choose not to restart PixInsight, the updates will be installed in the same way but some final steps will be carried out by the Core application the next time you run it.

If all went well, the PixInsight Core application will be relaunched, possibly updated/new modules and scripts will be installed and featured, respectively,  and you'll see a dialog box like the following one:

As an option, you can see the log file generated by the updater process. The log file provides exhaustive information about every file and directory that the updater has created, copied or removed on your system during the update installation task. This information can be of great help to diagnose/repair possible installation errors.

Software Updates and User Privileges

An important topic that must be carefully considered when designing a software update system is user access privileges. When PixInsight is installed on a system protected directory, the updater application must gain the required privileges. In the case of PixInsight, the policies applied are dependent on the operating system:

- Windows. PixInsight for Windows installs now on the standard Program Files folder by default. This is a protected directory requiring administrative privileges. The PixInsight Updater application uses the standard User Account Control (UAC) mechanism on Windows Vista and Windows 7 to request 'elevation' to the user upon execution. PixInsight Updater has been, as well as all executables and installation programs in all Windows distributions of PixInsight, digitally signed with an Authenticode certificate issued by VeriSign for Pleiades Astrophoto S.L.

- Mac OS X. On the Mac, we assume that the user has full write access on the /Applications folder, so the PixInsight Updater application requires no special permissions to work normally.

- FreeBSD and Linux. On these operating systems, the required permissions for the updater program depend on where PixInsight has been installed. If PixInsight has been installed on a directory under the user's $HOME directory, then no special permissions are required because the user already has full write access. However, if PixInsight has been installed under a system directory (for example, on /usr/local or usr/lib) the situation is completely different. In these cases the updater1 executable (which you'll find on the bin installation directory) requires a sticky SUID bit, which can only be set by root. This task will be performed automatically by the new UNIX/Linux installation program that will be released with the next version of PixInsight. For now, if you are interested in installing PixInsight manually in this way, let me know and we'll post a description of the required procedure on this forum.

[Harry page]

Hi

All looks excellent    thanks for your hard work 

So as I read this this will the last time I do a full download and install all future upgrades / bugfixes will under the auspices of the update function ?    8)


Regards Harry

[Emanuele]

Wow, Juan, that is some great work!! Thank you very much! This update is really appreciated due to the update system!
Long gone are the days of updating all the Macs by hand!

Thank you!

E.

[Harry page]

HI

All works first time  , sorry could not help looking 

Harry

[Jack Harvey]

Mac 64 bit worked perfectly.  This is really cool!  Thanks for all the work!

[Nigel Ball]

Update system worked fine on Windows 7 64-bit

Thanks for all the hard work Juan  8)

[Thorsten Lockert]

I've found that if you rename PixInsight.app (e.g. to "PixInsight 32.app" and/or "PixInsight 64.app"), update/relaunch fails when you quit the application. The updates are downloaded correctly, however.

Also, if such renaming is done after the initial launch, it fails to find the included modules until either the plist with the preferences is updated manually or removed.

Thorsten

[Enzo De Bernardini]

Una obra maestra realmente. El sistema de actualizaciones creo que será un punto muy atractivo para los potenciales nuevos usuarios, así como la documentación integrada. ¡Felicitaciones!, y realmente gracias por todo el esfuerzo. ¡Cada día mejor!

A truly masterpiece. The upgrade system I think will be a very attractive to potential new users, as well the integrated documentation. Congratulations!, and I really thank you for all the effort. Every day better!

Enzo.

[David Serrano]

Hi,

Congratulations for this piece of software. I follow your developments from the distance (thanks to this forum's RSS feed) and I must say I'm impressed that you can keep up with the maintaining/bug fixing of this big project in no less than 4 different architectures, and even find some time to create a documentation system, fill it with contents and now provide this update mechanism you describe. Well done!

Now, some quick questions about the update system, you know .

Third-party developers can --and I hope they will-- create their own repositories very easily. I'll describe the procedure to create a PixInsight repository in a new post. When a developer on the PixInsight/PCL or PJSR platforms creates a repository, (s)he should announce it on this forum --we are also glad to issue massive emails to announce new repositories to all PI users, so developers can count on us in this regard.

What stops me, as the hypothetical owner of a new repository, to develop a new PixInsight module which is actually a virus/trojan, and pretend it's an update so I can inject it in all PixInsight installations around the world? Does my repo download stuff only from the official Pleiades repo? Or is some kind of digitally-signed "Contents" file in place?

Installation of updates is performed when you exit the PixInsight Core application. [...] If you choose not to restart PixInsight, the updates will be installed in the same way but some final steps will be carried out by the Core application the next time you run it.

The usual procedure is wait to the next startup of the application to perform updates. Just out of curiosity, why does PixInsight does this upon exit, if some work will be left to the next startup anyway?

Software Updates and User Privileges
- FreeBSD and Linux. [...] the updater1 executable (which you'll find on the bin installation directory) requires a sticky SUID bit, which can only be set by root.

Hmm... this is an ugly one. Have you considered using gksu or similar?

[Juan Conejero]

Many thanks to everybody for your nice comments!

[Juan Conejero]

Hi Thorsten,

I've found that if you rename PixInsight.app (e.g. to "PixInsight 32.app" and/or "PixInsight 64.app"), update/relaunch fails when you quit the application. The updates are downloaded correctly, however.

Also, if such renaming is done after the initial launch, it fails to find the included modules until either the plist with the preferences is updated manually or removed.

You can't rename the application bundles on Mac OS X. They must be "PixInsight.app" for both the 32-bit and 64-bit version, for compatibility reasons. I'll try to overcome this limitation in a future version.

Please read this post for more information on how to use both versions from the /Applciations folder on the Mac.

[Juan Conejero]

Hi David,

What stops me, as the hypothetical owner of a new repository, to develop a new PixInsight module which is actually a virus/trojan, and pretend it's an update so I can inject it in all PixInsight installations around the world? Does my repo download stuff only from the official Pleiades repo? Or is some kind of digitally-signed "Contents" file in place?

Currently nothing stops you from doing that, except the fact that PixInsight users must trust you to add your repository URL to their PixInsight applications. Obviously this is not a very strong security scheme (laughs...)

This problem will be solved with a repository validation mechanism. Unfortunately this has not been implemented in this initial version of the update system due to urgency to release a new version of PI (too many pending bug fixes).

The repository validation system is relatively simple. When you create a new repository, we provide you with a small PHP script that you must install on your repository. This script knows how to transform special messages that update.pixinsight.com sends to request repository identification. If your repo answers as expected to the identification message, it will be cross-validated by the PI Core application and the packages will be downloaded normally; otherwise a red flashing window will appear with a strong thunderstorm sound, etc.

All data transfers between the three elements of the cross-validation process —your repo, the PI repo and the PI Core application— are strongly encrypted, so none of them can be supplanted. As long as you don't disclose your validation script, this system is secure and does not depend on third-party validation 'authorities', which I personally would prefer to avoid. What do you think?

The usual procedure is wait to the next startup of the application to perform updates. Just out of curiosity, why does PixInsight does this upon exit, if some work will be left to the next startup anyway?

In fact, I initially implemented PI's update system this way. Then I realized that this is not very secure, especially on some operating systems. Basically, we can't have any control on where the updates (and their associated control data, which are the critical part) have been downloaded and extracted. The most robust way to apply updates is applying them just when the application terminates, before not too many odd things can happen with the extracted update files.

As you probably know, I generally tend to be somewhat paranoiac with these things; I always try to make robust systems.

Hmm... this is an ugly one. Have you considered using gksu or similar?

Yes, but gksu depends on GTK+, and su cannot be used programmatically. sudo can be used via stdin, but is tricky and requires that the user be added to the sudoers list. In all cases the root password must be asked from the PI Core application, something that I don't like at all.

Finally, my conclusion is that the most efficient and secure way is implementing the updater application just as it is working on the rest of platforms, and rely on UNIX's native security mechanism: the SUID executable file attribute. The SUID is simple and secure. It can only be set by root to acquire root's rights (or by an installer program running as root). The simplicity and robustness of SUID is what makes UNIX much more secure and reliable than the rest of OSs.

[David Serrano]

The repository validation system is relatively simple. When you create a new repository, we provide you with a small PHP script that you must install on your repository. This script knows how to transform special messages that update.pixinsight.com sends to request repository identification. If your repo answers as expected to the identification message, it will be cross-validated by the PI Core application and the packages will be downloaded normally; otherwise a red flashing window will appear with a strong thunderstorm sound, etc.

All data transfers between the three elements of the cross-validation process —your repo, the PI repo and the PI Core application— are strongly encrypted, so none of them can be supplanted. As long as you don't disclose your validation script, this system is secure and does not depend on third-party validation 'authorities', which I personally would prefer to avoid. What do you think?

<Insert a sizable paragraph about signing the metadata by using some public key infrastructure here>. Err, first of all, do you have any way to integrate X.509 with PixInsight? . That way, update.pixinsight.com wouldn't have to be involved in the process, since PKI would ensure that other repos can be trusted.

In fact, I initially implemented PI's update system this way. Then I realized that this is not very secure, especially on some operating systems. Basically, we can't have any control on where the updates (and their associated control data, which are the critical part) have been downloaded and extracted. The most robust way to apply updates is applying them just when the application terminates, before not too many odd things can happen with the extracted update files.

If you talk about tampering with the already downloaded files, that can be done between PI downloading them and exiting, so I think I'm missing something here.

Moreover, if PKI is in place, the files can be trusted again at this point since each one would be digitally signed.

Yes, but gksu depends on GTK+, and su cannot be used programmatically. sudo can be used via stdin, but is tricky and requires that the user be added to the sudoers list. In all cases the root password must be asked from the PI Core application, something that I don't like at all.

Good points.

Well, being forced to use the suid bit, I would then try to make the affected binary as small and simple as possible. I understand that the program is already finished and in production by now , but FWIW my approach would be a small version of 'cp' which accepted commands from stdin. Here's a possible conversation:

Code: [Select]

-> Hello, I can prove I'm the PixInsight application and I'd like you to perform some tasks on my behalf (i.e. some kind of authentication)
<- OK
-> Now, copy /this/file to /usr/lib/foo
<- OK
-> Now, copy /this/other/file to /usr/lib/bar
<- OK
-> Nice job mate
The hard part is, of course, the authentication. Failing to do this well would result in a system wide root hole .

[Harry page]

Hi

I see the animation script supplied with the latest PI is way out of date     Can this sort of thing be done via Update

Also some of those small processes by carlos still have not made it in , I know you are busy but.............. pretty please


Harry

[Enzo De Bernardini]

Here is the las update done by Carlos M.:

http://pixinsight.com/forum/index.php?topic=1895.msg17881#msg17881

Regards,

Enzo.