Endor PixInsight File Server at the University of Valencia, Spain

[Juan Conejero]

Hi all,

In a previous post we announced PixInsight Files, a new data storage service intended to support shared data and images among PixInsight Forum users. Unfortunately, we underestimated the bandwidth and CPU requirements of this service, and had to cancel it after a few days due to server overload.

Today I am glad to announce the immediate availability of Endor, our new dedicated server at the Astronomical Observatory of the University of Valencia (OAUV):

http://endor.uv.es/

which can also be accessed through this redirected URL:

http://endor.pixinsight.org/ 

In its initial version, Endor is an Intel Core i7 2700K with 4 TB of SATA/600 RAID 1 storage and 16 GB of RAM, running Fedora 16 Linux x86_64 and Apache 2.2 server. We'll be increasing its performance and resources as necessary. Endor is connected to a 1 Gb/s line and is under our direct maintenance and control, with the help of personnel of the OAUV.

For now, the only public service running on Endor is PixInsight Files. As described in the initial announcement, the purpose of this service is to provide a high performance storage area where all PixInsight users can share images and working data, especially to support image processing challenges, tutorials and workshops.

To implement this file sharing service, we have chosen AjaXplorer, an AJAX/PHP file browser that works in a similar way to common operating system native file browsers such as Dolphin, Nautilus, Finder, Windows Explorer, etc. For more information you can read the end user AjaXplorer documentation.

To use PixInsight Files you must be a registered user of this new service. If you want to register, please send us a request at any of our support email addresses, or use the contact form on our website. To help us identify you, either use the same email address that you have associated with your license, and/or specify your user id. You can find your user id (a 16-digit number) in the email you received when you purchased your license, and also on the Resources > About PixInsight dialog of the PixInsight Core application. Unless you tell us otherwise, your PixInsight Files account will have the same user name and password as your current PixInsight user account (which you use to access our Software Distribution interface).

I would like to express our gratitude to the University of Valencia, for providing the connectivity to the Internet and the hosting facilities for the Endor server. A big thanks to Vicent Martínez, the Director of the OAUV, for his invaluable support and assistance to this project.

Thank you for your attention.

[Nocturnal]

I have to admit I was skeptical about the reported CPU load on the original server. How many people signed up? Why was the CPU load so high for simple file transfers? Unless you had some huge mega-pipe to the internet you were probably limited to 100 Mb/s load. A P4 based server can handle that without sweating. I suspect the fancy file manager was to blame, not the transfer load.

Regardless it's good news you have found a new home for the service of course.

[Juan Conejero]

I suspect the fancy file manager was to blame, not the transfer load.

Exactly. Although the bandwidth consumption was quite large, if we had implemented an FTP service there would have been no major problems. We want to offer something more usable and powerful than FTP, though, and this has a cost in terms of *CPU bandwidth*. Concurrent accesses to the AJAX/PHP file manager frozen the shared server where we have pixinsight.org hosted. This kind of things cannot be done on a shared hosting service; lesson learned!

Now we should have no problems with our own server. This is a good step forward for the PI community!

[Alejandro Tombolini]

Great, thanks for the effort!
Regards, Alejandro.

[Jules]

Muchas gracias Juan, espero que con la instalación.

Saludos

Julian

[troypiggo]

Hi Juan, I'm trying to share some FITS files using this server.  I have created a dir in the Endor server called "troypiggo".  In there I have created a dir called "NGC6188", and in that I have placed 3 FITS files.  If I click on each FITS file and click on the "share" button in toolbar, I get a dialog box.  I left the expiration as 0 (assume that means never expire?), and password empty.  Got the share link, posted them here.  If you click on those links, you get a download file screen, but if you click on the download file button a "Cannot find file" error message.

Any ideas?  What am I doing wrong?

[Lex]

Hi,

Just wanted to write something similar.
I do not think that you are doing something wrong because I had the same problem already, just like some other users also already did.
The sharing does not work properly IMO??

[Juan Conejero]

Hi Troy and Lex,

Don't share individual files; share a folder from your 'My Files' personal data area. Do the following:

- Select the folder and click the Share button (or right-click the folder and select the Share menu item).

- On the Share dialog box, the label is the same as the folder's name by default. Usually this is OK, but you can change it if you want.

- Leave the Rights as 'Read'. This will prevent anybody from modifying your data.

- If you want to share the folder with a few users only, select them (or write them) in the 'Share with' area.

- If you want to share the folder with everybody, select the 'public' user in the 'Share with' area. 'public' is a special user whose password is 'pixinsight' (don't forget to mention this as a reminder when you announce your data on the forum). It has read-only rights on the file sharing service.

- You can also create a temporary user, but using 'public' is simpler and more secure.

As I've said this is to share a folder from your 'My Files' data area. If you want to allow read/write access to everybody, upload your folder to the 'Forum Shared Files' area.

Let me know if this works as expected.

[troypiggo]

Thanks Juan.  Makes sense, just a couple of comments/issues.

I can't seem to delete or remove the previous one I uploaded to the folder "troypiggo/NGC6188" in "forum shared files" due to permissions.  Could you please remove that one?

Be nice to be able to provide a direct link to the files.  At the moment all we can do is to say "go to Endor's shared elements, shared repositories, NGC6188 public" (I had to add suffix public because as per above I couldn't remove the original one).