This message describes how to run PI on a Windows system rented at Amazon’s Elastic Cloud 2 (EC2) service that is avaiable as part of Amazon Web Services (AWS). At 0.30 EUR/hours this is a realistic alternative to replacing an outdated laptop.
Motivation
=======
So far, I did all of my PI work on my Toshiba Windows laptop. Win7x64 home edition, 4GB RAM, Intel Core 2 Duo T7300@2GHz. The new Drizzle functionality basically means that RAM is no longer sufficient, and general speed-especially for Batch Preprocessing- also was no longer really good.
- Option 1: By new system, sets me back 800-2000 EUR
- Option 2: Rent system at >=0.30 EUR/h at Amazon EC, if speed is needed.
This text investigates option 2.
Overview
=======
To get an instance (=a computer in EC2) running, you need to perform the following steps that are discussed in more detail below. I am trying to explain Amazon speak as we go along.
- You need to setup an account with AWS. Optionally, you can also set up identities (AWS user ids managed by you) using Identity and Access management (IAM)
- You create a cryptographic key pair that helps you with secure communication, and describe the internet access permission (~firewall) for your instances by defining a security group.
- You boot your instance by selecting an instance type (=hardware) and and Amazon Image (AIM, operating system software)
- After booting, you connect to this instance, configure users and install software
- Now you remote system is ready for use, allowing GUI access and copy between your local and remote drives.
- If necessary, you can switch the instance type to something stronger, shutdown/reboot the system, ... . You only (mostly) pay for what you use.
- I am including a couple of hints for optimal use and expected performance.
Once you have some experience with this, it will take you maybe 1 minute to re-launch a new instance, and maybe 10 minutes to completely configure an instance for PI. The first try may take 60 minutes or so.
Setup
====
The starting point of this description is
http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/IntroWindowsUserGuide.html . The documentation there gives a lot of detail that is not necessary to get started with PI.
1. Create AWS account and identity
Create account and log in on
http://aws.amazon.com. Following
http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/get-set-up-for-amazon-ec2-windows.html here. Note that this is the “super user” of anything you do in AWS.
- Make sure you select the right region (the computer center you use) on the top right of the AWS management console. In my case, that’s Ireland.
- In the following steps, you optionally can create identities (AWS users that are managed by you). You can manage their permissions (such as creating, launching, stopping instances) with a high degree of detail. As an alternative, just work with your AWS login instead of the identity georgAdmin that we create below.
- Create new identity (=user account that belongs to your main AWS account) in the IAM tool. You can also use your AWS account directly, but Amazon recommends doing this.
- Create new group of users “Administrators” with Administrators access rights.
- Create new identity (“georgAdmin”)
- We need not download credentials here, since we use AWS management console
- In the IAM tool, click on “users” on the newly created identity, and use” Security Credentials/Manage Password” to set a password for this new user.
- Take note of the AWS account ID (123456, can be seen for instance on your AWS account), and log onto AWS console with https://12345.signin.aws.amazon.com/console/ using your newly created identity and password
- You may want to create similar new identities for other family members or roles. But note that any action that is made in this account will be charged to your main AWS account.
2. Security
Define security settings. Create a key pair for identity "georgvAdmin". The key pair is used to communicate secrets (such as passwords) to you. Also set a Security Group (network permissions) for your instances. In the AWS console, select the EC2 tool.
- Using the new console, select the EC2 tool.
- Create a key pair, and save it in a save place on your local system.
- Create a Security Group (network permissions) that allows Windows Terminal Server Access (RDP) to your instance. Here: Name RDPWindows, no VPC, Allow Inbound RDP from Anywhere
2. Creating an Instance
An instance is a computer/system in Amazon speak. You need to create it using EC2 the EC2 tool. Following
http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/EC2Win_GetStarted.html here:
- Open EC2 tool
- Select Launch Instance.
- Select the AMI (=base software setup) to be Microsoft Windows Server 2012 R2 Base (currently ami-27539050). This is similar to Windows 8…
- Choose Instance type (=hardware of system). I am using m3.large here because it is the cheapest instance with >6GB RAM ($0.266 per Hour according to http://aws.amazon.com/ec2/pricing/)
- Select Review and Launch.
- Be sure to select the RDPWindows Security Group
- You may want to change the size of your root EBS device (30 GB default, roughly 12 GB free).
- Click “View Instances”. When your instance is running, you can log in. It may take a minute or so until you can actually log in, even if the console shows “running” as status.
3. First Login
In this first login, we will configure additional users and new passwords:
- Take note of the public DNS of your instance as displayed in the EC2 console, something like "ec2-54-217-186-177.eu-west-1.compute.amazonaws.com". You need this to talk to the system later.
- In EC console, press “Connect”. Download and launch the RDP file. This launches Windows Remote Desktop and connects to your instance. You’ll see a login screen.
- in EC2 consolse "Decode Password" (using your key pair).
- Using Remote Desktop and your password, Log in as "Adminstrator". Ignore the certificate error. You now should have a Windows desktop in front of you.
- Change your Administrator Password using Control Panel/User Accounts/Manage Accounts. You need to go to “Remove Accounts” to be able to change passwords and add new users !
- Create an ordinary user ("georg") in using Control Panel/User Accounts/Manage Accounts. You will be using this account for everything else.
- in Control Panel/System and Security/Allow Remote Access/Remote/Select Users you need to add the new user georg. Otherwise georg does not have remote access!
- Sign out
Normal login
=========
This is the normal login procedure (plus PI installation).
- Start "Remote Desktop Connection" from the Start Menu of the local system. Do not use the Connect button from EC2, it will not allow you to share disks.
- Enter the Public DNS name you took down above in General tab as host name.
- In Local Resources tab, share your disks.
- After this, you can copy files between the local system and the EC2 instance using Explorer.
- log in as the normal user (georg, not Administrator) using the name and password you set above
- Install PI as usual. Install your license file into your user directory as described in http://pixinsight.com/faq/index.html point 2.7. Launch and Update…
- If possible, configure the local disks (instance stores/ephemeral disks, Y/Z) as swap devices for PI (Edit/Global Preferences/Directories and Network). See performance section below. The root directories do not work, use something like Z:\SwapPi and Y:\SwapPi.
Using your Instance
==============
You can now use the system. Some hints:
- Work with this system. You’ll be charged ~0.30 EUR while it is running.
- When you are done, stop your instance in the EC2 console. This is similar to shutting down your personal computer.
- You can stop the system. You will no longer be charged for the instance.
- You will be charged for the storage on your C:-drive (EBS device) . With the default setup of m3.large this is 30GB (at $0.055 GB/month)
- When you relaunch your stopped instance, all your software/data/setttings on C: will be there.
- The data on the instance store (also called ephemeral storage, Z: 30GB SSD) will be lost on relaunch.
- Note that the DNS name of a relaunched instance will change.
- A stopped instance can also be launched on a difference instance type, for instance if you need faster hardware. Change it in EC2 console/Instance/Actions/Change Instance type. Note that you may not be able to use all instance storage in this case.
- You can also terminate the system. This is similar to throwing away your system. After removing your EBS store, all your data/user accounts, installed software, data will be lost. There are no costs because you don’t use anything. You’ll need to start over with your very first login as Administrator. (Note: You can preserve the EBS storage, but this is similar to stop/start).
Some Tips
=======
- Make sure you use the local disk (instance storage/ephemeral storage, for instance Z: but not C:) as your PI swap device. It makes a lot of difference – see Performance section below.
- Be sure to delete a Disk (EBS volume) that you no longer need when you terminate (instead of stop) an instance.
- When launching with a new instance type, sometimes the additional instance stores are not found (see performance section). I needed to create a new instance to get them with C3.2xlarge.
Costs
====
Costs: Study the Amazon pages
http://aws.amazon.com/ec2/pricing/ . Here only a short summary:
- The instance costs (depends on instance type, charged per launch in units of one hour) and data transfer into Amazon (0 cents/GB) are simple to calculate.
- Amazon charges for data stored on their storage systems (EBS) even when the instance is not running, something like $0.055GB/month. They also charge for data transfer from/to those systems ($0.055 per 1 million I/O requests) - which is kind of difficult to estimate, but never was a relevant factor for me.
- Amazon charged for data transfer out of Amazon, <=$0.12/GB
- Please add VAT to all prices.
Some performance data in the next post.
Georg
Edit: Making the creation of new identity optional. Adding overview section.
