hubertf's NetBSD Blog
Send interesting links to hubert at feyrer dot de!
 
[20130203] Managing Amazon/EC2 NetBSD instances with euca2ools
What and Why

Playing with ansible, its "ec2" module came to my attention: it is intended to manage virtual machines in Amazon's EC2 cloud. The idea is that you describe a system with the property "needs to run in Amazon's cloud", and ansible then starts the machine if it isn't there already. In order to get to the point where this can be played with, a working version of the euca2ools package was required first.

Packaging was mostly a no-brainer, and a package is currently under review and will end up in pkgsrc eventually. The more interesting part was to verify if the pkg actually worked as expected. This proved tricky for two reasons: 1) my overall lack of how to use the Amazon AWS command line tools (ec2-ami-tools, ec2-api-tools), and 2) the fact that euca2ools is mostly written for the Eucalyptus Cloud infrastructure, which just happens to be compatible with Amazon AWS. To give future parties something to google, here are the steps that to fire up a NetBSD machine in the Amazon cloud.

How - Prerequirements

A login for Amazon Web Services (AWS) is required, of which the Elastic Cloud Computing (EC2) Xen infrastructure is a part of. I won't go into details of this, please see the NetBSD wiki or my article ``NetBSD in der Cloud'' in the German FreeX 5/2012 magazine, pages 58-63, for details.

Before starting, a few environment variables have to be filled with authentication information. Log into the Amazon AWS Console, click on your name in the upper right corner to get to the "Security Credentials" page, and create an access key if not already present. Get the acces key ID and the secret key, and put them into environment variables EC2_ACCESS_KEY and EC2_SECRET_KEY:

% setenv EC2_ACCESS_KEY "AKxxxxxxxxxxxxxxxxxx"
% setenv EC2_SECRET_KEY "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
Next create and download a X.509 certificate - make sure to get both the file with the private key (pk-XXXX.pem) as well as the file with the public key (cert-XXX.pem). Set the environment variables EC2_CERT and EC2_PRIVATE_KEY to thos files, respectively:
% setenv EC2_CERT         .../cert-XXX.pem
% setenv EC2_PRIVATE_KEY  .../pk-XXX.pem
Last, euca2ools want to know what cloud infrastructure to use for virtual machines (EC2) and storage (S3). Coming from the Eucalyptus project, the tools can talk to cloud servers running Eucalyptus, OpenStack and Amazon AWS. Communication is via HTTP, and the environment variables EC2_URL and S3_URL have to be set accordingly:
% setenv EC2_URL          http://ec2.amazonaws.com
% setenv S3_URL           http://s3.amazonaws.com
Last, make sure your system's time is somewhat in sync with reality, else you will get funny error messages!

So much for the preparations, let's dive into euca2ools.

List Regions and Availability Zones

Amazon's service offers is spread across many data centers across different regions of the world. The list of regions is available via the "euca-describe-regions" command:

% euca-describe-regions
REGION  eu-west-1       ec2.eu-west-1.amazonaws.com
REGION  sa-east-1       ec2.sa-east-1.amazonaws.com
REGION  us-east-1       ec2.us-east-1.amazonaws.com
REGION  ap-northeast-1  ec2.ap-northeast-1.amazonaws.com
REGION  us-west-2       ec2.us-west-2.amazonaws.com
REGION  us-west-1       ec2.us-west-1.amazonaws.com
REGION  ap-southeast-1  ec2.ap-southeast-1.amazonaws.com
REGION  ap-southeast-2  ec2.ap-southeast-2.amazonaws.com
Inside one region, systems are grouped together in "availability zones" - usually data centers or separate security zones within (refer to the Amazon documentation for details). To list the availability zones in one region, use the "euca-describe-availability-zones" command:

% euca-describe-availability-zones
AVAILABILITYZONE        us-east-1a      available
AVAILABILITYZONE        us-east-1b      available
AVAILABILITYZONE        us-east-1c      available
AVAILABILITYZONE        us-east-1d      available
To specify what region to talk to there are two ways. The first is to specify the region name on any of the following commands with the "--region" option (yuck):

% euca-describe-availability-zones --region eu-west-1
AVAILABILITYZONE        eu-west-1a      available
AVAILABILITYZONE        eu-west-1b      available
AVAILABILITYZONE        eu-west-1c      available
The second one is to adjust your EC2_URL to point to your preferred region directly:
% setenv EC2_URL http://ec2.eu-west-1.amazonaws.com
% euca-describe-availability-zones --region eu-west-1
AVAILABILITYZONE        eu-west-1a      available
AVAILABILITYZONE        eu-west-1b      available
AVAILABILITYZONE        eu-west-1c      available
Systems and Listing Available Machine Images

Now that we have a basic overview of the cloud infrastructure with its regions and availability zones, the next questions are what hardware is available for running virtual machine instances on, and what operating systems can be put on.

Amazon lists available hardware configurations on their "instance types" web sites. Sizes range from Micro Instances with 613MB RAM, up to two CPU cores and no local harddisk (t1.micro) to Extra Large (XL) Instances with 15GB RAM, 8 CPU cores and 1.690 GB local harddisk. Many more configurations are available for situations that require much memory, much CPU, much IO, or do cluster computing with CPU and GPU.

As for the operating system and software to put on those virtual machine instances, there is a VERY wide choice available. The "euca-describe-images --all" command lists all available optione:

% euca-describe-images --all
...
IMAGE   ami-abd0d0df    101367081206/NetBSD-i386-6.0-20121015-1054 \
  101367081206    available       public          i386    machine \
  aki-64695810                    ebs
IMAGE   ami-7fc3c30b    101367081206/NetBSD-x86_64-6.0-20121014-1007 \
  101367081206    available       public          x86_64  machine \
  aki-62695816                    ebs
...
In the output, the configuration is identified by the Amazon Machine Identifier (AMI), e.g. "ami-7fc3c30b" for a NetBSD 6.0/amd64 instance. This image ID is required when defining what virtual machine instance to start.

Note that the "euca-describe-images" command depends on the region setting, so you will get (and need) different output depending on the region that you intend your instances to run in.

Setup SSH Access

When starting a NetBSD AMI, access will be via SSH to the root account. For that, a SSH key pair needs to be created with the "euca-add-keypair" command. The command can write the private key to a local file, be sure to protect it properly - it will be the only way of access to the system! Other interesting commands when managing SSH keys are "euca-describe-keypairs" and "euca-delete-keypair":

% euca-describe-keypairs
% euca-add-keypair -f key-eucaHF.pem eucaHF
KEYPAIR eucaHF  b8:e9:05:7e:3a:df:c7:8e:eb:6e:8d:72:ff:77:68:01:e2:03:7e:3e
% euca-describe-keypairs
KEYPAIR eucaHF  b8:e9:05:7e:3a:df:c7:8e:eb:6e:8d:72:ff:77:68:01:e2:03:7e:3e
% euca-delete-keypair eucaHF
KEYPAIR eucaHF
% euca-describe-keypairs
%
Of course we want to keep a key for logging with it, so let's re-run the important part:
% euca-add-keypair -f key-eucaHF.pem eucaHF
KEYPAIR eucaHF  9b:d4:15:09:bc:51:b1:76:5c:db:a3:93:52:f0:d8:08:87:a4:80:c7
% cat key-eucaHF.pem
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAn8rCLhqLyfke+NqeOkqb6BUIbfwBFm/9ddG8ghVt9CUmyKUMRrKFSyaTRreO
...
wA5a3XZuEFw83HdGrhaRgom2ZJ1SEk2889FpAA+yrhveKhDJIe6Zc2rM+crqUWBfnvs=
-----END RSA PRIVATE KEY-----
Manage Virtual Machine Instances

Now that everything is prepared, telling the cloud infrastructure to find physical hardware, put our preferred operating system on it, and start it is done with the "euca-run-instance" command:

% euca-run-instances -t t1.micro -k eucaHF ami-7fc3c30b
RESERVATION     r-2182506a      749335780469    default
INSTANCE        i-2ed60264      ami-7fc3c30b                    pending \
  eucaHF  0               t1.micro        2013-02-03T15:51:49.000Z        \
  us-east-1b      aki-62695816                    monitoring-disabled \
  ebs 
That's actually as complicate as it gets - one command that tells what hardware to use (t1.micro - can be omitted, a useful default will be chosen), what SSH key to use for the root account, and what machine image (AMI) to use are all used here. In return, the command prints a number of information from the freshly created instance. The one used in the following commands is the "instance id", "i-2ed60264" in this example.

When the above command was started, this is a good time to go back to the Amazon AWS console and have a look at your instances - you will find the one listed above there now, too! Instead of the web-based console, the "euca-describe-instances" command can be used:

% euca-describe-instances
RESERVATION     r-2182506a      749335780469    default
INSTANCE        i-2ed60264      ami-7fc3c30b    ec2-54-228-22-143.compute.amazonaws.com       \
  ip-10-226-194-20.compute.internal     running eucaHF  0   \
  t1.micro        2013-02-03T15:51:49.000Z        us-east-1b \
  aki-62695816                    monitoring-disabled     \
  54.228.22.143   10.226.194.20                   ebs
Now this is all nice and dandy, but we have just created a NetBSD machine in the Amazon cloud. Let's log in!!!1!

To do so, we need the private key file created with the "euca-add-keypair" command, and the host name. The latter is available in the list of instances - be sure to use the one within the "compute.anazonaws.com" domain:

% ssh -i key-eucaHF.pem -l root ec2-54-228-22-143.compute.amazonaws.com
The authenticity of host 'ec2-54-228-22-143.compute.amazonaws.com (54.228.22.143)'
can't be established.
ECDSA key fingerprint is f7:a9:f6:21:fc:d2:0e:46:03:41:f8:d5:c1:72:92:28.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ec2-54-228-22-143.compute.amazonaws.com,54.228.22.143' (ECDSA)
to the list of known hosts.
NetBSD 6.0 (XEN3_DOMU)
Welcome to NetBSD - Amazon EC2 image!

This system is running a snapshot of a stable branch of the NetBSD
operating system, adapted for running on the Amazon EC2 infrastructure.

The environment is very similar to one provided within a typical Xen domU
installation. It contains a small, autonomous environment (including a
compiler toolchain) that you can run to build your own system.

The file system is lightly populated so you have plenty of space to play with.
Should you need a src or pkgsrc tree, please use the "bootstrap" script found
under /usr to download them.  You can also use the script to set up
binary packages using "pkgin":

                /usr/bootstrap.sh [src|pkgsrc|binpkg]

This AMI sends email to the maintainer on first boot, to help get
an idea of what is in use at any given time.

You are encouraged to test this image as thoroughly as possible.  Should you
encounter any problem, please report it back to the development team using the
send-pr(1) utility (requires a working MTA).  If yours is not properly set up,
use the web interface at: http://www.NetBSD.org/support/send-pr.html

Thank you for helping us test and improve NetBSD's quality!
Terminal type is vt220.
We recommend that you create a non-root account and use su(1) for root access.
ip-10-226-194-20# uname -a
NetBSD ip-10-226-194-20.compute.internal 6.0 NetBSD 6.0 (XEN3_DOMU) amd64
ip-10-226-194-20# exit
From here, you are on your own - it's a NetBSD machine, after all.

One word of warning at this point: Amazon AWS is not for free (as you should be aware from the Preparations step). If you do not need machines any more, be sure to remove them from the cluster, else this may drive up your bill for nothing! You can use the "euca-terminate-instances" command to do just that:

% euca-terminate-instances i-2ed60264
INSTANCE        i-2ed60264
When you look at the output of "euca-describe-instances" now, you will see that the machine's state goes from "running" first to "shuting-down" then to "terminated" - the cloud infrastructure will eventually be cleaned up to not list the stale machines any more.

What's next?

As stated above, the whole goal of this exercise is to manage Amazon EC2 images from ansible. Weekend's mostly over and we will see where this journey is going. For the time being, I'm happy to hear about any comments of you using NetBSD on Amazon's EC2, and of my euca2ools package.

Appendix: euca2ools Cheat Sheet

Environment variables:
  setenv EC2_ACCESS_KEY "AKxxxxxxxxxxxxxxxxxx"
  setenv EC2_SECRET_KEY "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  setenv EC2_CERT         .../cert-XXX.pem
  setenv EC2_PRIVATE_KEY  .../pk-XXX.pem
  setenv EC2_URL          http://ec2.amazonaws.com
  setenv S3_URL           http://s3.amazonaws.com

Regions & availability zones:
  euca-describe-regions
  euca-describe-availability-zones
  euca-describe-availability-zones --region eu-west-1
Change default region:
  setenv EC2_URL		http://ec2.eu-west-1.amazonaws.com

AMIs:
  euca-describe-images --all

SSH Keypairs:
  euca-add-keypair		-f key-eucaHF.pem eucaHF
  euca-describe-keypairs	
  euca-delete-keypair		eucaHF

Instances:
  euca-run-instances -k eucaHF ami-7fc3c30b
  euca-describe-instances
  euca-describe-instances i-96a773dc
  ssh -i key-eucaHF.pem ec2-54-328-43-220.compute.amazonaws.com -l root
  euca-terminate-instances i-96a773dc


[Tags: , , , , ]


Disclaimer: All opinion expressed here is purely my own. No responsibility is taken for anything.

Access count: 17159113
Copyright (c) Hubert Feyrer