The cool thing with Amazon AWS is that you can use command line tools on your workstation to bring up and manage your services. In this post I’ll look into how you can launch a Linux VM from the command line and how you can attach a storage volume to it.

Before you can use the command line tools you’ll need to create a user that has the proper permissions to connect. This must be done in the IAM console:

aws_rds_iam_1

As I have no users available right now I’ll create one:

aws_rds_iam_2
aws_rds_iam_3

Be sure to download the credentials right now as you’ll not be able to get the “Secret Access Key” later. You’ll need this key when we configure the command line tools later.

aws_rds_iam_4

Once the user is created all the details are listed:

aws_rds_iam_5

To be sure I can do everything I want I attached the “Administrator Access” policy to the user just created:

aws_rds_iam_6

Having the user available we can now proceed and install the command line tools. You’ll need to have python available for that:

dwe@dwe:~/Downloads$ python --version
Python 2.7.6

The installation is quite easy:

dwe@dwe:~/Downloads$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
dwe@dwe:~/Downloads$ unzip awscli-bundle.zip
dwe@dwe:~/Downloads$ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
Running cmd: /usr/bin/python virtualenv.py --python /usr/bin/python /usr/local/aws
Running cmd: /usr/local/aws/bin/pip install --no-index --find-links file:///home/dwe/Downloads/awscli-bundle/packages awscli-1.9.20.tar.gz
You can now run: /usr/local/bin/aws --version
dwe@dwe:~/Downloads$ /usr/local/bin/aws --version
aws-cli/1.9.20 Python/2.7.6 Linux/4.2.0-23-generic botocore/1.3.20
dwe@dwe:~/Downloads

You can get help by issuing:

dwe@dwe:~/Downloads$ /usr/local/bin/aws rds help

This brings up the manual similar to the Linux man pages:

aws_rds_help

Now it is the time to configure the command line tools. You’ll need the contents of the credentials file you downloaded above:

dwe@dwe:~/Downloads$ /usr/local/bin/aws configure
AWS Access Key ID [None]: AKIAIZJEZEA4F4B6VWDQ
AWS Secret Access Key [None]: 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
Default region name [None]: eu-west-1
Default output format [None]: json

A list of Regions and endpoints can be found here.

For being able to launch a VM the next step is to create the ssh keys:

dwe@dwe:~/Downloads$ aws ec2 create-key-pair --key-name MyKeyPair --query 'KeyMaterial' --output text > MyKeyPair.pem
dwe@dwe:~/Downloads$ cat MyKeyPair.pem 
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAiwnxikaNFmlgDB1v0lQhlmmWvT1bfVpCjfa0tjOw6enp5q5B6eahN36q+eCT
W9jf6UTOakQxXxeNAXrbt/SWmVmjZzOOx1SyvqpU+dFk8eBI7MCuEoQyGfxAau6XyFB9v+Juzyy2
V2Z6kMtfwGSUST9uy9uoVmvH9J+aefJv6R5jQcv8c6AUIGPvOuO3LmHRT2IEuaLgC4bD332NuawB
y5ZIpDpiDojU6ENrk0Q6Z8fClbD+smWISx/cTV1Pj0zOpjNZDtnEnqxpZyhSmHdqX/BbN4lVECzJ
VypA0kbuHVu9tcYz+T7lsK3/Itlm8hdtVeFQBd6S/w5v3G/j2uNaRwIDAQABAoIBAAc2lNlfn6Iy
BVGblVmME9IB2Fgo/r4aGKnUyrtJIzx/bisRj0nbNTHz2FruU7bIIZWwhCTvNMCCMEaLYSrB/a4n
AFsySY39zuglXhM30czane3qcR10zkSa2ZyylB1IT73MuYL2Bt+x6VnpsyYAEdFbARyCYNFa9hyy
ywjYkwW9AbTxKewEwTN4DOfmfEXaP5t/slXm1afq3cRFPUnoClFhXwK8TSPX6Dz521UUn+JUq/xM
0B36yBtG1HtKJEvcf+cUAPQ8/+oyQgXI0RliKQQKUNg6g5Nmb/RoWbO6zmS/bGdbX5Ckz1IQ+UtM
J6oCdgCSQCXrc6l550TcFx+5l8ECgYEA79IdoWcLkrTuIjRuoDwA7Jit9SnbLkfaQwEBnjizO5j/
IzUSi9GectrkL7WvIt/ylfjneiAa3wx8U2pGDPA2cI0QjZi/u5amSf9a7EsU4m6LFLcH5CkgdDvW
mOIKkMdYku2FW9UGNmc1m5wLiESp47m0tB59cvLefb+K/WklYb0CgYEAlGs/m7aLE8ZSO9yjwkpu
8FP4SiuzjIMuyTdn8MXXU6TCfHdW1GuE4WIlYn8EJ0U7GUOULZ9TPPekJ7+n2yrF+KItonutTI4v
XgwH6XMWdDNnUScawSwqzMLCB2fDu/BhDCdxHRwPujbGcQ8gxYRO0Q/OlnhGgKfHFIpE+gjg8lMC
gYEA6HgdEN/6b/PDApUgx2Ji4vX9arFwLaSpBlprXxxHYXYlm9NObwp7NYrJtxW+92dul8H3YILO
iXho69MQpGoV23Rin196PDUEbKaDVJpTXEsbtrDVjW3wb7uxgfFbnIwgaAymQSZ2JzZU2Mqiwy0M
IhtZ91+26z5SPkL2UD8kZWECgYEAhSFWozQwuIIyjWOyuLrPnF+V3eIpYibhtrguUfkE1xB5K/BY
QJ5ZSVoiMqHAdgFRq63Eos/BeHSiGM1/ocZSYl4HFTJfFsaLko60IiGLyJu7Vz3+b7xQf+9K4B2h
o1lRk/dlLTlYmi47/noVaVbu4/SL2Mj2ZL0ahEAq8yU3seUCgYBnd8mJIBuWqKOPGWp5MSkco1OX
iIS2//BEaJiOHXxqow6uS/0TlTXk/qZavEoEiUZ91bhLM4ThH9q/yywBD2nu+YxUfbH2Lz9PeIeL
SiOrK6HBKmU3P2sVUxCLO/vQZN32rIK8U8Hv6h77ViUZyXRTvSxULduJsjRpHxIzVoWFmw==
-----END RSA PRIVATE KEY-----
dwe@dwe:~/Downloads$

Be sure to modify the permission of the generated key:

dwe@dwe:~/Downloads$ chmod 400 MyKeyPair.pem

What type of instance do we want to create? You can get the full list of available images by using the “describe-images” sub command:

dwe@dwe:~/.aws$ aws ec2 describe-images > /var/tmp/aws-images.json

As I wanted to know what “Amazon Linux” is based on I took this one:

"ImageId": "ami-025e5676"
"Description": "Amazon Linux AMI x86_64 EBS"

Knowing what we want to launch we can now bring up the instance:

dwe@dwe:~/Downloads$ aws ec2 run-instances --image-id ami-025e5676 --count 1 --instance-type t1.micro --key-name MyKeyPair
{
    "OwnerId": "505389097308", 
    "ReservationId": "r-1abaa1a3", 
    "Groups": [], 
    "Instances": [
        {
            "Monitoring": {
                "State": "disabled"
            }, 
            "PublicDnsName": "", 
            "KernelId": "aki-71665e05", 
            "State": {
                "Code": 0, 
                "Name": "pending"
            }, 
            "EbsOptimized": false, 
            "LaunchTime": "2016-01-16T04:57:30.000Z", 
            "PrivateIpAddress": "172.31.43.22", 
            "ProductCodes": [], 
            "VpcId": "vpc-6c634009", 
            "StateTransitionReason": "", 
            "InstanceId": "i-07560b8a", 
            "ImageId": "ami-025e5676", 
            "PrivateDnsName": "ip-172-31-43-22.eu-west-1.compute.internal", 
            "KeyName": "MyKeyPair", 
            "SecurityGroups": [
                {
                    "GroupName": "default", 
                    "GroupId": "sg-ee31498a"
                }
            ], 
            "ClientToken": "", 
            "SubnetId": "subnet-dffbb186", 
            "InstanceType": "t1.micro", 
            "NetworkInterfaces": [
                {
                    "Status": "in-use", 
                    "MacAddress": "0a:32:b2:89:70:67", 
                    "SourceDestCheck": true, 
                    "VpcId": "vpc-6c634009", 
                    "Description": "", 
                    "NetworkInterfaceId": "eni-0fa82155", 
                    "PrivateIpAddresses": [
                        {
                            "PrivateDnsName": "ip-172-31-43-22.eu-west-1.compute.internal", 
                            "Primary": true, 
                            "PrivateIpAddress": "172.31.43.22"
                        }
                    ], 
                    "PrivateDnsName": "ip-172-31-43-22.eu-west-1.compute.internal", 
                    "Attachment": {
                        "Status": "attaching", 
                        "DeviceIndex": 0, 
                        "DeleteOnTermination": true, 
                        "AttachmentId": "eni-attach-21ce50c7", 
                        "AttachTime": "2016-01-16T04:57:30.000Z"
                    }, 
                    "Groups": [
                        {
                            "GroupName": "default", 
                            "GroupId": "sg-ee31498a"
                        }
                    ], 
                    "SubnetId": "subnet-dffbb186", 
                    "OwnerId": "505389097308", 
                    "PrivateIpAddress": "172.31.43.22"
                }
            ], 
            "SourceDestCheck": true, 
            "Placement": {
                "Tenancy": "default", 
                "GroupName": "", 
                "AvailabilityZone": "eu-west-1a"
            }, 
            "Hypervisor": "xen", 
            "BlockDeviceMappings": [], 
            "Architecture": "x86_64", 
            "StateReason": {
                "Message": "pending", 
                "Code": "pending"
            }, 
            "RootDeviceName": "/dev/sda1", 
            "VirtualizationType": "paravirtual", 
            "RootDeviceType": "ebs", 
            "AmiLaunchIndex": 0
        }
    ]
}

A lot of json ouput 🙂 You can pass json files to command line utilities as well. The output fully describes your instance.

Having a look at the AWS EC2 console we can see that the instance in deed was created and is being initialized currently:
was_launch_instance_1

While the instance is set up it is good idea to give the instance a name:

dwe@dwe:~/Downloads$ aws ec2 create-tags --resources i-07560b8a --tags Key=Name,Value=MyFirstInstance

Going back to the console the name is set and the instance is ready:
was_launch_instance_2

Time to connect:

dwe@dwe:~/Downloads$ ssh -i /home/dwe/Downloads/MyKeyPair.pem [email protected]

       __|  __|_  )
       _|  (     /   Amazon Linux AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-ami/2012.09-release-notes/
Amazon Linux version 2015.09 is available.

Lets check if this instance is redhat based and we can use yum:

[ec2-user@ip-172-31-43-22 ~]$ yum search postgresql94-server
Loaded plugins: priorities, update-motd, upgrade-helper
=================================================== N/S matched: postgresql94-server ====================================================
postgresql94-server.x86_64 : The programs needed to create and run a PostgreSQL server

  Name and summary matches only, use "search all" for everything.

Cool. What devices do I have available?

[ec2-user@ip-172-31-43-22 ~]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1            2.0G  686M  1.3G  35% /
tmpfs                 298M     0  298M   0% /dev/shm

Only the root volume was created. Not much if we want to some real work on the instance. So lets create a new volume (10gb):

dwe@dwe:~/Downloads$ aws ec2 create-volume --size 10 --availability-zone eu-west-1a
{
    "AvailabilityZone": "eu-west-1a", 
    "Encrypted": false, 
    "VolumeType": "standard", 
    "VolumeId": "vol-fcb06c32", 
    "State": "creating", 
    "SnapshotId": "", 
    "CreateTime": "2016-01-16T05:17:33.615Z", 
    "Size": 10
}

Providing the name of the new volume it can be attached to the instance:

dwe@dwe:~/Downloads$ aws ec2 attach-volume --volume-id vol-fcb06c32 --instance-id i-07560b8a --device /dev/sdb
{
    "AttachTime": "2016-01-16T05:19:51.640Z", 
    "InstanceId": "i-07560b8a", 
    "VolumeId": "vol-fcb06c32", 
    "State": "attaching", 
    "Device": "/dev/sdb"
}

Seems this operation is online, lets check:

dwe@dwe:~/Downloads$ ssh -i /home/dwe/Downloads/MyKeyPair.pem [email protected]
Last login: Sat Jan 16 05:19:35 2016 from hsi-kbw-078-043-043-105.hsi4.kabel-badenwuerttemberg.de

       __|  __|_  )
       _|  (     /   Amazon Linux AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-ami/2012.09-release-notes/
Amazon Linux version 2015.09 is available.
[ec2-user@ip-172-31-43-22 ~]$ ls /dev/sd*
/dev/sda1  /dev/sdb

Really, really cool. Do I have root access to create a partition on my new device?

[ec2-user@ip-172-31-43-22 ~]$ sudo su -
[root@ip-172-31-43-22 ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x85f44e7c.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): p

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x85f44e7c

   Device Boot      Start         End      Blocks   Id  System

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): 
Using default value 1305

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@ip-172-31-43-22 ~]# ls -la /dev/sd*
lrwxrwxrwx 1 root root 5 Jan 16 04:58 /dev/sda1 -> xvda1
lrwxrwxrwx 1 root root 4 Jan 16 05:21 /dev/sdb -> xvdb
lrwxrwxrwx 1 root root 5 Jan 16 05:21 /dev/sdb1 -> xvdb1

Yes, I do. So creating a file system should be possible, too:

[root@ip-172-31-43-22 ~]# mkfs.ext4 /dev/sdb1
mke2fs 1.42.3 (14-May-2012)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
655360 inodes, 2620595 blocks
131029 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Can I mount it?

[root@ip-172-31-43-22 ~]# mkdir /opt/PostgreSQL
[root@ip-172-31-43-22 ~]# mount /dev/sdb1 /opt/PostgreSQL
[root@ip-172-31-43-22 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1            2.0G  686M  1.3G  35% /
tmpfs                 298M     0  298M   0% /dev/shm
/dev/xvdb1            9.9G  151M  9.2G   2% /opt/PostgreSQL

Yes, I have full control. So I could do whatever I want with this VM. Imagine how easy it becomes to bring up test systems this way. Create a little script which wraps all the commands and you’re done in a few minutes.

As a lot of sensitive information was displayed here lets terminate the instance:

dwe@dwe:~/.aws$ aws ec2 terminate-instances --instance-ids i-07560b8a 
{
    "TerminatingInstances": [
        {
            "InstanceId": "i-07560b8a", 
            "CurrentState": {
                "Code": 32, 
                "Name": "shutting-down"
            }, 
            "PreviousState": {
                "Code": 16, 
                "Name": "running"
            }
        }
    ]
}

Easy as well. Have fun with command line tools …

Post Views: 588
Thumbnail [60x60]
by
Daniel Westermann