Copying to\from AWS GovCloud

It’s important to note before migrating anything out of GovCloud or into it for that matter that AWS GovCloud is an ITAR compliant region and data is subject to ITAR controls. Be sure you are meeting proper controls before moving data into or out of GovCloud. Migrating AMIs and instances from a commercial region to AWS GovCloud and vice versa is a little different due to the fact that AWS GovCloud is an isolated AWS region. Rather than spending time on the why this is, let’s focus on the how to make migrations happen.

First, I am going to be focusing on Linux based instances if there is a demand I will come back and update this with Windows options later on.

Note: Remember to take an EBS Snapshot first.

To move an instance first you will need an instance created and built out as you want it. Think of this as your gold image, either STIGing, hardening, or just installing the application. Once the instance is built and you have taken your snapshot of the EBS volume, SSH in and elevate privilege. Run

fdisk –l

to get a list of volumes attached. You will be able to tell which of the volumes is the EBS volume that you want to create the image on by the size. Create an S3 bucket for the image transfer this will be used later in the region you are using for the image build.

The easy way to do the copy is to create a new EBS volume in addition to the root volume and mount it to the same instance. Once that’s done you go back to your CLI, and run

dd if=/dev/sdX conv=sync,noerror bs=64K | gzip -c  > /path/to/backup.img.gz

Note: “that /dev/sdX” will be replaced with the volume path of the EBS volume you want to copy to govcloud.

You can also then run

fdisk -l /dev/sdX > /path/to/list_fdisk.info

This will add the telemetry data about the partition table, it might be useful on the recreation on the govcloud side.

Once you the disk copy is done, run

aws s3 sync /path/to/backup.img.gz <target> [--options]

this will copy the img file to the target S3 bucket for you to copy down. You could also use SCP if you wanted to copy it down from your ssh console but S3 will save the img copy for later use.

Now then copy the image out of S3 down locally, then upload it into an S3 bucket on GovCloud.

Once in GovCloud create a new EC2 instances with an EBS volume that is the same size as the one copied. SSH in elevate priviledge run

fdisk –l

so you know what your ebs volume is named then run

gunzip -c /path/to/backup.img.gz | dd of=/dev/sdX

here your path to the image will be the path to the s3 bucket with the image name and the sdX will be the new ebs volume on the ec2 instance.

And that’s it.

If you don’t want to do a disk copy or feel that this is all too much it is worth noting that AWS partners CloudVelox and CloudEndure are able to do this for you for a fee.

Let me know if this is helpful or if you need to see the steps for Windows instances?

Very Busy Day so here’s intro to EC2 Types

Well I got through my checkride semi-successfully yesterday thanks for all of the well wishes. Today is election day so please get out to vote and execute your civic rights.

As for the topic of the blog, here is a high level intro to EC2 instance types. We will get deeper in the next post. EC2 is split into instance families as seen here.

ec2types

 

Each family has it’s unique characteristics and as such its own use cases. Let’s go through some of these. The leading letter determines the family class, the following number indicates the generation of the instance, then the last part denotes the size of the instance. For example:

M4.XLarge = (M Family)(4th Generation) (Extra Large meaning 4 vCPU and 16 GB of mem)

The M class is a multi-purpose instance type, used for any number of applications where a balanced memory to cpu approach is applicable.

The C class is compute optimized and allows you to perform advanced network configuration and clustering of compute resources.

The T class is the default class and is highly burstable and the default instance type.

The D and I class are intended for database instances that users want to manage instead of using database services managed by AWS. The D class is dense storage while the I class is configured for large in memory databases like SAP HANA.

The G and P classes are for graphic intensive workloads. The G class has high performance NVIDIA processors for multimedia and high-end graphics, and the P class has GPU Direct support for general graphics workloads.

The R class is a memory optimized class that is meant for high memory hog apps think things like SharePoint.

For further comparison check out this chart, and the FAQ.

Instance Types Matrix

Instance Type vCPU Memory (GiB)  Storage (GB) Networking Performance Physical Processor Clock Speed (GHz) Intel AVX Intel AVX2 Intel Turbo EBS OPT
Enhanced Networking
t2.nano 1 0.5 EBS Only Low Intel Xeon family up to 3.3 Yes Yes
t2.micro 1 1 EBS Only Low to Moderate Intel Xeon family Up to 3.3 Yes Yes
t2.small 1 2 EBS Only Low to Moderate Intel Xeon family Up to 3.3 Yes Yes
t2.medium 2 4 EBS Only Low to Moderate Intel Xeon family Up to 3.3 Yes Yes
t2.large 2 8 EBS Only Low to Moderate Intel Xeon family Up to 3.0 Yes Yes
m4.large 2 8 EBS Only Moderate Intel Xeon E5-2676 v3** 2.4 Yes Yes Yes Yes Yes
m4.xlarge 4 16 EBS Only High Intel Xeon E5-2676 v3** 2.4 Yes Yes Yes Yes Yes
m4.2xlarge 8 32 EBS Only High Intel Xeon E5-2676 v3** 2.4 Yes Yes Yes Yes Yes
m4.4xlarge 16 64 EBS Only High Intel Xeon E5-2676 v3** 2.4 Yes Yes Yes Yes Yes
m4.10xlarge 40 160 EBS Only 10 Gigabit Intel Xeon E5-2676 v3 2.4 Yes Yes Yes Yes Yes
m4.16xlarge 64 256 EBS Only 20 Gigabit Intel Xeon E5-2686 v4 2.3 Yes Yes Yes Yes Yes
m3.medium 1 3.75 1 x 4 SSD Moderate Intel Xeon E5-2670 v2* 2.5 Yes Yes
m3.large 2 7.5 1 x 32 SSD Moderate Intel Xeon E5-2670 v2* 2.5 Yes Yes
m3.xlarge 4 15 2 x 40 SSD High Intel Xeon E5-2670 v2* 2.5 Yes Yes Yes
m3.2xlarge 8 30 2 x 80 SSD High Intel Xeon E5-2670 v2* 2.5 Yes Yes Yes
c4.large 2 3.75 EBS Only Moderate Intel Xeon E5-2666 v3 2.9 Yes Yes Yes Yes Yes
c4.xlarge 4 7.5 EBS Only High Intel Xeon E5-2666 v3 2.9 Yes Yes Yes Yes Yes
c4.2xlarge 8 15 EBS Only High Intel Xeon E5-2666 v3 2.9 Yes Yes Yes Yes Yes
c4.4xlarge 16 30 EBS Only High Intel Xeon E5-2666 v3 2.9 Yes Yes Yes Yes Yes
c4.8xlarge 36 60 EBS Only 10 Gigabit Intel Xeon E5-2666 v3 2.9 Yes Yes Yes Yes Yes
c3.large 2 3.75 2 x 16 SSD Moderate Intel Xeon E5-2680 v2 2.8 Yes Yes Yes
c3.xlarge 4 7.5 2 x 40 SSD Moderate Intel Xeon E5-2680 v2 2.8 Yes Yes Yes Yes
c3.2xlarge 8 15 2 x 80 SSD High Intel Xeon E5-2680 v2 2.8 Yes Yes Yes Yes
c3.4xlarge 16 30 2 x 160 SSD High Intel Xeon E5-2680 v2 2.8 Yes Yes Yes Yes
c3.8xlarge 32 60 2 x 320 SSD 10 Gigabit Intel Xeon E5-2680 v2 2.8 Yes Yes Yes
p2.xlarge 4 61 EBS Only High Intel Xeon E5-2686 v4 2.3 Yes Yes Yes Yes Yes
p2.8xlarge 32 488 EBS Only 10 Gigabit Intel Xeon E5-2686 v4 2.3 Yes Yes Yes Yes Yes
p2.16xlarge 64 732 EBS Only 20 Gigabit Intel Xeon E5-2686 v4 2.3 Yes Yes Yes Yes Yes
g2.2xlarge 8 15 1 x 60 SSD High Intel Xeon  E5-2670 2.6 Yes Yes Yes
g2.8xlarge 32 60 2 x 120 SSD 10 Gigabit Intel Xeon E5-2670 2.6 Yes Yes  –
x1.16large 64 976 1 x 1,920 SSD 10 Gigabit Intel Xeon E7-8880 v3 2.3 Yes Yes Yes Yes Yes
x1.32xlarge 128 1,952 2 x 1,920 SSD 20 Gigabit Intel Xeon E7-8880 v3 2.3 Yes Yes Yes Yes Yes
r3.large 2 15.25 1 x 32 SSD Moderate Intel Xeon E5-2670 v2 2.5 Yes Yes Yes
r3.xlarge 4 30.5 1 x 80 SSD Moderate Intel Xeon E5-2670 v2 2.5 Yes Yes Yes Yes
r3.2xlarge 8 61 1 x 160 SSD High Intel Xeon E5-2670 v2 2.5 Yes Yes Yes Yes
r3.4xlarge 16 122 1 x 320 SSD High Intel Xeon E5-2670 v2 2.5 Yes Yes Yes Yes
r3.8xlarge 32 244 2 x 320 SSD 10 Gigabit Intel Xeon E5-2670 v2 2.5 Yes Yes Yes
i2.xlarge 4 30.5 1 x 800 SSD Moderate Intel Xeon E5-2670 v2 2.5 Yes Yes Yes Yes
i2.2xlarge 8 61 2 x 800 SSD High Intel Xeon E5-2670 v2 2.5 Yes Yes Yes Yes
i2.4xlarge 16 122 4 x 800 SSD High Intel Xeon E5-2670 v2 2.5 Yes Yes Yes Yes
i2.8xlarge 32 244 8 x 800 SSD 10 Gigabit Intel Xeon E5-2670 v2 2.5 Yes Yes Yes
d2.xlarge 4 30.5 3 x 2000 Moderate Intel Xeon E5-2676 v3 2.4 Yes Yes Yes Yes Yes
d2.2xlarge 8 61 6 x 2000 High Intel Xeon E5-2676 v3 2.4 Yes Yes Yes Yes Yes
d2.4xlarge 16 122 12 x 2000 High Intel Xeon E5-2676 v3 2.4 Yes Yes Yes Yes Yes
d2.8xlarge 36 244 24 x 2000 10 Gigabit Intel Xeon E5-2676 v3 2.4 Yes Yes Yes Yes Yes

Direct Connect as a distraction?

Today is my checkride at AWS. Essentially I will stand in-front of a panel of my peers and present the AWS 101 deck and be asked questions in a real world customer scenario. I wouldn’t say I am nervous but I am anxious to see if I am as far along as I need\hope to be. One of the things I expect to be covered in depth is direct connect (DX), so while I am prepping I figured I would write up a quick post on what DX is and how it works.

Direct Connect is a dedicated connection from customers on-prem or CoLo facilities into AWS. Seems simple enough right? You can get DX connections in 1Gbps or 10Gbps ports and can aggregate multiple ports together. DX is ordered via your AWS console, when you order the connections you will have a couple of things you need to do. First you will have to present the IP space that you will be associating with the DX. You will also need to work with your ISP to create a connection to an AWS peering point with an Amazon Partner Network partner such as Equinix.

AWS will validate that you own your IP addresses and that you have a connection at the peering point. Then an authorization to connect will be issued to the APN partner and the connection will be made. Direct connect requires that you have established 802.1q VLANs and a BGP capable router. AWS will advertise all public Autonomous System Numbers (ASNs) you will want to limit the scope to the regions in which you want to share IP space with for your VPC. Private or public ASNs are allowed.

VLANs extended from your on-prem environment to AWS will travers the DX connection along the VLAN that they are assigned to a designated VPC with the correlating VLAN space. However you will not be able to connect directly across multiple VPC’s unless you have established VPC peering points in AWS, or if you use lollipop networking where your on-prem router acts as the core for VPC to VPC communications. The DX connection is not an internet connection, VPC’s will still require a designated Internet Gateway connection with an elastic IP or can use the on-prem internet connection for internet ingress and egress. Think of it this way, you can use the same security perimeter and monitoring solution with a DX so you don’t have to change the way your business or organization runs their security today.

This is a complex topic and I have only just scratched the surface. For more information check out the AWS Direct Connect FAQ.