Many of you might know of Amazon’s Elastic Cloud Computing platform. If not, check this article I wrote about it recently.
Anyway, I had to migrate an Amazon computer from their Virginia Data center to their new Oregon Data center. The rental costs are the same but Oregon is closer to California, so it makes for more responsive computing. Unfortunately the Amazon Web Services webpage from where you control everything doesn’t offer a simple point and click option to achieve this. So unfortunately, I had to do it by hand. Rather unproductive 🙁
Newbie checkpoint
Before we move on, I must say, this one is a fairly technical article. I’ve written this more as a note to myself for future references than an article for widespread consumption. If you’re new to EC2, you probably don’t care about the details. This article I wrote on EC2 recently may be more appropriate.
However, if a Google search brought you here, then you know what you’re looking for.
I) Setting it up
We’re going to use Linux to copy the Windows volume, byte-for-byte across the network. It’s actually very simple, I’ve just given detailed instructions to assist newbies.
1. Backup your Windows EBS volume at the source region
- Shutdown your Windows EC2 instance.
- Detach Volume. volumes section -> the Windows volume(Right click) -> Detach Volume.
- Create snapshot. Right click it again -> create snapshot.
- Note the ‘zone’ of the volume (eg. ‘us-east-1b’) somewhere. You need it shortly
2. Create a Linux instance in your source region, in same zone I used Ubuntu, (search for “ubuntu/images” AMIs). Alternatively go to http://alestic.com/ and you’ll find links to the latest official Ubuntu AMIs on the top. I used AMI ami-20f97410 as reference.
- Pick instance type, t1.micro worked for me. My ubuntu 11.10 32bit instance was at 40% CPU throughout the disk read operation.
- Set instance in the SAME zone as your windows volume (eg. ‘us-east-1b’).
- Attach the windows volume to your Linux instance (right click-> attach -> pick linux instance) to something like /dev/sdf (Web UI may change to /dev/xvdf in future). If you don’t see your linux instance, the Windows volume and Linux instance are likely in different ‘zones’ (eg ‘us-east-1a’ vs ‘us-east-1b’).
- Ensure that port 9999 is open in the security groups
- Boot up the Linux instance
- do NOT mount the Windows volume (we only want the windows ‘hard drive’ attached to the ‘machine’, NOT the Linux OS to mount the filesystem present inside the ‘hard drive’ device)
- install cpipe via
sudo apt-get install cpipe
3. Create a blank EBS volume at your destination region
- Destination region (top left) -> Volumes -> Create Volume.
- I used the same size (30GB) but this is a chance to grow the volume.
- Note down the zone (eg. us-west-2a). Ensure it’s in the same region that you’d like your Windows EC2 instance to be.
4. Create a Linux instance in your destination region, in same zone Read the above section (#2) about the Linux instance. Again make sure the Linux and Windows volume zones are the same within the destination zone. My Ubuntu 11.10 32bit ‘server’ t1.micro instance was hitting 80% CPU during this disk write operation.
- Boot the instance.
- do NOT* mount the windows volume
- Attach the blank, destination windows volume to this Linux instance.
II) Start the copy process
1. Destination Log into the Linux EC2 instance and enter
sudo sh -c 'netcat -p 9999 -l > /dev/xvdf' 2. Source Log into the Linux EC2 instance and enter
sudo sh -c 'cpipe -vt -b 1024 < /dev/xvdf | netcat -q 1 dest-aws-ip-dns-address.com 9999' note : More new/current kernels will use /dev/xvdf for the Windows volume. Older kernels refer to it as /dev/sdf
note2 netcat is sometimes aliased to nc, in case you’re using another Linux distro. cpipe just gives your feedback about progress and tranfer rates. netcat runs over TCP/IP, so it’s robust against dropped/corrupt packets. You’d still want to perform a windows disk check on the destination once the transfer completes.
3. Wait I was getting an exact 3.00MB/sec (Mbytes, not Mbits) throughout this operation from the east to west coast. My 30GB image took 2hours, 50 minutes.
4. Create new Windows EC2 instance We have a volume but a EC2 instance/machine is needed to actually boot anything. If go use the AWS console GUI to go volume->snapshot->AMI, AWS incorrectly creates a Linux (?!?) VM that doesn’t boot because the EC2 configuration is now garbage. We work around that by
- Create a new Windows EC2 instance in the SAME zone as your volume. You will choose stock Windows AMI’s but pick one closest to your actual Windows volume. My source image was Windows 2003 R2, 32bits, EBS backed, so I fired off a (generic) Windows 2003 R2, 32bit, EBS backed AMI too.
- Boot this instance completely. Verify by logging in via RDP.
- Shutdown this instance
- Detach the EBS volume created in #1 just before
- Attach the EBS volume you copied over as /dev/sda1 (NOT the default /dev/xvdf in the Web UI dialog)
- Reboot.
- Safety measure: Check the resulting system’s drive (chkdsk).
- Enjoy!
5. Cleanup Remember that you’ve ONLY done the following
- Migrated the volume
- Recreated the EC2 instance
If you terminate the above, it’s gone forever, no AMI to relaunch it. So, I would highly recommend
- Build an AMI out of the currently working instance (right click instance -> create image AMI). This is now your new “day 0” reference point.
- Make a snapshot of the volume too. Yeah, I know the AMI backup above helps, but I keep weekly backups, so I’d have a “day 7” reference next week anyway. Plus since snapshots are differential, I did this right away.
Conclusion
It’s a LONG writeup, but I left no details for newbies. The above worked 100% for me migrating from US-East (Virginia) to US-West (Oregon). I’m in CA and did this to bring the VM closer to me without hitting northern CA’s higher charges.