Well, I felt real good tonight when I got the EC2 library for WPF to cross compile to Silverlight, which in of itself was not as easy as it should have been, but only to get block by security violations errors which I can only assume that is because EC2 does not provide the proper policy files.

Point is, I know this, but was so lost in the "fun" of the cross compile effort that I spaced out that this would be a problem.  So, its onto needing to write a back end web service to proxy the calls over to EC2 from the Silverlight client.  I've done this before, and its relatively easy, and will actually allow me to use the WPF library that I wrote already, but it's a little dissappointing none the less.

Amazon released the AWS Console about a week ago and I've been using it almost exclusively since instead of ElasticFox.  For the most part I think it is better, but at the same time I notice things that are not quite as good.  A plus is that it is accessible via any browser and any system, so you can always access your EC2 configurations no matter where or what browser you are running.  A negative is that it appears to be slower than ElasticFox, and I have found myself annoyingly having to wait at various times.

One thing I like is that it opens into a dashboard.  It's not a whole lot of useful information right now, but it is nice and will allow a nice overview once more features are added.

I've started the development of this library.  It's been an interesting experience using the existing Amazon EC2 C# library as a base for this library.  All I've got to say about that library is it really is coded weird;  I'm not going to say its coded badly, lets just leave it at weird.  Lets just say it really appears to be coded by a non .net person.

If you remember from an earlier post, I have a few requirements for this library:

• It should compile and run under both WPF and Silverlight with the same code base
  
• It should make asynchronous calls the norm (since that is the model for Silverlight)
  
• It should use the WebClient class, the norm for Silverlight

Yes, I'm kind of coding to the least common denominator of Silverlight (as versus to WPF), but it is a better model in my mind anyway.

One of the first issues I had was in determining how to issue REST requests for EC2 using the WebClient class.  Believe it or not, I really did not find a lot of info on this while googling around.  In the end, it turned out to be pretty easy.  The code to do this is exemplified in this picture:


In short, this method takes a dictionary of key value pairs which are mapped to REST parameters.  Each caller of this method (like the AttachVolume method) will set its parameters (at a minimum the Action key).  It then adds in the required parameters in the call (access keys, signatures, ...), most of which are passed in on the constructor of this object and others calculated on the fly.

The WebClient object is then created, and the proxy added if in use, and then two headers that are required.  A callback is setup for handling the async response, and then the data is passed up with a POST method using the UploadDataAsync method.  I happen to pass a custom object as the UserData that encapsulates a callback passed from the calling method, and that can be seen in the explanations to come as well as by examining the async handler.

That's pretty much it to send the REST command to EC2.  Now lets see how to construct a command that uses this technique.

I've been building this out with an immediate need which requires me to be able to attach and detach volumes from an EC2 instance, so I have focused on the Attach-Volume, Detach-Volume, as well as the Describe-Volumes commands (the last is needed to track the attach/detach status).  As an example, lets look at the code to detach a volume:

As part of all of this research into Amazon cloud computing I've started to program against the API's to see how they work.  Eventually I'd like to build a client that does a lot more than what can be done with ElasticFox, which is a great tool, but falls short when you try to do things that require multiple steps.  For example, one scenario that I'm already repeating all the time is the following:

1) Launch AMI
2) Mount 2 volumes to it (one step for each)

This isn't very difficult to do, but imaging if I'm building out managed services on behalf of other clients of mine, and the # of clients gets large, such that this will actually start to take a lot of time in the aggregate.  I'd like to have a tool that can do this automatically, and it is possible to program this to be done through the web services.

Another scenario that I'm thinking will be useful is to be able to do automated backups of AMI's and / or volumes, and perhaps also roll back AMI's and / or volumes on a regular schedule.  Again, not too hard with ElasticFox, but again a manual process that doesn't scale well.

So, I've started programming a WPF client using one of the C# api's available on the Amazon site, the C# Library for EC2.  It's been pretty easy to use so far, and I wanted something C# based instead of pure SOAP / REST.  There are deficiencies in the library that I will address later, but I wanted to show what I've done so far even though it's really just proving to myself that this works.

The following is a control that I put together to manipulate S3 buckets.  It's a listbox that is databound to the results from the 'listAllMyBuckets' method on the AWSAuthConnection class in the C# library:

Indeed, these are the two buckets that I have right now.

For reference, I include the XAML and the code behind for the control:

In my previous post I discussed how to use ElasticFox to start a windows instance in EC2.  In that post in order to keep it simple I avoided discussing security groups.  Security groups in EC2 are basically just instructions to EC2 on how to open firewall ports to allow communications to your system.

In ElasticFox, open the Security Groups tab and you will see something similar to the follow, although this is showing a security group called '42Spikes' that I've already added and run this site in:

My last post mentioned that I was trying to move my blog over to Amazon EC2.  I have that completed (if you are reading this, it came out of the EC2 cloud), and as promised I am now writing up how to do it.

First, this was not without challenge, of which the primary one is just how to operate EC2.  Specifically, here are the challenges I came across and which I will explain (although not all in this single post):

• Creating a key pair,
• Finding a windows server AMI to run,
• Getting an elastic IP,
  
• Setting up security zones,
• Starting the ami,
• Getting the server password,
  
• Getting remote access to the server,
• Configuring the server,
• Oh, terminating and rebooting starts over from scratch with the image and therefore blows away all of your config,
• How to bundle your OS into your own AMI, hence saving your configuration, but
• That only saves the C drive, which on an Amazon machine is fairly small,
• How to create your own volumes and attach them to your system,
  
• Using the volumes to store your application data (hence using it as persistent storage in S3), and
• Restarting your AMI and attaching the volumes

This is a fairly non-trivial list and documentation is pretty sparse on how to do all of this, and hence this blog entry.

When I got started, the best resource I found was Tim Heuer's blog which showed how to get started with EC2.  I therefore give him credit here and you can read that post here.

The first thing to do is get an already existing tool to work with EC2.  The one that Tim recommended and that I am using at this point is ElasticFox, an extension to Firefox.  The reason for needing a tool like this is that Amazon does not provide a client for managing EC2.  They only provide a web service API, which is actually quite great as you can programmatically do anything with EC2 from outside (or inside) the Amazon cloud.  But it does leave you wondering how to get going quickly, and if you are a Firefox user like I am just go and get ElasticFox.

note: ElasticFox is a great tool and you can do most things with it, but it does fall short for automating a sequence of tasks, which I can see will be quite useful.  For example, say you want to start one of your AMI's, attach volumes, assign the elastic IP?  You need to do these all manually.  Hence, I'm working on a WPF application to manage EC2, which will also allow you to do workflow.  I'll be posting that to codeplex soon and I'll let you all know when I do this.

One you have ElasticFox installed, open firefox and select Tools -> ElasticFox.  Since this is the first time, you will need to enter your Amazon EC2 information.  To do this, press the 'Credentials' button:


At which point you will see a form like this (albeit empty if it is your first time):


Enter your account name, access key and secret access key, and press 'Add'.  This information will be used by elasticfox to identify you to them and to access your resources.

When that is completed, ElasticFox will present to you similar to the following:

What you see here is a list of AMI's (virtual machines in the amazon library), which is also filtered to just show only those with windows in the title.  The naming convention of this is somewhat convoluted, but you will get used to it.  Notice that I have one selected, and that is the AMI I'm using to run my blog.  It is a windows 2003 R2 instance with IIS and SQL Server 2005 express installed.  In the 'Your Instances' list, you can see the virtual systems that you have created and their status.  Here you see the one that I am currently running the blog upon.

Now before we can do anything (like starting an instance), you must now create a keypair.  This keypair is used in the web service call that launches the instances and the private key is used by ssh to authenticate.  The keypair can easily be created in elastic fox, and it will pass it to the service calls for you.  The easiest way to create one is to use the 'KeyPairs' tab create a keypair specifying a name and then save the returned private key in a safe place on your file system.

So, open the keypair tab and you will see something similar to the following, although if this is your first time the list will be empty:


Now press on the green key button, and you will be presented with the following form:


I've entered a name in the field, but name yours anything you want.  When you press 'OK', elastic fox passes the name to amazon, and it returns a '.pem' key file to elastic fox, and elasticfox will ask you to save the file somewhere.  Please do this, as this keypair / file will be used later (specifically to start an instance)

Fundamentally, you can just right click on any AMI and select 'Launch Instance(s) of this AMI' and you will see a new instance start, go through initialization, and eventually reach the 'Running" state.  The running state is when the system is, well, running, and you can RDP in to the box (if your security zone is configured to allow RDP - I'll explain this later).

To get the system into the running state, it will take several minutes of initialization time.  I don't know the specifics, but what I can guess from experience is going on is the following:

• Allocation of the AMI to a physical server,
• Sys prepping the AMI so it has a unique system name and IP configurations,
• Changes made to the Amazon firewalls to allow access into the system
• Database entries created to allow manipulation of the virtual system via the web services

To be honest, I skipped a few things here, specifically the security zone configuration and the launch instance dialog.  Security zone info I'll cover in another post, and I'll briefly explain the launch instance dialog now.  When you select 'Launch Instance(s) of this AMI' you will be presented with this dialog asking you some information that EC2 needs to know to start the instance:

IMHO, at this point, the most important thing to select is the Security Groups to run the instance in.  I'll cover them in another post, but in short they are instructions to EC2 on how to open (or keep closed) tunnels through the firewall.  The default security group will basically shut down all access, even RDP.  But, if you try to connect to the system from ElasticFox, it will see you don't have port 3389 open and will ask you if it is ok if it is ok for it to add 3389 to the defautl zone.  So, leaving default here will allow you to RDP to the system (after ElasticFox confirms with you opening 3389), but will block everything else.  A good test of this is to try (since this instance by default installs IIS) to connect to the public DNS name of the instance with HTTP.  This will be blocked.  You can open port 80 later by creating a new security group.

Another thing of importance here is the specification of the keypair.  You can see here that I've selected one of my existing keypairs.  Again, I'm not sure of the details, but some investigation leads me to believe that firefox passes the .pem file associated to the key back to EC2 when you start the instance.  Note that I believe the .pem file is sent is because if you either don't save the file, OR your delete/move the file, you can not start an instance.

There is other information asked here, like the instance type, min and max # of instances, your key pair, availability zone, (I'll comment on these other options in another post) ...  but to keep it simple at this point you can at this point just press launch and you will get a machine that is getting spun up (I'll explain all these in other posts):

I've moved my blog over to a windows virtual system hosted on Amazon EC2.  Part of the process was also to update to the latest DasBlog, which I did.  It was an interesting experience and I'll write it up over the holidays. 

I'm still wondering whether or not I should move the whole blog over to Wordpress.  I'm going to work on that too.  Overall I'm real happy with EC2, except for the price of a windows system which I figure will be over $100 per month.  That's quite good for a business, but too much for a personal blog.

I do like having overall control of the blog though.  I think I noticed somewhere that the windows instance in EC2 was actually a linux machine virtualizing windows 2003.  I might be good if I can just virtualize an ubuntu instance, and then run windows 2003 inside of it (from my own licenses).  The linux boxes are much less expensive than the windows ones.

I saw a presentation the other day on Amazon EC2 which got me quite intrigued in the use of this service for doing “cloud computing”.  I currently host several sites for a couple of people on a dedicated windows server at a hosting center, and I’ve lately been wondering about other options so as to make things both easier and to save money, and this looked good at least at first sight.

The basic problem with it for my situation is that I’m running services that run on NET on windows servers, and the EC2 services only support Linux servers.  This immediately made me start to think about how I can get around this.  My first thoughts of course went to VMWare, but that would also necessitate buying VMWare in addition to Windows.  So, I did a little googling and I found this link:

http://www.howtoforge.com/amazon_elastic_compute_cloud_qemu

Interesting stuff, so I figured I’d give running windows with QEMU on an Ubuntu install.  I run VMWare Fusion on my mac, so I went and downloaded an Ubuntu appliance from VMware and had that up and running within a few minutes, as well as copying over some windows 2003 server .iso’s (which you can see on the desktop):



First thing I had to do (via the link) is install yum, as it wasn’t installed by default in this ubuntu appliance.  Well, it appears that Ubutnu doesn’t use yum, but something called apt-get, so where told to use yum in the link, I used this command:

sudo apt-get install qemu

The next step is to create a disk image for qemu to install windows into.  This is done with the following:

qemu-img create -f qcow win2003.img 4G

And then after arguing with the system about the amount of shared memory (I still don’t really have this worked out, but I got it running with 128m), I got the following command to start the virtual machine as can be seen in this image:


I will have to say that I am impressed with this; running virtualization software within virtualization software!

I’m going to let this run for a little to get the install done and I’ll continue the post more later.