I’ve uploaded my slides and demo code for Chicago Code Camp: Intro to Google App Engine. Enjoy!

Once you have Virtual Box up and running, you’ll want to install the Virtual Box Guest Additions. To do this, follow these steps:

1. Start your Ubuntu VM and log in.
2. Press the right CTRL key and select Devices->Install Guest Additions.
3. Open up a terminal window (Applications->Accessories->Terminal)
4. type in: cd /media/cdrom
5. type in: sudo bash ./autorun.sh
6. A new window will come up and work for a while. When it is done, it will tell you to reboot the VM. Reboot the VM when you see this message.
7. Log in again when the reboot is complete.
8. Press [Right Ctrl]+F. You now have full screen mode activated.

I’m posting this because the top rated instructions for Virtual Box and Ubuntu 9.04 that enable Full Screen mode have too many steps.

Linux seems to be the OS of choice for utility computing providers. EC2 charges 3 cents/hour for Linux instances (when buying $325 or $500 hours in bulk) compared to the 12.5 cents per hour charged for the comparable Windows instance. Given the price savings, that’s incentive enough for me to try building an app on the Linux system. OK, and the fact that knowing Linux will open up a wealth of Open Source tools to me AND I’ll be able to use the Advanced Packaging Tool (aka apt) to get and install applications. If I go the Windows route, I would lose the apt path, and that’s unacceptable since the Windows installations are a bit more difficult than the Linux ones.

I’m also not willing to give up Windows 7 as my main OS, which means I need a VM to run things. My filter process to pick out a VM host is simple: if I can install Ubuntu AND have the wireless card in my Dell XPS 13 work, I’m done with the search. Here’s how the search panned out:

Windows Virtual PC

Installed but wouldn’t run on Windows 7. I couldn’t find any other VM software on my installed image, and I gave up after 20 minutes of digging around the Internet and my PC. FAIL.

Total time spent: 45 minutes.

VMWare Workstation

This seemed to be the next, obvious choice. Everything installed great, but I couldn’t figure out how to share my wireless card with the Ubuntu VM. After 20 minutes, I again called off the search and moved on to something else. Remember, I’m just trying to get Ubuntu to work on my machine and talk to the Internet over my wireless card. If I fail on everything I try, I’ll do more research, but I gave up here too.

Total time spent: 55 minutes.

Virtual Box

Installed quickly, then allowed me to get Ubuntu set up. Wireless worked immediately-no thinking required on my part.

Total time spent: 20 minutes.

So, if you are running a 64-bit Windows 7 instance (or maybe something else?) and need to run Ubuntu 9.04, I can state that downloading an Ubuntu ISO and installing using the defaults onto a Virtual Box VM just works. I’m not going to investigate any other options-I’m happy enough with how things have been working, including speed of execution.

.and that is going to change, because it isn’t good to keep such things a secret!

You out there. Did you know that Chicago is a major technology hub. We just hide the technology in mundane things: finance, insurance, shipping, and other forms of commerce. Amazingly, a lot of this tech is profitable. Very profitable.

Trading firms are coming up with some incredible innovations in automated trading to notice when the market is moving a certain way and then take advantage of the few seconds where noticing is profitable and turn that observation into profit. Parallelization, grid computing, and more is happening in a wealth of areas. Unfortunately, broadcasting these discoveries and strategies is a losing proposition because many of these trading strategies only remain profitable while no one else has figured out the pattern. As soon as the pattern becomes common knowledge, the systems become too optimized to do anything good with the application and the developers have to find the next neat thing before their competition picks up on the fact. Due to the speed with which this cycle executes, the trading firms wind up staying quiet about the cool thing they figured out. That’s right, by the time the new algorithm or technique would be newsworthy, it’s a dead end.

Thankfully, that scene is broadening. Chicago has a lot of stellar talent that figures out all sorts of hard computer science problems. Warehousing systems and delivery companies constantly work on ways to find optimal solutions to the traveling salesman problem in p-algorithms type systems (instead of np) by finding heuristics that get good enough solutions.

Chicago is also home to Motorola’s cell phone division-a company that has brought out a number of innovative phones (though they have experienced a bit of a drought lately). Boeing is here-the company that makes air travel possible over much of the globe.

For you web geeks, talent abounds in the city. 37signals calls Chicago home. So does EveryBlock. By the way, if you use Google App Engine and Python, it’s likely you use Django which was co-authored by EveryBlock’s founder.

Crowd sourcing is at home in Chicago. crowdSPRING and threadless take crowdsourcing to a whole new level. Guess what, they are in Chicago too!

fastroot provides an awesome hosting experience. That job is made easy since the fattest Internet pipes in the US flow through Chicago. A lot of Internet research that happens between University of Wisconsin-Madison and University of Illinois Champaign-Urbana (top 10 CS schools for the longest time!). Those high speed pipes flow through Chicago. Another reason if financial. Because of the markets in Chicago, data has to flow as fast as possible. Without solid infrastructure, the financial markets couldn’t work. If you are founding an Internet business, you want to host in Chicago to minimize transmission delays to the coasts.

There are plenty more examples, but I think you are getting the idea. If you aren’t moving to Chicago for your startup (lower rent, cost of living, and cost of labor than any other tech hub), you should be asking yourself why you haven’t looked into Chicago. I think you’ll be surprised at what you find.

I’m at IIW this week and have had my days jam packed with info, meeting folks, and getting stuff done. As a result, I haven’t posted much this week. As you may know, VS 2010 Beta 1 dropped this week. I’ve been busy chasing down sources to find out if Azure works with .NET 4.0. I would hope it does, but no word. I’m downloading the beta now and will try putting up a .NET 4.0 app later tomorrow. I’ll post if I find success.

Wish me luck!

UPDATE!

I got a response back from one of my sources, before VS2010 finished downloading. Here’s what I have:

“Yes – [Azure will allow .NET 4.0] but you have to include ALL the .NET 4.0 assemblies (i.e. it’s a real pain), as Windows Azure only has .NET 3.5SP1 deployed today.  Once .NET 4.0 is officially released, Windows Azure will be updated.”

So, I won’t be running this experiment and will continue on the happy path where I use .NET 2008 SP1 and smile about it:)

In case you missed it, GAE updated their Java SDK to version 1.2.1. Responses can now be up to 10 MB in size. A full list of changes can be found here.

You can download the source for my F# AWS application from here. The remaining posts start on April 13, 2009, and run through today (with a few interruptions from me on other topics).

I wish I could say that their is something amazing that I did differently from everyone else who has hosted on EC2, but that’s just not so. Once you get an EC2 instance running, the rest of the process is identical to connecting to a virtual machine anywhere. If you need to add additional Windows components, the directions in this post are dynamite. If you use the management console, the directions on the Amazon article are pretty easy to map to actions in the UI without much thought. So, you are deploying to a virtual machine that is easily duplicated across the Amazon EC2 infrastructure. I would say that the strongest reason to use EC2 is that you already know how to use it. A second reason to use EC2 over your own data center is this: if you are using S3 and SimpleDB, you don’t pay for data transfers within the Amazon data center (but you do pay for data entering and leaving the data center!).

I installed the F# SDK on to the VM, added my web app to the inetpub/wwwroot folder, and made the application into an application root in the IIS Manager. I’d go into details, but these things are all well understood to Web developers. (If you need an assist, e-mail me.)

If you are hosting files and other items on Amazon, it will make sense to use EC2 to host your applications. If you don’t want to go out and buy a bunch of servers, rack them, and find a lot of high quality bandwidth, EC2 makes sense. If you are hosting your applications on an individual server that you rent by the month, you definitely should switch to EC2 (it’s almost the same thing only much less expensive).

Overall, I wasn’t ‘wowed’ by EC2. It’s a VM running on a machine I rent-nothing more. It’s strength is it’s simplicity.

All that said, it’s not my favorite environment for hosting applications on the web. EC2 doesn’t simplify my life-I still have to maintain patches on my own VMs. That’s not horrible and it does give me more control over when updates happen, but it doesn’t simplify things. Perhaps I haven’t been burned often enough by other people’s patches?

(You can review my description of S3 here.)

Because Amazon’s Simple Storage Service, S3, has been around for quite a while, the community has built a large number of libraries to access S3 programmatically. The C# space is pretty well crowded as well, and many folks wrote libraries that are good enough for their needs, then released those libraries to the world. The trick in picking an S3 library does not revolve around picking a best library. Instead, it involves finding one written by someone who had a similar set of needs. When looking for a library, think about how you want to add, update, and delete objects in S3. Write down the use cases. Then, download a set of libraries and keep messing with them until you find one that matches your needs. My planned usage involves:

1. Create a bucket once, when the application first runs. (This step can be done as part of initial setup, then never done again.)
2. All the objects for a given user have a prefix mapping to the user’s e-mail address.
3. Given a user ID, I need to be able to query for that user’s complete set of images (and no one else’s!)
4. Given a picture ID, I need to delete the image from S3.

Overall, pretty simplistic. However, I found that many libraries didn’t look at how to iterate over a given prefix. After kissing a number of frogs and getting no where, I found a library called LitS3. Given that I didn’t have any other complex requirements, like setting ACLs on objects, I was able to use this library and nothing else to insert, delete and query objects in my S3 bucket. LitS3 provides a simple object, LitS3.S3Service, to manipulate S3. You initialize the S3Service instance with the access key and secret so that the instance can manipulate your buckets. My initialization code is pretty basic and config driven:

Let’s take a look at the code I wrote to satisfy my 4 objectives using this object.

Create a Bucket

The S3Service class has a method, CreateBucket(string bucketName), to create an S3 bucket. Instead of figuring out how to create the bucket via a properly formatted URL, I cheated and wrote this code that should only need to run once, just before running any other code:

After that, I set createBucket to false so that this line wouldn’t run any longer. On Amazon, you get charged by how much work their systems have to do. This incents you, the developer, to use as little extra resources as possible. You can execute the call to create a bucket as often as you like without any harm. However, this is an expensive call and calling it too often will cost you, literally. If I was creating the bucket more frequently, I’d probably store the setting in SimpleDB and have the application key off of the data in SimpleDB instead.

Adding an Object to S3 (based on User ID)

When a user uploads an image, they use the ASP.NET FileUpload control. The server-side code converts the image to a JPEG that fits in a 300×300 pixel square and stores that version in S3. To save an object to S3, you need to hand over an array of bytes, a bucket name, a key name, a MIME type, and an ACL for the item. In this implementation, all images will be saved as being publically visible. The Identity of a user is their e-mail address. S3 doesn’t like some characters

List Object Keys (Based on User ID)

Given a user ID, LitS3 makes it trivial to retrieve all of the names of objects for the user. The code is a one liner:

Each ID, coupled with the rule that all im
ages live right under a prefix based on the user’s e-mail address, gives me an easy way to then calculate the full path to each image. The images are returned using an ImageItem structure who can retrieve it’s other fields (stored in SimpleDB). Since we covered SimpleDB earlier (overview, insert/update, query, and delete), I’ll skip that information for ImageItem. ImageItem has the following DataContract:

(plus some other functions to read and write to SimpleDB)

The application takes the result of ListObjects and creates a Sequence of ImageItems to return to the caller (using WCF + JSON).

Deleting an Object from S3

To remove an object from S3, you just need to have the key to the object and proper authorization. The owner of the bucket always has proper authorization to delete an object. When performing any cleanup, make sure to also remove any data stored elsewhere about the object.

With that last bit, we can manage objects in S3.

I ordered a laptop from the Dell Outlet a few days ago. The Dell Outlet has a well known issue in their system: a package’s order status will correctly move to Shipped when the package is assigned a FedEx tracking number and moved to the loading dock but the carrier and tracking information will forever say Data Temporarily Unavailable. I say FedEx because most of us using the outlet will also opt for the free 3-5 day shipping which, in 2009, is provided by FedEx Ground for United States shipments.

Knowing that they use FedEx, you can look up the information without calling up Dell customer service. How do you do this? When you placed your order, you should have received an order number. If you don’t have the information handy, login to the Dell site under the My Account link (usually on the top of the page on the Dell site). Then, look at Order Status and select your order. You should eventually see a page with this information on it:

(yes, the Customer Number, Purchase ID, and Order Number have been edited.)

Copy the Order Number and head over to FedEx’s site to their Track by Reference page. You probably don’t have and definitely don’t need an Account Number, so skip the first field. Then, put the Order Number into the field labeled Enter reference. For the approximate ship date, enter in the Order Date-accuracy isn’t important since the date is only used to scope the reference/Order number to a subset of all FedEx shipments. Set Destination Country to United States and put in your zip code/postal code in the Destination postal code box. Lastly, click on Track and you should see your Dell Outlet item on your PC.

Of course, you can also call up customer support at Dell, but do you really want to wait for an answer when, through this method, you can easily compulsively check to see if you new laptop|netbook|PC is at your door yet? I know what I did!