Simon Willison's Weblog: cloud-computing

RAIC? What's that?

2010-04-20T08:43:20+00:00

“Redundant Array of Independent Cloud providers”. Solve the cloud lock-in problem by storing data with multiple different providers from the start.

Via Matt Biddulph

Tags: cloud-computing, raic

PiCloud

2010-02-26T18:25:41+00:00

PiCloud

An interesting twist on cloud computing for Python. “import cloud; cloud.call(my_function, arguments)” serialises my_function and its arguments, pushes it up to one of their EC2 servers and hands you back a job ID which you can poll (or block on) for a response. They suggest using it for long running tasks such as web crawling or image processing.

Tags: cloud, cloud-computing, picloud, python

Quoting Jeremy from Reddit

2010-01-07T22:10:37+00:00

Since we moved to EC2, the number of unique users has gone up 50%, and pageviews are up more than 100%. To support this growth, we have added 30% more ram and 50% more CPU, yet because of Amazon's constant price reductions, we are actually paying less per month now than when we started.

— Jeremy from Reddit

Tags: amazon, cloud-computing, ec2, pricing, reddit

Tile Drawer

2009-08-26T09:32:56+00:00

Tile Drawer

The most inspired use of EC2 I’ve seen yet: center a map on an area, pick a Cascadenik stylesheet URL (or write and link to your own) and Tile Drawer gives you an Amazon EC2 AMI and a short JSON snippet. Launch the AMI with the JSON as the “user data” parameter and you get your own OpenStreetMap tile rendering server, which self-configures on startup and starts rendering and serving tiles using your custom design.

Via Mike Migurski

Tags: amazon, cascadenik, cloud-computing, ec2, json, mapnik, mapping, michal-migurski, openstreetmap, userdata

NASA NEBULA Services

2009-07-28T12:10:29+00:00

NASA NEBULA Services

NASA’s new NEBULA cloud computing platform appears to be built entirely on open source infrastructure, including Python, Django, Fabric, Eucalyptus, RabbitMQ, Trac and Solr.

Via Grig Gheorghiu

Tags: cloud-computing, django, eucalyptus, fabric, nasa, nebula, open-source, python, rabbitmq, solr, trac

EC2: Creating an Image

2009-05-19T19:50:55+00:00

EC2: Creating an Image

Here’s the easier way of creating your own AMI: start with a running instance in EC2, then customise it to fit your purposes and create a new bundle (and then AMI) using the ec2-bundle-vol command.

Tags: amazon, ami, cloud-computing, ec2

HOWTO Building a self-bundling Debian AMI

2009-05-19T19:49:22+00:00

HOWTO Building a self-bundling Debian AMI

Not as terrifying as you would have thought. Also contains some neat hints as to how some of the more magical parts of EC2 work (like the way your SSH public key automatically ends up in /root/.ssh/authorized_keys).

Tags: amazon, ami, cloud-computing, debian, ec2

New Features for EC2: Elastic Load Balancing, Auto Scaling, and Amazon CloudWatch

2009-05-18T10:07:56+00:00

New Features for EC2: Elastic Load Balancing, Auto Scaling, and Amazon CloudWatch

EC2 now fulfils the promise of “magic scaling in the cloud” out of the box—CloudWatch monitors performance of your EC2 instances without needing to install any monitoring software, Auto Scaling allows you to configure “scaling triggers” which start up new instances based on information from CloudWatch, and Elastic Load Balancing balances requests across all available instances.

Tags: amazon, autoscaling, cloud-computing, cloudwatch, ec2, elasticloadbalancing, scaling

Amazon Elastic MapReduce

2009-04-02T10:25:37+00:00

Amazon Elastic MapReduce

Hadoop as a service. Basically a web based GUI around Hadoop—you could roll this yourself on EC2 but for a small markup on regular EC2 prices you get to avoid the extra work setting everything up. Data processing scripts can be written in Java, Ruby, Perl, Python, PHP, R, or C++ and are loaded in to S3 before firing off the job.

Via Joe Drumgoole

Tags: amazon, amazon-web-services, cloud-computing, ec2, hadoop, mapreduce, s3

maps from scratch

2009-03-15T13:20:23+00:00

maps from scratch

An idea whose time has come: using EC2 AMIs for tutorial sessions to give everyone a pre-configured environment.

Tags: cloud-computing, ec2, mapping, michal-migurski, tutorials

Introducing the Karmic Koala, our mascot for Ubuntu 9.10

2009-02-21T17:19:18+00:00

Introducing the Karmic Koala, our mascot for Ubuntu 9.10

Ubuntu 9.10 will have a strong focus on cloud computing, including tools for easily creating EC2 AMIs and Eucalyptus, an open-source system for running an EC2-compatible cloud in your own data centre.

Via Simon Wardley

Tags: cloud-computing, ec2, eucalyptus, karmickoala, linux, mark-shuttleworth, ubuntu

Google App Engine: A roadmap update!

2009-02-09T19:00:38+00:00

Google App Engine: A roadmap update!

Receiving e-mail, background tasks and XMPP. I predict a bunch of sites will start building small parts of their overall functionality on App Engine when some of these features land (much easier than hosting your own custom XMPP server).

Tags: cloud-computing, email, google, google-app-engine, python, xmpp

Manage Amazon EC2 With New Web-Based AWS Management Console

2009-01-09T09:34:49+00:00

Manage Amazon EC2 With New Web-Based AWS Management Console

Finally! I’m amazed it took Amazon so long to do this. Managing EC2 instances from a custom Firefox extension was pretty bizarre. It’s a very nice interface, built on top of YUI. Unfortunately you still have to manage your entire virtual server farm using a single shared Amazon account.

Tags: amazon, aws, cloud-computing, ec2, javascript, yui

How Tarsnap uses Amazon Web Services

2008-12-14T19:35:28+00:00

How Tarsnap uses Amazon Web Services

Useful case study, including some thoughts on SimpleDB.

Via programming

Tags: amazon-web-services, aws, cloud-computing, ec2, s3, simpledb, tarsnap

Trying out Windows on EC2

2008-10-24T09:57:55+00:00

Trying out Windows on EC2

Phillip Pearson provides the missing documentation.

Tags: amazonaws, cloud-computing, ec2, phillip-pearson, windows

Windows Server and SQL Server on EC2

2008-10-23T15:54:27+00:00

Windows Server and SQL Server on EC2

Launched today, the pricing includes rental of the Windows license. Regular Windows is 25% to 50% more expensive than Linux, but SQL Server comes in at a hefty $1.10 per hour, which is $9636 per year (nearly three times as much as a Linux server running an open source database).

Via AWS Blog

Tags: cloud-computing, ec2, open-source, pricing, sqlserver, windows

Quoting Tim Bray

2008-10-15T17:09:55+00:00

Are we so deranged here in the twenty-first century that we’re going to re-enact, wide-eyed, the twin tragedies of the great desktop-suite lock-in and the great proprietary-SQL lock-in? You know, the ones where you give a platform vendor control over your IT budget? Gimme a break.

— Tim Bray

Tags: cloud-computing, lockin, tim-bray