Simon Willison's Weblog: load-balancing

Load Balancing

2024-07-13T22:51:45+00:00

Sam Rose built this interactive essay explaining how different load balancing strategies work. It's part of a series that includes memory allocation, bloom filters and more.

Tags: algorithms, load-balancing, explorables, sam-rose

The power of two random choices, visualized

2024-02-06T22:21:47+00:00

The power of two random choices, visualized

Grant Slatton shares a visualization illustrating “a favorite load balancing technique at AWS”: pick two nodes at random and then send the task to whichever of those two has the lowest current load score.

Why just two nodes? “The function grows logarithmically, so it’s a big jump from 1 to 2 and then tapers off *real* quick.”

Tags: aws, load-balancing, scaling

Scaling React Server-Side Rendering

2019-12-30T22:26:24+00:00

Scaling React Server-Side Rendering

Outstanding, detailed essay from 2017 on challenges and solutions for scaling React server-side rendering at Kijiji, Canada’s largest classified site (owned by eBay). There’s a lot of great stuff in here, including a detailed discussion of different approaches to load balancing, load shedding, component caching, client-side rendering fallbacks and more.

Via Hacker News

Tags: load-balancing, scaling, react

Round-robin Django setup with nginx

2009-12-21T15:43:42+00:00

Round-robin Django setup with nginx

An nginx trick I didn’t know: a low proxy_connect_timeout value (e.g. 2 seconds) combined with the proxy_next_upstream setting means that if one of your backends breaks a user won’t even see an error, they’ll just have a short delay before getting a response from a working server.

Tags: django, load-balancing, nginx, ops, sysadmin

Simple CouchDB multi-master clustering via Nginx

2009-11-19T16:37:36+00:00

Simple CouchDB multi-master clustering via Nginx

An impressive combination. CouchDB can be easily set up in a multi-master configuration, where writes to one master are replicated to the other and vice versa. This makes setting up a reliable CouchDB cluster is as simple as putting two such servers behind a single nginx proxy.

Tags: cluster, couchdb, load-balancing, multimaster, nginx, replication

Quoting Matt Palmer

2009-11-03T10:45:05+00:00

I loathe [hardware load balancers]. They’re expensive, restrictive, slow, and generally cause you a lot more pain and suffering than they’re worth. At my last job, one of my projects was to convert most of one of our existing clusters from a load-balancing appliance to use keepalived. Why would we do this? Because the $100k worth of appliance wasn’t capable of doing the job that $15k worth of commodity hardware and an installation of keepalived were handling with ease.

— Matt Palmer

Tags: keepalived, load-balancing, matt-palmer, ops, sysadmin

Load Balancing in Amazon EC2 with HAProxy

2009-02-05T23:12:24+00:00

Load Balancing in Amazon EC2 with HAProxy

Solid tutorial introduction to HAProxy.

Tags: ec2, griggheorghiu, haproxy, load-balancing

Minimal nginx conf to split get/post requests

2008-10-14T16:33:38+00:00

Minimal nginx conf to split get/post requests

Interesting idea for master-slave replication balancing where GET v.s. POST is load-balanced by nginx, presumably to different backend servers that are configured to talk to either a slave or a master. This won’t deal very will with replication lag though—you really want a user’s session to be bound to the master server for the next few GET requests after data is modified to ensure they see the effects of their updates. UPDATE: Amit fixed my complaint with a neat hack based around a cookie with a max age of 10 seconds.

Tags: load-balancing, masterslave, mysql, nginx, replication

mod_rpaf for Apache

2008-06-24T17:02:55+00:00

mod_rpaf for Apache

A more secure alternative to Django’s equivalent middleware: sets the REMOTE_ADDR of incoming requests from whitelisted load balancers to the X-Forwarded-For header, without any risk that if the load balancers are missing attackers could abuse it to spoof their IP addresses.

Tags: apache, django, http, load-balancing, middleware, modrpaf, rpaf, security, xforwardedfor

Scaling your website with the Perlbal web server (PDF)

2008-06-17T22:39:29+00:00

Scaling your website with the Perlbal web server (PDF)

Perlbal documentation is pretty thin on the ground; this is a really useful introduction from Frank Wiles.

Via Frank Wiles

Tags: frank-wiles, load-balancing, pdf, perlbal

Load Balancer Update

2008-05-01T10:06:13+00:00

Load Balancer Update

WordPress.com has switched from Pound to nginx for load balancing, resulting in a significant drop in CPU usage. I’ve been using nginx on my little VPS for over a year now with no complaints, nice to know it scales up as well as down.

Tags: load-balancing, nginx, pound, wordpress, wordpresscom

A Fair Proxy Balancer for Nginx and Mongrel

2007-12-09T14:57:44+00:00

A Fair Proxy Balancer for Nginx and Mongrel

nginx uses round robin for proxying by default; this extension module ensures requests are queued up and sent through to backend mongrel servers that aren’t currently busy. I don’t see any reason this wouldn’t work with servers other than mongrel.

Tags: fair, load-balancing, mongrel, nginx, proxies

Client Side Load Balancing for Web 2.0 Applications

2007-10-05T23:29:32+00:00

Client Side Load Balancing for Web 2.0 Applications

I recall that early versions of Netscape picked a random server from a hard-coded list each time a user clicked the “What’s New” button, back before server-side scaling techniques were well understood.

Via SitePoint

Tags: digitalweb, leizhu, load-balancing, netscape, scaling, sitepoint