Simon Willison's Weblog: caching

Adding dynamic features to an aggressively cached website

2026-01-28T22:10:08+00:00

My blog uses aggressive caching: it sits behind Cloudflare with a 15 minute cache header, which guarantees it can survive even the largest traffic spike to any given page. I've recently added a couple of dynamic features that work in spite of that full-page caching. Here's how those work.

Edit links that are visible only to me

This is a Django site and I manage it through the Django admin.

I have four types of content - entries, link posts (aka blogmarks), quotations and notes. Each of those has a different model and hence a different Django admin area.

I wanted an "edit" link on the public pages that was only visible to me.

The button looks like this:

I solved conditional display of this button with localStorage. I have a tiny bit of JavaScript which checks to see if the localStorage key ADMIN is set and, if it is, displays an edit link based on a data attribute:

document.addEventListener('DOMContentLoaded', () => {
  if (window.localStorage.getItem('ADMIN')) {
    document.querySelectorAll('.edit-page-link').forEach(el => {
      const url = el.getAttribute('data-admin-url');
      if (url) {
        const a = document.createElement('a');
        a.href = url;
        a.className = 'edit-link';
        a.innerHTML = '<svg>...</svg> Edit';
        el.appendChild(a);
        el.style.display = 'block';
      }
    });
  }
});

If you want to see my edit links you can run this snippet of JavaScript:

localStorage.setItem('ADMIN', '1');

My Django admin dashboard has a custom checkbox I can click to turn this option on and off in my own browser:

Those admin edit links are a very simple pattern. A more interesting one is a feature I added recently for navigating randomly within a tag.

Here's an animated GIF showing those random tag navigations in action (try it here):

On any of my blog's tag pages you can click the "Random" button to bounce to a random post with that tag. That random button then persists in the header of the page and you can click it to continue bouncing to random items in that same tag.

A post can have multiple tags, so there needs to be a little bit of persistent magic to remember which tag you are navigating and display the relevant button in the header.

Once again, this uses localStorage. Any click to a random button records both the tag and the current timestamp to the random_tag key in localStorage before redirecting the user to the /random/name-of-tag/ page, which selects a random post and redirects them there.

Any time a new page loads, JavaScript checks if that random_tag key has a value that was recorded within the past 5 seconds. If so, that random button is appended to the header.

This means that, provided the page loads within 5 seconds of the user clicking the button, the random tag navigation will persist on the page.

You can see the code for that here.

And the prompts

I built the random tag feature entirely using Claude Code for web, prompted from my iPhone. I started with the /random/TAG/ endpoint (full transcript):

Build /random/TAG/ - a page which picks a random post (could be an entry or blogmark or note or quote) that has that tag and sends a 302 redirect to it, marked as no-cache so Cloudflare does not cache it

Use a union to build a list of every content type (a string representing the table out of the four types) and primary key for every item tagged with that tag, then order by random and return the first one

Then inflate the type and ID into an object and load it and redirect to the URL

Include tests - it should work by setting up a tag with one of each of the content types and then running in a loop calling that endpoint until it has either returned one of each of the four types or it hits 1000 loops at which point fail with an error

Then:

I do not like that solution, some of my tags have thousands of items

Can we do something clever with a CTE?

Here's the something clever with a CTE solution we ended up with.

For the "Random post" button (transcript):

Look at most recent commit, then modify the /tags/xxx/ page to have a "Random post" button which looks good and links to the /random/xxx/ page

Then:

Put it before not after the feed icon. It should only display if a tag has more than 5 posts

And finally, the localStorage implementation that persists a random tag button in the header (transcript):

Review the last two commits. Make it so clicking the Random button on a tag page sets a localStorage value for random_tag with that tag and a timestamp. On any other page view that uses the base item template add JS that checks for that localStorage value and makes sure the timestamp is within 5 seconds. If it is within 5 seconds it adds a "Random name-of-tag" button to the little top navigation bar, styled like the original Random button, which bumps the localStorage timestamp and then sends the user to /random/name-of-tag/ when they click it. In this way clicking "Random" on a tag page will send the user into an experience where they can keep clicking to keep surfing randomly in that topic.

Tags: caching, django, javascript, localstorage, ai, cloudflare, generative-ai, llms, ai-assisted-programming

tidwall/pogocache

2025-07-21T23:58:53+00:00

tidwall/pogocache

New project from Josh Baker, author of the excellent tg C geospatial libarry (covered previously) and various other interesting projects:

Pogocache is fast caching software built from scratch with a focus on low latency and cpu efficency.

Faster: Pogocache is faster than Memcache, Valkey, Redis, Dragonfly, and Garnet. It has the lowest latency per request, providing the quickest response times. It's optimized to scale from one to many cores, giving you the best single-threaded and multithreaded performance.

Faster than Memcache and Redis is a big claim! The README includes a design details section that explains how the system achieves that performance, using a sharded hashmap inspired by Josh's shardmap project and clever application of threads.

Performance aside, the most interesting thing about Pogocache is the server interface it provides: it emulates the APIs for Redis and Memcached, provides a simple HTTP API and lets you talk to it over the PostgreSQL wire protocol as well!

psql -h localhost -p 9401
=> SET first Tom;
=> SET last Anderson;
=> SET age 37;

$ curl http://localhost:9401/last
Anderson

Via Show HN

Tags: c, caching, http, memcached, postgresql, redis

Double-keyed Caching: How Browser Cache Partitioning Changed the Web

2025-01-09T19:00:56+00:00

Double-keyed Caching: How Browser Cache Partitioning Changed the Web

Addy Osmani provides a clear explanation of how browser cache partitioning has changed the landscape of web optimization tricks.

Prior to 2020, linking to resources on a shared CDN could provide a performance boost as the user's browser might have already cached that asset from visiting a previous site.

This opened up privacy attacks, where a malicious site could use the presence of cached assets (based on how long they take to load) to reveal details of sites the user had previously visited.

Browsers now maintain a separate cache-per-origin. This has had less of an impact than I expected: Chrome's numbers show just a 3.6% increase in overall cache miss rate and 4% increase in bytes loaded from the network.

The most interesting implication here relates to domain strategy: hosting different aspects of a service on different subdomains now incurs additional cache-related performance costs compared to keeping everything under the same domain.

Via Hacker News

Tags: browsers, caching, performance, web-performance, addy-osmani

Weeknotes: Page caching and custom templates for Datasette Cloud

2024-01-07T20:45:11+00:00

My main development focus this week has been adding public page caching to Datasette Cloud, and exploring what custom template support might look like for that service.

Datasette Cloud primarily provides private "spaces" for teams to collaborate on data. A team can invite additional members, upload CSV files, use the API to ingest data, run enrichments, share private comments and browse and query the data together.

The overall goal is to help teams find stories in their data.

Originally I planned Datasette Cloud as an exclusively private collaboration space, but with hindsight this was a mistake. Datasette has been a tool for publishing data right from the start, and Datasette Cloud users quickly started asking for ways to share their data with the world.

I started with a plugin for this, datasette-public, allowing tables to be selectively made visible to unauthenticated users.

This raised a couple of challenges though. First, I worry about sudden spikes of traffic. Each Datasette Cloud user gets their own dedicated Fly container to ensure performance issues are isolated and don't affect other users, but I still don't like the idea of a big public traffic spike taking down a user's site.

Secondly, some users expressed interest in customizing the display of their public Datasette instance. The open source Datasette application has extensive support for this, but allowing users to run arbitrary HTML and JavaScript on a hosted service is a major risk for XSS holes.

This week I've been exploring a way to address both of these issues.

Full page caching for unauthorized users

I've used this trick multiple times through my career - at Lanyrd, at Eventbrite and even for my own personal blog. If a user is signed out, serve them pages through a simple full-page cache - something like Varnish. Set a short TTL on that cache - maybe as short as 15s - such that cached content doesn't have time to go stale.

Good caches include support for dog-pile prevention, also known as request coalescing. If 10 requests come in for the same page at exactly the same moment, the cache bundles them together and makes just a single request to the backend, then serves the result to all 10 waiting clients.

How to implement this for Datasette Cloud? My current plan is to use a separate domain - .datasette.site - for the publicly visible pages of each site. So simon.datasette.cloud (my personal Datasette Cloud space) would have simon.datasette.site as its public domain.

I got this working as a proof-of-concept this week. I actually got it working twice: I figured out how to run a dedicated Varnish instance on Fly, and then I realized that Cloudflare also now offer wildcard DNS support so I tried that out too.

I have both mechanisms up and running at the moment, on two separate domains. I'll likely go with the Cloudflare option to reduce the number of moving parts I'm responsible for myself, but having both means I can compare them to see which one is likely to work best.

Custom templates based on host

The other reason I decided to explore *.datasette.site was the security issue I mentioned earlier.

XSS attacks, where malicious JavaScript executes on a trusted domain, are a major security risk.

I plan to explore additional layers of protection against these such as CSP headers, but my general rule is to NEVER allow even a chance of untrusted JavaScript executing on a domain where authenticated users are able to perform privileged actions.

My current plan is to have *.datasette.site work as an entirely cookie-free domain. Any functionality that requires authentication will be handled by the privileged *.datasette.cloud domain instead.

This means I can allow users to provide their own custom templates for their public Datasette instance, without worrying that any mistakes in those templates could lead to a security breach elsewhere within the service.

There was just one catch: this meant I needed Datasette to be able to use different templates depending on host that the content was being served on.

After wasting a bunch of time trying to get this to work through monkey-patching, I realized the solution was to add a new plugin hook. jinja2_environment_from_request(datasette, request, env) is now implemented on main and should be out in a new alpha release pretty soon. The documentation for that hook includes an example that hints at how I'm using it for Datasette Cloud.

Fun further applications of this pattern

I'm wary of adding features to Datasette that only serve Datasette Cloud. In this case, I realized that the new plugin hook opens up some interesting possibilities for other users of Datasette.

I run a bunch of projects on top of Datasette myself - til.simonwillison.net and www.niche-museums.com are two examples of my sites that are actually templated Datasette instances.

Currently, those sites are hosted separately - which means I'm paying to run Datasette multiple times.

With the ability to serve different templates based on host, I've realized I could instead serve a single Datasette instance for multiple sites, each with their own custom templates.

Taking advantage of CNAMEs - or even wildcard DNS - means I could run a whole family of weird personal projects on a single instance without any incremental cost for each new project!

Releases

datasette-upgrade 0.1a0 - 2024-01-06
Upgrade Datasette instance configuration to handle new features

TILs

GitHub Actions, Issues and Pages to build a daily planner - 2024-01-02

Tags: caching, security, varnish, xss, datasette, cloudflare, weeknotes, datasette-cloud

Cloudflare does not consider vary values in caching decisions

2023-11-20T05:08:52+00:00

Cloudflare does not consider vary values in caching decisions

Here’s the spot in Cloudflare’s documentation where they hide a crucially important detail:

“Cloudflare does not consider vary values in caching decisions. Nevertheless, vary values are respected when Vary for images is configured and when the vary header is vary: accept-encoding.”

This means you can’t deploy an application that uses content negotiation via the Accept header behind the Cloudflare CDN—for example serving JSON or HTML for the same URL depending on the incoming Accept header. If you do, Cloudflare may serve cached JSON to an HTML client or vice-versa.

There’s an exception for image files, which Cloudflare added support for in September 2021 (for Pro accounts only) in order to support formats such as WebP which may not have full support across all browsers.

Tags: caching, http, cloudflare

New HTTP standards for caching on the modern web

2021-10-21T22:40:50+00:00

New HTTP standards for caching on the modern web

Cache-Status is a new HTTP header (RFC from August 2021) designed to provide better debugging information about which caches were involved in serving a request—“Cache-Status: Nginx; hit, Cloudflare; fwd=stale; fwd-status=304; collapsed; ttl=300” for example indicates that Nginx served a cache hit, then Cloudflare had a stale cached version so it revalidated from Nginx, got a 304 not modified, collapsed multiple requests (dogpile prevention) and plans to serve the new cached value for the next five minutes. Also described is $Target-Cache-Control: which allows different CDNs to respond to different headers and is already supported by Cloudflare and Akamai (Cloudflare-CDN-Cache-Control: and Akamai-Cache-Control:).

Via Hacker News

Tags: caching, dogpile, http, cloudflare

Last Mile Redis

2021-07-17T02:44:55+00:00

Last Mile Redis

Fly.io article about running a local redis cache in each of their geographic regions—“Cache data overlaps a lot less than you assume it will. For the most part, people in Singapore will rely on a different subset of data than people in Amsterdam or São Paulo or New Jersey.” But then they note that Redis has the ability to act as both a replica of a primary AND a writable server at the same time (“replica-read-only no”), which actually makes sense for a cache—it lets you cache local data but send out cluster-wide cache purges if necessary.

Via Hacker News

Tags: caching, redis, fly

nginx proxy-cache-lock

2017-11-14T21:53:07+00:00

nginx proxy-cache-lock

Crucially important feature hidden away in the nginx documentation: proxy_cache_lock enables request coalescing, or dog-pile protection: it means that if a hundred simultaneous requests all suffer the same cache miss, only one request is made to the backend and the answer is then sent back to all hundred requests at once. I’ve leaned heavily on this feature in Varnish for years—useful to know that nginx has the same capability.

Via How Discord Resizes 150 Million Images Every Day with Go and C++

Tags: caching, dogpile, nginx, varnish

Display your events on your own website with Lanyrd Badges

2011-01-13T20:38:00+00:00

Display your events on your own website with Lanyrd Badges

We’ve launched badges for Lanyrd—JavaScript that lets you embed a top bar or a content “splat” showing events you plan to attend, talks you’ve given in the past and other various combinations. I’m quite pleased with the implementation—the badges are configured using classes on a link to your Lanyrd profile, and the badges themselves are served through a combination of Amazon CloudFront for the initial script and a Varnish cache for the badge data itself to keep things nice and snappy.

Tags: badges, caching, cloudfront, javascript, varnish, lanyrd, recovered

Reddit is now running on Cassandra

2010-03-13T00:14:27+00:00

Reddit is now running on Cassandra

Migrating their persistent cache over from memcacheDB to Cassandra took one developer just ten days.

Tags: caching, cassandra, memcachedb, reddit

Cache Machine: Automatic caching for your Django models

2010-03-11T19:35:32+00:00

Cache Machine: Automatic caching for your Django models

This is the third new ORM caching layer for Django I’ve seen in the past month! Cache Machine was developed for zamboni, the port of addons.mozilla.org to Django. Caching is enabled using a model mixin class (to hook up some post_delete hooks) and a custom caching manager. Invalidation works by maintaining a “flush list” of dependent cache entries for each object—this is currently stored in memcached and hence has potential race conditions, but a comment in the source code suggests that this could be solved by moving to redis.

Tags: cachemachine, caching, django, memcached, mozilla, orm, ormcaching, python, redis

Announcing django-cachebot

2010-03-06T12:48:39+00:00

Announcing django-cachebot

The ORM caching space around Django is heating up. django-cachebot is used in production at mingle.com and takes a more low level approach to cache invalidation than Johnny Cache, enabling you to specifically mark the querysets you wish to cache and providing some advanced options for cache invalidation. Unfortunately it currently relies on a patch to Django core to enable its own manager.

Tags: cachebot, caching, django, mingle, orm, ormcaching

Is johnny-cache for you?

2010-03-02T11:44:08+00:00

Is johnny-cache for you?

“Using Johnny is really adopting a particular caching strategy. This strategy isn’t always a win; it can impact performance negatively”—but for a high percentage of Django sites there’s a very good chance it will be a net bonus.

Tags: caching, django, johnnycache, performance, python

jmoiron.net: Johnny Cache

2010-03-01T11:48:54+00:00

jmoiron.net: Johnny Cache

The blog entry announcing Johnny Cache (“a drop-in caching library/framework for Django that will cache all of your querysets forever in a consistent and safe manner”) to the world.

Tags: caching, django, johnnycache, python, querysets

Johnny Cache

2010-02-28T22:55:15+00:00

Johnny Cache

Clever twist on ORM-level caching for Django. Johnny Cache (great name) monkey-patches Django’s QuerySet classes and caches the result of every single SELECT query in memcached with an infinite expiry time. The cache key includes a “generation” ID for each dependent database table, and the generation is changed every single time a table is updated. For apps with infrequent writes, this strategy should work really well—but if a popular table is being updated constantly the cache will be all but useless. Impressively, the system is transaction-aware—cache entries created during a transaction are held in local memory and only pushed to memcached should the transaction complete successfully.

Tags: caching, databases, django, memcached, orm, ormcaching, performance, python

Internet Explorer Cookie Internals (FAQ)

2010-02-26T12:25:24+00:00

Internet Explorer Cookie Internals (FAQ)

Grr... IE 6, 7 and 8 don’t support the max-age cookie argument, forcing you to use an explicit expiry date instead. This appears to affect the cache busting cookie pattern, where you set a cookie to expire in 30 seconds for any user who posts content and use the presence of that cookie to skip caches and/or send their queries to a master instead of slave database. If you have to use expires, users with incorrect system clocks may get inconsistent results. Anyone know of a workaround?

Tags: cachebusting, caching, cookies, internet-explorer

High-end Varnish-tuning

2009-10-20T09:25:42+00:00

High-end Varnish-tuning

Tuning the Varnish HTTP cache to serve 27K requests/second on a single core 2.2GHz Opteron.

Tags: caching, http, performance, varnish

Caching in ASP.NET with the SqlCacheDependency Class

2009-08-18T12:15:44+00:00

Caching in ASP.NET with the SqlCacheDependency Class

Interesting cache invalidation concept: set up dependencies between cache entries and tables or rows in the database, then use triggers (which I presume are automatically created for you) to clear your cache.

Tags: aspdotnet, caching, invalidation

Memcached 1.4.0 released

2009-07-17T22:26:48+00:00

Memcached 1.4.0 released

The big new feature is the (optional) binary protocol, which enables other features such as CAS-everywhere and efficient client-side replication. Maintainer Dustin Sallings has also released some useful sounding EC2 instances which automatically assign nearly all of their RAM to memcached on launch and shouldn’t need any further configuration.

Tags: ami, binary, caching, cas, dustin-sallings, ec2, memcached, performance, scaling

Yahoo! proposal to open source "Traffic Server" via the ASF

2009-07-07T12:37:02+00:00

Yahoo! proposal to open source "Traffic Server" via the ASF

Traffic Server is a “fast, scalable and extensible HTTP/1.1 compliant caching proxy server” (presumably equivalent to things like Squid and Varnish) originally acquired from Inktomi and developed internally at Yahoo! for the past three years, which has been benchmarked handling 35,000 req/s on a single box. No source code yet but it looks like the release will arrive pretty soon.

Tags: apache, asf, caching, open-source, proxies, squid, trafficserver, varnish, yahoo

cache-money

2009-06-28T15:17:30+00:00

cache-money

A “write-through caching library for ActiveRecord”, maintained by Nick Kallen from Twitter. Queries hit memcached first, and caches are automatically kept up-to-date when objects are created, updated and deleted. Only some queries are supported—joins and comparisons won’t hit the cache, for example.

Tags: activerecord, cachemoney, caching, memcached, rails, twitter

Twitter, an Evolving Architecture

2009-06-28T15:09:44+00:00

Twitter, an Evolving Architecture

The most detailed write-up of Twitter’s current architecture I’ve seen, explaining the four layers of cache (all memcached) used by the Twitter API.

Tags: architecture, caching, memcached, twitter

Django tip: Caching and two-phased template rendering

2009-05-19T01:34:35+00:00

Django tip: Caching and two-phased template rendering

Neat trick for expensive pages which can be mostly cached with the exception of the “logged in as” bit—run them through the template system twice, caching the intermediary generated template.

Tags: adrian-holovaty, caching, django, performance, python, template

mmalone's django-caching

2009-05-07T07:36:48+00:00

mmalone's django-caching

Mike Malone shares code used by Pownce to add QuerySet level caching to Django. It’s a smart implementation—a CachingQuerySet class inspects the arguments passed to get(), and if they’re just a straight forward exact PK lookup hits memcache for the object before hitting the database. Signals are used to invalidate the cache.

Tags: caching, django, mike-malone, pownce, querysets, signals

hash_ring 1.2

2009-05-05T13:45:08+00:00

hash_ring 1.2

A Python library for consistent hashing with memcached, using MD5 and the same algorithm as libketama. Exposes an interface that is identical to regular memcache making this a drop-in replacement.

Tags: amir-salihefendic, caching, consistenthashing, django, hashring, libketama, md5, memcached, python

How search.twitter.com uses Varnish

2009-03-02T17:08:13+00:00

How search.twitter.com uses Varnish

Includes examples of the configuration options they use.

Tags: caching, search, twitter, varnish

Sharding Counters on Google App Engine

2009-01-27T20:27:41+00:00

Sharding Counters on Google App Engine

“While the datastore for App Engine scales to support a huge number of entities it is important to note that you can only expect to update any single entity, or entity-group, about five times a second”. This article explains a technique for sharding writes across multiple counters in detail, including a way to keep a memcache counter updated at the same time for faster reads.

Tags: caching, google, google-app-engine, memcache, python, scaling, sharding

ETags And Modification Times In Django

2008-12-13T09:49:42+00:00

ETags And Modification Times In Django

Part of Malcolm’s series of tutorials on implementing advanced HTTP concepts in Django.

Tags: caching, django, etags, http, malcolm-tredinnick

REST, I just don't get it

2008-08-15T08:20:04+00:00

REST, I just don't get it

Read the comments for some excellent practical reasons to care about REST, including cache management (PUT and DELETE can expire the cache entries for the corresponding GET), the ability to add or move parts of the server API without redeploying client libraries and the idempotency of GET / PUT / DELETE and HEAD (repeated POST operations may have side-effects).

Tags: caching, damien-katz, delete, get, idempotency, post, put, rest

Facelift Image Replacement

2008-08-05T18:36:19+00:00

Facelift Image Replacement

Like sIFR but with JavaScript and a PHP text rendering component. I question the need for the JavaScript if you’re already generating the images on the server, but the actual generation script is nicely done—it makes smart use of ImageMagick and caches the generated images.

Tags: caching, facelift, imagemagick, imagereplacement, javascript, php, sifr

Simon Willison's Weblog: caching

Adding dynamic features to an aggressively cached website

Edit links that are visible only to me

Random navigation within a tag

And the prompts

tidwall/pogocache

Double-keyed Caching: How Browser Cache Partitioning Changed the Web

Weeknotes: Page caching and custom templates for Datasette Cloud

Full page caching for unauthorized users

Custom templates based on host

Fun further applications of this pattern

Releases

TILs

Cloudflare does not consider vary values in caching decisions

New HTTP standards for caching on the modern web

Last Mile Redis

nginx proxy-cache-lock

Display your events on your own website with Lanyrd Badges

Reddit is now running on Cassandra

Cache Machine: Automatic caching for your Django models

Announcing django-cachebot

Is johnny-cache for you?

jmoiron.net: Johnny Cache

Johnny Cache

Internet Explorer Cookie Internals (FAQ)

High-end Varnish-tuning

Caching in ASP.NET with the SqlCacheDependency Class

Memcached 1.4.0 released

Yahoo! proposal to open source "Traffic Server" via the ASF

cache-money

Twitter, an Evolving Architecture

Django tip: Caching and two-phased template rendering

mmalone's django-caching

hash_ring 1.2

How search.twitter.com uses Varnish

Sharding Counters on Google App Engine

ETags And Modification Times In Django

REST, I just don't get it

Facelift Image Replacement