Simon Willison's Weblog: starlette

Starlette 1.0 skill

2026-03-23T00:05:00+00:00

See Experimenting with Starlette 1.0 with Claude skills.

Experimenting with Starlette 1.0 with Claude skills

2026-03-22T23:57:44+00:00

Starlette 1.0 is out! This is a really big deal. I think Starlette may be the Python framework with the most usage compared to its relatively low brand recognition because Starlette is the foundation of FastAPI, which has attracted a huge amount of buzz that seems to have overshadowed Starlette itself.

Kim Christie started working on Starlette in 2018 and it quickly became my favorite out of the new breed of Python ASGI frameworks. The only reason I didn't use it as the basis for my own Datasette project was that it didn't yet promise stability, and I was determined to provide a stable API for Datasette's own plugins... albeit I still haven't been brave enough to ship my own 1.0 release (after 26 alphas and counting)!

Then in September 2025 Marcelo Trylesinski announced that Starlette and Uvicorn were transferring to their GitHub account, in recognition of their many years of contributions and to make it easier for them to receive sponsorship against those projects.

The 1.0 version has a few breaking changes compared to the 0.x series, described in the release notes for 1.0.0rc1 that came out in February.

The most notable of these is a change to how code runs on startup and shutdown. Previously that was handled by on_startup and on_shutdown parameters, but the new system uses a neat lifespan mechanism instead based around an async context manager:

@contextlib.asynccontextmanager
async def lifespan(app):
    async with some_async_resource():
        print("Run at startup!")
        yield
        print("Run on shutdown!")

app = Starlette(
    routes=routes,
    lifespan=lifespan
)

If you haven't tried Starlette before it feels to me like an asyncio-native cross between Flask and Django, unsurprising since creator Kim Christie is also responsible for Django REST Framework. Crucially, this means you can write most apps as a single Python file, Flask style.

This makes it really easy for LLMs to spit out a working Starlette app from a single prompt.

There's just one problem there: if 1.0 breaks compatibility with the Starlette code that the models have been trained on, how can we have them generate code that works with 1.0?

I decided to see if I could get this working with a Skill.

Building a Skill with Claude

Regular Claude Chat on claude.ai has skills, and one of those default skills is the skill-creator skill. This means Claude knows how to build its own skills.

So I started a chat session and told it:

Clone Starlette from GitHub - it just had its 1.0 release. Build a skill markdown document for this release which includes code examples of every feature.

I didn't even tell it where to find the repo, Starlette is widely enough known that I expected it could find it on its own.

It ran git clone https://github.com/encode/starlette.git which is actually the old repository name, but GitHub handles redirects automatically so this worked just fine.

The resulting skill document looked very thorough to me... and then I noticed a new button at the top I hadn't seen before labelled "Copy to your skills". So I clicked it:

And now my regular Claude chat has access to that skill!

A task management demo app

I started a new conversation and prompted:

Build a task management app with Starlette, it should have projects and tasks and comments and labels

And Claude did exactly that, producing a simple GitHub Issues clone using Starlette 1.0, a SQLite database (via aiosqlite) and a Jinja2 template.

Claude even tested the app manually like this:

cd /home/claude/taskflow && timeout 5 python -c "
import asyncio
from database import init_db
asyncio.run(init_db())
print('DB initialized successfully')
" 2>&1

pip install httpx --break-system-packages -q \
  && cd /home/claude/taskflow && \
  python -c "
from starlette.testclient import TestClient
from main import app

client = TestClient(app)

r = client.get('/api/stats')
print('Stats:', r.json())

r = client.get('/api/projects')
print('Projects:', len(r.json()), 'found')

r = client.get('/api/tasks')
print('Tasks:', len(r.json()), 'found')

r = client.get('/api/labels')
print('Labels:', len(r.json()), 'found')

r = client.get('/api/tasks/1')
t = r.json()
print(f'Task 1: \"{t[\"title\"]}\" - {len(t[\"comments\"])} comments, {len(t[\"labels\"])} labels')

r = client.post('/api/tasks', json={'title':'Test task','project_id':1,'priority':'high','label_ids':[1,2]})
print('Created task:', r.status_code, r.json()['title'])

r = client.post('/api/comments', json={'task_id':1,'content':'Test comment'})
print('Created comment:', r.status_code)

r = client.get('/')
print('Homepage:', r.status_code, '- length:', len(r.text))

print('\nAll tests passed!')
"

For all of the buzz about Claude Code, it's easy to overlook that Claude itself counts as a coding agent now, fully able to both write and then test the code that it is writing.

Here's what the resulting app looked like. The code is here in my research repository.

Tags: open-source, python, ai, asgi, kim-christie, generative-ai, llms, ai-assisted-programming, claude, coding-agents, skills, agentic-engineering, starlette

Streaming File Upload Prototype

2025-12-14T20:17:00+00:00

Research: Streaming File Upload Prototype

Tags: starlette

Building, launching, and scaling ChatGPT Images

2025-05-13T23:52:22+00:00

Building, launching, and scaling ChatGPT Images

Gergely Orosz landed a fantastic deep dive interview with OpenAI's Sulman Choudhry (head of engineering, ChatGPT) and Srinivas Narayanan (VP of engineering, OpenAI) to talk about the launch back in March of ChatGPT images - their new image generation mode built on top of multi-modal GPT-4o.

The feature kept on having new viral spikes, including one that added one million new users in a single hour. They signed up 100 million new users in the first week after the feature's launch.

When this vertical growth spike started, most of our engineering teams didn't believe it. They assumed there must be something wrong with the metrics.

Under the hood the infrastructure is mostly Python and FastAPI! I hope they're sponsoring those projects (and Starlette, which is used by FastAPI under the hood.)

They're also using some C, and Temporal as a workflow engine. They addressed the early scaling challenge by adding an asynchronous queue to defer the load for their free users (resulting in longer generation times) at peak demand.

There are plenty more details tucked away behind the firewall, including an exclusive I've not been able to find anywhere else: OpenAI's core engineering principles.

Ship relentlessly - move quickly and continuously improve, without waiting for perfect conditions

Own the outcome - take full responsibility for products, end-to-end

Follow through - finish what is started and ensure the work lands fully

I tried getting o4-mini-high to track down a copy of those principles online and was delighted to see it either leak or hallucinate the URL to OpenAI's internal engineering handbook!

Gergely has a whole series of posts like this called Real World Engineering Challenges, including another one on ChatGPT a year ago.

Via @GergelyOrosz

Tags: python, scaling, ai, openai, generative-ai, chatgpt, llms, gergely-orosz, starlette

Building Python tools with a one-shot prompt using uv run and Claude Projects

2024-12-19T07:00:37+00:00

I've written a lot about how I've been using Claude to build one-shot HTML+JavaScript applications via Claude Artifacts. I recently started using a similar pattern to create one-shot Python utilities, using a custom Claude Project combined with the dependency management capabilities of uv.

(In LLM jargon a "one-shot" prompt is a prompt that produces the complete desired result on the first attempt. Confusingly it also sometimes means a prompt that includes a single example of the desired output format. Here I'm using the first of those two definitions.)

I'll start with an example of a tool I built that way.

I had another round of battle with Amazon S3 today trying to figure out why a file in one of my buckets couldn't be accessed via a public URL.

Out of frustration I prompted Claude with a variant of the following (full transcript here):

I can't access the file at EXAMPLE_S3_URL. Write me a Python CLI tool using Click and boto3 which takes a URL of that form and then uses EVERY single boto3 trick in the book to try and debug why the file is returning a 404

It wrote me this script, which gave me exactly what I needed. I ran it like this:

uv run debug_s3_access.py \
  https://test-public-bucket-simonw.s3.us-east-1.amazonaws.com/0f550b7b28264d7ea2b3d360e3381a95.jpg

You can see the text output here.

Inline dependencies and uv run

Crucially, I didn't have to take any extra steps to install any of the dependencies that the script needed. That's because the script starts with this magic comment:

# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "click",
#     "boto3",
#     "urllib3",
#     "rich",
# ]
# ///

This is an example of inline script dependencies, a feature described in PEP 723 and implemented by uv run. Running the script causes uv to create a temporary virtual environment with those dependencies installed, a process that takes just a few milliseconds once the uv cache has been populated.

This even works if the script is specified by a URL! Anyone with uv installed can run the following command (provided you trust me not to have replaced the script with something malicious) to debug one of their own S3 buckets:

uv run http://tools.simonwillison.net/python/debug_s3_access.py \
  https://test-public-bucket-simonw.s3.us-east-1.amazonaws.com/0f550b7b28264d7ea2b3d360e3381a95.jpg

Writing these with the help of a Claude Project

The reason I can one-shot scripts like this now is that I've set up a Claude Project called "Python app". Projects can have custom instructions, and I used those to "teach" Claude how to take advantage of inline script dependencies:

You write Python tools as single files. They always start with this comment:
# /// script
# requires-python = ">=3.12"
# ///
These files can include dependencies on libraries such as Click. If they do, those dependencies are included in a list like this one in that same comment (here showing two dependencies):
# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "click",
#     "sqlite-utils",
# ]
# ///

That's everything Claude needs to reliably knock out full-featured Python tools as single scripts which can be run directly using whatever dependencies Claude chose to include.

I didn't suggest that Claude use rich for the debug_s3_access.py script earlier but it decided to use it anyway!

I've only recently started experimenting with this pattern but it seems to work really well. Here's another example - my prompt was:

Starlette web app that provides an API where you pass in ?url= and it strips all HTML tags and returns just the text, using beautifulsoup

Here's the chat transcript and the raw code it produced. You can run that server directly on your machine (it uses port 8000) like this:

uv run https://gist.githubusercontent.com/simonw/08957a1490ebde1ea38b4a8374989cf8/raw/143ee24dc65ca109b094b72e8b8c494369e763d6/strip_html.py

Then visit http://127.0.0.1:8000/?url=https://simonwillison.net/ to see it in action.

Custom instructions

The pattern here that's most interesting to me is using custom instructions or system prompts to show LLMs how to implement new patterns that may not exist in their training data. uv run is less than a year old, but providing just a short example is enough to get the models to write code that takes advantage of its capabilities.

I have a similar set of custom instructions I use for creating single page HTML and JavaScript tools, again running in a Claude Project:

Never use React in artifacts - always plain HTML and vanilla JavaScript and CSS with minimal dependencies.

CSS should be indented with two spaces and should start like this:
<style>
* {
  box-sizing: border-box;
}
Inputs and textareas should be font size 16px. Font should always prefer Helvetica.

JavaScript should be two space indents and start like this:
<script type="module">
// code in here should not be indented at the first level

Most of the tools on my tools.simonwillison.net site were created using versions of this custom instructions prompt.

Tags: aws, cli, python, s3, ai, prompt-engineering, generative-ai, llms, ai-assisted-programming, claude, claude-artifacts, uv, rich, prompt-to-app, starlette

Python Developers Survey 2023 Results

2024-09-03T02:47:45+00:00

Python Developers Survey 2023 Results

The seventh annual Python survey is out. Here are the things that caught my eye or that I found surprising:

25% of survey respondents had been programming in Python for less than a year, and 33% had less than a year of professional experience.

37% of Python developers reported contributing to open-source projects last year - a new question for the survey. This is delightfully high!

6% of users are still using Python 2. The survey notes:

Almost half of Python 2 holdouts are under 21 years old and a third are students. Perhaps courses are still using Python 2?

In web frameworks, Flask and Django neck and neck at 33% each, but FastAPI is a close third at 29%! Starlette is at 6%, but that's an under-count because it's the basis for FastAPI.

The most popular library in "other framework and libraries" was BeautifulSoup with 31%, then Pillow 28%, then OpenCV-Python at 22% (wow!) and Pydantic at 22%. Tkinter had 17%. These numbers are all a surprise to me.

pytest scores 52% for unit testing, unittest from the standard library just 25%. I'm glad to see pytest so widely used, it's my favourite testing tool across any programming language.

The top cloud providers are AWS, then Google Cloud Platform, then Azure... but PythonAnywhere (11%) took fourth place just ahead of DigitalOcean (10%). And Alibaba Cloud is a new entrant in sixth place (after Heroku) with 4%. Heroku's ending of its free plan dropped them from 14% in 2021 to 7% now.

Linux and Windows equal at 55%, macOS is at 29%. This was one of many multiple-choice questions that could add up to more than 100%.

In databases, SQLite usage was trending down - 38% in 2021 to 34% for 2023, but still in second place behind PostgreSQL, stable at 43%.

The survey incorporates quotes from different Python experts responding to the numbers, it's worth reading through the whole thing.

Via PSF news

Tags: open-source, postgresql, python, sqlite, surveys, pytest, psf, pydantic, starlette

AI-enhanced development makes me more ambitious with my projects

2023-03-27T14:38:56+00:00

The thing I'm most excited about in our weird new AI-enhanced reality is the way it allows me to be more ambitious with my projects.

As an experienced developer, ChatGPT (and GitHub Copilot) save me an enormous amount of "figuring things out" time. For everything from writing a for loop in Bash to remembering how to make a cross-domain CORS request in JavaScript - I don't need to even look things up any more, I can just prompt it and get the right answer 80% of the time.

This doesn't just make me more productive: it lowers my bar for when a project is worth investing time in at all.

In the past I've had plenty of ideas for projects which I've ruled out because they would take a day - or days - of work to get to a point where they're useful. I have enough other stuff to build already!

But if ChatGPT can drop that down to an hour or less, those projects can suddenly become viable.

Which means I'm building all sorts of weird and interesting little things that previously I wouldn't have invested the time in.

I'll describe my latest one of these mini-projects in detail.

Using ChatGPT to build a system to archive ChatGPT messages

I use ChatGPT a lot, and I want my own archive of conversations I've had with it.

There's no API for retrieving previous conversations at the moment, but if you open up the browser DevTools and watch the network tab you can see it loading JSON any time you navigate to an older conversation.

This gave me an idea for a solution...

What if I could run some JavaScript on the ChatGPT site that grabbed that JSON data and sent it off to my own server?

I couldn't find the raw data in JavaScript variables available to the console, so I came up with an alternative approach: could I patch the default fetch() function to intercept calls made by the ChatGPT code to that endpoint, and intercept the data that way?

Normally this is where the project would end. Figuring out how to patch fetch() like that is non-trivial - where I'm using the software engineer's definition of "non-trivial" as meaning "I don't know how to do that off the top of my head". I'd have to do some thinking and research! I'm already tempted to drop this idea and work on something else.

But now I have ChatGPT... so I prompted it:

Write a new JavaScript fetch() function which calls the old one but also saves the JSON that is retrieved - if it is used to retrieve JSON - in a window.fetchedData object

It gave me some code, but it wasn't quite what I wanted. So I added a follow-up prompt:

No I need to replace the old window.fetch() function entirely - I want to patch it so that I can intercept calls made by other existing code

And it produced the following:

// Define a global object to store fetched JSON data
window.fetchedData = {};

// Store a reference to the original fetch function
const originalFetch = window.fetch;

// Define a new fetch function that wraps the original fetch
window.fetch = async function(url, options) {
  try {
    // Call the original fetch function
    const response = await originalFetch(url, options);

    // Check if the response has a JSON content type
    const contentType = response.headers.get('content-type');
    if (contentType && contentType.includes('application/json')) {
      // If the response is JSON, clone the response so we can read it twice
      const responseClone = response.clone();
      // Parse the JSON data and save it to the fetchedData object
      const jsonData = await responseClone.json();
      window.fetchedData[url] = jsonData;
    }

    // Return the original response
    return response;
  } catch (error) {
    // Handle any errors that occur during the fetch
    console.error('Error fetching and saving JSON:', error);
    throw error;
  }
};

That's exactly what I wanted!

I pasted it into the DevTools console on the ChatGPT site and clicked on some links. It worked as intended.

Here's that full ChatGPT transcript that got me to this point.

So now I have a prototype! I've proved to myself that I can intercept the JSON data fetched by ChatGPT's own web application code.

I only wanted to run my code on requests that matched https://chat.openai.com/backend-api/conversation/... - I could write a regex for that, but I'd have to remember to escape the necessary characters. ChatGPT did that for me too:

const pattern = /^https:\/\/chat\.openai\.com\/backend-api\/conversation\//;

So now I have the key building blocks I need for my solution: I can intercept JSON fetches and then filter to just the data from the endpoint I care about.

But I need CORS support

My plan was to have my intercepting fetch() call POST the JSON data to my own Datasette Cloud instance, similar to how I record scraped Hacker News listings as described in this post about Datasette's new write API.

One big problem: this means that code running on the chat.openai.com domain needs to POST JSON to another server. And that means the other server needs to be serving CORS headers.

Datasette Cloud doesn't (yet) support CORS - and I wasn't about to implement a new production feature there just so I could solve a problem for this prototype.

What I really needed was some kind of CORS proxy... a URL running somewhere which forwards traffic on to Datasette Cloud but adds CORS headers to enable it to be called from elsewhere.

This represents another potential blocking point: do I really want to implement an entire proxy web application just for this little project?

Here's my next ChatGPT prompt:

Write a web app in python Starlette which has CORS enabled - including for the auth header - and forwards all incoming requests to any path to another server host specified in an environment variable

I like Starlette and I know it has CORS support and is great for writing proxies. I was hopeful that GPT-4 had seen its documentation before the September 2021 cut-off date.

ChatGPT wrote me some very solid code! It used the python-dotenv package though, and I didn't feel like using that dependency. So I added:

Rewrite that to use os.environ

The exception handling looked wrong to me, so I said:

Catch httpx.HTTPError as the only exception you care about

And after just those three prompts it produced the following:

import os
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.cors import CORSMiddleware
from starlette.routing import Route
from starlette.requests import Request
from starlette.responses import JSONResponse
import httpx

TARGET_SERVER_HOST = os.environ.get("TARGET_SERVER_HOST")

middleware = [
    Middleware(
        CORSMiddleware,
        allow_origins=["*"],
        allow_credentials=True,
        allow_methods=["*"],
        allow_headers=["*"],
    )
]

app = Starlette(middleware=middleware)

async def forward_request(request: Request):
    try:
        async with httpx.AsyncClient() as client:
            target_url = f"{TARGET_SERVER_HOST}{request.url.path}"
            method = request.method
            headers = dict(request.headers)
            if "host" in headers:
                del headers["host"]
            if request.method in ["POST", "PUT", "PATCH"]:
                request_body = await request.json()
            else:
                request_body = None

            response = await client.request(
                method, target_url, headers=headers, json=request_body
            )
            return JSONResponse(response.json(), status_code=response.status_code)
    except httpx.HTTPError as e:
        return JSONResponse({"error": str(e)}, status_code=500)

app.add_route("/{path:path}", forward_request, methods=["GET", "POST", "PUT", "PATCH", "DELETE"])

if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app, host="0.0.0.0", port=8000)

I tested this locally... and spotted a bug. It would fail with a 500 error if the incoming request included a content-length header. I fixed that myself by adding this code:

if "content-length" in headers:
    del headers["content-length"]

My finished code is here. Here's the ChatGPT transcript.

I deployed this to Vercel using the method described in this TIL - and now I had a working proxy server.

Creating the tables and a token

ChatGPT had got me a long way. The rest of my implementation was now a small enough lift that I could quickly finish it by myself.

I created two tables in my Datasette Cloud instance by executing the following SQL (using the datasette-write plugin):

create table chatgpt_conversation (
  id text primary key,
  title text,
  create_time float,
  moderation_results text,
  current_node text,
  plugin_ids text
);
create table chatgpt_message (
  id text primary key,
  conversation_id text references chatgpt_conversation(id),
  author_role text,
  author_metadata text,
  create_time float,
  content text,
  end_turn integer,
  weight float,
  metadata text,
  recipient text
);

Then I made myself a Datasette API token with permission to insert-row and update-row just for those two tables, using the new finely grained permissions feature in the 1.0 alpha series.

The last step was to combine this all together into a fetch() function that did the right thing. I wrote this code by hand, using the ChatGPT prototype as a starting point:

const TOKEN = "dstok_my-token-here";

// Store a reference to the original fetch function
window.originalFetch = window.fetch;

// Define a new fetch function that wraps the original fetch

window.fetch = async function (url, options) {
  try {
    // Call the original fetch function
    const response = await originalFetch(url, options);

    // Check if the response has a JSON content type
    const contentType = response.headers.get("content-type");
    if (contentType && contentType.includes("application/json")) {
      // If the response is JSON, clone the response so we can read it twice
      const responseClone = response.clone();
      // Parse the JSON data and save it to the fetchedData object
      const jsonData = await responseClone.json();
      // NOW: if url for https://chat.openai.com/backend-api/conversation/...
      // do something very special with it
      const pattern =
        /^https:\/\/chat\.openai\.com\/backend-api\/conversation\/(.*)/;
      const match = url.match(pattern);
      if (match) {
        const conversationId = match[1];
        console.log("conversationId", conversationId);
        console.log("jsonData", jsonData);
        const conversation = {
          id: conversationId,
          title: jsonData.title,
          create_time: jsonData.create_time,
          moderation_results: JSON.stringify(jsonData.moderation_results),
          current_node: jsonData.current_node,
          plugin_ids: JSON.stringify(jsonData.plugin_ids),
        };
        fetch(
          "https://starlette-cors-proxy-simonw-datasette.vercel.app/data/chatgpt_conversation/-/insert",
          {
            method: "POST",
            headers: {
              "Content-Type": "application/json",
              Authorization: `Bearer ${TOKEN}`,
            },
            mode: "cors",
            body: JSON.stringify({
              row: conversation,
              replace: true,
            }),
          }
        )
          .then((d) => d.json())
          .then((d) => console.log("d", d));
        const messages = Object.values(jsonData.mapping)
          .filter((m) => m.message)
          .map((message) => {
            m = message.message;
            let content = "";
            if (m.content) {
              if (m.content.text) {
                content = m.content.text;
              } else {
                content = m.content.parts.join("\n");
              }
            }
            return {
              id: m.id,
              conversation_id: conversationId,
              author_role: m.author ? m.author.role : null,
              author_metadata: JSON.stringify(
                m.author ? m.author.metadata : {}
              ),
              create_time: m.create_time,
              content: content,
              end_turn: m.end_turn,
              weight: m.weight,
              metadata: JSON.stringify(m.metadata),
              recipient: m.recipient,
            };
          });
        fetch(
          "https://starlette-cors-proxy-simonw-datasette.vercel.app/data/chatgpt_message/-/insert",
          {
            method: "POST",
            headers: {
              "Content-Type": "application/json",
              Authorization: `Bearer ${TOKEN}`,
            },
            mode: "cors",
            body: JSON.stringify({
              rows: messages,
              replace: true,
            }),
          }
        )
          .then((d) => d.json())
          .then((d) => console.log("d", d));
      }
    }

    // Return the original response
    return response;
  } catch (error) {
    // Handle any errors that occur during the fetch
    console.error("Error fetching and saving JSON:", error);
    throw error;
  }
};

The fiddly bit here was writing the JavaScript that reshaped the ChatGPT JSON into the rows: [array-of-objects] format needed by the Datasette JSON APIs. I could probably have gotten ChatGPT to help with that - but in this case I pasted the SQL schema into a comment and let GitHub Copilot auto-complete parts of the JavaScript for me as I typed it.

And it works

Now I can paste the above block of code into the browser console on chat.openai.com and any time I click on one of my older conversations in the sidebar the fetch() will be intercepted and the JSON data will be saved to my Datasette Cloud instance.

It's a lot more than just this project

This ChatGPT archiving problem is just one example from the past few months of things I've built that I wouldn't have tackled without AI-assistance.

It took me longer to write this up than it did to implement the entire project from start to finish!

When evaluating if a new technology is worth learning and adopting, I have two criteria:

Does this let me build things that would have been impossible to build without it?
Can this reduce the effort required for some projects such that they tip over from "not worth it" to "worth it" and I end up building them?

Large language models like GPT3/4/LLaMA/Claude etc clearly meet both of those criteria - and their impact on point two keeps on getting stronger for me.

Some more examples

Here are a few more examples of projects I've worked on recently that wouldn't have happened without at least some level of AI assistance:

I used ChatGPT to generate me the OpenAI schema I needed to build the datasette-chatgpt-plugin plugin, allowing human language questions in ChatGPT to be answered by SQL queries executed against Datasette.
Using ChatGPT to write AppleScript describes how I used ChatGPT to finally figure out enough AppleScript to liberate my notes data, resulting in building apple-notes-to-sqlite.
datasette-paste-table isn't in a usable state yet, but I built the first interactive prototype for that using ChatGPT.
Convert git log output to JSON using jq is something I figured out using ChatGPT - transcript here.
Learning Rust with ChatGPT, Copilot and Advent of Code describes one of my earlier efforts to use ChatGPT to help learn a completely new (to me) programming language.
Reading thermometer temperatures over time from a video describes a project I built using ffmpg and Google Cloud Vision.
Interactive row selection prototype with Datasette explains a more complex HTML and JavaScript UI prototype I worked on.

Tags: projects, ai, generative-ai, chatgpt, github-copilot, applescript, llms, ai-assisted-programming, starlette

asgi-gzip 0.2

2022-10-13T23:47:49+00:00

Release: asgi-gzip 0.2

Tags: starlette

Automatically opening issues when tracked file content changes

2022-04-28T17:18:14+00:00

I figured out a GitHub Actions pattern to keep track of a file published somewhere on the internet and automatically open a new repository issue any time the contents of that file changes.

Extracting GZipMiddleware from Starlette

Here's why I needed to solve this problem.

I want to add gzip support to my Datasette open source project. Datasette builds on the Python ASGI standard, and Starlette provides an extremely well tested, robust GZipMiddleware class that adds gzip support to any ASGI application. As with everything else in Starlette, it's really good code.

The problem is, I don't want to add the whole of Starlette as a dependency. I'm trying to keep Datasette's core as small as possible, so I'm very careful about new dependencies. Starlette itself is actually very light (and only has a tiny number of dependencies of its own) but I still don't want the whole thing just for that one class.

So I decided to extract the GZipMiddleware class into a separate Python package, under the same BSD license as Starlette itself.

The result is my new asgi-gzip package, now available on PyPI.

What if Starlette fixes a bug?

The problem with extracting code like this is that Starlette is a very effectively maintained package. What if they make improvements or fix bugs in the GZipMiddleware class? How can I make sure to apply those same fixes to my extracted copy?

As I thought about this challenge, I realized I had most of the solution already.

Git scraping is the name I've given to the trick of running a periodic scraper that writes to a git repository in order to track changes to data over time.

It may seem redundant to do this against a file that already lives in version control elsewhere - but in addition to tracking changes, Git scraping can offfer a cheap and easy way to add automation that triggers when a change is detected.

I need an actionable alert any time the Starlette code changes so I can review the change and apply a fix to my own library, if necessary.

Since I already run all of my projects out of GitHub issues, automatically opening an issue against the asgi-gzip repository would be ideal.

My track.yml workflow does exactly that: it implements the Git scraping pattern against the gzip.py module in Starlette, and files an issue any time it detects changes to that file.

Starlette haven't made any changes to that file since I started tracking it, so I created a test repo to try this out.

Here's one of the example issues. I decided to include the visual diff in the issue description and have a link to it from the underlying commit as well.

How it works

The implementation is contained entirely in this track.yml workflow. I designed this to be contained as a single file to make it easy to copy and paste it to adapt it for other projects.

It uses actions/github-script, which makes it easy to do things like file new issues using JavaScript.

Here's a heavily annotated copy:

name: Track the Starlette version of this

# Run on repo pushes, and if a user clicks the "run this action" button,
# and on a schedule at 5:21am UTC every day
on:
  push:
  workflow_dispatch:
  schedule:
  - cron:  '21 5 * * *'

# Without this block I got this error when the action ran:
# HttpError: Resource not accessible by integration
permissions:
  # Allow the action to create issues
  issues: write
  # Allow the action to commit back to the repository
  contents: write

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: actions/github-script@v6
      # Using env: here to demonstrate how an action like this can
      # be adjusted to take dynamic inputs
      env:
        URL: https://raw.githubusercontent.com/encode/starlette/master/starlette/middleware/gzip.py
        FILE_NAME: tracking/gzip.py
      with:
        script: |
          const { URL, FILE_NAME } = process.env;
          // promisify pattern for getting an await version of child_process.exec
          const util = require("util");
          // Used exec_ here because 'exec' variable name is already used:
          const exec_ = util.promisify(require("child_process").exec);
          // Use curl to download the file
          await exec_(`curl -o ${FILE_NAME} ${URL}`);
          // Use 'git diff' to detect if the file has changed since last time
          const { stdout } = await exec_(`git diff ${FILE_NAME}`);
          if (stdout) {
            // There was a diff to that file
            const title = `${FILE_NAME} was updated`;
            const body =
              `${URL} changed:` +
              "\n\n```diff\n" +
              stdout +
              "\n```\n\n" +
              "Close this issue once those changes have been integrated here";
            const issue = await github.rest.issues.create({
              owner: context.repo.owner,
              repo: context.repo.repo,
              title: title,
              body: body,
            });
            const issueNumber = issue.data.number;
            // Now commit and reference that issue number, so the commit shows up
            // listed at the bottom of the issue page
            const commitMessage = `${FILE_NAME} updated, refs #${issueNumber}`;
            // https://til.simonwillison.net/github-actions/commit-if-file-changed
            await exec_(`git config user.name "Automated"`);
            await exec_(`git config user.email "actions@users.noreply.github.com"`);
            await exec_(`git add -A`);
            await exec_(`git commit -m "${commitMessage}" || exit 0`);
            await exec_(`git pull --rebase`);
            await exec_(`git push`);
          }

In the asgi-gzip repository I keep the fetched gzip.py file in a tracking/ directory. This directory isn't included in the Python package that gets uploaded to PyPI - it's there only so that my code can track changes to it over time.

More interesting applications

I built this to solve my "tell me when Starlette update their gzip.py file" problem, but clearly this pattern has much more interesting uses.

You could point this at any web page to get a new GitHub issue opened when that page content changes. Subscribe to notifications for that repository and you get a robust , shared mechanism for alerts - plus an issue system where you can post additional comments and close the issue once someone has reviewed the change.

There's a lot of potential here for solving all kinds of interesting problems. And it doesn't cost anything either: GitHub Actions (somehow) remains completely free for public repositories!

Update: October 13th 2022

Almost six months after writing about this... it triggered for the first time!

Here's the issue that the script opened: #4: tracking/gzip.py was updated.

I applied the improvement (Marcelo Trylesinski and Kai Klingenberg updated Starlette's code to avoid gzipping if the response already had a Content-Encoding header) and released version 0.2 of the package.

Tags: github, gzip, projects, python, datasette, asgi, github-actions, git-scraping, github-issues, starlette

Exploring the SameSite cookie attribute for preventing CSRF

2021-08-03T21:09:02+00:00

In reading Yan Zhu's excellent write-up of the JSON CSRF vulnerability she found in OkCupid one thing puzzled me: I was under the impression that browsers these days default to treating cookies as SameSite=Lax, so I would expect attacks like the one Yan described not to work in modern browsers.

This lead me down a rabbit hole of exploring how SameSite actually works, including building an interactive SameSite cookie exploration tool along the way. Here's what I learned.

Background: Cross-Site Request Forgery

I've been tracking CSRF (Cross-Site Request Forgery) on this blog since 2005(!)

A quick review: let's say you have a page in your application that allows a user to delete their account, at https://www.example.com/delete-my-account. The user has to be signed in with a cookie in order to activate that feature.

If you created that page to respond to GET requests, I as an evil person could create a page at https://www.evil.com/force-you-to-delete-your-account that does this:

<img src="https://www.example.com/delete-my-account">

If I can get you to visit my page, I can force you to delete your account!

But you're smarter than that, and you know that GET requests should be idempotent. You implement your endpoint to require a POST request instead.

Turns out I can still force-delete accounts, if I can trick a user into visiting a page with the following evil HTML on it:

<form action="https://www.example.com/delete-my-account" method="POST">
<input type="submit" value="Delete my account">
</form>
<script>document.forms[0].submit()</script>

The form submits with JavaScript the instant they load the page!

CSRF is an extremely common and nasty vulnerability - especially since it's a hole by default: if you don't know what CSRF is, you likely have it in your application.

Traditionally the solution has been to use CSRF tokens - hidden form fields which "prove" that the user came from a form on your own site, and not a form hosted somewhere else. OWASP call this the Double Submit Cookie pattern.

Web frameworks like Django implement CSRF protection for you. I built asgi-csrf to help add CSRF token protection to ASGI applications.

Clearly it would be better if we didn't have to worry about CSRF at all.

As far as I can tell, work on specifying the SameSite cookie attribute started in June 2016. The idea was to add an additional attribute to cookies that specifies the policy for if they should be included in requests made to a domain from pages hosted on another domain.

Today, all modern browsers support SameSite. MDN has SameSite documentation, but a summary is:

SameSite=None - the cookie is sent in "all contexts" - more-or-less how things used to work before SameSite was invented. Update: One major edge-case here is that Safari apparently ignores None if the "Prevent cross-site tracking" privacy preference is turned on - and since that is on by default, this means that SameSite=None is effectively useless if you care about Safari or Mobile Safari users.
SameSite=Strict - the cookie is only sent for requests that originate on the same domain. Even arriving on the site from an off-site link will not see the cookie, unless you subsequently refresh the page or navigate within the site.
SameSite=Lax - cookie is sent if you navigate to the site through following a link from another domain but not if you submit a form. This is generally what you want to protect against CSRF attacks!

The attribute is specified by the server in a set-cookie header that looks like this:

set-cookie: lax-demo=3473; Path=/; SameSite=lax

Why not habitually use SameSite=Strict? Because then if someone follows a link to your site their first request will be treated as if they are not signed in at all. That's bad!

So explicitly setting a cookie with SameSite=Lax should be enough to protect your application from CSRF vulnerabilities... provided your users have a browser that supports it.

(Can I Use reports 93.95% global support for the attribute - not quite high enough for me to stop habitually using CSRF tokens, but we're getting there.)

What if the SameSite attribute is missing?

Here's where things get interesting. If a cookie is set without a SameSite attribute at all, how should the browser treat it?

Over the past year, all of the major browsers have been changing their default behaviour. The goal is for a missing SameSite attribute to be treated as if it was SameSite=Lax - providing CSRF protection by default.

I have found it infuriatingly difficult to track down if and when this change has been made:

Chrome/Chromium offer the best documentation - they claim to have ramped up the new default to 100% of users in August 2020. WebViews in Android still have the old default behaviour, which is scheduled to be fixed in Android 12 (not yet released).
Firefox have a blog entry from August 2020 which says "Starting with Firefox 79 (June 2020), we rolled it out to 50% of the Firefox Beta user base" - but I've not been able to find any subsequent updates. Update 26th August 2024: It turns out Firefox didn't ship this after all, going with their own Total Cookie Protection solution instead, which rolled out in April 2023.
I have no idea at all what's going on with Safari!

I started a Twitter thread to try and collect more information, so please reply there if you know what's going on in more detail.

The Chrome 2-minute twist

Assuming all of the above, the mystery remained: how did Yan's exploit fail to be prevented by browsers?

After some back-and-forth about this on Twitter Yan proposed that the answer may be this detail, tucked away on the Chrome Platform Status page for Feature: Cookies default to SameSite=Lax.

Note: Chrome will make an exception for cookies set without a SameSite attribute less than 2 minutes ago. Such cookies will also be sent with non-idempotent (e.g. POST) top-level cross-site requests despite normal SameSite=Lax cookies requiring top-level cross-site requests to have a safe (e.g. GET) HTTP method. Support for this intervention ("Lax + POST") will be removed in the future.

It looks like OkCupid were setting their authentication cookie without a SameSite attribute... which opened them up to a form-based CSRF attack but only for the 120 seconds following the cookie being set!

Building a tool to explore SameSite browser behaviour

I was finding this all very confusing, so I built a tool.

The code lives in simonw/samesite-lax-demo on GitHub, but the tool itself has two sides:

A server-side Python (Starlette) web application for setting cookies with different SameSite attributes. This is hosted on Vercel at https://samesite-lax-demo.vercel.app/
An HTML page on a different domain that links to that cookied site, provides a POST form targetting it, embeds an image from it and executes some fetch() requests against it. This is at https://simonw.github.io/samesite-lax-demo/

Hosting on two separate domains is critical for the tool to show what is going on. I chose Vercel and GitHub Pages because they are both trivial to set up to continuously deploy changes from a GitHub repository.

Using the tool in different browsers helps show exactly what is going on with regards to cross-domain cookies.

A few of the things I observed using the tool:

SameSite=Strict works as you would expect. It's particularly interesting to follow the regular <a href=...> link from the static site to the application and see how the strict cookie is NOT visible upon arrival - but becomes visible when you refresh that page.
I included a dynamically generated SVG in a <img src="/cookies.svg"> image tag, which shows the cookies (using SVG <text>) that are visible to the request. That image shows all four types of cookie when embedded on the Vercel domain, but when embedded on the GitHub pages domain it differs wildly:
- Firefox 89 shows both the SameSite=None and the missing SameSite cookies
- Chrome 92 shows just the SameSite=None cookie
- Safari 14.0 shows no cookies at all!
Chrome won't let you set a SameSite=None cookie without including the Secure attribute.
I also added some JavaScript that makes a cross-domain fetch(..., {credentials: "include"}) call against a /cookies.json endpoint. This didn't send any cookies at all until I added server-side headers access-control-allow-origin: https://simonw.github.io and access-control-allow-credentials: true. Having done that, I got the same results across the three browsers as for the <img test described above.

Safari ignoring SameSite=None looked like it was this bug: Cookies with SameSite=None or SameSite=invalid treated as Strict - it's marked as fixed but it's not clear to me if the fix has been released yet - I still saw that behaviour on my macOS 10.15.6 laptop or my iOS 14.7.1 iPhone.

Update: krinchan on Hacker News has an answer here:

The Safari "bug" is a new setting that's turned on by default: "Prevent cross-site tracking". It treats all cookies as SameSite=Lax, even cookies with SameSite=None.

Full Third-Party Cookie Blocking and More on the WebKit blog has more about this.

Most excitingly, I was able to replicate the Chrome two minute window bug using the tool! Each cookie has its value set to the timestamp when it was created, and I added code to display how many seconds ago the cookie was set. Here's an animation showing how Chrome on a form submission navigation can see the cookie that was set with SameSite missing at 114 seconds old, but that cookie is no longer visible once it passes 120 seconds.

Consider your subdomains

One last note about CSRF that you should consider: SameSite=Lax still allows form submissions from subdomains of your primary domain to carry their cookies.

This means that if you have a XSS vulnerability on one of your subdomains the security of your primary domain will be compromised.

Since it's common for subdomains to host other applications that may have their own security concerns, ditching CSRF tokens for Lax cookies may not be a wise step!

A Login CSRF attack is when a malicious forces a user to sign into an account controlled by the attacker. Why do this? Because if that user then saves sensitive information the attacker can see it.

Imagine I trick you into signing into an e-commerce account I control and saving your credit card details. I could then later sign in myself and buy things on your card!

Here's how that would work: Say the site's login form makes a POST to https://www.example.com/login with username and password as the form fields. If those credentials match, the site sets an authentication cookie.

I can set up my evil website with the following form:

<form action="https://www.example.com/login">
  <input type="hidden" name="username" value="my-username">
  <input type="hidden" name="password" value="my-password">
</form>
<script>document.forms[0].submit()</script>

I trick you into visiting my evil pge and you're now signed in to that site using an account that I control. I cross my fingers and hope you don't notice the "you are signed in as X" message in the UI.

An interesting thing about Login CSRF is that, since it involves setting a cookie but not sending a cookie, SameSite=Lax would seem to make no difference at all. You need to look to other mechanisms to protect against this attack.

But actually, you can use SameSite=Lax to prevent these. The trick is to only allow logins from users that are carrying at least one cookie which you have set in that way - since you know that those cookies could not have been sent if the user originated in a form on another site.

Another (potentially better) option: check the HTTP Origin header on the oncoming request.

Final recommendations

As an application developer, you should set all cookies with SameSite=Lax unless you have a very good reason not to. Most web frameworks do this by default now - Django shipped support for this in Django 2.1 in August 2018.

Do you still need CSRF tokens as well? I think so: I don't like the idea of users who fire up an older browser (maybe borrowing an obsolete computer) being vulnerable to this attack, and I worry about the subdomain issue described above.

And if you work for a browser vendor, please make it easier to find information on what the default behaviour is and when it was shipped!

Tags: chrome, cookies, csrf, security, samesite, starlette

Datasette 0.31

2019-11-12T06:11:57+00:00

Datasette 0.31

Released today: this version adds compatibility with Python 3.8 and breaks compatibility with Python 3.5. Since Glitch support Python 3.7.3 now I decided I could finally give up on 3.5. This means Datasette can use f-strings now, but more importantly it opens up the opportunity to start taking advantage of Starlette, which makes all kinds of interesting new ASGI-based plugins much easier to build.

Tags: glitch, projects, python, datasette, asgi, starlette

Porting Datasette to ASGI, and Turtles all the way down

2019-06-23T21:39:00+00:00

This evening I finally closed a Datasette issue that I opened more than 13 months ago: #272: Port Datasette to ASGI. A few notes on why this is such an important step for the project.

ASGI is the Asynchronous Server Gateway Interface standard. It’s been evolving steadily over the past few years under the guidance of Andrew Godwin. It’s intended as an asynchronous replacement for the venerable WSGI.

Turtles all the way down

Ten years ago at EuroDjangoCon 2009 in Prague I gave a talk entitled Django Heresies. After discussing some of the design decisions in Django that I didn’t think had aged well, I spent the last part of the talk talking about Turtles all the way down. I wrote that idea up here on my blog (see also these slides).

The key idea was that Django would be more interesting if the core Django contract - a function that takes a request and returns a response - was extended to more places in the framework. The top level site, the reusable applications, middleware and URL routing could all share that same contract. Everything could be composed from the same raw building blocks.

I’m excited about ASGI because it absolutely fits the turtles all the way down model.

The ASGI contract is an asynchronous function that takes three arguments:

async def application(scope, receive, send):
    ...

scope is a serializable dictionary providing the context for the current connection. receive is an awaitable which can be used to recieve incoming messages. send is an awaitable that can be used to send replies.

It’s a pretty low-level set of primitives (and less obvious than a simple request/response) - and that’s because ASGI is about more than just the standard HTTP request/response cycle. This contract works for HTTP, WebSockets and potentially any other protocol that needs to asynchronously send and receive data.

It’s an extremely elegant piece of protocol design, informed by Andrew’s experience with Django Channels, SOA protocols (we are co-workers at Eventbrite where we’ve both been heavily involved in Eventbrite’s SOA mechanism) and Andrew’s extensive conversations with other maintainers in the Python web community.

The ASGI protocol really is turtles all the way down - it’s a simple, well defined contract which can be composed together to implement all kinds of interesting web architectural patterns.

My asgi-cors library was my first attempt at building an ASGI turtle. The implementation is a simple Python decorator which, when applied to another ASGI callable, adds HTTP CORS headers based on the parameters you pass to the decorator. The library has zero installation dependencies (it has test dependencies on pytest and friends) and can be used on any HTTP ASGI project.

Building asgi-cors completely sold me on ASGI as the turtle pattern I had been desiring for over a decade!

Datasette plugins and ASGI

Which brings me to Datasette.

One of the most promising components of Datasette is its plugin mechanism. Based on pluggy (extracted from pytest), Datasette Plugins allow new features to be added to Datasette without needing to change the underlying code. This means new features can be built, packaged and shipped entirely independently of the core project. A list of currently available plugins can be found here.

WordPress is very solid blogging engine. Add in the plugin ecosystem around it and it can be used to build literally any CMS you can possibly imagine.

My dream for Datasette is to apply the same model: I want a strong core for publishing and exploring data that’s enhanced by plugins to solve a huge array of data analysis, visualization and API-backed problems.

Datasette has a range of plugin hooks already, but I’ve so far held back on implementing the most useful class of hooks: hooks that allow developers to add entirely new URL routes exposing completely custom functionality.

The reason I held back is that I wanted to be confident that the contract I was offering was something I would continue to support moving forward. A plugin system isn’t much good if the core implementation keeps on changing in backwards-incompatible ways.

ASGI is the exact contract I’ve been waiting for. It’s not quite ready yet, but you can follow #520: prepare_asgi plugin hook (thoughts and suggestions welcome!) to be the first to hear about this hook when it lands. I’m planning to use it to make my asgi-cors library available as a plugin, after which I’m excited to start exploring the idea of bringing authentication plugins to Datasette (and to the wider ASGI world in general).

I’m hoping that many Datasette ASGI plugins will exist in a form that allows them to be used by other ASGI applications as well.

I also plan to use ASGI to make components of Datasette itself available to other ASGI applications. If you just want a single instance of Datasette’s table view to be embedded somewhere in your URL configuration you should be able to do that by routing traffic directly to the ASGI-compatible view class.

I’m really excited about exploring the intersection of ASGI turtles-all-the-way-down and pluggy’s powerful mechanism for gluing components together. Both WSGI and Django’s reusable apps have attempted to create a reusable ecosystem in the past, to limited levels of success. Let’s see if ASGI can finally make the turtle dream come true.

Automatically playing science communication games with transfer learning and fastai

2018-10-29T03:16:33+00:00

This weekend was the 9th annual Science Hack Day San Francisco, which was also the 100th Science Hack Day held worldwide.

Natalie and I decided to combine our interests and build something fun.

I’m currently enrolled in Jeremy Howard’s Deep Learning course so I figured this was a great opportunity to try out some computer vision.

Natalie runs the SciComm Games calendar and accompanying @SciCommGames bot to promote and catalogue science communication hashtag games on Twitter.

Hashtag games? Natalie explains them here - essentially they are games run by scientists on Twitter to foster public engagement around an animal or topic by challenging people to identify if a photo is a #cougarOrNot or participate in a #TrickyBirdID or identify #CrowOrNo or many others.

Combining the two… we decided to build a bot that automatically plays these games using computer vision. So far it’s just trying #cougarOrNot - you can see the bot in action at @critter_vision.

Training data from iNaturalist

In order to build a machine learning model, you need to start out with some training data.

I’m a big fan of iNaturalist, a citizen science project that encourages users to upload photographs of wildlife (and plants) they have seen and have their observations verified by a community. Natalie and I used it to build owlsnearme.com earlier this year - the API in particular is fantastic.

iNaturalist has over 5,000 verified sightings of felines (cougars, bobcats, domestic cats and more) in the USA.

The raw data is available as a paginated JSON API. The medium sized photos are just the right size for training a neural network.

I started by grabbing 5,000 images and saving them to disk with a filename that reflected their identified species:

Bobcat_9005106.jpg
Domestic-Cat_10068710.jpg
Bobcat_15713672.jpg
Domestic-Cat_6755280.jpg
Mountain-Lion_9075705.jpg

Building a model

I’m only one week into the fast.ai course so this really isn’t particularly sophisticated yet, but it was just about good enough to power our hack.

The main technique we are learning in the course is called transfer learning, and it really is shockingly effective. Instead of training a model from scratch you start out with a pre-trained model and use some extra labelled images to train a small number of extra layers.

The initial model we are using is ResNet-34, a 34-layer neural network trained on 1,000 labelled categories in the ImageNet corpus.

In class, we learned to use this technique to get 94% accuracy against the Oxford-IIIT Pet Dataset - around 7,000 images covering 12 cat breeds and 25 dog breeds. In 2012 the researchers at Oxford were able to get 59.21% using a sophisticated model - it 2018 we can get 94% with transfer learning and just a few lines of code.

I started with an example provided in class, which loads and trains images from files on disk using a regular expression that extracts the labels from the filenames.

My full Jupyter notebook is inaturalist-cats.ipynb - the key training code is as follows:

from fastai import *
from fastai.vision import *
cat_images_path = Path('/home/jupyter/.fastai/data/inaturalist-usa-cats/images')
cat_fnames = get_image_files(cat_images_path)
cat_data = ImageDataBunch.from_name_re(
    cat_images_path,
    cat_fnames,
    r'/([^/]+)_\d+.jpg$',
    ds_tfms=get_transforms(),
    size=224
)
cat_data.normalize(imagenet_stats)
cat_learn = ConvLearner(cat_data, models.resnet34, metrics=error_rate)
cat_learn.fit_one_cycle(4)
# Save the generated model to disk
cat_learn.save("usa-inaturalist-cats")

Calling cat_learn.save("usa-inaturalist-cats") created an 84MB file on disk at /home/jupyter/.fastai/data/inaturalist-usa-cats/images/models/usa-inaturalist-cats.pth - I used scp to copy that model down to my laptop.

This model gave me a 24% error rate which is pretty terrible - others on the course have been getting error rates less than 10% for all kinds of interesting problems. My focus was to get a model deployed as an API though so I haven’t spent any additional time fine-tuning things yet.

Deploying the model as an API

The fastai library strongly encourages training against a GPU, using pytorch and PyCUDA. I’ve been using n1-highmem-8 Google Cloud Platform instance with an attached Tesla P4, then running everything in a Jupyter notebook there. This costs around $0.38 an hour - fine for a few hours of training, but way too expensive to permanently host a model.

Thankfully, while a GPU is essential for productively training models it’s not nearly as important for evaluating them against new data. pytorch can run in CPU mode for that just fine on standard hardware, and the fastai README includes instructions on installing it for a CPU using pip.

I started out by ensuring I could execute my generated model on my own laptop (since pytorch doesn’t yet work with the GPU built into the Macbook Pro). Once I had that working, I used the resulting code to write a tiny Starlette-powered API server. The code for that can be found in in cougar.py.

fastai is under very heavy development and the latest version doesn’t quite have a clean way of loading a model from disk without also including the initial training images, so I had to hack around quite a bit to get this working using clues from the fastai forums. I expect this to get much easier over the next few weeks as the library continues to evolve based on feedback from the current course.

To deploy the API I wrote a Dockerfile and shipped it to Zeit Now. Now remains my go-to choice for this kind of project, though unfortunately their new (and brilliant) v2 platform imposes a 100MB image size limit - not nearly enough when the model file itself weights in at 83 MB. Thankfully it’s still possible to specify their v1 cloud which is more forgiving for larger applications.

Here’s the result: an API which can accept either the URL to an image or an uploaded image file: https://cougar-or-not.now.sh/ - try it out with a cougar and a bobcat.

The Twitter Bot

Natalie built the Twitter bot. It runs as a scheduled task on Heroku and works by checking for new #cougarOrNot tweets from Dr. Michelle LaRue, extracting any images, passing them to my API and replying with a tweet that summarizes the results. Take a look at its recent replies to get a feel for how well it is doing.

Amusingly, Dr. LaRue frequently tweets memes to promote upcoming competitions and marks them with the same hashtag. The bot appears to think that most of the memes are bobcats! I should definitely spend some time tuning that model.

Science Hack Day was great fun. A big thanks to the organizing team, and congrats to all of the other participants. I’m really looking forward to the next one.

Plus… we won a medal!

Enjoyed #scienceHackday this weekend, made & launched a cool machine learning hack to process images & work out if they have a cougar in them or not! #CougarOrNot @critter_vision

... we won a medal!

Bot code: https://t.co/W2jZcGCnFr
Machine learning API: https://t.co/swNiKlcTp0 pic.twitter.com/dcdIhNZy63
— Natbat (@Natbat) October 29, 2018

Tags: computer-vision, machine-learning, natalie-downe, inaturalist, fastai, transferlearning, jeremy-howard, starlette