Simon Willison's Weblog: async

asyncinject 0.7

2026-06-11T06:28:09+00:00

I built this utility library to support an asyncio dependency injection pattern a few years ago. I was using it with Datasette and Claude Fable 5 spotted some bugs in the dependency which it then fixed for me. It's a very proactive model!

Tags: async, projects, python, claude-mythos

llm-all-models-async 0.1

2026-03-31T20:52:02+00:00

Release: llm-all-models-async 0.1

LLM plugins can define new models in both sync and async varieties. The async variants are most common for API-backed models - sync variants tend to be things that run the model directly within the plugin.

My llm-mrchatterbox plugin is sync only. I wanted to try it out with various Datasette LLM features (specifically datasette-enrichments-llm) but Datasette can only use async models.

So... I had Claude spin up this plugin that turns sync models into async models using a thread pool. This ended up needing an extra plugin hook mechanism in LLM itself, which I shipped just now in LLM 0.30.

Tags: async, python, llm

Quoting Kumar Aditya

2025-09-11T03:07:16+00:00

In Python 3.14, I have implemented several changes to fix thread safety of asyncio and enable it to scale effectively on the free-threaded build of CPython. It is now implemented using lock-free data structures and per-thread state, allowing for highly efficient task management and execution across multiple threads. In the general case of multiple event loops running in parallel, there is no lock contention and performance scales linearly with the number of threads. [...]

For a deeper dive into the implementation, check out the internal docs for asyncio.

— Kumar Aditya, Scaling asyncio on Free-Threaded Python

Tags: async, gil, python, scaling, threads

Textual v4.0.0: The Streaming Release

2025-07-22T00:32:53+00:00

Textual v4.0.0: The Streaming Release

Will McGugan may no longer be running a commercial company around Textual, but that hasn't stopped his progress on the open source project.

He recently released v4 of his Python framework for building TUI command-line apps, and the signature feature is streaming Markdown support - super relevant in our current age of LLMs, most of which default to outputting a stream of Markdown via their APIs.

I took an example from one of his tests, spliced in my async LLM Python library and got some help from o3 to turn it into a streaming script for talking to models, which can be run like this:

uv run http://tools.simonwillison.net/python/streaming_textual_markdown.py \
'Markdown headers and tables comparing pelicans and wolves' \
-m gpt-4.1-mini

Tags: async, python, markdown, ai, will-mcgugan, generative-ai, llms, textual, llm, uv

files-to-prompt 0.5

2025-02-14T04:14:21+00:00

files-to-prompt 0.5

My files-to-prompt tool (originally built using Claude 3 Opus back in April) had been accumulating a bunch of issues and PRs - I finally got around to spending some time with it and pushed a fresh release:

New -n/--line-numbers flag for including line numbers in the output. Thanks, Dan Clayton. #38

Fix for utf-8 handling on Windows. Thanks, David Jarman. #36

--ignore patterns are now matched against directory names as well as file names, unless you pass the new --ignore-files-only flag. Thanks, Nick Powell. #30

I use this tool myself on an almost daily basis - it's fantastic for quickly answering questions about code. Recently I've been plugging it into Gemini 2.0 with its 2 million token context length, running recipes like this one:

git clone https://github.com/bytecodealliance/componentize-py
cd componentize-py
files-to-prompt . -c | llm -m gemini-2.0-pro-exp-02-05 \
  -s 'How does this work? Does it include a python compiler or AST trick of some sort?'

I ran that question against the bytecodealliance/componentize-py repo - which provides a tool for turning Python code into compiled WASM - and got this really useful answer.

Here's another example. I decided to have o3-mini review how Datasette handles concurrent SQLite connections from async Python code - so I ran this:

git clone https://github.com/simonw/datasette
cd datasette/datasette
files-to-prompt database.py utils/__init__.py -c | \
  llm -m o3-mini -o reasoning_effort high \
  -s 'Output in markdown a detailed analysis of how this code handles the challenge of running SQLite queries from a Python asyncio application. Explain how it works in the first section, then explore the pros and cons of this design. In a final section propose alternative mechanisms that might work better.'

Here's the result. It did an extremely good job of explaining how my code works - despite being fed just the Python and none of the other documentation. Then it made some solid recommendations for potential alternatives.

I added a couple of follow-up questions (using llm -c) which resulted in a full working prototype of an alternative threadpool mechanism, plus some benchmarks.

One final example: I decided to see if there were any undocumented features in Litestream, so I checked out the repo and ran a prompt against just the .go files in that project:

git clone https://github.com/benbjohnson/litestream
cd litestream
files-to-prompt . -e go -c | llm -m o3-mini \
  -s 'Write extensive user documentation for this project in markdown'

Once again, o3-mini provided a really impressively detailed set of unofficial documentation derived purely from reading the source.

Tags: async, projects, python, sqlite, ai, datasette, webassembly, litestream, generative-ai, llms, ai-assisted-programming, llm, gemini, llm-reasoning, files-to-prompt

In search of a faster SQLite

2024-12-15T18:09:17+00:00

In search of a faster SQLite

Turso developer Avinash Sajjanshetty (previously) shares notes on the April 2024 paper Serverless Runtime / Database Co-Design With Asynchronous I/O by Turso founder and CTO Pekka Enberg, Jon Crowcroft, Sasu Tarkoma and Ashwin Rao.

The theme of the paper is rearchitecting SQLite for asynchronous I/O, and Avinash describes it as "the foundational paper behind Limbo, the SQLite rewrite in Rust."

From the paper abstract:

We propose rearchitecting SQLite to provide asynchronous byte-code instructions for I/O to avoid blocking in the library and de-coupling the query and storage engines to facilitate database and serverless runtime co-design. Our preliminary evaluation shows up to a 100x reduction in tail latency, suggesting that our approach is conducive to runtime/database co-design for low latency.

Via lobste.rs

Tags: async, sqlite, rust, limbo

googleapis/python-genai

2024-12-12T16:21:46+00:00

googleapis/python-genai

Google released this brand new Python library for accessing their generative AI models yesterday, offering an alternative to their existing generative-ai-python library.

The API design looks very solid to me, and it includes both sync and async implementations. Here's an async streaming response:

async for response in client.aio.models.generate_content_stream(
    model='gemini-2.0-flash-exp',
    contents='Tell me a story in 300 words.'
):
    print(response.text)

It also includes Pydantic-based output schema support and some nice syntactic sugar for defining tools using Python functions.

Tags: async, google, python, ai, generative-ai, llms, gemini, llm-tool-use, pydantic

llm-gemini 0.4

2024-11-18T07:37:17+00:00

llm-gemini 0.4

New release of my llm-gemini plugin, adding support for asynchronous models (see LLM 0.18), plus the new gemini-exp-1114 model (currently at the top of the Chatbot Arena) and a -o json_object 1 option to force JSON output.

I also released llm-claude-3 0.9 which adds asynchronous support for the Claude family of models.

Tags: async, google, plugins, projects, python, ai, generative-ai, llms, llm, anthropic, claude, gemini

LLM 0.18

2024-11-17T20:40:27+00:00

LLM 0.18

New release of LLM. The big new feature is asynchronous model support - you can now use supported models in async Python code like this:

import llm

model = llm.get_async_model("gpt-4o")
async for chunk in model.prompt(
    "Five surprising names for a pet pelican"
):
    print(chunk, end="", flush=True)

Also new in this release: support for sending audio attachments to OpenAI's gpt-4o-audio-preview model.

Tags: async, projects, python, ai, generative-ai, llms, llm

Free Threaded Python With Asyncio

2024-10-09T20:38:19+00:00

Free Threaded Python With Asyncio

Jamie Chang expanded my free-threaded Python experiment from a few months ago to explore the interaction between Python's asyncio and the new GIL-free build of Python 3.13.

The results look really promising. Jamie says:

Generally when it comes to Asyncio, the discussion around it is always about the performance or lack there of. Whilst peroformance is certain important, the ability to reason about concurrency is the biggest benefit. [...]

Depending on your familiarity with AsyncIO, it might actually be the simplest way to start a thread.

This code for running a Python function in a thread really is very pleasant to look at:

result = await asyncio.to_thread(some_function, *args, **kwargs)

Jamie also demonstrates asyncio.TaskGroup, which makes it easy to execute a whole bunch of threads and wait for them all to finish:

async with TaskGroup() as tg:
    for _ in range(args.tasks):
        tg.create_task(to_thread(cpu_bound_task, args.size))

Tags: async, gil, python

aiolimiter

2024-02-20T01:15:27+00:00

aiolimiter

I found myself wanting an asyncio rate limiter for Python today—so I could send POSTs to an API endpoint no more than once every 10 seconds. This library worked out really well—it has a very neat design and lets you set up rate limits for things like “no more than 50 items every 10 seconds”, implemented using the leaky bucket algorithm.

Tags: async, python, rate-limiting

Writing a chat application in Django 4.2 using async StreamingHttpResponse, Server-Sent Events and PostgreSQL LISTEN/NOTIFY

2023-05-19T15:42:03+00:00

Writing a chat application in Django 4.2 using async StreamingHttpResponse, Server-Sent Events and PostgreSQL LISTEN/NOTIFY

Excellent tutorial by Víðir Valberg Guðmundsson on implementing chat with server-sent events using the newly async-capable StreamingHttpResponse from Django 4.2x.

He uses PostgreSQL’a LISTEN/NOTIFY mechanism which can be used asynchronously in psycopg3—at the cost of a separate connection per user of the chat.

The article also covers how to use the Last-Event-ID header to implement reconnections in server-sent events, transmitting any events that may have been missed during the time that the connection was dropped.

Via lobste.rs

Tags: async, django, postgresql, asgi

Django 4.2 released

2023-04-03T14:14:39+00:00

Django 4.2 released

“This version has been designated as a long-term support (LTS) release, which means that security and data loss fixes will be applied for at least the next three years.” Some neat new async features, including improvements to async streaming responses.

Tags: async, django

Why Rust’s postfix await syntax is good

2022-05-15T14:27:53+00:00

Why Rust’s postfix await syntax is good

C J Silverio explains postfix await in Rust—where you can write a line like this, with the ? causing any errors to be caught and turned into an error return from your function:

let count = fetch_all_animals().await?.filter_for_hedgehogs().len();

Via @ceejbot

Tags: async, rust

Weeknotes: Parallel SQL queries for Datasette, plus some middleware tricks

2022-04-27T19:01:30+00:00

A promising new performance optimization for Datasette, plus new datasette-gzip and datasette-total-page-time plugins.

Parallel SQL queries in Datasette

From the start of the project, Datasette has been built on top of Python's asyncio capabilities - mainly to benefit things like streaming enormous CSV files.

This week I started experimenting with a new way to take advantage of them, by exploring the potential to run multiple SQL queries in parallel.

Consider this Datasette table page:

That page has to execute quite a few SQL queries:

A select count(*) ... to populate the 3,283 rows heading at the top
Queries against each column to decide what the "suggested facets" should be (details here)
For each of the selected facets (in this case repos and committer) a select name, count(*) from ... group by name order by count(*) desc query
The actual select * from ... limit 101 query used to display the actual table

It ends up executing more than 30 queries! Which may seem like a lot, but Many Small Queries Are Efficient In SQLite.

One thing that's interesting about the above list of queries though is that they don't actually have any dependencies on each other. There's no reason not to run all of them in parallel - later queries don't depend on the results from earlier queries.

I've been exploring a fancy way of executing parallel code using pytest-style dependency injection in my asyncinject library. But I decided to do a quick prototype to see what this would look like using asyncio.gather().

It turns out that simpler approach worked surprisingly well!

You can follow my research in this issue, but the short version is that as-of a few days ago the Datasette main branch runs many of the above queries in parallel.

This trace (using the datasette-pretty-traces plugin) illustrates my initial results:

As you can see, the grey lines for many of those SQL queries are now overlapping.

You can add the undocumented ?_noparallel=1 query string parameter to disable parallel execution to compare the difference:

One thing that gives me pause: for this particular Datasette deployment (on the cheapest available Cloud Run instance) the overall performance difference between the two is very small.

I need to dig into this deeper: on my laptop I feel like I'm seeing slightly better results, but definitely not conclusively. It may be that multiple cores are not being used effectively here.

Datasette runs SQL queries in a pool of threads. You might expect Python's infamous GIL (Global Interpreter Lock) to prevent these from executing across multiple cores - but I checked, and the GIL is released in Python's C code the moment control transfers to SQLite. And since SQLite can happily run multiple threads, my hunch is that this means parallel queries should be able to take advantage of multiple cores. Theoretically at least!

I haven't yet figured out how to prove this though, and I'm not currently convinced that parallel queries are providing any overall benefit at all. If you have any ideas I'd love to hear them - I have a research issue open, comments welcome!

Update 28th April 2022: Research continues, but it looks like there's little performance benefit from this. Current leading theory is that this is because of the GIL - while the SQLite C code releases the GIL, much of the activity involved in things like assembling Row objects returned by a query still uses Python - so parallel queries still end up mostly blocked on a single core. Follow the issue for more details. I started a discussion on the SQLite Forum which has some interesting clues in it as well.

Further update: It's definitely the GIL. I know because I tried running it against Sam Gross's nogil Python fork and the parallel version soundly beat the non-parallel version! Details in this comment.

datasette-gzip

I've been putting off investigating gzip support for Datasette for a long time, because it's easy to add as a separate layer. If you run Datasette behind Cloudflare or an Apache or Nginx proxy configuring gzip can happen there, with very little effort and fantastic performance.

Then I noticed that my Global Power Plants demo returned an HTML table page that weighed in at 420KB... but gzipped was just 16.61KB. Turns out HTML tables have a ton of repeated markup and compress REALLY well!

More importantly: Google Cloud Run doesn't gzip for you. So all of my Datasette instances that were running on Cloud Run without also using Cloudflare were really suffering.

So this morning I released datasette-gzip, a plugin that gzips content if the browser sends an Accept-Encoding: gzip header.

The plugin is an incredibly thin wrapper around the thorougly proven-in-production GZipMiddleware. So thin that this is the full implementation:

from datasette import hookimpl
from starlette.middleware.gzip import GZipMiddleware

@hookimpl(trylast=True)
def asgi_wrapper(datasette):
    return GZipMiddleware

This kind of thing is exactly why I ported Datasette to ASGI back in 2019 - and why I continue to think that the burgeoning ASGI ecosystem is the most under-rated piece of today's Python web development environment.

The plugin's tests are a lot more interesting.

That @hookimpl(trylast=True) line is there to ensure that this plugin runs last, after ever other plugin has executed.

This is necessary because there are existing ASGI plugins for Datasette (such as the new datasette-total-page-time) which modify the generated request.

If the gzip plugin runs before they do, they'll get back a blob of gzipped data rather than the HTML that they were expecting. This is likely to break them.

I wanted to prove to myself that trylast=True would prevent these errors - so I ended up writing a test that demonstrated that the plugin registered with trylast=True was compatible with a transforming content plugin (in the test it just converts everything to uppercase) whereas tryfirst=True would instead result in an error.

Thankfully I have an older TIL on Registering temporary pluggy plugins inside tests that I could lean on to help figure out how to do this.

The plugin is now running on my latest-with-plugins demo instance. Since that instance loads dozens of different plugins it ends up serving a bunch of extra JavaScript and CSS, all of which benefits from gzip:

datasette-total-page-time

To help understand the performance improvements introduced by parallel SQL queries I decided I wanted the Datasette footer to be able to show how long it took for the entire page to load.

This is a tricky thing to do: how do you measure the total time for a page and then include it on that page if the page itself hasn't finished loading when you render that template?

I came up with a pretty devious middleware trick to solve this, released as the datasette-total-page-time plugin.

The trick is to start a timer when the page load begins, and then end that timer at the very last possible moment as the page is being served back to the user.

Then, inject the following HTML directly after the closing </html> tag (which works fine, even though it's technically invalid):

<script>
let footer = document.querySelector("footer");
if (footer) {
    let ms = 37.224;
    let s = ` &middot; Page took ${ms.toFixed(3)}ms`;
    footer.innerHTML += s;
}
</script>

This adds the timing information to the page's <footer> element, if one exists.

You can see this running on this latest-with-plugins page.

Releases this week

datasette-gzip: 0.1 - 2022-04-27
Add gzip compression to Datasette
datasette-total-page-time: 0.1 - 2022-04-26
Add a note to the Datasette footer measuring the total page load time
asyncinject: 0.5 - (7 releases total) - 2022-04-22
Run async workflows using pytest-fixtures-style dependency injection
django-sql-dashboard: 1.1 - (35 releases total) - 2022-04-20
Django app for building dashboards using raw SQL queries
shot-scraper: 0.13 - (14 releases total) - 2022-04-18
Tools for taking automated screenshots of websites

TIL this week

Tags: async, gzip, python, datasette, weeknotes

PHP 8.1 release notes

2021-11-25T19:53:30+00:00

PHP 8.1 release notes

PHP is gaining “Fibers” for lightweight cooperative concurrency—very similar to Python asyncio. Interestingly you don’t need to use separate syntax like “await fn()” to call them—calls to non-blocking functions are visually indistinguishable from calls to blocking functions. Considering how much additional library complexity has emerged in Python world from having these two different colours of functions it’s noteworthy that PHP has chosen to go in a different direction here.

Via Hacker News

Tags: async, php

Finding and reporting an asyncio bug in Python 3.10

2021-10-09T04:46:30+00:00

I found a bug in Python 3.10 today! Some notes on how I found it and my process for handling it once I figured out what was going on.

Testing Datasette against Python 3.10

I finally got around to attempting to upgrade Datasette to the just-released Python 3.10 this morning. I started by adding "3.10" to the matrix of Python versions that Datasette is tested against by GitHub Actions:

strategy:
  matrix:
    python-version: ["3.6", "3.7", "3.8", "3.9", "3.10"]

That line previously looked like this:

python-version: [3.6, 3.7, 3.8, 3.9]

I switched to using quoted strings here based on a tip from Jeff Triplett, who pointed out that 3.10 is actually a floating point number in YAML syntax which will be treated as referring to Python 3.1 instead!

This one-line change in a pull request was all it took to run the tests against the new version of Python... and they failed, spectacularly.

Annoyingly the root error wasn't displayed directly in the tests, because it was triggered and then caught by Datasette's own error handling and turned into a 500 HTTP error. So, I needed to run Python 3.10 locally and it the debugger.

Running Python 3.10 locally

I occasionally use pyenv to manage multiple Python versions on my machine. I have this installed through Homebrew.

I had to run brew upgrade pyenv first, because my installed version didn't know about Python 3.10. Then I ran this:

pyenv install 3.10.0

Which gave me a Python 3.10 installation on my Mac in ~/.pyenv/versions/3.10.0/bin/python.

Finally, I created a new virtual environment using this version of Python.

~/.pyenv/versions/3.10.0/bin/python \
  -m venv /tmp/py310
# Now activate that virtual environment:
source /tmp/py310/bin/activate

I put this in my /tmp directory so I don't have to remember to clean it up later.

Having done this, I installed my Datasette test dependencies into the new environment and used pytest to run my tests:

% cd ~/Dropbox/Development/datasette
% pip install -e '.[test]'

Running pytest -x --pdb stopped at the first failing test and dropped me into a debugger where I could finally access the full traceback, which looked something like this:

  File "datasette/datasette/views/base.py", line 122, in dispatch_request
    await self.ds.refresh_schemas()
  File "datasette/datasette/app.py", line 344, in refresh_schemas
    await self._refresh_schemas()
  File "datasette/datasette/app.py", line 349, in _refresh_schemas
    await init_internal_db(internal_db)
  File "datasette/datasette/utils/internal_db.py", line 5, in init_internal_db
    await db.execute_write(
  File "datasette/datasette/database.py", line 102, in execute_write
    return await self.execute_write_fn(_inner, block=block)
  File "datasette/datasette/database.py", line 113, in execute_write_fn
    reply_queue = janus.Queue()
  File "janus/__init__.py", line 39, in __init__
    self._async_not_empty = asyncio.Condition(self._async_mutex)
  File "lib/python3.10/asyncio/locks.py", line 234, in __init__
    raise ValueError("loop argument must agree with lock")
ValueError: loop argument must agree with lock

So, something is going long with asyncio.Condition(), which is being called by janus.Queue().

Some background on Janus

The Janus library referenced here describes itself as a "Thread-safe asyncio-aware queue for Python".

I use it in Datasette for the write queue - a mechanism where writes can be safely made to a SQLite database by first queueing them up in memory.

Janus provides a mechanism that lets asyncio event tasks send and receive messages to Python threads, and vice-versa, via a queue class that is modelled on Python's own standard library queues. It's really neat.

Tracking the investigation in issue comments

Any time I'm investigating a bug I make sure that there's an associated GitHub issue or pull request, to give me somewhere to build up detailed notes of my investigation.

I used my PR #1481 for this. Now that I had an error message - "loop argument must agree with lock" - I did some initial Google searches and found to my surprise that it had very few existing mentions - my first clue that this might be related to the new Python 3.10 release itself.

I tracked down the relevant source code in both Python and the Janus library based on the traceback and linked to them from the issue comments. I pasted in copies of the relevant code too, since GitHub only magically embeds code from the same repository, not code from other repositories. You can see those comments here.

I always like to include copies of the code most likely to be the cause of a bug in an issue comment, to save myself from having to dig around for it again later on and to act as a historical record once the bug has been fixed.

I also started an issue in the Janus repo, with the title Error with Python 3.10: "ValueError: loop argument must agree with lock". I linked this to my own issue and posted relevant research in both places.

The two lines of code in Janus that caused the bug where these ones (note my added comment):

self._async_mutex = asyncio.Lock()
# "loop argument must agree with lock" exception is raised here:
self._async_not_empty = asyncio.Condition(self._async_mutex)

This is where I began to suspect a Python bug. The above code can be simplified to this:

asyncio.Condition(asyncio.Lock())

This is a documented way of using conditions in Python: you can instantiate conditions with an optional lock, which allows that lock to be shared by more than one condition.

So I tried that in the Python 3.10 interpreter... and it didn't throw the error:

% ~/.pyenv/versions/3.10.0/bin/python
Python 3.10.0 (default, Oct  7 2021, 13:45:58) [Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> print(asyncio.Condition(asyncio.Lock()))
<asyncio.locks.Condition object at 0x1062521d0 [unlocked]>

From browsing the source code I could see that there was something going on here involving the event loop - so on a hunch I tried running that code in an event loop instead, like so:

>>> async def example():
...     print(asyncio.Condition(asyncio.Lock()))
... 
>>> asyncio.run(example())
Traceback (most recent call last):
  ...
  File "<stdin>", line 2, in example
  File "/Users/simon/.pyenv/versions/3.10.0/lib/python3.10/asyncio/locks.py", line 234, in __init__
    raise ValueError("loop argument must agree with lock")
ValueError: loop argument must agree with lock

There's the exception!

So it looked like this might be a genuine Python 3.10 bug. After all, I knew that this code worked in prior versions of Python (since Janus and Datasette worked fine there). In fact I could confirm it like so:

~ % python3.9
Python 3.9.7 (default, Sep  3 2021, 12:45:31) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> async def example():
...     print(asyncio.Condition(asyncio.Lock()))
... 
>>> asyncio.run(example())
<asyncio.locks.Condition object at 0x1049aab50 [unlocked]>

Reporting the bug to the Python issue tracker

Python bugs are tracked on bugs.python.org - thankfully they support sign in with GitHub (and Google and even OpenID) so I signed in with my GitHub account to report the bug.

I filed Issue 45416 entitled:

"loop argument must agree with lock" instantiating asyncio.Condition

And included a brief description with my minimal Python script that demonstrated the bug.

Then I added a comment pointing back to one of the related GitHub issues where I had been figuring this out, and another comment that linked to the most relevant test in Python's own test suite, which I found using GitHub search for the term Condition.

Any time I file an issue against a project - my own or someone else's - I always like to do a bit of extra research and link to both the likely code at fault and the likely test.

As a maintainer, this saves me a bunch of time - if the issue links straight to the code and tests I don't have to spend any time digging around looking for them.

My hope is that this saves time for the people I am reporting the bug to as well, which increases the chance that they'll dig into it and help me find a fix!

Then I tweeted about it. I have a bunch of followers with relevant experience, so I thought it was time to get a few more eyes on the problem.

I think I /may/ have found a bug in Python 3.10, which is exciting!

asyncio.Condition(asyncio.Lock()) raises a ValueError error if executed while an event loop is running

Means Datasette doesn't work on 3.10 (yet)https://t.co/ofd9MmzuJS pic.twitter.com/ICQz8lf39M
- Simon Willison (@simonw) October 8, 2021

I included a screenshot of my steps-to-reproduce script to save anyone who might be casually interested from having to click through to the issue itself.

Łukasz Langa to the rescue

Łukasz is the "CPython Developer in Residence", which means he is paid by the Python Software Foundation to work on Python full-time.

Less than an hour after I filed the issue he posted a comment that both confirmed the bug and provided two short-term workarounds for it!

I really liked his first workaround, which looks like this:

>>> l = asyncio.Lock()
>>> getattr(l, '_get_loop', lambda: None)()
<_UnixSelectorEventLoop running=True closed=False debug=False>

It works by calling the lock._get_loop() method, but only if it exists - so even though that's an undocumented internal method this should be safe even when run against future Python versions that might remove the method.

You can see that method here - it safely populates the self._loop property, which helps work around the bug.

Submitting a PR to Janus

The best place to apply this workaround is to Janus - so I submitted a PR there which adds the workaround and updates their CI configuration to test against Python 3.10: Janus PR #359. The GitHub Actions tests for that PR are now passing against Python 3.10.

Their README links to a gitter chat so I dropped a link to my PR in there too. Hopefully that can be merged soon!

What I should have done

I'm feeling pretty good about this situation - the bug is set to be fixed, we have a great workaround thanks to Łukasz and I'm optimistic that the upstream library from Datasette will land the fix soon.

If only I'd done this a few weeks earlier!

The Python 3.10 release schedule shipped the first alpha a year ago, in October 2020. It totalled 7 alphas, 4 betas and 2 release candidates before the final 3.10 release on Monday 4th October 2021.

If I'd taken the time to get Datasette's test suite running against Python 3.10 for any of those pre-releases, I could have helped spot this bug before it landed in the final release!

So lesson learned: Python has alphas and betas and RCs for a reason. For future Pythons I'm going to pay attention to testing against these for my projects before the final release lands.

Tags: async, debugging, python, weeknotes, github-actions

Introducing Partytown 🎉: Run Third-Party Scripts From a Web Worker

2021-09-23T18:29:14+00:00

Introducing Partytown 🎉: Run Third-Party Scripts From a Web Worker

This is just spectacularly clever. Partytown is a 6KB JavaScript library that helps you move gnarly poorly performing third-party scripts out of your main page and into a web worker, so they won’t destroy your page performance. The really clever bit is in how it provides sandboxed access to the page DOM: it uses a devious trick where a proxy object provides getters and setters which then make blocking API calls to a separate service worker, using the mostly-forgotten xhr.open(..., false) parameter that turns off the async default for an XMLHttpRequest call.

Via @Rich_Harris

Tags: async, javascript, webworkers, service-workers

Quoting Using async and await in Flask 2.0

2021-05-12T17:59:51+00:00

Async functions require an event loop to run. Flask, as a WSGI application, uses one worker to handle one request/response cycle. When a request comes in to an async view, Flask will start an event loop in a thread, run the view function there, then return the result.

Each request still ties up one worker, even for async views. The upside is that you can run async code within a view, for example to make multiple concurrent database queries, HTTP requests to an external API, etc. However, the number of requests your application can handle at one time will remain the same.

— Using async and await in Flask 2.0

Tags: async, flask, python, wsgi

New Major Versions Released! Flask 2.0, Werkzeug 2.0, Jinja 3.0, Click 8.0, ItsDangerous 2.0, and MarkupSafe 2.0

2021-05-12T17:37:14+00:00

New Major Versions Released! Flask 2.0, Werkzeug 2.0, Jinja 3.0, Click 8.0, ItsDangerous 2.0, and MarkupSafe 2.0

Huge set of releases from the Pallets team. Python 3.6+ required and comprehensive type annotations. Flask now supports async views, Jinja async templates (used extensively by Datasette) “no longer requires patching”, Click has a bunch of new code around shell tab completion, ItsDangerous supports key rotation and so much more.

Tags: async, flask, jinja, python

unasync

2021-02-27T22:20:08+00:00

unasync

Today I started wondering out loud if one could write code that takes an asyncio Python library and transforms it into the synchronous equivalent by using some regular expressions to strip out the “await ...” keywords and suchlike. Turns out that can indeed work, and Ratan Kulshreshtha built it! unasync uses the standard library tokenize module to run some transformations against an async library and spit out the sync version automatically. I’m now considering using this for sqlite-utils.

Via @simonw

Tags: async, python

datasette-ripgrep: deploy a regular expression search engine for your source code

2020-11-28T06:51:06+00:00

This week I built datasette-ripgrep - a web application for running regular expression searches against source code, built on top of the amazing ripgrep command-line tool.

datasette-ripgrep demo

I've deployed a demo version of the application here:

ripgrep.datasette.io/-/ripgrep?pattern=pytest

The demo runs searches against the source code of every one of my GitHub repositories that start with datasette - 61 repos right now - so it should include all of my Datasette plugins plus the core Datasette repository itself.

Since it's running on top of ripgrep, it supports regular expressions. This is absurdly useful. Some examples:

Every usage of the .plugin_config( method: plugin_config\(
Everywhere I use async with httpx.AsyncClient (usually in tests): async with.*AsyncClient
All places where I use a Jinja | filter inside a variable: \{\{.*\|.*\}\}

I usually run ripgrep as rg on the command-line, or use it within Visual Studio Code (fun fact: the reason VS Code's "Find in Files" is so good is it's running ripgrep under the hood).

So why have it as a web application? Because this means I can link to it, bookmark it and use it on my phone.

Why build this?

There are plenty of great existing code search tools out there already: I've heard great things about livegrep, and a quick Google search shows a bunch of other options.

Aside from being a fun project, datasette-ripgrep has one key advantage: it gets to benefit from Datasette's publishing mechanism, which means it's really easy to deploy.

That ripgrep.datasette.io demo is deployed by checking out the source code to be searched into a all directory and then using the following command:

datasette publish cloudrun \
    --metadata metadata.json \
    --static all:all \
    --install=datasette-ripgrep \
    --service datasette-ripgrep \
    --apt-get-install ripgrep

all is a folder containing the source code to be searched. metadata.json contains this:

{
    "plugins": {
        "datasette-ripgrep": {
            "path": "/app/all",
            "time_limit": 3.0
        }
    }
}

That's all there is to it! The result is a deployed code search engine, running on Google Cloud Run.

(If you want to try this yourself you'll need to be using the just-released Datasette 0.52.)

The GitHub Action workflow that deploys the demo also uses my github-to-sqlite tool to fetch my repos and then shallow-clones the ones that begin with datasette.

If you have your own Google Cloud Run credentials, you can run your own copy of that workflow against your own repositories.

A different kind of Datasette plugin

Datasette is a tool for publishing SQLite databases, so most Datasette plugins integrate with SQLite in some way.

datasette-ripgrep is different: it makes no use of SQLite at all, but instead takes advantage of Datasette's URL routing, datasette publish deployments and permissions system.

The plugin implementation is currently 134 lines of code, excluding tests and templates.

While the plugin doesn't use SQLite, it does share a common philosophy with Datasette: the plugin bundles the source code that it is going to search as part of the deployed application, in a similar way to how Datasette usually bundles one or more SQLite database files.

As such, it's extremely inexpensive to run and can be deployed to serverless hosting. If you need to scale it, you can run more copies.

This does mean that the application needs to be re-deployed to pick up changes to the searchable code. I'll probably set my demo to do this on a daily basis.

Controlling processes from asyncio

The trickiest part of the implementation was figuring out how to use Python's asyncio.create_subprocess_exec() method to safely run the rg process in response to incoming requests.

I don't want expensive searches to tie up the server, so I implemented two limits here. The first is a time limit: by default, searches have a second to run after which the rg process will be terminated and only results recieved so far will be returned. This is achieved using the asyncio.wait_for() function.

I also implemented a limit on the number of matching lines that can be returned, defaulting to 2,000. Any more than that and the process is terminated early.

Both of these limits can be customized using plugin settings (documented in the README). You can see how they are implemented in the async def run_ripgrep(pattern, path, time_limit=1.0, max_lines=2000) function.

Highlighted linkable line numbers

The other fun implementation detail is the way the source code listings are displayed. I'm using CSS to display the line numbers in a way that makes them visible without them breaking copy-and-paste (inspired by this article by Sylvain Durand).

code:before {
    content: attr(data-line);
    display: inline-block;
    width: 3.5ch;
    -webkit-user-select: none;
    color: #666;
}

The HTML looks like this:

<pre><code id="L1" data-line="1">from setuptools import setup</code>
<code id="L2" data-line="2">import os</code>
<code id="L3" data-line="3">&nbsp;</code>
<code id="L4" data-line="4">VERSION = &#34;0.1&#34;</code>
...

I wanted to imitate GitHub's handling of line links, where adding #L23 to the URL both jumps to that line and causes the line to be highlighted. Here's a demo of that - I use the following JavaScript to update the contents of a <style id="highlightStyle"></style> element in the document head any time the URL fragment changes:

<script>
var highlightStyle = document.getElementById('highlightStyle');
function highlightLineFromFragment() {
    if (/^#L\d+$/.exec(location.hash)) {
        highlightStyle.innerText = `${location.hash} { background-color: yellow; }`;
    }
}
highlightLineFromFragment();
window.addEventListener("hashchange", highlightLineFromFragment);
</script>

It's the simplest way I could think of to achieve this effect.

Update 28th November 2020: Louis Lévêque on Twitter suggested using the CSS :target selector instead, which is indeed MUCH simpler - I deleted the above JavaScript and replaced it with this CSS:

:target {
    background-color: #FFFF99;
}

Next steps for this project

I'm pleased to have got datasette-ripgrep to a workable state, and I'm looking forward to using it to answer questions about the growing Datasette ecosystem. I don't know how much more time I'll invest in this - if it proves useful then I may well expand it.

I do think there's something really interesting about being able to spin up this kind of code search engine on demand using datasette publish. It feels like a very useful trick to have access to.

Better URLs for my TILs

My other project this week was an upgrade to til.simonwillison.net: I finally spent the time to design nicer URLs for the site.

Before:

til.simonwillison.net/til/til/javascript_manipulating-query-params.md

After:

til.simonwillison.net/javascript/manipulating-query-params

The implementation for this takes advantage of a feature I sneaked into Datasette 0.49: Path parameters for custom page templates. I can create a template file called pages/{topic}/{slug}.html and Datasette use that template to handle 404 errors that match that pattern.

Here's the new pages/{topic}/{slug}.html template for my TIL site. It uses the sql() template function from the datasette-template-sql plugin to retrieve and render the matching TIL, or raises a 404 if no TIL can be found.

I also needed to setup redirects from the old pages to the new ones. I wrote a TIL on edirects for Datasette explaining how I did that.

TIL this week

Redirects for Datasette

Releases this week

datasette-ripgrep 0.2 - 2020-11-27
datasette-ripgrep 0.1 - 2020-11-26
datasette-atom 0.8.1 - 2020-11-25
datasette-ripgrep 0.1a1 - 2020-11-25
datasette-ripgrep 0.1a0 - 2020-11-25
datasette-graphql 1.2.1 - 2020-11-24

Tags: async, css, projects, python, regular-expressions, datasette, weeknotes, cloudrun, ripgrep, baked-data

Quoting Carlton Gibson

2020-09-27T15:09:26+00:00

Inevitably we got round to talking about async.

As much of an unneeded complication as it is for so many day-to-day use-cases, it’s important for Python because, if and when you do need the high throughput handling of these io-bound use-cases, you don’t want to have to switch language.

The same for Django: most of what you’re doing has no need of async but you don’t want to have to change web framework just because you need a sprinkling of non-blocking IO.

— Carlton Gibson

Tags: async, django, python, carlton-gibson

The "await me maybe" pattern for Python asyncio

2020-09-02T23:09:41+00:00

I've identified a pattern for handling potentially-asynchronous callback functions in Python which I'm calling the "await me maybe" pattern. It works by letting you return a value, a callable function that returns a value OR an awaitable function that returns that value.

Background

Datasette has been built on top of Python 3 asyncio from the very start - initially using Sanic, and as-of Datasette 0.29 using a custom mini-framework on top of ASGI 3, usually running under Uvicorn.

Datasette also has a plugin system, built on top of Pluggy.

Pluggy is a beautifully designed mechanism for plugins. It works based on decorated functions, which are called at various points by Datasette itself.

A simple plugin that injects a new JavaScript file into a page coud look like this:

from datasette import hookimpl

@hookimpl
def extra_js_urls():
    return [
        "https://code.jquery.com/jquery-3.5.1.min.js"
    ]

Datasette can then gather together all of the extra JavaScript URLs that should be injected into a page by running this code:

urls = []
for url in pm.hook.extra_js_urls(
    template=template.name,
    datasette=datasette,
):
    urls.extend(url)

What's up with the template= and datasette= parameters that are passed here?

Pluggy implements a form of dependency injection, where plugin hook functions can optionally list additional parameters that they would like to have access to.

The above simple example didn't need any extra information. But imagine a plugin that only wants to inject jQuery on the table.html template page:

@hookimpl
def extra_js_urls(template):
    if template == "table.html":
        return [
            "https://code.jquery.com/jquery-3.5.1.min.js"
        ]

Datasette actually provides several more optional argument for these plugin functions - see the plugin hooks documentation for full details.

What if we need to await something?

The datasette object that can be passed to plugin hooks is special: it provides an object that can be used for the following:

Executing SQL against databases connected to Datasette
Looking up Datasette metadata and configuration settings, including plugin configuration
Rendering templates using the template environment configured by Datasette
Performing checks against the Datasette permissions system

Here's the problem: many of those methods on Datasette are awaitable - await datasette.render_template(...) for example. But Pluggy is built around regular non-awaitable Python functions.

If my def extra_js_urls() plugin function needs to execute a SQL query to decide what JavaScript to include, it won't be able to - because you can't use await inside a regular Python function.

That's where the "await me maybe" pattern comes in.

The basic idea is that a function can return a value, OR a function-that-returns-a-value, OR an awaitable-function-that-returns-a-value.

If we want our extra_js_urls(datasette) hook to execute a SQL query in order to decide what URLs to return, it can look like this:

@hookimpl
def extra_js_urls(datasette):
    async def inner():
        db = datasette.get_database()
        results = await db.execute("select url from js_files")
        return [r[0] for r in results]

    return inner

Note that Python lets you define an async def inner() function inside the body of a regular function, which is what we're doing here.

The code that calls the plugin hook in Datasette can then look like this:

urls = []
for url in pm.hook.extra_js_urls(
    template=template.name,
    datasette=datasette,
):
    if callable(url):
        url = url()
    if asyncio.iscoroutine(url):
        url = await url
    urls.append(url)

I use this pattern in a bunch of different places in Datasette, so today I refactored that into a utility function:

import asyncio

async def await_me_maybe(value):
    if callable(value):
        value = value()
    if asyncio.iscoroutine(value):
        value = await value
    return value

This commit includes a bunch of examples where this function is called, for example this code which gathers extra body scripts to be included at the bottom of the page:

body_scripts = []
for extra_script in pm.hook.extra_body_script(
    template=template.name,
    database=context.get("database"),
    table=context.get("table"),
    columns=context.get("columns"),
    view_name=view_name,
    request=request,
    datasette=self,
):
    extra_script = await await_me_maybe(extra_script)
    body_scripts.append(Markup(extra_script))

Tags: async, patterns, plugins, python, datasette

Waiting in asyncio

2020-05-26T15:28:10+00:00

Waiting in asyncio

Handy cheatsheet explaining the differences between asyncio.gather(), asyncio.wait_for(), asyncio.as_completed() and asyncio.wait() by Hynek Schlawack.

Tags: async, python, hynek-schlawack

Django: Added support for asynchronous views and middleware

2020-03-19T03:43:40+00:00

Django: Added support for asynchronous views and middleware

An enormously consequential feature just landed in Django, and is set to ship as part of Django 3.1 in August. Asynchronous views will allow Django applications to define views using “async def myview(request)”—taking full advantage of Python’s growing asyncio ecosystem and providing enormous performance improvements for Django sites that do things like hitting APIs over HTTP. Andrew has been puzzling over this for ages and it’s really exciting to see it land in a form that should be usable in a stable Django release in just a few months.

Via @andrewgodwin

Tags: andrew-godwin, async, django

Async Support - HTTPX

2020-01-10T04:49:59+00:00

Async Support - HTTPX

HTTPX is the new async-friendly HTTP library for Python spearheaded by Tom Christie. It works in both async and non-async mode with an API very similar to requests. The async support is particularly interesting - it's a really clean API, and now that Jupyter supports top-level await you can run (await httpx.AsyncClient().get(url)).text directly in a cell and get back the response. Most excitingly the library lets you pass an ASGI app directly to the client and then perform requests against it - ideal for unit tests.

Via @_tomchristie

Tags: async, http, python, asgi, kim-christie, httpx

Quoting Andrew Godwin

2019-05-10T02:00:03+00:00

... the overall conclusion I reach is that we have so much to gain from making Django async-capable that it is worth the large amount of work it will take. I also believe, crucially, that we can undertake this change in an iterative, community-driven way that does not rely solely on one or two long-time contributors burning themselves out.

— Andrew Godwin

Tags: andrew-godwin, async, django

Quoting Tom Christie

2018-10-08T14:43:16+00:00

The ASGI specification provides an opportunity for Python to hit a productivity/performance sweet-spot for a wide range of use-cases, from writing high-volume proxy servers through to bringing large-scale web applications to market at speed.

— Tom Christie

Tags: async, python, asgi, kim-christie

Changelog 2018-06-12 / Observable

2018-06-13T15:50:01+00:00

Changelog 2018-06-12 / Observable

The ability to download an Observable notebook as a stand-alone ES module and run it anywhere using their open source runtime is fascinating, but it’s also worth reading the changelog for some of the new clever tricks they are pulling using await—“await visibility();” in a notebook cell will cause execution to pause until the cell scrolls into view for example.

Tags: async, javascript, observable