vietnam crab exportersoft-shell crab

Simon Willison’s Weblog

Subscribe
Atom feed

Elsewhere

Filters: Sorted by date

Sighting 2:24 PM – 2:39 PM — White-crowned Sparrow, Turkey Vulture, in San Mateo County, CA, US
White-crowned Sparrow
White-crowned Sparrow
White-crowned Sparrow
White-crowned Sparrow
Turkey Vulture
Turkey Vulture
Turkey Vulture
Turkey Vulture

An embarrassingly tiny release. The pyproject.toml had pinned to datasette==1.0a27, inadvertently making this plugin incompatible with all other Datasette versions. It's now datasette>=1.0a27 instead.

Sighting 7:34 PM – 7:36 PM — Pacific Harbor Seal, California Brown Pelican, in Monterey Bay National Marine Sanctuary, CA, US, CA
Pacific Harbor Seal
Pacific Harbor Seal
Pacific Harbor Seal
Pacific Harbor Seal
California Brown Pelican
California Brown Pelican
California Brown Pelican
California Brown Pelican

I'll write more about this one soon, but it's a big release. Three highlights from the release notes:

  • New "Create table" interface in the database actions menu, backed by the /<database>/-/create JSON API. It can define columns, primary keys, custom column types, NOT NULL constraints, literal defaults, expression defaults and single-column foreign keys. (#2787)
  • New "Alter table" table action and /<database>/<table>/-/alter JSON API for changing existing tables: add, rename, reorder and drop columns; change column types, defaults, NOT NULLconstraints, primary keys and foreign keys; and rename the table. The alter table dialog also includes a "Drop table" button. (#2788)
  • New Template context documentation listing the variables available to custom templates for Datasette's core pages. Variables documented there are treated as a stable API for custom templates until Datasette 2.0. The documentation is generated from dataclass definitions next to the view code, with tests that compare the documented fields against the actual contexts rendered by the database, table, query and row pages. (#1510, #2127, #1477, #2803)

Here's a rough video demo I made of the new create/alter table feature as part of reviewing the PR:

I've been pondering if Datasette Lite - the Python Datasette application run entirely in the browser using Pyodide and WebAssembly - might be able to edit persistent SQLite files stored on the user's computer.

That's what OFPS (Origin Private File System) is for, so I had Claude Code for web build me this playground UI to try it out in different browsers.

Sighting 9:39 AM — Common Raven, in Monterey Bay National Marine Sanctuary, CA, US, CA
Common Raven
Common Raven
Common Raven
Common Raven
Common Raven
Common Raven
Common Raven
Common Raven
Sighting 5:30 PM – 6:23 PM — California Brown Pelican, Common Loon, Pacific Harbor Seal, in Monterey Bay National Marine Sanctuary, CA, US, CA
California Brown Pelican
California Brown Pelican
Common Loon
Common Loon
Pacific Harbor Seal
Pacific Harbor Seal
Sighting 7:48 PM – 8:04 PM — California Brown Pelican, Great Blue Heron, in Monterey Bay National Marine Sanctuary, CA, US, CA
California Brown Pelican
California Brown Pelican
Great Blue Heron
Great Blue Heron
California Brown Pelican
California Brown Pelican
Sighting 6:55 PM – 7:02 PM — California Brown Pelican, Pacific Harbor Seal, in Monterey Bay National Marine Sanctuary, CA, US, CA
California Brown Pelican
California Brown Pelican
Pacific Harbor Seal
Pacific Harbor Seal

This release expands datasette-acl from table-only permissions toward a general resource-sharing system.

Alex Garcia did most of the work for this release - we're fleshing out the plugin that will allow multi-user Datasette instances finely grained control over who can access which resources within Datasette.

A progressive enchantment Web Component that turns this markup:

<click-to-play>
  <a href="URL to GIF">
    <img src="URL to first frame" alt="...">
  </a>
</click-to-play>

Into a still frame with a click to play button which loads the GIF on demand. For when you don't want big GIFs to be loaded unless people want to play them.

Here's an example that demonstrates the new row editing tools in Datasette - in fact I built this Web Component for that post.

Sighting 6:38 PM – 7:10 PM — California Brown Pelican, Botta's Pocket Gopher, Great Blue Heron, California Sea Lion, Pacific Harbor Seal, in Monterey Bay National Marine Sanctuary, CA, US, CA
California Brown Pelican
California Brown Pelican
Botta's Pocket Gopher
Botta's Pocket Gopher
Great Blue Heron
Great Blue Heron
California Sea Lion
California Sea Lion
Pacific Harbor Seal
Pacific Harbor Seal
California Brown Pelican
California Brown Pelican

Quoting the release notes:

The big feature in this alpha is tools to insert, edit and delete rows within the Datasette interface. These features are available on table pages, and edit and delete are also available as action items on the row page.

The inspiration for this feature - which is long overdue - was Datasette Agent. I added SQL write support to that the other day which highlighted how absurd it was that you could insert and edit ties via the chat interface but not in the regular Datasette UI!

A very experimental alpha plugin which lets you do this:

datasette tailscale mydata.db \
  --ts-authkey tskey-auth-xxxx --ts-hostname datasette-preview

This starts a localhost Datasette server with a Tailscale sidecar that connects it to your Tailnet, such that http://datasette-preview/ serves Datasette.

It's using the Python bindings for the experimental tailscale-rs library. I filed an issue asking if there's a cleaner way of setting up the proxy mechanism.

Sighting 6:56 PM — Pacific Harbor Seal, in Monterey Bay National Marine Sanctuary, CA, US, CA
Pacific Harbor Seal
Pacific Harbor Seal
Pacific Harbor Seal
Pacific Harbor Seal
Pacific Harbor Seal
Pacific Harbor Seal
Pacific Harbor Seal
Pacific Harbor Seal
None

I'm using Cloudflare's CAPTCHA (they call it a "Web Application Firewall > Custom rules > Managed Challenge" these days) to prevent crawlers from aggresively spidering my faceted search engine on this site, but I got fed up of even simple ?q=term searches triggering the challenge.

After some mucking around with Claude Code it turns out you can register the following rule instead, so the CAPTCHA only kicks in for search URLs containing at least one ampersand:

(http.request.uri.path wildcard r"/search/*" and http.request.uri.query contains "&")

And now /search/?q=lemur works without triggering a CAPTCHA!

Also included: notes on trying out the Cloudflare MCP with Claude Code, though it turned out not to be able to edit the rules in question so I had Claude Code switch to the Cloudflare API instead.

  • Fixed a bug where users without the create-app permission could still create apps. #27
  • Fixed a bug where it was impossible to grant permission to edit an app to users who were not the app's owner. The rules for edit/delete are now the same as view: if the app is private only the owner can modify it, otherwise permission is controlled by Datasette's regular permission system. #29
  • Custom network/CSP origins for apps are now guarded by a new apps-set-csp permission, with an optional allowed_csp_origins plugin allow-list for non-privileged users. The Datasette Agent app creation tool enforces the same rules. #24
  • Stored query picker now supports keyboard navigation and shows the three most recent accessible stored queries when focused.
  • #fragment links inside apps are no longer intercepted by the external-link confirmation modal. #23
  • Fixed link confirmation modal and logging panels in ?full=1 full-screen mode. #26
  • New tool, execute_write_sql, which requests user approval and then writes to a database - taking user permissions into account. #27

I added a mechanism for asking user approval in datasette agent 0.2a0. The new execute_write_sql tool can now prompt the user for all kinds of useful operations. Here's an example where I add some pelican sightings to my pelican_sightings table:

Screenshot of a chat interface showing a write SQL confirmation dialog. User message (blue bubble): "I saw 4 pelicans flying over the harbor". Collapsed tool section: "► Tool: execute_write_sql". A yellow-bordered confirmation card reads: "Confirm write SQL batch / Database: pelicans / Statements execute in order. If one statement fails, later statements will not be executed. / Statement 1 / INSERT INTO pelican_sightings (number_of_pelicans, notes) VALUES (:number_of_pelicans, :notes); / number_of_pelicans 4 / notes Flying over the harbor". A table with columns "Operation, Database, Table, Required permissions" shows row: "insert, pelicans, pelican_sightings" with permission buttons "insert-row", "update-row", "delete-row". Below: "Execute 1 write SQL statement against database 'pelicans'? / Asked by tool: execute_write_sql" with "Yes" (blue) and "No" (gray) buttons.

The new version also enhances the datasette agent chat terminal mode to support approvals, and adds several new options including --unsafe mode for auto-approving them:

  • datasette agent chat can execute tools that require user approval. #30
  • Three new options for datasette agent chat - --root to run as root, --yes to approve all ask user questions, and --unsafe for both.
  • Tools can now provide plain text alternatives to HTML, for display in the datasette agent chat CLI. #31

The datasette agent chat content.db -m gpt-5.5 --unsafe command can now be used to chat directly with a specific database and directly modify it through prompts like "create a notes table", "add a note about X" etc.

Sighting 5:38 PM — California Brown Pelican, in Monterey Bay National Marine Sanctuary, CA, US, CA
California Brown Pelican
California Brown Pelican
California Brown Pelican
California Brown Pelican
California Brown Pelican
California Brown Pelican
Sighting 7:11 PM – 7:19 PM — California Brown Pelican, Great Blue Heron, California Sea Lion, in Monterey Bay National Marine Sanctuary, CA, US, CA
Great Blue Heron
Great Blue Heron
California Sea Lion
California Sea Lion
California Brown Pelican
California Brown Pelican
California Brown Pelican
California Brown Pelican

It would be neat if arbitrary SQL queries in Datasette could be rendered with additional information based on which columns from which tables were included in the results.

To build that, we would need to be able to look at a SQL query like select users.name, orders.total from users join orders on orders.user_id = users.id and programmatically identify the table.column for each result - navigating not just joins but also more complex syntax like CTEs.

I decided to set Claude Code (Opus 4.8, since Fable is currently banned by the US government) on the problem. It found several promising solutions - one using apsw, another that uses ctypes to access the SQLite sqlite3_column_table_name() C function (which is not otherwise exposed to Python), and one using clever interrogation of the output of EXPLAIN.

Sighting 7:49 PM — Harbor Seal, in Monterey Bay National Marine Sanctuary, CA, US, CA
Harbor Seal
Harbor Seal
Harbor Seal
Harbor Seal

This alpha is a significant step on the road to a stable 1.0, finally extending the ?_extra= pattern I introduced in Datasette 1.0a3 to cover queries and rows in addition to tables. That pattern is also now documented!

I wrote a whole lot more about the new release on the Datasette project blog: Datasette 1.0a33 with JSON extras in the API.

Because API explorer tools are almost free to build now I had Claude Fable 5 in Claude Code (for the plan) and GPT-5.5 xhigh in Codex Desktop (for the implementation) build me this custom extras API explorer to help demonstrate the feature:

Screenshot of a web application titled "Datasette extras explorer". A URL input field contains https://latest.datasette.io/fixtures/facetable.json with a teal Explore button next to it. Below, a left panel labeled EXTRAS (30) lists checkboxes: all_columns - All columns in the table, regardless of _col/_nocol filtering; column_types - Column type assignments for this table; columns (checked) - Column names returned by this query; count - Total count of rows matching these filters; count_sql - SQL query used to calculate the total count; custom_table_templates - Custom template names considered for this table; database - Database name; database_color - Color assigned to the database. A right panel labeled RESPONSE shows GET /fixtures/fac… with Copy JSON and Copy URL buttons, then a dark JSON viewer showing 200 - 9.9 KB - 114ms and JSON: "ok": true, "next": null, "columns": (highlighted array) "pk", "created", "planet_int", "on_earth", "state", "_city_id", "_neighborhood", "tags", "complex_array", "distinct_some_null", "n", "rows": list of objects.

Tool Datasette extras explorer — Query Datasette JSON endpoints and discover available extras by entering a URL and selecting which extras to request. The interface displays the returned JSON response with syntax highlighting and allows you to examine which extras are supported by your Datasette instance, with support for sharing configurations via URL hash parameters.

I built this utility library to support an asyncio dependency injection pattern a few years ago. I was using it with Datasette and Claude Fable 5 spotted some bugs in the dependency which it then fixed for me. It's a very proactive model!

Research Running untrusted queries: Datasette/SQLite vs psycopg/PostgreSQL — Exploring how untrusted SQL queries are safely run in Datasette (using SQLite) and whether similar protections can be applied with psycopg and PostgreSQL, this project shows that both can provide robust safeguards against data corruption and resource exhaustion. Datasette leverages hard read-only file modes and a VM progress handler for query timeouts, while PostgreSQL's privilege system enforces SELECT-only access and its `statement_timeout` cancels resource-intensive or sleeping queries.

Highlights from the release notes:

  • Tools can now ask the user questions mid-execution. Tools that declare a context parameter receive a ToolContext object, and await context.ask_user(...) can ask a yes/no, multiple-choice (options=[...]) or free-text (free_text=True) question. While a question is unanswered the agent turn suspends: the question renders as a form in the chat UI and persists to the internal database, so suspended conversations survive a server restart. Once answered, the tool re-executes from the top with stored answers replayed, so call ask_user() before performing side effects. #20
  • New built-in save_query tool: the agent can save SQL it has written as a Datasette stored query. Saving always requires human approval - the agent shows the full SQL plus the proposed name, database and visibility, and nothing is stored until you click Yes. #20

The ask_user() feature was enabled by the new LLM alpha I built yesterday with the help of Claude Fable 5.