<?xml version="1.0" encoding="utf-8"?>
<feed xml:lang="en-us" xmlns="http://www.w3.org/2005/Atom"><title>Simon Willison's Weblog: permissions</title><link href="http://simonwillison.net/" rel="alternate"/><link href="http://simonwillison.net/tags/permissions.atom" rel="self"/><id>http://simonwillison.net/</id><updated>2024-04-16T19:49:16+00:00</updated><author><name>Simon Willison</name></author><entry><title>Quoting wkirby on Hacker News</title><link href="https://simonwillison.net/2024/Apr/16/wkirby-on-hacker-news/#atom-tag" rel="alternate"/><published>2024-04-16T19:49:16+00:00</published><updated>2024-04-16T19:49:16+00:00</updated><id>https://simonwillison.net/2024/Apr/16/wkirby-on-hacker-news/#atom-tag</id><summary type="html">
    &lt;blockquote cite="https://news.ycombinator.com/item?id=40052729#40054080"&gt;&lt;p&gt;Permissions have three moving parts, who wants to do it, what do they want to do, and on what object. Any good permission system has to be able to efficiently answer any permutation of those variables. Given this person and this object, what can they do? Given this object and this action, who can do it? Given this person and this action, which objects can they act upon?&lt;/p&gt;&lt;/blockquote&gt;
&lt;p class="cite"&gt;&amp;mdash; &lt;a href="https://news.ycombinator.com/item?id=40052729#40054080"&gt;wkirby on Hacker News&lt;/a&gt;&lt;/p&gt;

    &lt;p&gt;Tags: &lt;a href="https://simonwillison.net/tags/hacker-news"&gt;hacker-news&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/permissions"&gt;permissions&lt;/a&gt;&lt;/p&gt;



</summary><category term="hacker-news"/><category term="permissions"/></entry><entry><title>Datasette 1.0a2: Upserts and finely grained permissions</title><link href="https://simonwillison.net/2022/Dec/15/datasette-1a2/#atom-tag" rel="alternate"/><published>2022-12-15T17:58:19+00:00</published><updated>2022-12-15T17:58:19+00:00</updated><id>https://simonwillison.net/2022/Dec/15/datasette-1a2/#atom-tag</id><summary type="html">
    &lt;p&gt;I've released the third alpha of Datasette 1.0. The &lt;a href="https://docs.datasette.io/en/latest/changelog.html#a2-2022-12-14"&gt;1.0a2 release&lt;/a&gt; introduces upsert support to the new JSON API and makes some major improvements to the Datasette permissions system.&lt;/p&gt;
&lt;p&gt;Here are the annotated releases (&lt;a href="https://simonwillison.net/series/datasette-release-notes/"&gt;see previous&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;You can install and try out the alpha using:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;pip install datasette==1.0a2
&lt;/code&gt;&lt;/pre&gt;
&lt;h4&gt;Upserts for the JSON API&lt;/h4&gt;
&lt;blockquote&gt;
&lt;p&gt;New &lt;code&gt;/db/table/-/upsert&lt;/code&gt; API, &lt;a href="https://docs.datasette.io/en/latest/json_api.html#tableupsertview"&gt;documented here&lt;/a&gt;. Upsert is an update-or-insert: existing rows will have specified keys updated, but if no row matches the incoming primary key a brand new row will be inserted instead. (&lt;a href="https://github.com/simonw/datasette/issues/1878"&gt;#1878&lt;/a&gt;)&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;I &lt;a href="https://simonwillison.net/2022/Dec/2/datasette-write-api/"&gt;wrote about the new JSON Write API&lt;/a&gt; when I released the first alpha a couple of weeks ago.&lt;/p&gt;
&lt;p&gt;The API can be used to create and drop tables, and to insert, update and delete rows in those tables.&lt;/p&gt;
&lt;p&gt;The new &lt;code&gt;/db/table/-/upsert&lt;/code&gt; API adds &lt;a href="https://docs.datasette.io/en/latest/json_api.html#tableupsertview"&gt;upsert support&lt;/a&gt; to Datasette.&lt;/p&gt;
&lt;p&gt;An &lt;em&gt;upsert&lt;/em&gt; is a update-or-insert. Consider the following:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;POST /books/authors/-/upsert
Authorization: Bearer $TOKEN
Content-Type: application/json
&lt;/code&gt;&lt;/pre&gt;
&lt;div class="highlight highlight-source-json"&gt;&lt;pre&gt;{
  &lt;span class="pl-ent"&gt;"rows"&lt;/span&gt;: [
    {
      &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;1&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Ursula K. Le Guin&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"born"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;1929-10-21&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    },
    {
      &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;2&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Terry Pratchett&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"born"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;1948-04-28&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    },
    {
      &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;3&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Neil Gaiman&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"born"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;1960-11-10&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    }
  ]
}&lt;/pre&gt;&lt;/div&gt;
&lt;p&gt;This table has a primary key of &lt;code&gt;id&lt;/code&gt;. The above API call will create three records if the table is empty. But if the table already has records matching any of those primary keys, their &lt;code&gt;name&lt;/code&gt; and &lt;code&gt;born&lt;/code&gt; columns will be updated to match the incoming data.&lt;/p&gt;
&lt;p&gt;Upserts can be a really convenient way of synchronizing data with an external data source. I had a couple of enquiries about them when I published the first alpha, so I decided to make them a key feature for this release.&lt;/p&gt;
&lt;h4&gt;Ignore and replace for the create table API&lt;/h4&gt;
&lt;blockquote&gt;
&lt;ul&gt;
&lt;li&gt;The &lt;code&gt;/db/-/create&lt;/code&gt; API for &lt;a href="https://docs.datasette.io/en/latest/json_api.html#tablecreateview"&gt;creating a table&lt;/a&gt; now accepts &lt;code&gt;"ignore": true&lt;/code&gt; and &lt;code&gt;"replace": true&lt;/code&gt; options when called with the &lt;code&gt;"rows"&lt;/code&gt; property that creates a new table based on an example set of rows. This means the API can be called multiple times with different rows, setting rules for what should happen if a primary key collides with an existing row. (&lt;a href="https://github.com/simonw/datasette/issues/1927"&gt;#1927&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;/db/-/create&lt;/code&gt; API now requires actor to have &lt;code&gt;insert-row&lt;/code&gt; permission in order to use the &lt;code&gt;"row"&lt;/code&gt; or &lt;code&gt;"rows"&lt;/code&gt; properties. (&lt;a href="https://github.com/simonw/datasette/issues/1937"&gt;#1937&lt;/a&gt;)&lt;/li&gt;
&lt;/ul&gt;
&lt;/blockquote&gt;
&lt;p&gt;This feature is a little less obvious, but I think it's going to be really useful.&lt;/p&gt;
&lt;p&gt;The &lt;code&gt;/db/-/create&lt;/code&gt; API can be used to create a new table. You can feed it &lt;a href="https://docs.datasette.io/en/latest/json_api.html#creating-a-table"&gt;an explicit list of columns&lt;/a&gt;, but you can also give it one or more rows and have it &lt;a href="https://docs.datasette.io/en/latest/json_api.html#creating-a-table-from-example-data"&gt;infer the correct schema&lt;/a&gt; based on those examples.&lt;/p&gt;
&lt;p&gt;Datasette inherits this feature &lt;a href="https://sqlite-utils.datasette.io/en/stable/cli.html#inserting-json-data"&gt;from sqlite-utils&lt;/a&gt; - I've been finding this an incredibly productive way to work with SQLite databases for a few years now.&lt;/p&gt;
&lt;p&gt;The real magic of this feature is that you can pipe data into Datasette without even needing to first check that the appropriate table has been created. It's a really fast way of getting from data to a populated database and a working API.&lt;/p&gt;
&lt;p&gt;Prior to 1.0a2 you could call &lt;code&gt;/db/-/create&lt;/code&gt; with &lt;code&gt;"rows"&lt;/code&gt; more than once and it would &lt;em&gt;probably&lt;/em&gt; work... unless you attempted to insert rows with primary keys that were already in use - in which case you would get an error. This limited the utility of the feature.&lt;/p&gt;
&lt;p&gt;Now you can pass &lt;code&gt;"ignore": true&lt;/code&gt; or &lt;code&gt;"replace": true&lt;/code&gt; to the API call, to tell Datasette what to do if it encounters a primary key that already exists in the table.&lt;/p&gt;
&lt;p&gt;Heres an example using the author data from above:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;POST /books/-/create
Authorization: Bearer $TOKEN
Content-Type: application/json
&lt;/code&gt;&lt;/pre&gt;
&lt;div class="highlight highlight-source-json"&gt;&lt;pre&gt;{
  &lt;span class="pl-ent"&gt;"table"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;authors&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"pk"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;id&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"replace"&lt;/span&gt;: &lt;span class="pl-c1"&gt;true&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"rows"&lt;/span&gt;: [
    {
      &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;1&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Ursula K. Le Guin&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"born"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;1929-10-21&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    },
    {
      &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;2&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Terry Pratchett&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"born"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;1948-04-28&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    },
    {
      &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;3&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Neil Gaiman&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"born"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;1960-11-10&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    }
  ]
}&lt;/pre&gt;&lt;/div&gt;
&lt;p&gt;This will create the &lt;code&gt;authors&lt;/code&gt; table if it does not exist and ensure that those three rows exist in it, in their exact state. If a row already exists it will be replaced.&lt;/p&gt;
&lt;p&gt;Note that this is subtly different from an upsert. An upsert will only update the columns that were provided in the incoming data, leaving any other columns unchanged. A replace will replace the entire row.&lt;/p&gt;
&lt;h4 id="finely-grained-permissions"&gt;Finely grained permissions&lt;/h4&gt;
&lt;p&gt;This is the most significant area of improvement in this release.&lt;/p&gt;
&lt;blockquote&gt;
&lt;ul&gt;
&lt;li&gt;New &lt;a href="https://docs.datasette.io/en/latest/plugin_hooks.html#plugin-register-permissions"&gt;register_permissions(datasette)&lt;/a&gt; plugin hook. Plugins can now register named permissions, which will then be listed in various interfaces that show available permissions. (&lt;a href="https://github.com/simonw/datasette/issues/1940"&gt;#1940&lt;/a&gt;)&lt;/li&gt;
&lt;/ul&gt;
&lt;/blockquote&gt;
&lt;p&gt;Prior to this, permissions were just strings - things like &lt;code&gt;"view-instance"&lt;/code&gt; or &lt;code&gt;"view-table"&lt;/code&gt; or &lt;code&gt;"insert-row"&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Plugins can introduce their own permissions - many do already, like &lt;a href="https://datasette.io/plugins/datasette-edit-schema"&gt;datasette-edit-schema&lt;/a&gt; which adds a &lt;code&gt;"edit-schema"&lt;/code&gt; permission.&lt;/p&gt;
&lt;p&gt;In order to start building UIs for managing permissions, I needed Datasette to know what they were!&lt;/p&gt;
&lt;p&gt;The &lt;a href="https://docs.datasette.io/en/latest/plugin_hooks.html#plugin-register-permissions"&gt;register_permissions() hook&lt;/a&gt; lets them do exactly that, and Datasette core uses it to register its own default set of permissions too.&lt;/p&gt;
&lt;p&gt;Permissions are registered using the following named tuple:&lt;/p&gt;
&lt;pre&gt;&lt;span class="pl-v"&gt;Permission&lt;/span&gt; &lt;span class="pl-c1"&gt;=&lt;/span&gt; &lt;span class="pl-s1"&gt;collections&lt;/span&gt;.&lt;span class="pl-en"&gt;namedtuple&lt;/span&gt;(
    &lt;span class="pl-s"&gt;"Permission"&lt;/span&gt;, (
        &lt;span class="pl-s"&gt;"name"&lt;/span&gt;, &lt;span class="pl-s"&gt;"abbr"&lt;/span&gt;, &lt;span class="pl-s"&gt;"description"&lt;/span&gt;,
        &lt;span class="pl-s"&gt;"takes_database"&lt;/span&gt;, &lt;span class="pl-s"&gt;"takes_resource"&lt;/span&gt;, &lt;span class="pl-s"&gt;"default"&lt;/span&gt;
    )
)&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;abbr&lt;/code&gt; is an abbreviation - e.g. &lt;code&gt;insert-row&lt;/code&gt; can be abbreviated to &lt;code&gt;ir&lt;/code&gt;. This is useful for creating things like signed API tokens where space is at a premium.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;takes_database&lt;/code&gt; and &lt;code&gt;takes_resource&lt;/code&gt; are booleans that indicate whether the permission can optionally be applied to a specific database (e.g. &lt;code&gt;execute-sql&lt;/code&gt;) or to a "resource", which is the name I'm now using for something that could be a SQL table, a SQL view or a &lt;a href="https://docs.datasette.io/en/stable/sql_queries.html#canned-queries"&gt;canned query&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The &lt;code&gt;insert-row&lt;/code&gt; permission for example can be granted to the whole of Datasette, or to all tables in a specific database, or to specific tables.&lt;/p&gt;
&lt;p&gt;Finally, the &lt;code&gt;default&lt;/code&gt; value is a boolean that indicates whether the permission should be default-allow (&lt;code&gt;view-instance&lt;/code&gt; for example) or default-deny (&lt;code&gt;create-table&lt;/code&gt; and suchlike).&lt;/p&gt;
&lt;p&gt;This next feature explains why I needed those permission names to be known to Datasette:&lt;/p&gt;
&lt;blockquote&gt;
&lt;ul&gt;
&lt;li&gt;The &lt;code&gt;/-/create-token&lt;/code&gt; page can now be used to create API tokens which are restricted to just a subset of actions, including against specific databases or resources. See &lt;a href="https://docs.datasette.io/en/latest/authentication.html#createtokenview"&gt;API Tokens&lt;/a&gt; for details. (&lt;a href="https://github.com/simonw/datasette/issues/1947"&gt;#1947&lt;/a&gt;)&lt;/li&gt;
&lt;/ul&gt;
&lt;/blockquote&gt;
&lt;p&gt;Datasette now has finely grained permissions for API tokens!&lt;/p&gt;
&lt;p&gt;This is the feature I always want when I'm working with other APIs: the ability to create a token that can only perform a restricted subset of actions.&lt;/p&gt;
&lt;p&gt;When I'm working with the GitHub API for example I frequently find myself wanting to create a "personal access token" that only has the ability to read issues from a specific repository. It's infuriating how many APIs leave this ability out.&lt;/p&gt;
&lt;p&gt;The &lt;code&gt;/-/create-token&lt;/code&gt; interface (which you can try out on &lt;a href="https://latest.datasette.io/"&gt;latest.datasette.io&lt;/a&gt; by first &lt;a href="https://latest.datasette.io/login-as-root"&gt;signing in as root&lt;/a&gt; and then &lt;a href="https://latest.datasette.io/-/create-token"&gt;visiting this page&lt;/a&gt;) lets you create an API token that can act on your behalf... and then optionally specify a subset of actions that the token is allowed to perform.&lt;/p&gt;
&lt;p&gt;Thanks to the new permissions registration system, the UI on that page knows which permissions can be applied to which entities within Datasette itself.&lt;/p&gt;
&lt;p&gt;Here's a partial screenshot of the UI:&lt;/p&gt;
&lt;p&gt;&lt;img src="https://static.simonwillison.net/static/2022/create-api-token.jpg" alt="Create an API token. This token will allow API access with the same abilities as your current user, root. Form lets you select Expires after X hours, 5 - and there's an expanded menu item for Restrict actions that can be performed using this token. Below that are lists of checkboxes - the first is for All databases and tables, with a list of every permission known to Datasette. Next is All tables in fixtures, which lists just permissions that make sense for a specific database. Finally is Specific tables, which lists fixtures: primary_key_multiple_columns with a much shorter list of permissions that can apply just to tables." style="max-width: 100%;" /&gt;&lt;/p&gt;
&lt;p&gt;Select a subset of permissions, hit "Create token" and the result will be an access token you can copy and paste into another application, or use to call Datasette with &lt;code&gt;curl&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Here's an example token I created that grants &lt;code&gt;view-instance&lt;/code&gt; permission against all of Datasette, and &lt;code&gt;view-database&lt;/code&gt; and &lt;code&gt;execute-sql&lt;/code&gt; permission against the &lt;code&gt;ephemeral&lt;/code&gt; database, which in that demo is hidden from anonymous Datasette users.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;dstok_eyJhIjoicm9vdCIsInQiOjE2NzEwODUzMDIsIl9yIjp7ImEiOlsidmkiXSwiZCI6eyJlcGhlbWVyYWwiOlsidmQiLCJlcyJdfX19.1uw0xyx8UND_Y_vTVg5kEF6m3GU&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;Here's a screenshot of the screen I saw when I created it:&lt;/p&gt;
&lt;p&gt;&lt;img src="https://static.simonwillison.net/static/2022/token-created.jpg" alt="Create an API token. Your API token: a copy-to-clipboard box with a long token in it. The token details area is expanded to show JSON that represents the permissions granted with that token." style="max-width: 100%;" /&gt;&lt;/p&gt;
&lt;p&gt;I've expanded the "token details" section to show the JSON that is bundled inside the signed token. The &lt;code&gt;"_r"&lt;/code&gt; block records the specific permissions granted by the token:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;"a": ["vi"]&lt;/code&gt; indicates that the &lt;code&gt;view-instance&lt;/code&gt; permission is granted against all of Datasette.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;"d": {"ephemeral": ["vd", "es"]}&lt;/code&gt; indicates that the &lt;code&gt;view-database&lt;/code&gt; and &lt;code&gt;execute-sql&lt;/code&gt; permissions are granted against the &lt;code&gt;ephemeral&lt;/code&gt; database.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The token also contains the ID of the user who created it (&lt;code&gt;"a": "root"&lt;/code&gt;) and the time that the token was created (&lt;code&gt;"t": 1671085302&lt;/code&gt;). If the token was set to expire that expiry duration would be baked in here as well.&lt;/p&gt;
&lt;p&gt;You can see the effect this has on the command-line using &lt;code&gt;curl&lt;/code&gt; like so:&lt;/p&gt;
&lt;p&gt;&lt;code&gt;curl 'https://latest.datasette.io/ephemeral.json?sql=select+3+*+5&amp;amp;_shape=array'&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;This will return a forbidden error. But if you add the signed token:&lt;/p&gt;
&lt;p&gt;&lt;code&gt;curl 'https://latest.datasette.io/ephemeral.json?sql=select+3+*+5&amp;amp;_shape=array' -H 'Authorization: Bearer dstok_eyJhIjoicm9vdCIsInQiOjE2NzEwODUzMDIsIl9yIjp7ImEiOlsidmkiXSwiZCI6eyJlcGhlbWVyYWwiOlsidmQiLCJlcyJdfX19.1uw0xyx8UND_Y_vTVg5kEF6m3GU'&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;You'll get back a JSON response:&lt;/p&gt;
&lt;div class="highlight highlight-source-json"&gt;&lt;pre&gt;[{&lt;span class="pl-ent"&gt;"3 * 5"&lt;/span&gt;: &lt;span class="pl-c1"&gt;15&lt;/span&gt;}]&lt;/pre&gt;&lt;/div&gt;
&lt;h4&gt;The datasette create-token CLI tool&lt;/h4&gt;
&lt;blockquote&gt;
&lt;ul&gt;
&lt;li&gt;Likewise, the &lt;code&gt;datasette create-token&lt;/code&gt; CLI command can now create tokens with &lt;a href="https://docs.datasette.io/en/latest/authentication.html#authentication-cli-create-token-restrict"&gt;a subset of permissions&lt;/a&gt;. (&lt;a href="https://github.com/simonw/datasette/issues/1855"&gt;#1855&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;New &lt;a href="https://docs.datasette.io/en/latest/internals.html#create-token-actor-id-expires-after-none-restrict-all-none-restrict-database-none-restrict-resource-none"&gt;datasette.create_token()&lt;/a&gt; API method for programmatically creating signed API tokens. (&lt;a href="https://github.com/simonw/datasette/issues/1951"&gt;#1951&lt;/a&gt;)&lt;/li&gt;
&lt;/ul&gt;
&lt;/blockquote&gt;
&lt;p&gt;The other way you can create Datasette tokens is on the command-line, using &lt;a href="https://docs.datasette.io/en/latest/authentication.html#datasette-create-token"&gt;the datasette create-token command&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;That's been upgraded to support finely grained permissions too.&lt;/p&gt;
&lt;p&gt;Here's how you'd create a token for the same set of permissions as my ephemeral example above:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;datasette create-token root \
  --all view-instance \
  --database ephemeral view-database \
  --database ephemeral execute-sql \
  --secret MY_DATASETTE_SECRET
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In order to sign the token you need to pass in the &lt;code&gt;--secret&lt;/code&gt; used by the server - although it will pick that up from the &lt;code&gt;DATASETTE_SECRET&lt;/code&gt; environment variable if it's available.&lt;/p&gt;
&lt;p&gt;This has the interesting side-effect that you can use that command to create valid tokens for other Datasette instances, provided you know the secret they're using. I think this ability will be really useful for people like myself who run lots of different Datasette instances on stateless hosting platforms such as Vercel and Google Cloud Run.&lt;/p&gt;
&lt;h4&gt;Configuring permissions in metadata.json/yaml&lt;/h4&gt;
&lt;blockquote&gt;
&lt;ul&gt;
&lt;li&gt;Arbitrary permissions can now be configured at the instance, database and resource (table, SQL view or canned query) level in Datasette's &lt;a href="https://docs.datasette.io/en/latest/metadata.html#metadata"&gt;Metadata&lt;/a&gt; JSON and YAML files. The new &lt;code&gt;"permissions"&lt;/code&gt; key can be used to specify which actors should have which permissions. See &lt;a href="https://docs.datasette.io/en/latest/authentication.html#authentication-permissions-other"&gt;Other permissions in metadata&lt;/a&gt; for details. (&lt;a href="https://github.com/simonw/datasette/issues/1636"&gt;#1636&lt;/a&gt;)&lt;/li&gt;
&lt;/ul&gt;
&lt;/blockquote&gt;
&lt;p&gt;Datasette has long had the ability to set permissions for viewing databases and tables using blocks of configuration in the increasingly poorly named &lt;a href="https://docs.datasette.io/en/stable/metadata.html"&gt;metadata.json/yaml files&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;As I've built new plugins that introduce new permissions, I've found myself wishing for an easier way to say "user X is allowed to perform action Y" for arbitrary other permissions.&lt;/p&gt;
&lt;p&gt;The new &lt;code&gt;"permissions"&lt;/code&gt; key in metadata.json/yaml files allows you to do that.&lt;/p&gt;
&lt;p&gt;Here's how to specify that the user with &lt;code&gt;"id": "simon"&lt;/code&gt; is allowed to use the API to create tables and insert data into the &lt;code&gt;docs&lt;/code&gt; database:&lt;/p&gt;
&lt;div class="highlight highlight-source-yaml"&gt;&lt;pre&gt;&lt;span class="pl-ent"&gt;databases&lt;/span&gt;:
  &lt;span class="pl-ent"&gt;docs&lt;/span&gt;:
    &lt;span class="pl-ent"&gt;permissions&lt;/span&gt;:
      &lt;span class="pl-ent"&gt;create-table&lt;/span&gt;:
        &lt;span class="pl-ent"&gt;id&lt;/span&gt;: &lt;span class="pl-s"&gt;simon&lt;/span&gt;
      &lt;span class="pl-ent"&gt;insert-row&lt;/span&gt;:
        &lt;span class="pl-ent"&gt;id&lt;/span&gt;: &lt;span class="pl-s"&gt;simon&lt;/span&gt;&lt;/pre&gt;&lt;/div&gt;
&lt;p&gt;Here's a demo you can run on your own machine. Save the above to &lt;code&gt;permissions.yaml&lt;/code&gt; and run the following in one terminal window:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;datasette docs.db --create --secret sekrit -m permissions.yaml
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This will create the &lt;code&gt;docs.db&lt;/code&gt; database if it doesn't already exist, and start Datasette with the &lt;code&gt;permissions.yaml&lt;/code&gt; metadata file.&lt;/p&gt;
&lt;p&gt;It sets &lt;code&gt;--secret&lt;/code&gt; to a known value (you should always use a random secure secret in production) so we can easily use it with &lt;code&gt;create-token&lt;/code&gt; in the next step:&lt;/p&gt;
&lt;p&gt;Then in another terminal window run:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;export TOKEN=$(
  datasette create-token simon \
  --secret sekrit
)
curl -XPOST http://localhost:8001/docs/-/create \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "table": "demo",
    "row": {"id": 1, "name": "Simon"},
    "pk": "id"
  }'
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The first line creates a token that can act on behalf of the &lt;code&gt;simon&lt;/code&gt; actor. The second &lt;code&gt;curl&lt;/code&gt; line then uses that token to create a table using the &lt;code&gt;/-/create&lt;/code&gt; endpoint.&lt;/p&gt;
&lt;p&gt;Run this, then visit &lt;a href="http://localhost:8001/docs/demo"&gt;http://localhost:8001/docs/demo&lt;/a&gt; to see the newly created table.&lt;/p&gt;
&lt;h4&gt;What's next?&lt;/h4&gt;
&lt;p&gt;With the 1.0a2 release I'm reasonably confident that Datasette 1.0 is new-feature-complete. There's still a &lt;strong&gt;lot&lt;/strong&gt; of work to do before the final release, but the remaining work is far more intimidating: I need to make clean backwards-incompatible breakages to a whole host of existing features in order to ship a 1.0 that I can keep stable for as long as possible.&lt;/p&gt;
&lt;p&gt;First up: I'm going to &lt;a href="https://github.com/simonw/datasette/issues/1914"&gt;redesign Datasette's default API output&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The current default JSON output for a simple table looks like this:&lt;/p&gt;
&lt;div class="highlight highlight-source-json"&gt;&lt;pre&gt;{
  &lt;span class="pl-ent"&gt;"database"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;fixtures&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"table"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;facet_cities&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"is_view"&lt;/span&gt;: &lt;span class="pl-c1"&gt;false&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"human_description_en"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;sorted by name&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"rows"&lt;/span&gt;: [
    [
      &lt;span class="pl-c1"&gt;3&lt;/span&gt;,
      &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Detroit&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    ],
    [
      &lt;span class="pl-c1"&gt;2&lt;/span&gt;,
      &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Los Angeles&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    ],
    [
      &lt;span class="pl-c1"&gt;4&lt;/span&gt;,
      &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Memnonia&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    ],
    [
      &lt;span class="pl-c1"&gt;1&lt;/span&gt;,
      &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;San Francisco&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    ]
  ],
  &lt;span class="pl-ent"&gt;"truncated"&lt;/span&gt;: &lt;span class="pl-c1"&gt;false&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"filtered_table_rows_count"&lt;/span&gt;: &lt;span class="pl-c1"&gt;4&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"expanded_columns"&lt;/span&gt;: [],
  &lt;span class="pl-ent"&gt;"expandable_columns"&lt;/span&gt;: [],
  &lt;span class="pl-ent"&gt;"columns"&lt;/span&gt;: [
    &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;id&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
    &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;name&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
  ],
  &lt;span class="pl-ent"&gt;"primary_keys"&lt;/span&gt;: [
    &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;id&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
  ],
  &lt;span class="pl-ent"&gt;"units"&lt;/span&gt;: {},
  &lt;span class="pl-ent"&gt;"query"&lt;/span&gt;: {
    &lt;span class="pl-ent"&gt;"sql"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;select id, name from facet_cities order by name limit 101&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
    &lt;span class="pl-ent"&gt;"params"&lt;/span&gt;: {}
  },
  &lt;span class="pl-ent"&gt;"facet_results"&lt;/span&gt;: {},
  &lt;span class="pl-ent"&gt;"suggested_facets"&lt;/span&gt;: [],
  &lt;span class="pl-ent"&gt;"next"&lt;/span&gt;: &lt;span class="pl-c1"&gt;null&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"next_url"&lt;/span&gt;: &lt;span class="pl-c1"&gt;null&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"private"&lt;/span&gt;: &lt;span class="pl-c1"&gt;false&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"allow_execute_sql"&lt;/span&gt;: &lt;span class="pl-c1"&gt;true&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"query_ms"&lt;/span&gt;: &lt;span class="pl-c1"&gt;6.718471999647591&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"source"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;tests/fixtures.py&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"source_url"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;https://github.com/simonw/datasette/blob/main/tests/fixtures.py&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"license"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Apache License 2.0&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;,
  &lt;span class="pl-ent"&gt;"license_url"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;https://github.com/simonw/datasette/blob/main/LICENSE&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
}&lt;/pre&gt;&lt;/div&gt;
&lt;p&gt;In addition to being &lt;em&gt;really&lt;/em&gt; verbose, you'll note that the rows themselves are represented like this:&lt;/p&gt;
&lt;div class="highlight highlight-source-json"&gt;&lt;pre&gt;[
  [
    &lt;span class="pl-c1"&gt;3&lt;/span&gt;,
    &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Detroit&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
  ],
  [
    &lt;span class="pl-c1"&gt;2&lt;/span&gt;,
    &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Los Angeles&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
  ],
  [
    &lt;span class="pl-c1"&gt;4&lt;/span&gt;,
    &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Memnonia&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
  ],
  [
    &lt;span class="pl-c1"&gt;1&lt;/span&gt;,
    &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;San Francisco&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
  ]
]&lt;/pre&gt;&lt;/div&gt;
&lt;p&gt;I originally designed it this way because I thought saving on repeating the column names for every row would be more efficient.&lt;/p&gt;
&lt;p&gt;In practice, every single time I've used Datasette's API I've found myself using the &lt;code&gt;?_shape=array&lt;/code&gt; parameter, which outputs this format instead:&lt;/p&gt;
&lt;div class="highlight highlight-source-json"&gt;&lt;pre&gt;[
  {
    &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;3&lt;/span&gt;,
    &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Detroit&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
  },
  {
    &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;2&lt;/span&gt;,
    &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Los Angeles&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
  },
  {
    &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;4&lt;/span&gt;,
    &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Memnonia&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
  },
  {
    &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;1&lt;/span&gt;,
    &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;San Francisco&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
  }
]&lt;/pre&gt;&lt;/div&gt;
&lt;p&gt;It's just so much more convenient to work with!&lt;/p&gt;
&lt;p&gt;So the new default format will look like this:&lt;/p&gt;
&lt;div class="highlight highlight-source-json"&gt;&lt;pre&gt;{
  &lt;span class="pl-ent"&gt;"rows"&lt;/span&gt;: [
    {
      &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;3&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Detroit&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    },
    {
      &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;2&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Los Angeles&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    },
    {
      &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;4&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;Memnonia&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    },
    {
      &lt;span class="pl-ent"&gt;"id"&lt;/span&gt;: &lt;span class="pl-c1"&gt;1&lt;/span&gt;,
      &lt;span class="pl-ent"&gt;"name"&lt;/span&gt;: &lt;span class="pl-s"&gt;&lt;span class="pl-pds"&gt;"&lt;/span&gt;San Francisco&lt;span class="pl-pds"&gt;"&lt;/span&gt;&lt;/span&gt;
    }
  ]
}&lt;/pre&gt;&lt;/div&gt;
&lt;p&gt;The &lt;code&gt;rows&lt;/code&gt; key is there so I can add extra keys to the output, based on additional &lt;code&gt;?_extra=&lt;/code&gt; request parameters. You'll be able to get back everything you can get in the current full-fat table API, but you'll have to ask for it.&lt;/p&gt;
&lt;p&gt;There are a ton of other changes I want to make to Datasette as a whole - things like renaming &lt;code&gt;metadata.yaml&lt;/code&gt; to &lt;code&gt;config.yaml&lt;/code&gt; to reflect that it's gone way beyond its origins as a way of attaching metadata to a database.&lt;/p&gt;
&lt;p&gt;The &lt;a href="https://github.com/simonw/datasette/milestone/7"&gt;1.0 milestone&lt;/a&gt; is a dumping ground for many of these ideas. It's not a canonical reference though: I'd be very surprised if everything currently in that milestone makes it into the final 1.0 release.&lt;/p&gt;
&lt;p&gt;As I get closer to 1.0 though I'll be refining that milestone so it should get more accurate over time.&lt;/p&gt;
&lt;p&gt;Once again: &lt;strong&gt;now is the time&lt;/strong&gt; to be providing feedback on this stuff! The &lt;a href="https://datasette.io/discord"&gt;Datasette Discord&lt;/a&gt; is a particularly valuable way for me to get feedback on the work so far, and my plans for the future.&lt;/p&gt;
    
        &lt;p&gt;Tags: &lt;a href="https://simonwillison.net/tags/api"&gt;api&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/permissions"&gt;permissions&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/projects"&gt;projects&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/upsert"&gt;upsert&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/datasette"&gt;datasette&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/annotated-release-notes"&gt;annotated-release-notes&lt;/a&gt;&lt;/p&gt;
    

</summary><category term="api"/><category term="permissions"/><category term="projects"/><category term="upsert"/><category term="datasette"/><category term="annotated-release-notes"/></entry><entry><title>Weeknotes: Datasette internals</title><link href="https://simonwillison.net/2020/Dec/27/weeknotes-datasette-internals/#atom-tag" rel="alternate"/><published>2020-12-27T03:38:51+00:00</published><updated>2020-12-27T03:38:51+00:00</updated><id>https://simonwillison.net/2020/Dec/27/weeknotes-datasette-internals/#atom-tag</id><summary type="html">
    &lt;p&gt;I've been working on some fundamental changes to Datasette's internal workings - they're not quite ready for a release yet, but they're shaping up in an interesting direction.&lt;/p&gt;
&lt;p&gt;One of my goals for Datasette is to be able to handle a truly enormous variety of data in one place. The &lt;a href="https://github.com/simonw/datasette/issues/417"&gt;Datasette Library&lt;/a&gt; ticket tracks this effort - I'd like a newsroom (or any other information-based organization) to be able to keep hundreds of databases with potentially thousands of tables all in a single place.&lt;/p&gt;
&lt;p&gt;SQLite databases are just files on disk, so if you have a TB of assorted databases of all shapes and sizes I'd like to be able to present them in a single Datasette instance.&lt;/p&gt;
&lt;p&gt;If you have a hundred database files each with a hundred tables, that's 10,000 tables total. This implies a need for &lt;a href="https://github.com/simonw/datasette/issues/461"&gt;pagination of the homepage&lt;/a&gt;, plus the ability to search and filter within the tables that are available to the Datasette instance.&lt;/p&gt;
&lt;p&gt;Sounds like the kind of problem I'd normally solve with Datasette!&lt;/p&gt;
&lt;p&gt;So in &lt;a href="https://github.com/simonw/datasette/issues/1150"&gt;issue #1150&lt;/a&gt; I've implemented the first part of a solution. On startup, Datasette now creates an in-memory SQLite database representing all of the connected databases and tables, plus those tables' columns, indexes and foreign keys.&lt;/p&gt;
&lt;p&gt;For a demo, first &lt;a href="https://latest.datasette.io/login-as-root"&gt;sign in as root&lt;/a&gt; to the &lt;code&gt;latest.datasette.io&lt;/code&gt; demo instance and then visit the private &lt;a href="https://latest.datasette.io/_internal"&gt;_internal database&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;img src="https://static.simonwillison.net/static/2020/_internal__columns__132_rows.png" style="max-width:100%;" alt="Screenshot of the columns table, which has columns describing the columns that make up all of the tables." /&gt;&lt;/p&gt;
&lt;p&gt;This new internal database is currently private because I don't want to expose any metadata about tables that may themselves be covered by Datasette's permissions mechanism.&lt;/p&gt;
&lt;p&gt;The new in-memory database represents the schemas of the underlying database files - but what if those change, as new tables are added or modified?&lt;/p&gt;
&lt;p&gt;SQLite has a neat trick to help with this: &lt;a href="https://www.sqlite.org/pragma.html#pragma_schema_version"&gt;PRAGMA schema_version&lt;/a&gt; returns an integer representing the current version of the schema, which changes any time a table is created or modified.&lt;/p&gt;
&lt;p&gt;This means I can cache the table schemas and only recalculate them if something has changed. Running &lt;code&gt;PRAGMA schema_version&lt;/code&gt; against a connection to a database is an extremely fast query.&lt;/p&gt;
&lt;p&gt;I first used this trick &lt;a href="https://github.com/simonw/datasette-graphql/issues/51"&gt;in datasette-graphql&lt;/a&gt; to cache the results of GraphQL schema introspection, so I'm confident it will work here too.&lt;/p&gt;
&lt;h4&gt;The problem with permissions&lt;/h4&gt;
&lt;p&gt;There's one really gnarly challenge I still need to solve here: permissions.&lt;/p&gt;
&lt;p&gt;Datasette's &lt;a href="https://docs.datasette.io/en/stable/authentication.html#permissions"&gt;permissions system&lt;/a&gt;, added &lt;a href="https://simonwillison.net/2020/Jun/12/annotated-release-notes/"&gt;in Datasette 0.44&lt;/a&gt; back in June, works against plugin hooks. Permissions plugins can answer questions along the lines of "is the current authenticated actor allowed to perform the action &lt;code&gt;view-table&lt;/code&gt; against table X?". Every time that question is asked, the plugins are queried via &lt;a href="https://docs.datasette.io/en/stable/plugin_hooks.html#permission-allowed-datasette-actor-action-resource"&gt;a plugin hook&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;When I designed the permissions system I forgot a crucial lesson I've learned about permissions systems before: at some point, you're going to need to answer the question "show me the list of all Xs that this actor has permission to act on".&lt;/p&gt;
&lt;p&gt;That time has now come. If I'm going to render a paginated homepage for Datasette listing 10,000+ tables, I need an efficient way to calculate the subset of those 10,000 tables that the current user is allowed to see.&lt;/p&gt;
&lt;p&gt;Looping through and calling that plugin hook 10,000 times isn't going to cut it.&lt;/p&gt;
&lt;p&gt;So I'm starting to rethink permissions a bit - I'm glad I've not hit Datasette 1.0 yet!&lt;/p&gt;
&lt;p&gt;In my favour is SQLite. Efficiently answering permissions questions in bulk, in a generic, customizable way, is a really hard problem. It's made a lot easier by the presence of a relational database.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://github.com/simonw/datasette/issues/1152"&gt;Issue #1152&lt;/a&gt; tracks some of my thinking on this. I have a hunch that the solution is going to involve a new plugin hook, potentially along the lines of "return a fragment of SQL that identifies databases or tables that this user can access". I can then run that SQL against the new &lt;code&gt;_internal&lt;/code&gt; database (potentially combined with SQL returned by other plugins answering the same hook, using an &lt;code&gt;AND&lt;/code&gt; or a &lt;code&gt;UNION&lt;/code&gt;) to construct a set of tables that the user can access.&lt;/p&gt;
&lt;p&gt;If I can do that, and then run the query against an in-memory SQLite database, I should be able to provide a paginated, filtered interface to 10,000+ tables on the homepage that easily fulfills my performance goals.&lt;/p&gt;
&lt;p&gt;I'd like to ship the new &lt;code&gt;_internal&lt;/code&gt; database in a release quite soon, so I may end up implementing a slower version of this first. It's definitely an interesting problem.&lt;/p&gt;
    
        &lt;p&gt;Tags: &lt;a href="https://simonwillison.net/tags/permissions"&gt;permissions&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/datasette"&gt;datasette&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/weeknotes"&gt;weeknotes&lt;/a&gt;&lt;/p&gt;
    

</summary><category term="permissions"/><category term="datasette"/><category term="weeknotes"/></entry><entry><title>Datasette 0.44: The annotated release notes</title><link href="https://simonwillison.net/2020/Jun/12/annotated-release-notes/#atom-tag" rel="alternate"/><published>2020-06-12T03:11:06+00:00</published><updated>2020-06-12T03:11:06+00:00</updated><id>https://simonwillison.net/2020/Jun/12/annotated-release-notes/#atom-tag</id><summary type="html">
    &lt;p&gt;I just released &lt;strong&gt;Datasette 0.44&lt;/strong&gt; to PyPI. With &lt;a href="https://github.com/simonw/datasette/compare/0.43...0.44"&gt;128 commits&lt;/a&gt; since 0.43 this is the biggest release in a long time - and likely the last major release of new features before Datasette 1.0.&lt;/p&gt;

&lt;p&gt;You can read the &lt;a href="https://datasette.readthedocs.io/en/latest/changelog.html#v0-44"&gt;full release notes here&lt;/a&gt;, but I've decided to try something a little different for this release and write up some annotations here on my blog.&lt;/p&gt;

&lt;blockquote&gt;&lt;h4&gt;Writable canned queries&lt;/h4&gt;&lt;p&gt;Datasette's &lt;a href="https://datasette.readthedocs.io/en/latest/sql_queries.html#canned-queries"&gt;Canned queries&lt;/a&gt; feature lets you define SQL queries in &lt;code&gt;metadata.json&lt;/code&gt; which can then be executed by users visiting a specific URL. &lt;a href="https://latest.datasette.io/fixtures/neighborhood_search"&gt;https://latest.datasette.io/fixtures/neighborhood_search&lt;/a&gt; for example.&lt;/p&gt;&lt;p&gt;Canned queries were previously restricted to &lt;code&gt;SELECT&lt;/code&gt;, but Datasette 0.44 introduces the ability for canned queries to execute &lt;code&gt;INSERT&lt;/code&gt; or &lt;code&gt;UPDATE&lt;/code&gt; queries as well, using the new &lt;code&gt;"write": true&lt;/code&gt; property (&lt;a href="https://github.com/simonw/datasette/issues/800"&gt;#800&lt;/a&gt;)&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I originally intended this to be the main feature in the release.&lt;/p&gt;

&lt;p&gt;Datasette 0.37 added the ability for plugins to &lt;a href="https://simonwillison.net/2020/Feb/26/weeknotes-datasette-writes/"&gt;write to the database&lt;/a&gt;. This marked a pretty huge philosophical shift for Datasette: from a read-only publishing system to a framework for building interactive SQL-driven applications. But you needed a plugin, such as my &lt;a href="https://github.com/simonw/datasette-upload-csvs"&gt;datasette-upload-csvs&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;I realized &lt;a href="https://github.com/simonw/datasette/issues/698"&gt;back in March&lt;/a&gt; that canned queries could provide a simple, sensible way to start adding write functionality to Datasette core. A query could be configured like this:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;{
  "databases": {
    "my-database": {
      "queries": {
        "add_twitter_handle": {
          "sql": "insert into twitter_handles (username) values (:username)",
          "write": true
        }
      }
    }
  }
}&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;And Datasette could then provide a form interface at &lt;code&gt;/my-database/add_twitter_handle&lt;/code&gt; for executing that query. How hard could that be to implement?&lt;/p&gt;

&lt;p&gt;The problem with "simple" features like this is that they open up a cascade of other features.&lt;/p&gt;

&lt;p&gt;If you're going to have write queries, you probably need to restrict who can execute them - which means authentication and permissions. If forms are performing POSTs you need CSRF protection. If users are changing state you need a way to show them messages telling them what happened.&lt;/p&gt;

&lt;p&gt;So let's talk about authentication and permissions.&lt;/p&gt;

&lt;blockquote&gt;&lt;h4&gt;Authentication&lt;/h4&gt;&lt;p&gt;Prior to this release the Datasette ecosystem has treated authentication as exclusively the realm of plugins, most notably through &lt;a href="https://github.com/simonw/datasette-auth-github"&gt;datasette-auth-github&lt;/a&gt;.&lt;/p&gt;&lt;p&gt;0.44 introduces &lt;a href="https://datasette.readthedocs.io/en/latest/authentication.html#authentication"&gt;Authentication and permissions&lt;/a&gt; as core Datasette concepts (&lt;a href="https://github.com/simonw/datasette/issues/699"&gt;#699&lt;/a&gt;). This makes it easier for different plugins can share responsibility for authenticating requests - you might have one plugin that handles user accounts and another one that allows automated access via API keys, for example.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I demonstrated with &lt;a href="https://github.com/simonw/datasette-auth-github"&gt;datasette-auth-github&lt;/a&gt; and &lt;a href="https://github.com/simonw/datasette-auth-existing-cookies"&gt;datasette-auth-existing-cookies&lt;/a&gt; that authentication could exist as completely separate layer from Datasette call (thanks to the magic of ASGI middleware). But I'd started to run into limitations of this approach.&lt;/p&gt;

&lt;p&gt;Crucially, I wanted to be able to support more than one kind of authentication. Users might get authenticated with cookies (via a SSO mechanism such as GitHub's) but API clients need API keys. Now the different authentication plugins need to make sure they don't accidentally intefere with each other's logic.&lt;/p&gt;

&lt;p&gt;Authentication in web applications always comes down to the same thing: inspecting aspects of the incoming HTTP request (headers, cookies, querystring variables etc) and deciding if they prove that the request is coming from a specific user or API integration.&lt;/p&gt;

&lt;p&gt;I decided to use the word "actor" for this, since "user or API integration" is a bit of a mess. This means that authentication can work entirely via a new plugin hook, &lt;a href="https://datasette.readthedocs.io/en/stable/plugins.html#actor-from-request-datasette-request"&gt;actor_from_request&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Here's a really simple implementation of that hook, copied directly from the documentation:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;from datasette import hookimpl
import secrets

SECRET_KEY = "this-is-a-secret"

@hookimpl
def actor_from_request(datasette, request):
    authorization = request.headers.get("authorization") or ""
    expected = "Bearer {}".format(SECRET_KEY)

    if secrets.compare_digest(authorization, expected):
        return {"id": "bot"}&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;It returns an "actor dictionary" describing the authenticated actor. Datasette currently has no opinion at all on what shape this dictionary should take, though I expect conventions to emerge over time.&lt;/p&gt;

&lt;p&gt;I have &lt;a href="https://github.com/simonw/datasette/commit/9f236c4c00689a022fd1d508f2b809ee2305927f"&gt;a new policy&lt;/a&gt; of never releasing a new plugin hook without also building a real-world plugin with it. For &lt;code&gt;actor_from_request&lt;/code&gt; I've released &lt;a href="https://github.com/simonw/datasette-auth-tokens"&gt;datasette-auth-tokens&lt;/a&gt;, which lets you create secret API tokens in a configuration file and specify which actions they are allowed to perfom.&lt;/p&gt;

&lt;blockquote&gt;&lt;h4&gt;Permissions&lt;/h4&gt;&lt;p&gt;Datasette also now has a built-in concept of &lt;a href="https://datasette.readthedocs.io/en/stable/authentication.html#authentication-permissions"&gt;Permissions&lt;/a&gt;. The permissions system answers the following question:&lt;/p&gt;&lt;blockquote&gt;&lt;p&gt;Is this &lt;strong&gt;actor&lt;/strong&gt; allowed to perform this &lt;strong&gt;action&lt;/strong&gt;, optionally against this particular &lt;strong&gt;resource&lt;/strong&gt;?&lt;/p&gt;&lt;/blockquote&gt;&lt;p&gt;You can use the new &lt;code&gt;"allow"&lt;/code&gt; block syntax in &lt;code&gt;metadata.json&lt;/code&gt; (or &lt;code&gt;metadata.yaml&lt;/code&gt;) to set required permissions at the instance, database, table or canned query level. For example, to restrict access to the &lt;code&gt;fixtures.db&lt;/code&gt; database to the &lt;code&gt;"root"&lt;/code&gt; user:&lt;/p&gt;&lt;pre&gt;&lt;code&gt;{
    "databases": {
        "fixtures": {
            "allow": {
                "id" "root"
            }
        }
    }
}&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;See &lt;a href="https://datasette.readthedocs.io/en/stable/authentication.html#authentication-permissions-allow"&gt;Defining permissions with "allow" blocks&lt;/a&gt; for more details.&lt;/p&gt;&lt;p&gt;Plugins can implement their own custom permission checks using the new &lt;a href="https://datasette.readthedocs.io/en/stable/plugins.html#plugin-hook-permission-allowed"&gt;permission_allowed(datasette, actor, action, resource)&lt;/a&gt; hook.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Authentication on its own isn't enough: you also need a way of deciding if an authenticated actor has permission to perform a specific action.&lt;/p&gt;

&lt;p&gt;I was dreading adding permissions to Datasette. I have a long-running feud with Amazon IAM and the Google Cloud equivalent: I've been using AWS for over a decade and I still get completely lost any time I try to figure out the minimum set of permissions for something.&lt;/p&gt;

&lt;p&gt;But... I want Datasette permissions to be flexible. My dream for Datasette is to nurture a growing ecosystem of plugins that can solve data collaboration and analysis problems way beyond what I've imagined myself.&lt;/p&gt;

&lt;p&gt;Thanks to Datasette's plugin hooks, I think I've found a way to provide powerful plugins with minimum additional footprint to Datasette itself.&lt;/p&gt;

&lt;p&gt;The key is the new &lt;a href="https://datasette.readthedocs.io/en/stable/plugins.html#plugin-hook-permission-allowed"&gt;permission_allowed()&lt;/a&gt; hook, which lets plugins receive an actor, action and optional resource and allows them to reply with "allow", "deny" or "I don't know, ask someone else".&lt;/p&gt;

&lt;p&gt;Its partner is the &lt;a href="https://datasette.readthedocs.io/en/stable/internals.html#await-permission-allowed-actor-action-resource-none-default-false"&gt;datasette.permission_allowed(actor, action, resource)&lt;/a&gt; method, which plugins (and Datasette core) can call to check if the current actor is allowed to perform an action against a given resource (a specific database, table or canned query).&lt;/p&gt;

&lt;p&gt;I invented a JSON/YAML based syntax for defining simple permission rules. If you want to provide access to a table to a specific user you can do so like this:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;{
    "allow": {
        "id": "simonw"
    }
}&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Or you can grant access to any actor with a role of "staff" like so:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;{
    "allow": {
        "role": "staff"
    }
}&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;What are roles here? They're nothing at all to Datasette itself. Datasette authentication plugins can create actors of any shape, so if your plugin decides that "role" is a useful concept it can just bake it into the Datasette.&lt;/p&gt;

&lt;p&gt;You can read more about &lt;a href="https://datasette.readthedocs.io/en/stable/authentication.html#defining-permissions-with-allow-blocks"&gt;how these "allow" blocks work&lt;/a&gt; in the documentation.&lt;/p&gt;

&lt;h4&gt;/-/permissions debug tool&lt;/h4&gt;

&lt;p&gt;Given my ongoing battle with opaque permission systems, I'm determined to try and make Datasette's take on permissions as transparent as possible. The new &lt;code&gt;/-/permissions&lt;/code&gt; page - visible only to authenticated users with the &lt;code&gt;debug-permissions&lt;/code&gt; permission - shows a rolling log of the last 200 permission checks carried out by that instance. My hope is that instance administrators and plugin authors can use this to figure out exactly what is going on.&lt;/p&gt;

&lt;p&gt;&lt;img src="https://static.simonwillison.net/static/2020/Debug_permissions.png" alt="/-/permissions screenshot" style="max-width: 100%" /&gt;&lt;/p&gt;

&lt;h4&gt;datasette-permissions-sql&lt;/h4&gt;

&lt;p&gt;&lt;a href="https://github.com/simonw/datasette-permissions-sql"&gt;datasette-permissions-sql&lt;/a&gt; is my new proof-of-concept plugin that puts the permission hook to use.&lt;/p&gt;

&lt;p&gt;It exercises a Datasette pattern I find really interesting: using SQL queries to configure custom behaviour. I first started eploring this in &lt;a href="https://github.com/simonw/datasette-atom"&gt;datasette-atom&lt;/a&gt; and &lt;a href="https://github.com/simonw/datasette-ics"&gt;datasette-ics&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;This is best illustrated by an example &lt;code&gt;metadata.yaml&lt;/code&gt; file. I prefer YAML over JSON for anything that includes a SQL query because YAML has support for multi-line strings:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;databases:
  mydatabase:
    queries:
      promote_to_staff:
        sql: |-
          UPDATE users
          SET is is_staff=1
          WHERE id=:id
        write: true
plugins:
  datasette-permissions-sql:
  - action: view-query
    resource:
    - mydatabase
    - promote_to_staff
    sql: |-
      SELECT * FROM users
      WHERE is_staff = 1
      AND id = :actor_id&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This block does two things. It configures a writable canned query called &lt;code&gt;promote_to_staff&lt;/code&gt;. It then uses &lt;code&gt;datasette-permissions-sql&lt;/code&gt; to define a permission rule that says that only authenticated actors who's &lt;code&gt;id&lt;/code&gt; appears in the &lt;code&gt;users&lt;/code&gt; table with &lt;code&gt;is_staff=1&lt;/code&gt; are allewod to execute that canned query&lt;/p&gt;

&lt;p&gt;This is the beginnings of a full user management system in just a few lines of configuration. I'm really excited about exploring this concept further.&lt;/p&gt;

&lt;blockquote&gt;&lt;h4&gt;register_routes() plugin hooks&lt;/h4&gt;&lt;p&gt;Plugins can now register new views and routes via the &lt;a href="https://datasette.readthedocs.io/en/stable/plugins.html#plugin-register-routes"&gt;register_routes()&lt;/a&gt; plugin hook (&lt;a href="https://github.com/simonw/datasette/issues/819"&gt;#819&lt;/a&gt;). View functions can be defined that accept any of the current &lt;code&gt;datasette&lt;/code&gt; object, the current &lt;code&gt;request&lt;/code&gt;, or the ASGI &lt;code&gt;scope&lt;/code&gt;, &lt;code&gt;send&lt;/code&gt; and &lt;code&gt;receive&lt;/code&gt; objects.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I thought the &lt;a href="https://datasette.readthedocs.io/en/stable/plugins.html#asgi-wrapper-datasette"&gt;asgi_wrapper() hook&lt;/a&gt; might be enough to allow plugins to add their own custom routes and views, but the more I worked with it the more I wanted something a bit more high level.&lt;/p&gt;

&lt;p&gt;Inspired by &lt;a href="https://docs.pytest.org/ "&gt;pytest&lt;/a&gt;, the view functions you can define using &lt;code&gt;register_routes()&lt;/code&gt; benefit from a simple form of dependency injection. Any of the following counts as a valid view function - it will be called with the arguments it requests:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;async def hello(request):
    return Response.html("Hello {}".format(
        html.escape(request.args.get("name"))
    ))

async def page(request, datasette):
    page = await datasette.get_databse().execute(
        "select body from pages where id = :id", {
            "id: request.url_vars["page_id"]
        }
    ).first()
    return Response.html(body)

async def asgi_hello_world(scope, receive, send):
    assert scope["type"] == "http"
    await send(
        {
            "type": "http.response.start",
            "status": 200,
            "headers": [
                [b"content-type", b"application/json"]
            ],
        }
    )
    await send({
        "type": "http.response.body",
        "body": b'{"hello": "world"}'
    })&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Here's &lt;a href="https://github.com/simonw/datasette/blob/b906030235efbdff536405d66078f4868ce0d3bd/datasette/utils/__init__.py#L850-L871"&gt;the code that makes this work&lt;/a&gt; - utility functions that pass in just the arguments that match a function's signature.&lt;/p&gt;

&lt;p&gt;Tucked away at the end of the 0.44 release notes is this:&lt;/p&gt;

&lt;blockquote&gt;&lt;h4&gt;The road to Datasette 1.0&lt;/h4&gt;&lt;p&gt;I've assembled a &lt;a href="https://github.com/simonw/datasette/milestone/7"&gt;milestone for Datasette 1.0&lt;/a&gt;. The focus of the 1.0 release will be the following:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Signify confidence in the quality/stability of Datasette&lt;/li&gt;&lt;li&gt;Give plugin authors confidence that their plugins will work for the whole 1.x release cycle&lt;/li&gt;&lt;li&gt;Provide the same confidence to developers building against Datasette JSON APIs&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;If you have thoughts about what you would like to see for Datasette 1.0 you can join &lt;a href="https://github.com/simonw/datasette/issues/519"&gt;the conversation on issue #519&lt;/a&gt;.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;It's time to start working towards the big 1.0!&lt;/p&gt;
    
        &lt;p&gt;Tags: &lt;a href="https://simonwillison.net/tags/authentication"&gt;authentication&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/permissions"&gt;permissions&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/plugins"&gt;plugins&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/projects"&gt;projects&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/releasenotes"&gt;releasenotes&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/datasette"&gt;datasette&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/weeknotes"&gt;weeknotes&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/annotated-release-notes"&gt;annotated-release-notes&lt;/a&gt;&lt;/p&gt;
    

</summary><category term="authentication"/><category term="permissions"/><category term="plugins"/><category term="projects"/><category term="releasenotes"/><category term="datasette"/><category term="weeknotes"/><category term="annotated-release-notes"/></entry><entry><title>Advice on specifying more granular permissions with Google Cloud IAM</title><link href="https://simonwillison.net/2020/May/28/advice-specifying-more-granular-permissions-google-cloud-iam/#atom-tag" rel="alternate"/><published>2020-05-28T22:44:24+00:00</published><updated>2020-05-28T22:44:24+00:00</updated><id>https://simonwillison.net/2020/May/28/advice-specifying-more-granular-permissions-google-cloud-iam/#atom-tag</id><summary type="html">
    
&lt;p&gt;&lt;strong&gt;&lt;a href="https://dev.to/googlecloud/advice-on-specifying-more-granular-permissions-with-google-cloud-iam-4b70"&gt;Advice on specifying more granular permissions with Google Cloud IAM&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;
My single biggest frustration working with both Google Cloud and AWS is permissions: more specifically, figuring out what the smallest set of permissions are that I need to assign in order to achieve different goals. Katie McLaughlin’s new series aims to address exactly that problem. I learned a ton from this that I’ve previously missed, and there’s plenty of actionable advice on tooling that can be used to help figure this stuff out.

    &lt;p&gt;&lt;small&gt;&lt;/small&gt;Via &lt;a href="https://twitter.com/glasnt/status/1266123072580358144"&gt;Katie McLaughlin&lt;/a&gt;&lt;/small&gt;&lt;/p&gt;


    &lt;p&gt;Tags: &lt;a href="https://simonwillison.net/tags/permissions"&gt;permissions&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/cloudrun"&gt;cloudrun&lt;/a&gt;&lt;/p&gt;



</summary><category term="permissions"/><category term="cloudrun"/></entry><entry><title>grant XXX on * ?</title><link href="https://simonwillison.net/2010/Mar/16/grant/#atom-tag" rel="alternate"/><published>2010-03-16T18:26:30+00:00</published><updated>2010-03-16T18:26:30+00:00</updated><id>https://simonwillison.net/2010/Mar/16/grant/#atom-tag</id><summary type="html">
    
&lt;p&gt;&lt;strong&gt;&lt;a href="http://www.depesz.com/index.php/2007/10/19/grantall/"&gt;grant XXX on * ?&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;
PostgreSQL doesn’t have a way to say “this user is allowed to select/update/etc on all tables in database X”. That kind of sucks. UPDATE: This is fixed in PostgreSQL 9, see the comments.

    &lt;p&gt;&lt;small&gt;&lt;/small&gt;Via &lt;a href="http://serverfault.com/questions/60508/grant-select-to-all-tables-in-postgresql"&gt;GRANT SELECT to all tables in postgresql - Server Fault&lt;/a&gt;&lt;/small&gt;&lt;/p&gt;


    &lt;p&gt;Tags: &lt;a href="https://simonwillison.net/tags/databases"&gt;databases&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/grant"&gt;grant&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/permissions"&gt;permissions&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/postgresql"&gt;postgresql&lt;/a&gt;, &lt;a href="https://simonwillison.net/tags/sql"&gt;sql&lt;/a&gt;&lt;/p&gt;



</summary><category term="databases"/><category term="grant"/><category term="permissions"/><category term="postgresql"/><category term="sql"/></entry></feed>