Simon Willison's Weblog: elections

Follow the Crypto

2024-07-15T22:06:29+00:00

Very smart new site from Molly White tracking the huge increase in activity from Cryptocurrency-focused PACs this year. These PACs have already raised $203 million and spent $38 million influencing US elections in 2024.

Right now Molly's rankings show that the "Fairshake" cryptocurrency PAC is second only to the Trump-supporting "Make America Great Again Inc" in money raised by Super PACs this year - though it's 9th in the list that includes other types of PAC.

Molly's data comes from the FEC, and the code behind the site is all open source.

There's lots more about the project in the latest edition of Molly's newsletter:

Did you know that the cryptocurrency industry has spent more on 2024 elections in the United States than the oil industry? More than the pharmaceutical industry?

In fact, the cryptocurrency industry has spent more on 2024 elections than the entire energy sector and the entire health sector. Those industries, both worth hundreds of billions or trillions of dollars, are being outspent by an industry that, even by generous estimates, is worth less than $20 billion.

Via Citation Needed

Tags: data-journalism, elections, politics, blockchain, molly-white

UK Parliament election results, now with Datasette

2024-07-05T23:36:18+00:00

UK Parliament election results, now with Datasette

The House of Commons Library maintains a website of UK parliamentary election results data, currently listing 2010 through 2019 and with 2024 results coming soon.

The site itself is a Rails and PostgreSQL app, but I was delighted to learn today that they're also running a Datasette instance with the election results data, linked to from their homepage!

The raw data is also available as CSV files in their GitHub repository. Here's their Datasette configuration, which includes a copy of their SQLite database.

Tags: elections, sqlite, datasette

Weeknotes: sqlite-utils 3.0 alpha, Git scraping in the zeitgeist

2020-11-07T02:17:55+00:00

Natalie and I decided to escape San Francisco for election week, and have been holed up in Fort Bragg on the Northern California coast. I've mostly been on vacation, but I did find time to make some significant changes to sqlite-utils. Plus notes on an exciting Git scraping project.

Better search in the sqlite-utils 3.0 alpha

I practice semantic versioning with sqlite-utils, which means it only gets a major version bump if I break backwards compatibility in some way.

My goal is to avoid breaking backwards compatibility as much as possible, and I was proud to have made it all the way to version 2.23 representing 23 new feature releases since the 2.0 release without breaking any documented features!

Sadly this run has come to an end: I realized that the table.search() method was poorly designed, and I also needed to grab back the -c command-line option (a shortcut for --csv output) to be used for another purpose.

The chances that either of these changes will break anyone are pretty small, but semantic versioning dictates a major version bump so here we are.

I shipped a 3.0 alpha today, which should hopefully become a stable release very shortly (milestone here).

The big new feature is sqlite-utils search - a command-line tool for executing searches against a full-text search enabled table:

$ sqlite-utils search 24ways-fts4.db articles maps -c title
[{"rowid": 163, "title": "Get To Grips with Slippy Maps", "rank": -10.028754920576421},
 {"rowid": 220, "title": "Finding Your Way with Static Maps", "rank": -9.952534352591737},
 {"rowid": 27, "title": "Putting Design on the Map", "rank": -5.667327088267961},
 {"rowid": 168, "title": "Unobtrusively Mapping Microformats with jQuery", "rank": -4.662224207228984},

Here's full documentation for the new command.

Notably, this command works against both FTS4 and FTS5 tables in SQLite - despite FTS4 not shipping with a built-in ranking function. I'm using my sqlite-fts4 package for this, which I described back in January 2019 in Exploring search relevance algorithms with SQLite.

Git scraping to predict the election

It's not quite over yet but the end is in sight, and one of the best tools to track the late arriving vote counts is this Election 2020 results site built by Alex Gaynor and a growing cohort of contributors.

The site is a beautiful example of Git scraping in action, and I'm thrilled that it links to my article in the README!

Take a look at the repo to see how it works. Short version: this GitHub Action workflow grabs the latest snapshot of this undocumented New York Times JSON API once every five minutes and commits it to the repository. It then runs this Python script which iterates through the Git history and generates an HTML summary showing the different batches of new votes that were reported and their impact on the overall race.

The resulting report is published to GitHub pages - resulting in a site that can handle a great deal of traffic and is updated entirely by code running in scheduled actions.

This is a perfect use-case for Git scraping: it takes a JSON endpoint that represents the current state of the world and turns it into a sequence of historic snapshots, then uses those snapshots to build a unique and useful new source of information to help people understand what's going on.

Releases this week

sqlite-utils 3.0a0 - 2020-11-07
sqlite-fts4 1.0.1 - 2020-11-06
sqlite-fts4 1.0 - 2020-11-06
csvs-to-sqlite 1.2 - 2020-11-03
datasette 0.51.1 - 2020-11-01

Tags: alex-gaynor, elections, weeknotes, git-scraping, sqlite-utils

nyt-2020-election-scraper

2020-11-06T14:24:36+00:00

nyt-2020-election-scraper

Brilliant application of git scraping by Alex Gaynor and a growing team of contributors. Takes a JSON snapshot of the NYT’s latest election poll figures every five minutes, then runs a Python script to iterate through the history and build an HTML page showing the trends, including what percentage of the remaining votes each candidate needs to win each state. This is the perfect case study in why it can be useful to take a “snapshot if the world right now” data source and turn it into a git revision history over time.

Tags: alex-gaynor, data-journalism, elections, git, new-york-times, git-scraping

Dealing with election results data

2009-06-12T18:06:05+00:00

Dealing with election results data

Alf Eaton loaded the Guardian’s European election results spreadsheet in to Google’s new Fusion Tables tool.

Tags: alf-eaton, datablog, datastore, elections, fusiontables, google, guardian

Exactly how well did the BNP do where you live?

2009-06-11T11:37:56+00:00

Exactly how well did the BNP do where you live?

Guardian journalists spent a day and a half calling round different local authorities to get a proper breakdown of the European election results (which are only officially published in aggregate) and published the results as a spreadsheet on the Datablog.

Tags: bnp, datablog, datastore, elections, guardian

Obama v McCain - battleground graph

2008-11-03T20:40:26+00:00

Obama v McCain - battleground graph

Paul Crowley provides the smartest election visualisation I’ve seen this cycle, using the current projections from fivethirtyeight.com and with a promise of a frequently updated version as the actual results roll in.

Via LShift Ltd.

Tags: elections, graph, paul-crowley, visualisation

ORG verdict on London Elections: "Insufficient evidence" to declare confidence in results

2008-07-02T10:36:26+00:00

ORG verdict on London Elections: "Insufficient evidence" to declare confidence in results

Electronic voting strikes again. Also of interest: the audit conducted by KPMG can’t be published due to “commercial confidentiality”.

Tags: audit, elections, electronicvoting, kpmg, london, openrightsgroup, org

LJWorld.com: Kansas Democratic Presidential Caucuses

2008-02-08T11:17:45+00:00

LJWorld.com: Kansas Democratic Presidential Caucuses

The most beautiful election results page I’ve ever seen. Love the typography and the Google Charts integration.

Via Matt Croydon

Tags: design, elections, google-charts, kansas, ljworld, matt-croydon, typography

Open Rights Group: Our first two years

2007-11-25T22:05:06+00:00

Open Rights Group: Our first two years

ORG’s review of the past two years shows just how worthwhile a cause they have become—highlights include their hugely successful campaign against copyright term extension and their involvement in this year’s e-voting trials.

Tags: copyright, digitalrights, elections, evoting, openrightsgroup, org

Election endorsements

2004-10-31T19:44:19+00:00

My ex-colleague Jacob Kaplan-Moss has put together a fantastic site listing the presidential endorsements published by American newspapers in the run up to the election. I was looking for something like this just the other day so it was great to find the answer so close to home. I was depressed but not at all surprised to see my former employer endorse Bush, but it's interesting to see that of the four Kansan papers listed two endorsed Kerry, despite that state's huge Republican majority.

Tags: elections, jacob-kaplan-moss, kansas