Simon Willison's Weblog: rye

Quoting Armin Ronacher

2024-08-21T12:08:15+00:00

There is an elephant in the room which is that Astral is a VC funded company. What does that mean for the future of these tools? Here is my take on this: for the community having someone pour money into it can create some challenges. For the PSF and the core Python project this is something that should be considered. However having seen the code and what uv is doing, even in the worst possible future this is a very forkable and maintainable thing. I believe that even in case Astral shuts down or were to do something incredibly dodgy licensing wise, the community would be better off than before uv existed.

— Armin Ronacher

Tags: armin-ronacher, open-source, python, rye, uv, astral

uv: Unified Python packaging

2024-08-20T22:45:16+00:00

uv: Unified Python packaging

Huge new release from the Astral team today. uv 0.3.0 adds a bewildering array of new features, as part of their attempt to build "Cargo, for Python".

It's going to take a while to fully absorb all of this. Some of the key new features are:

uv tool run cowsay, aliased to uvx cowsay - a pipx alternative that runs a tool in its own dedicated virtual environment (tucked away in ~/Library/Caches/uv), installing it if it's not present. It has a neat --with option for installing extras - I tried that just now with uvx --with datasette-cluster-map datasette and it ran Datasette with the datasette-cluster-map plugin installed.
Project management, as an alternative to tools like Poetry and PDM. uv init creates a pyproject.toml file in the current directory, uv add sqlite-utils then creates and activates a .venv virtual environment, adds the package to that pyproject.toml and adds all of its dependencies to a new uv.lock file (like this one). That uv.lock is described as a universal or cross-platform lockfile that can support locking dependencies for multiple platforms.
Single-file script execution using uv run myscript.py, where those scripts can define their own dependencies using PEP 723 inline metadata. These dependencies are listed in a specially formatted comment and will be installed into a virtual environment before the script is executed.
Python version management similar to pyenv. The new uv python list command lists all Python versions available on your system (including detecting various system and Homebrew installations), and uv python install 3.13 can then install a uv-managed Python using Gregory Szorc's invaluable python-build-standalone releases.

It's all accompanied by new and very thorough documentation.

The paint isn't even dry on this stuff - it's only been out for a few hours - but this feels very promising to me. The idea that you can install uv (a single Rust binary) and then start running all of these commands to manage Python installations and their dependencies is very appealing.

If you’re wondering about the relationship between this and Rye - another project that Astral adopted solving a subset of these problems - this forum thread clarifies that they intend to continue maintaining Rye but are eager for uv to work as a full replacement.

Via @charliermarsh

Tags: packaging, python, rust, rye, uv, astral

uv: Python packaging in Rust

2024-02-15T19:57:13+00:00

uv: Python packaging in Rust

"uv is an extremely fast Python package installer and resolver, written in Rust, and designed as a drop-in replacement for pip and pip-tools workflows."

From Charlie Marsh and Astral, the team behind Ruff, who describe it as a milestone in their pursuit of a "Cargo for Python".

Also in this announcement: Astral are taking over stewardship of Armin Ronacher's Rye packaging tool, another Rust project.

uv is reported to be 8-10x faster than regular pip, increasing to 80-115x faster with a warm global module cache thanks to copy-on-write and hard links on supported filesystems - which saves on disk space too.

It also has a --resolution=lowest option for installing the lowest available version of dependencies - extremely useful for testing, I've been wanting this for my own projects for a while.

Also included: uv venv - a fast tool for creating new virtual environments with no dependency on Python itself.

Via @charliermarsh

Tags: armin-ronacher, pip, python, rust, rye, ruff, uv, astral, charlie-marsh

Rye: Added support for marking virtualenvs ignored for cloud sync

2024-02-10T06:50:00+00:00

Rye: Added support for marking virtualenvs ignored for cloud sync

A neat feature in the new Rye 0.22.0 release. It works by using an xattr Rust crate to set the attributes “com.dropbox.ignored” and “com.apple.fileprovider.ignore#P” on the folder.

Via Rye 0.22.0 release notes

Tags: python, dropbox, rust, rye

Quoting Armin Ronacher

2024-02-04T15:12:06+00:00

Rye lets you get from no Python on a computer to a fully functioning Python project in under a minute with linting, formatting and everything in place.

[...] Because it was demonstrably designed to avoid interference with any pre-existing Python configurations, Rye allows for a smooth and gradual integration and the emotional barrier of picking it up even for people who use other tools was shown to be low.

— Armin Ronacher

Tags: armin-ronacher, python, rye

Weeknotes: Miscellaneous research into Rye, ChatGPT Code Interpreter and openai-to-sqlite

2023-05-01T05:12:09+00:00

I gave myself some time off stressing about my core responsibilities this week after PyCon, which meant allowing myself to be distracted by some miscellaneous research projects.

Rye

Rye is a new experimental twist on Python packaging from Armin Ronacher. He's been quite apologetic about it, asking Should Rye Exist? - Python packaging is a crowded space right now!

Personally, I think a working prototype of an interesting idea is always worthwhile. My experience is that running code increases the quality of the discussion around an idea enormously, because it gives people something concrete to talk about.

Rye has some really interesting ideas. By far my favourite is how it bundles Python itself: it doesn't depend on a system Python, instead downloading a standalone Python build from the python-build-standalone project and stashing it away in a ~/.rye directory.

I love this. Getting Python running on a system is often way harder than it should be. Rye provides a single binary (written in Rust) which can bootstrap a working Python environment, without interfering with the system Python or any other Python environments that might already be installed.

I wrote up a few notes on Rye in a TIL earlier this week, mainly detailing how it works and where it puts things.

I also released Datasette 0.64.3 with a tiny fix to ensure it would install cleanly using rye install datasette.

ChatGPT Code Interpreter

I've been having a whole lot of fun exploring this. I wrote about how I've been using it to run micro-benchmarks a few weeks ago - today I figured out a pattern for installing additional Python packages (despite its lack of an internet connection) and even uploading binaries for Deno and Lua to grant it the ability to run code in other languages!

I think it's the most interesting thing in all of ChatGPT/LLM world at the moment, which is a big statement.

openai-to-sqlite

Inspired by a Datasette Office Hours conversation on Friday I decided to see if I could figure out a way to run simple sentiment analysis against data in a SQLite database using any of my various tools.

I ended up adding a new mechanism to my openai-to-sqlite CLI tool - it can now execute SQL queries that can update existing tables with the results of a chatgpt() API call using a custom SQL function.

I wrote more about that in Enriching data with GPT3.5 and SQLite SQL functions.

Upgraded social media cards for my TILs

My Today I Learned site has had social media cards - images that show up in link previews when URLs are shared - for a long time now. Since few of my TILs have images of their own it generates these as screenshots of the pages themselves.

Until recently it stored these images as PNG files directly in the SQLite database itself. Vercel has a 50MB size limit on deployments and the other day the screenshots finally tipped the database over that limit.

To fix it, I moved the images out of the SQLite database and put them in an S3 bucket instead. This also meant I could increase their size and resolution - they are now generated with the shot-scraper --retina option which doubles their size to 1600x800 pixels.

This ended up being a fun exercise in combining both shot-scraper and my s3-credentials CLI tools. I wrote up full details of how the new screenshot system works in a new TIL, Social media cards generated with shot-scraper.

Next week: a webinar on Prompt Injection

My other blog entry this week introduced The Dual LLM pattern for building AI assistants that can resist prompt injection - my latest thinking on how we might be able to build AI assistants even without a robust solution to the prompt injection problem.

I have a speaking engagement lined up for next week: the LangChain Prompt Injection Webinar.

I'll be discussing prompt injection attacks against LLMs on a panel with Willem Pienaar, Kojin Oshiba and Jonathan Cohen and Christopher Parisien from NVIDIA.

I think it will be an interesting conversation. I'm going to reiterate my argument that You can’t solve AI security problems with more AI - a position that I'm not sure is shared by the other members of the panel!

Entries this week

Releases this week

s3-credentials 0.15 - 2023-04-30
A tool for creating credentials for accessing S3 buckets
openai-to-sqlite 0.3 - 2023-04-29
Save OpenAI API results to a SQLite database
datasette 0.64.3 - 2023-04-27
An open source multi-tool for exploring and publishing data
shot-scraper 1.2 - 2023-04-27
A command-line utility for taking automated screenshots of websites
datasette-explain 0.1a2 - 2023-04-24
Explain and validate SQL queries as you type them into Datasette

TIL this week

Expanding ChatGPT Code Interpreter with Python packages, Deno and Lua - 2023-05-01
Social media cards generated with shot-scraper - 2023-04-30
Deno KV - 2023-04-28
The location of the pip cache directory - 2023-04-28
A few notes on Rye - 2023-04-27

Tags: projects, weeknotes, prompt-injection, chatgpt, rye, code-interpreter, coding-agents

Rye

2023-04-24T04:02:42+00:00

Rye

Armin Ronacher's take on a Python packaging tool. There are a lot of interesting ideas in this one - it's written in Rust, configured using pyproject.toml and has some very strong opinions, including completely hiding pip from view and insisting you use rye add package instead. Notably, it doesn't use the system Python at all: instead, it downloads a pre-compiled standalone Python from Gregory Szorc's python-build-standalone project - the same approach I used for the Datasette Desktop Electron app.

Armin warns that this is just an exploration, with no guarantees of future maintenance - and even has an issue open titled Should Rye exist?

Tags: armin-ronacher, packaging, pypi, python, rust, rye