http://simonwillison.net/2018-09-25T23:02:57+00:00Simon Willison2018-09-25T23:02:57+00:002018-09-25T23:02:57+00:00https://simonwillison.net/2018/Sep/25/dropbox/#atom-tag

<p><strong><a href="https://blogs.dropbox.com/tech/2018/09/how-we-rolled-out-one-of-the-largest-python-3-migrations-ever/">How we rolled out one of the largest Python 3 migrations ever</a></strong></p> “If you’re using Dropbox today, the application is powered by a Dropbox-customized variant of Python 3.5” <p>Tags: <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a>, <a href="https://simonwillison.net/tags/dropbox">dropbox</a></p>

2018-03-01T16:06:12+00:002018-03-01T16:06:12+00:00https://simonwillison.net/2018/Mar/1/wagtail/#atom-tag

<p><strong><a href="https://wagtail.io/blog/wagtail-2/">Wagtail 2.0</a></strong></p> The leading Django content management system just released it’s 2.0 version—now Django 2.0 and Python 3 only, and with a new rich text editor based on Draft.js. I really like Wagtail—it’s full-feature,d mature, well-documented and philosophically aligned with how I think a CMS should work. I also like this line about the new Python 3 requirement: “Call us reckless neophiles, but we think that, nine years in, Python 3 is looking pretty solid.” <p>Tags: <a href="https://simonwillison.net/tags/cms">cms</a>, <a href="https://simonwillison.net/tags/django">django</a>, <a href="https://simonwillison.net/tags/python3">python3</a>, <a href="https://simonwillison.net/tags/wagtail">wagtail</a></p>

2018-02-20T14:39:26+00:002018-02-20T14:39:26+00:00https://simonwillison.net/2018/Feb/20/moving-a-large-and-old-codebase-to-python3/#atom-tag

<p><strong><a href="https://medium.com/@boxed/moving-a-large-and-old-codebase-to-python3-33a5a13f8c99">Moving a large and old codebase to Python3</a></strong></p> Really interesting case study full of good ideas. The codebase in this case was 240,000 lines of Python and Django written over the course of 15 years. The team used Python-Modernize to aid their transition to a six-compatible codebase first. <p><small></small>Via <a href="https://www.reddit.com/r/programming/comments/7yuc30/moving_a_large_and_old_codebase_to_python3/">r/programming</a></small></p> <p>Tags: <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a></p>

2018-02-20T00:30:03+00:002018-02-20T00:30:03+00:00https://simonwillison.net/2018/Feb/20/async-simplified/#atom-tag

<p><strong><a href="https://www.aeracode.org/2018/02/19/python-async-simplified/">Python &amp; Async Simplified</a></strong></p> Andrew Godwin: “Python’s async framework is actually relatively simple when you treat it at face value, but a lot of tutorials and documentation discuss it in minute implementation detail, so I wanted to make a higher-level overview that deliberately ignores some of the small facts and focuses on the practicalities of writing projects that mix both kinds of code.” ‪This is really useful: clearly explains the two separate worlds of Python (sync and async functions) and describes Andrew’s clever sync_to_async and async_to_sync decorators as well.‬ <p>Tags: <a href="https://simonwillison.net/tags/andrew-godwin">andrew-godwin</a>, <a href="https://simonwillison.net/tags/async">async</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a></p>

2018-02-02T19:06:48+00:002018-02-02T19:06:48+00:00https://simonwillison.net/2018/Feb/2/asgiref-asynctosync-and-synctoasync/#atom-tag

<p><strong><a href="https://github.com/django/asgiref/blob/538ce5ee92962307db1082552839df7d0037176a/asgiref/sync.py">asgiref: AsyncToSync and SyncToAsync</a></strong></p> Andrew’s classes in asgiref that can turn a synchronous callable into an awaitable (that runs in a thread pool) or an awaitable callable into a synchronous callable, using Python 3 futures and asyncio. <p><small></small>Via <a href="https://www.aeracode.org/2018/02/02/channels-20/">Andrew Godwin</a></small></p> <p>Tags: <a href="https://simonwillison.net/tags/andrew-godwin">andrew-godwin</a>, <a href="https://simonwillison.net/tags/async">async</a>, <a href="https://simonwillison.net/tags/python3">python3</a></p>

2018-02-02T18:19:54+00:002018-02-02T18:19:54+00:00https://simonwillison.net/2018/Feb/2/channels-2/#atom-tag

<p><strong><a href="https://www.aeracode.org/2018/02/02/channels-20/">Channels 2.0</a></strong></p> Andrew just shipped Channels 2.0—a major rewrite and redesign of the Channels project he started back in 2014. Channels brings async to Django, providing a logical, standardized way of supporting things like WebSockets and asynchronous execution on top of a Django application. Previously it required you to run a separate Twisted server and redis/RabbitMQ queue, but thanks to Python 3 async everything can now be deployed as a single process. And the new ASGI spec means its turtles all the way down! Everything from URL routing to view functions to middleware can be composed together using the same ASGI interface. <p>Tags: <a href="https://simonwillison.net/tags/andrew-godwin">andrew-godwin</a>, <a href="https://simonwillison.net/tags/async">async</a>, <a href="https://simonwillison.net/tags/django">django</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a>, <a href="https://simonwillison.net/tags/websockets">websockets</a></p>

2017-12-15T02:22:30+00:002017-12-15T02:22:30+00:00https://simonwillison.net/2017/Dec/15/monkeytype/#atom-tag

<p><strong><a href="https://engineering.instagram.com/let-your-code-type-hint-itself-introducing-open-source-monkeytype-a855c7284881">Let your code type-hint itself: introducing open source MonkeyType</a></strong></p> Instagram have open sourced their tool for automatically adding type annotations to your Python 3 code via runtime tracing. By default it logs the types it sees to a SQLite database, which means you can browse them with Datasette! <p>Tags: <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a>, <a href="https://simonwillison.net/tags/static-typing">static-typing</a></p>

2017-12-02T23:13:20+00:002017-12-02T23:13:20+00:00https://simonwillison.net/2017/Dec/2/python-3-readiness/#atom-tag

<p><strong><a href="http://py3readiness.org/">Python 3 Readiness</a></strong></p> 345 of the 360 most popular Python packages are now compatible with Python 3. I’d love to see a version of this graph over time. <p><small></small>Via <a href="https://news.ycombinator.com/item?id=15832924">My post on Hacker News</a></small></p> <p>Tags: <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a></p>

2017-10-21T22:22:40+00:002017-10-21T22:22:40+00:00https://simonwillison.net/2017/Oct/21/python3/#atom-tag

<p>This blog is now running on Python 3! Admittedly this is nearly nine years after <a href="https://www.python.org/download/releases/3.0/">the first release of Python 3.0</a>, but it’s the first Python 3 project I’ve deployed myself so I’m pretty excited about it.</p> <p>Library authors like to use <a href="https://pypi.python.org/pypi/six">six</a> to allow them to write code that supports both Python 2 and Python 3 at the same time… but my blog isn’t a library, so I used the <a href="https://docs.python.org/3/library/2to3.html">2to3 conversion tool</a> that ships with Python instead.</p> <p>And… it worked pretty well! I ran the following command from my project’s root directory:</p> <pre><code>2to3 -w -n blog/ config/ redirects/ feedstats/ </code></pre> <p>The <code>-w</code> option causes the files to be over-written in place. Since everything is already in git, there was no reason to have 2to3 show my a diff without applying it. Likewise, the <code>-n</code> option tells 2to3 not to bother saving backups of the files it modifies.</p> <p>Here’s <a href="https://github.com/simonw/simonwillisonblog/commit/615efeba55c0c32a8147bda49e207a7a52ddb674">the initial commit</a> containing mostly the 2to3 changes.</p> <p>Next step: run the tests! My test suite may be very thin, but it does at least check that the app can run its migrations, start up and serve a few basic pages without errors. One of my migrations was failing due to rogue bytestrings but that was <a href="https://github.com/simonw/simonwillisonblog/commit/f00224e7375098bb500b56b35c6f40dbc4955abc#diff-15a9a47c9ee1f93f3a07a4dbe0cf4214">an easy fix</a>.</p> <p>At this point I started to lean heavily on my <a href="https://simonwillison.net/2017/Oct/17/free-continuous-deployment/">continuous integration setup built on Travis CI</a>. All of my Python 3 work took place <a href="https://github.com/simonw/simonwillisonblog/tree/python3">in a branch</a>, and all it took was a <a href="https://github.com/simonw/simonwillisonblog/commit/f00224e7375098bb500b56b35c6f40dbc4955abc#diff-354f30a63fb0907d4ad57269548329e3">one line change to my .travis.yml</a> for Travis to start running the tests for that branch using Python 3.</p> <p>With the basic tests working, I made my first deploy to my Heroku staging instance - after first <a href="https://github.com/simonw/simonwillisonblog/commit/54b31e98e35031afbbf8c18f3c2446a0af8b5c65">modifying my Heroku runtime.txt</a> to tell it to use Python 3.6.2. My staging environment allowed me to sanity check that everything would work OK when deployed to Heroku.</p> <p>At this point I got a bit lazy. The responsible thing to do would have been extensive manual testing plus systematic unit test coverage of core functionality. My blog is hardly a critical piece of infrastructure though, so I went with the faster option: put it all live and <a href="https://simonwillison.net/2017/Oct/17/free-continuous-deployment/#Step_4_Monitor_errors_with_Sentry_75">use Sentry to see if anything breaks</a>.</p> <p>This is where Heroku’s ability to deploy a specific branch came in handy: one click to deploy my python3 branch, keep an eye on Sentry (via push notifications from <a href="https://simonwillison.net/2017/Oct/17/free-continuous-deployment/#Step_5_Hook_it_all_together_with_Slack_97">my private slack channel</a>) and then one click to deploy my master branch again for an instant rollback in case of errors. Which I had to do instantly, because it turned out I had stored some data in Django’s cache using Python 2 pickle and was trying to read it back out again using Python 3.</p> <p>I fixed that by <a href="https://github.com/simonw/simonwillisonblog/commit/41f7a112721ec5772ad189c4293da081291a604a">bumping my cache VERSION setting</a> and deployed again. This deploy lasted a few minute longer before Sentry started to fill up with encoding errors and I rolled it back again.</p> <p>The single biggest difference between Python 2 and Python 3 is <a href="https://docs.python.org/3.0/whatsnew/3.0.html#text-vs-data-instead-of-unicode-vs-8-bit">how strings are handled</a>. Python 3 strings are unicode sequences. Learning to live in a world where strings are all unicode and byte strings are the rare, deliberate exceptions takes some getting used to.</p> <p>The key challenge for my blog actually came from <a href="https://github.com/simonw/simonwillisonblog/blob/45d7acd56af475119d2738e736d9b4cb19a9e8eb/blog/templatetags/entry_tags.py">my custom markup handling template tags</a>. 15 years ago <a href="https://simonwillison.net/2002/Jun/16/myFirstXhtmlMindBomb/">I made the decision</a> to <a href="https://simonwillison.net/2003/Jan/6/xhtmlIsJustFine/">store all of my blog entries</a> as valid XHTML fragments. This meant I could use XML processors - back then in PHP, today <a href="https://docs.python.org/3/library/xml.etree.elementtree.html">Python’s ElementTree</a> - to perform various transformations on my content.</p> <p>ElementTree in Python 2 can only consume bytestrings. In Python 3 it expects unicode strings. Cleaning this up took a while, eventually inspiring me to <a href="https://github.com/simonw/simonwillisonblog/commit/7295cddd1a6ab2c7bc6fcf3da410ab6ea0954791">refactor my custom template tags completely</a>. In the process I realized that my blog templates were mostly written back before Django’s template language implemented autoescape (<a href="https://simonwillison.net/2008/Jul/22/alpha/">in Django 1.0</a>), so my code was littered with unnecessary <code>|escape</code> and <code>|safe</code> filters. Those are all gone now.</p> <p>Sentry lets you mark an exception as “resolved” when you think you’ve fixed it - if it occurs again after that it will be re-reported to your Slack channel and added back to the Sentry list of unresolved issues. Once Sentry was clear (especially given Googlebot had crawled my older pages) I could be pretty confident there were no more critical 500-causing errors.</p> <p>That left logic errors, of which only one has cropped up so far: the “zero years ago” bug. Entries on my homepage include a relative date representation, e.g. “three days ago”. Python 3 <a href="https://www.python.org/dev/peps/pep-0238/">changed how the divison operator works on integers</a> - <code>3 / 2 == 1.5</code> where in Python 2 it gets truncated to <code>1</code>. As a result, every entry on my homepage showed “zero years ago”. Thankfully this was <a href="https://github.com/simonw/simonwillisonblog/commit/d6e6eeb93ac02aa33c59b151f2a20e26d41f18b1">a one-line fix</a>.</p> <p>All in all this process was much less painful than I expected. It took me longer to write this blog entry than it did to actually make the conversion (thanks to 2to3 doing most of the tedious work), and the combination of Travis CI, Sentry and Heroku allowed me to ship aggressively with the knowledge that I could promptly identify and resolve any issues that came up.</p> <p>Next upgrade: <a href="https://www.djangoproject.com/weblog/2017/oct/16/django-20-beta-1-released/">Django 2.0</a>!</p> <p>Tags: <a href="https://simonwillison.net/tags/continuous-integration">continuous-integration</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a>, <a href="https://simonwillison.net/tags/heroku">heroku</a>, <a href="https://simonwillison.net/tags/travis">travis</a>, <a href="https://simonwillison.net/tags/sentry">sentry</a></p>

2010-07-04T23:21:00+00:002010-07-04T23:21:00+00:00https://simonwillison.net/2010/Jul/4/python/#atom-tag

<p><strong><a href="http://www.python.org/download/releases/2.7/">Python 2.7 Release</a></strong></p> Includes three of my favourite features from Python 3: odicts, set literals and set and dictionary comprehensions. <p>Tags: <a href="https://simonwillison.net/tags/dictionaries">dictionaries</a>, <a href="https://simonwillison.net/tags/odict">odict</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a>, <a href="https://simonwillison.net/tags/sets">sets</a>, <a href="https://simonwillison.net/tags/recovered">recovered</a></p>

2010-01-28T13:28:07+00:002010-01-28T13:28:07+00:00https://simonwillison.net/2010/Jan/28/python3/#atom-tag

<p><strong><a href="http://www.dabeaz.com/blog/2010/01/reexamining-python-3-text-io.html">Reexamining Python 3 Text I/O</a></strong></p> Python 3.1’s IO performance is a huge improvement over 3.0, but still considerably slower than 2.6. It turns out it’s all to do with Python 3’s unicode support: When you read a file in to a string, you’re asking Python to decode the bytes in to UTF-8 (the new default encoding) at the same time. If you open the file in binary mode Python 3 will read raw bytes in to a bytestring instead, avoiding the conversion overhead and performing only 4% slower than the equivalent code in Python 2.6.4. <p>Tags: <a href="https://simonwillison.net/tags/david-beazley">david-beazley</a>, <a href="https://simonwillison.net/tags/io">io</a>, <a href="https://simonwillison.net/tags/performance">performance</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a>, <a href="https://simonwillison.net/tags/text">text</a>, <a href="https://simonwillison.net/tags/unicode">unicode</a></p>

2010-01-07T12:41:48+00:002010-01-07T12:41:48+00:00https://simonwillison.net/2010/Jan/7/voidspace/#atom-tag

<p><strong><a href="http://www.voidspace.org.uk/python/weblog/arch_d7_2010_01_02.shtml#e1146">Current State of Unladen Swallow (Towards a Faster Python)</a></strong></p> The Unladen Swallow team are now planning to merge their work back in to the mainline Python 3 branch, adding a powerful incentive for Python developers to port their old Python 2 code (since the official Python 2.x line is extremely unlikely to have Unladen Swallow merged in to it). <p>Tags: <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a>, <a href="https://simonwillison.net/tags/unladenswallow">unladenswallow</a></p>

2009-06-28T15:02:09+00:002009-06-28T15:02:09+00:00https://simonwillison.net/2009/Jun/28/python31/#atom-tag

<p><strong><a href="http://docs.python.org/3.1/whatsnew/3.1.html">What&#x27;s New In Python 3.1</a></strong></p> Lots of stuff, but the best bits are an ordered dictionary type (congrats, Armin), a Counter class for counting unique items in an iterable (I do this on an almost daily basis) and a bunch of performance improvements including a rewrite of the Python 3.0 IO system in C. <p>Tags: <a href="https://simonwillison.net/tags/armin-ronacher">armin-ronacher</a>, <a href="https://simonwillison.net/tags/performance">performance</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a>, <a href="https://simonwillison.net/tags/python31">python31</a>, <a href="https://simonwillison.net/tags/releases">releases</a></p>

2009-01-26T18:10:13+00:002009-01-26T18:10:13+00:00https://simonwillison.net/2009/Jan/26/hooray/#atom-tag

<p><strong><a href="http://diveintopython3.org/">Dive into Python 3</a></strong></p> Mark Pilgrim’s seminal work taught me Python nearly eight years ago. Now he’s updating it to cover Python 3. It’s just a table of contents at the moment, but the chapter on “Packaging Python libraries” has me very excited. <p>Tags: <a href="https://simonwillison.net/tags/diveintopython">diveintopython</a>, <a href="https://simonwillison.net/tags/mark-pilgrim">mark-pilgrim</a>, <a href="https://simonwillison.net/tags/packaging">packaging</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a></p>

2008-12-06T10:00:54+00:002008-12-06T10:00:54+00:00https://simonwillison.net/2008/Dec/6/cmlenz/#atom-tag

<blockquote cite="http://www.cmlenz.net/archives/2008/12/py3k"><p>I don't think that Python 3.0 is a bad thing. But that it's displayed so prominently on the Python web site, without any kind of warning that it's not going to work with 99% of the Python code out there, scares the hell out of me. People are going to download and install 3.0 by default, and nothing's going to work. They're going to complain, and many are going to simply walk away.</p></blockquote> <p class="cite">&mdash; <a href="http://www.cmlenz.net/archives/2008/12/py3k">Christopher Lenz</a></p> <p>Tags: <a href="https://simonwillison.net/tags/christopher-lenz">christopher-lenz</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a></p>

2008-12-04T12:40:57+00:002008-12-04T12:40:57+00:00https://simonwillison.net/2008/Dec/4/whatsnew/#atom-tag

<p><strong><a href="http://docs.python.org/dev/3.0/whatsnew/3.0.html">What&#x27;s New In Python 3.0</a></strong></p> Lots. <p>Tags: <a href="https://simonwillison.net/tags/guido-van-rossum">guido-van-rossum</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a></p>

2008-12-04T12:38:41+00:002008-12-04T12:38:41+00:00https://simonwillison.net/2008/Dec/4/python/#atom-tag

<p><strong><a href="http://www.python.org/download/releases/3.0/">Python 3.0</a></strong></p> “We are pleased to announce the release of Python 3.0 (final), a new production-ready release, on December 3rd, 2008.” <p>Tags: <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a>, <a href="https://simonwillison.net/tags/releases">releases</a></p>

2007-10-09T02:08:13+00:002007-10-09T02:08:13+00:00https://simonwillison.net/2007/Oct/9/strings/#atom-tag

<p><strong><a href="http://pyside.blogspot.com/2007/10/string-types-in-python-3.html">String types in Python 3</a></strong></p> bytes are now immutable (just like the bytestrings they are replacing) and a new mutable buffer type has been introduced. <p>Tags: <a href="https://simonwillison.net/tags/buffers">buffers</a>, <a href="https://simonwillison.net/tags/bytes">bytes</a>, <a href="https://simonwillison.net/tags/bytestrings">bytestrings</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a>, <a href="https://simonwillison.net/tags/strings">strings</a>, <a href="https://simonwillison.net/tags/unicode">unicode</a></p>

2007-09-03T01:38:34+00:002007-09-03T01:38:34+00:00https://simonwillison.net/2007/Sep/3/sam/#atom-tag

<p><strong><a href="http://intertwingly.net/blog/2007/09/01/2to3">Sam Ruby: 2to3</a></strong></p> Sam’s report on an attempt to port the Universal Feed Parser to Python 3.0. The 2to3 tool does most of the work, but it seems the unicode changes can be pretty tricky. <p>Tags: <a href="https://simonwillison.net/tags/2to3">2to3</a>, <a href="https://simonwillison.net/tags/feedparser">feedparser</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a>, <a href="https://simonwillison.net/tags/sam-ruby">sam-ruby</a>, <a href="https://simonwillison.net/tags/unicode">unicode</a></p>

2007-09-01T01:41:34+00:002007-09-01T01:41:34+00:00https://simonwillison.net/2007/Sep/1/new/#atom-tag

<p><strong><a href="http://docs.python.org/dev/3.0/whatsnew/3.0.html">What&#x27;s New in Python 3.0</a></strong></p> They’re definitely taking advantage of the break in backwards compatibility—lots of niggling inconsistencies are finally being cleaned up. <p>Tags: <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a></p>

2007-08-31T20:36:39+00:002007-08-31T20:36:39+00:00https://simonwillison.net/2007/Aug/31/python/#atom-tag

<p><strong><a href="http://python.org/download/releases/3.0/">Python 3.0a1 released</a></strong></p> Wow, that was a pretty fast turnaround. Betas are planned for 2008, with a final release scheduled for August. <p>Tags: <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/python3">python3</a></p>