Simon Willison's Weblog: hashing

Design and code review requested for Django string signing / signed cookies

2010-01-04T13:24:50+00:00

Design and code review requested for Django string signing / signed cookies

Do you know your way around web app security and cryptography (in particular signing things using hmac and sha1)? We’d appreciate your help reviewing the usage of these concepts in Django’s proposed string signing and signed cookie implementations.

Tags: code-review, cryptography, django, hashing, hmac, python, security, sha1

Cryptographic Right Answers

2009-06-11T22:16:25+00:00

Cryptographic Right Answers

Best practise recommendations for cryptography: “While some people argue that you should never use cryptographic primitives directly and that trying to teach people cryptography just makes them more likely to shoot themselves in their proverbial feet, I come from a proud academic background and am sufficiently optimistic about humankind that I think it’s a good idea to spread some knowledge around.”

Tags: aes, colinpercival, cryptography, hashing, security

Bloom Filter Resources

2008-10-19T22:22:19+00:00

Bloom Filter Resources

A continuation of the discussion about how to transfer information about a large number of recently updated resources around in an efficient way, Joe provides working code illustrating a simple approach using bloom filters.

Tags: bloom-filters, hashing, joe-gregorio, rest

Hash Collisions (The Poisoned Message Attack)

2008-04-04T19:24:36+00:00

Hash Collisions (The Poisoned Message Attack)

Demonstrates the MD5 weakness by providing two deliberately engineered PostScript documents with the same MD5 hash but radically different rendered output.

Tags: collisions, hashing, md5, postscript, security

Consistent Hashing

2008-03-18T01:00:34+00:00

Consistent Hashing

Beautifully clear explanation of consistent hashing, a simple technique that allows you to add new caching servers to a cluster without re-hashing your keys and hence invalidating all of your caches.

Tags: caching, consistenthashing, hashing, scaling

In rainbows

2007-10-23T22:39:15+00:00

In rainbows

Dopplr generates a unique colour for each city using an MD5 hash. The colours are then used in subtle but intelligent ways throughout the design—right down to the favicon.

Tags: colour, design, dopplr, favicons, hashing, matt-biddulph, matt-jones, md5

libketama

2007-04-20T06:50:51+00:00

libketama

A consistent hashing algorithm for memcache clients, from the team at last.fm.

Via Les Orchard

Tags: hashing, lastfm, les-orchard, memcache

Stopping spambots with hashes and honeypots

2007-01-23T13:39:15+00:00

Stopping spambots with hashes and honeypots

Ned’s analysis of how spambots work, along with some relatively simple tricks that should fool most of them.

Tags: commentspam, hashing, ned-batchelder, spam, spambots

Schneier on Security: Cryptanalysis of SHA-1

2005-02-19T15:12:39+00:00

Schneier on Security: Cryptanalysis of SHA-1

If you want to understand the “breaking” of SHA-1, this is the place to go. Surprisingly accessible.

Tags: bruce-schneier, cryptanalysis, hashing, security, sha

Schneier on Security: SHA-1 Broken

2005-02-16T04:47:31+00:00

Schneier on Security: SHA-1 Broken

Whoa.

Tags: bruce-schneier, hashing, security, sha

Signing comments on blogs

2003-07-22T21:10:50+00:00

Adrian Holovaty has implemented reserved comment names in his blog, a feature that prevents anyone apart from him from using the names "Adrian", "Adrian H." or "Adrian Holovaty" when posting a comment. François Nonnenmacher suggests extending the idea to allow people to "confirm" their authorship of comments on any blog using a TrackBack sent to their site that in turn causes them to be sent an alert email, which they can then use to confirm their comment. I like his idea of authentication based on URLs (email addresses are no good; they should not be publically displayed for fear of spam harvesters) but I think I've come up with an alternative authentication scheme that removes the need for the user to manually confirm authorship. This is pretty complicated, so bare with me.

The comment author enter's their comment in to a form on the site. They see a standard icon indicating that the blog in question supports comment signing. Rather than manually entering their name and URL, they activate a bookmarklet that they have previously added to their browser.
The bookmarklet fills in the name and URL fields for them. It also takes the comment, appends a secret key (stored in the bookmarklet) and finds the MD5 hash of the new string, using the Javascript MD5 library. It inserts this hash in to a hidden field in the comment form.
The user can now submit the new comment. That's all they have to do.
The weblog server now kicks in to action. If the comment has not been signed (there is no hash in the hidden field) it adds the comment normally, noting that it should be displayed as an "unsigned" comment on the comments page. End of story.
If it has been signed, the server has some work to do. First it must start loading the URL indicated by the user on the comment form. It is looking for a <link rel="signature"> element, which will provide the URL of a signature authenticating web service. If the </head> tag is reached, the system can assume the link element does not exist and can mark the comment as "unsigned",
If the web service is found, the server can now send it the comment and the User's site URL. The web service (which knows the user's secret key) will respond with a hash created in the same way as the one constructed by the bookmarklet.
If the hash returned by the web service matches the hash provided by the bookmarklet, the comment is considered "signed". The server can store it as such, and later display it with an icon or style that indicates it is a signed comment. If they do not match, the server can either store the comment as "unsigned" or even flag it as "untrusted", since it was incorrectly signed.

As you can see, it's a relatively complicated system. The comment authors must have a custom bookmarklet and add a tag to their home page indicating their authenticating web service URL. Note that they do not need to host the authentication web service themselves - they can instead point to one run by someone else who they trust (trust here is essential as the web service must know the user's private key). Meanwhile, the blogging system needs to be able to perform HTTP requests.

The key advantage of my system is that, being based on MD5, it is relatively easy to implement (as opposed to a system based on something like PGP). Provided no one points out any immediate flaws, I would happily construct a prototype in PHP. I'm sure a Perl implementation for Moveable Type users would not prove much of a challenge to any talented plugin author.

Security wise, it strikes me that the weakest link is the client side bookmarklet which comment authors would need to use. However, comment signing is not the most critical security application in the world and comment authors could easily change their password by updating their bookmarklet and alerting their signature web-service provider (which could even be themselves) of the change.

And if the signature idea doesn't win any favour, the idea of having a bookmarklet to fill in your name and URL in blog comment forms is one I've been meaning to share for some time.

Tags: adrian-holovaty, blogging, hashing, security, trackback, site-upgrades

Hashing client-side data

2003-02-08T23:53:25+00:00

Via Scott, a clever PHP technique for ensuring data sent to the browser as a cookie or hidden form variable isn't tampered with by the user:

If you're expecting to receive data in a cookie or a hidden form field that you've previously sent to a client, make sure it hasn't been tampered with by sending a hash of the data and a secret word along with the data. Put the hash in a hidden form field (or in the cookie) along with the data. When you receive the data and the hash, re-hash the data and make sure the new hash matches the old one.

A further explanation and example code can be found in PHP and the OWASP Top Ten Security Vulnerabilities, a handy article describing how PHP coders can combat the top ten web application security problems highlighted by a recent report from OWASP. Incidentally, OWASP still haven't fixed the cross site scripting vulnerability on their own site, discovered by Tom Gilder several weeks ago.

Incidentally, while the hashing method is clever and should be nice and secure I personally advocate not sending the user any information unless absolutely necessary - use sessions and store sensitive data on the server instead. I suppose you could always use the hash to add an extra layer of security to the session identifier though.

Tags: hashing, owasp, php, scott-johnson, security