Simon Willison's Weblog: media

Thoughts on AI safety in this era of increasingly powerful open source LLMs

2023-04-10T18:41:25+00:00

This morning, VentureBeat published a story by Sharon Goldman: With a wave of new LLMs, open source AI is having a moment — and a red-hot debate. It covers the explosion in activity around openly available Large Language Models such as LLaMA - a trend I've been tracking in my own series LLMs on personal devices - and talks about their implications with respect to AI safety.

I talked to Sharon for this story last week. Here's the resulting excerpt:

The latest wave of open-source LLMs are much smaller and not as cutting-edge as ChatGPT, but “they get the job done,” said Simon Willison, an open-source developer and co-creator of Django, free and open-source, Python-based web framework.

“Before LLaMA came along, I think lots of people thought that in order to run a language model that was of any use at all, you needed $16,000 worth of video cards and a stack of 100 GPUs,” he told VentureBeat. “So the only way to access these models was through OpenAI or other organizations.”

But now, he explained, open-source LLMs can run on a laptop. “It turns out maybe we don’t need the cutting-edge for a lot of things,” he said.

To expand on this point: when I said "It turns out maybe we don’t need the cutting-edge for a lot of things" I was thinking specifically about tricks like the ReAct pattern, where LLMs are given the ability to use additional tools to run things like calculations or to search for information online or in private data.

This pattern is getting a LOT of attention right now: ChatGPT Plugins is one implementation, and new packages are coming out every few days such as Auto-GPT that implement variations on this theme.

An open question for me: how powerful does your LLM need to be in order to run this pattern? My hunch is that if you have an LLM that is powerful enough to produce reasonable summaries of text, it should also be powerful enough to use as part of that pattern.

Which means that a LLM running on a laptop should be enough to create truly impressive tool-enabled AI assistants - without any need to rely on cloud AI providers like OpenAI.

However, the ethical implications of using these open source LLM models are complicated and difficult to navigate, said Willison. OpenAI, for example, has extra filters and rules in place to prevent writing things like a Hitler manifesto, he explained. “But once you can run it on your own laptop and do your own additional training, you could potentially train a fascist language model — in fact, there are already projects on platforms like 4chan that aim to train ‘anti-woke’ language models,” he said.

This is concerning because it opens the door to harmful content creation at scale. Willison pointed to romance scams as an example: Now, with language models, scammers could potentially use them to convince people to fall in love and steal their money on a massive scale,” he said.

Currently, Willison says he leans towards open source AI. “As an individual programmer, I use these tools on a daily basis and my productivity has increased, allowing me to tackle more ambitious problems,” he said. “I don’t want this technology to be controlled by just a few giant companies; it feels inherently wrong to me given its impact.”

I wrote about this more here: AI-enhanced development makes me more ambitious with my projects

This is yet another example of a theme I keep coming back to: in AI, multiple things are true at the same time. The potential for harm is enormous, and the current systems have many flaws - but they are also incredibly empowering on an individual level if you can learn how to effectively use them.

But, he still expressed concern. “What if I’m wrong?” he said. “What if the risks of misuse outweigh the benefits of openness? It’s difficult to balance the pros and cons.”

This is a real challenge for me. Sci-fi paperclip scenarios aside, most of the arguments I hear from AI critics feel entirely correct to me. There are so many risks and harmful applications of this technology.

Maybe we can regulate its use in a way that helps mitigate the worst risks... but legislation is difficult to get right, and the pace at which AI is moving appears to be far beyond that of any governmental legislative process.

My current plan is to keep helping people learn how to use these tools in as positive and productive a way as possible. I hope I don't come to regret it.

Tags: ethics, media, open-source, ai, openai, generative-ai, chatgpt, llama, local-llms, llms, ai-ethics

What AI can do for you on the Theory of Change podcast

2023-04-02T00:17:59+00:00

Matthew Sheffield invited me on his show Theory of Change to talk about how AI models like ChatGPT, Bing and Bard work and practical applications of things you can do with them.

The episode is available on SoundCloud and various podcast platforms (here's Apple Podcasts), or you can watch it on YouTube. I've also embedded the video below.

Our full conversation is nearly an hour and twenty minutes long! There's a transcript on the site which includes additional links.

I'll quote one portion from towards the end of the interview, about ways to learn more about how to use these models:

WILLISON: Websites pop up every day that claim to help you with AI, to be honest, at a rate that’s too far to even evaluate them and figure out which ones are good and which ones are snake oil. The thing that matters is actually interacting with these systems. You should be playing with Google Bard, and ChatGPT, and Microsoft Bing, and trying things out with a very skeptical approach.

Always assume that anything that it does, it could be making things up. It could be tricking you into thinking that it’s capable of something that it’s not. But that’s where you have to learn to experiment. You have to try different things, give it a URL, and then give it a broken URL and see how it differs between them.

Because that really is the most reliable way to get stuff done here. To sort of build that crucial mental model of what these things can do, and what they can’t. And it’s full of pitfalls. It’s so easy to fall into traps. So you do need to read around this stuff and find communities of people who are experimenting in it with, with you and, and so on.

Unfortunately, I don’t think there’s an easy answer to the question yet of how to learn to use these effectively, partly because ChatGPT isn’t even four months old yet. It’s four-month birthday’s on the 30th of March. All of this stuff is so new, we’re all figuring it out together. The key thing is, because it’s all so new, you need to hang out with other people.

You need to get involved with communities who are figuring this out. Share what you learn, see what other people learn, and basically try and help society as a whole come to terms with what these things even are and what we can do with them.

[...]

So that’s, I think, one of my sort of big personal ethical concerns is you should share your prompts. There are websites where you can sell prompts to people. No, no, no, no. Don’t do that. Share your prompts with other people. You get them to share the prompts back. We are all in this together. And sharing the prompts that work for you and the prompts that don’t is the fastest way that you can learn, and the fastest way that you can help other people learn as well.

A shorter version of the above: share your prompts! We're all in this together. We have so much that we still need to figure out.

Tags: bing, media, podcasts, ai, generative-ai, chatgpt, bard, llms, podcast-appearances

A conversation about prompt engineering with CBC Day 6

2023-03-18T16:00:24+00:00

I'm on Canadian radio this morning! I was interviewed by Peter Armstrong for CBC Day 6 about the developing field of prompt engineering.

You can listen here on the CBC website.

CBC also published this article based on the interview, which includes some of my answers that didn't make the audio version: These engineers are being hired to get the most out of AI tools without coding.

Here's my own lightly annotated transcript (generated with the help of Whisper).

Peter: AI Whisperer, or more properly known as Prompt Engineers, are part of a growing field of humans who make their living working with AI

Their job is to craft precise phrases to get a desired outcome from an AI

Some experts are skeptical about how much control AI whisperers actually have

But more and more companies are hiring these prompt engineers to work with AI tools

There are even online marketplaces where freelance engineers can sell the prompts they've designed

Simon Willison is an independent researcher and developer who has studied AI prompt engineering

Good morning, Simon. Welcome to Day 6

Simon: Hi, it's really great to be here

Peter: So this is a fascinating and kind of perplexing job

What exactly does a prompt engineer do?

Simon: So we have these new AI models that you can communicate to with English language

You type them instructions in English and they do the thing that you ask them to do, which feels like it should be the easiest thing in the world

But it turns out actually getting great results out of these things, using these for the kinds of applications people want to sort of summarization and extracting facts requires a lot of quite deep knowledge as to how to use them and what they're capable of and how to get the best results out of them

So, prompt engineering is essentially the discipline of becoming an expert in communicating with these things

It's very similar to being a computer programmer except weird and different in all sorts of new ways that we're still trying to understand

Peter: You've said in some of your writing and talking about this that it's important for prompt engineers to resist what you call superstitious thinking

What do you mean by that?

My piece In defense of prompt engineering talks about the need to resist superstitious thinking.

Simon: It's very easy when talking to one of these things to think that it's an AI out of science fiction, to think that it's like the Star Trek computer and it can understand and do anything

And that's very much not the case

These systems are extremely good at pretending to be all powerful, all knowing things, but they have massive, massive flaws in them

So it's very easy to become superstitious, to think, oh wow, I asked it to read this web page, I gave it a link to an article and it read it

It didn't read it!

This is a common misconception that comes up when people are using ChatGPT. I wrote about this and provided some illustrative examples in ChatGPT can’t access the internet, even though it really looks like it can.

A lot of the time it will invent things that look like it did what you asked it to, but really it's sort of imitating what would look like a good answer to the question that you asked it

Peter: Well, and I think that's what's so interesting about this, that it's not sort of core science computer programming

There's a lot of almost, is it fair to call it intuition

Like what makes a prompt engineer good at being a prompt engineer?

Simon: I think intuition is exactly right there

The way you get good at this is firstly by using these things a lot

It takes a huge amount of practice and experimentation to understand what these things can do, what they can't do, and just little tweaks in how you talk to them might have huge effect in what they say back to you

Peter: You know, you talked a little bit about the assumption that we can't assume this is some all-knowing futuristic AI that knows everything and yet you know we already have people calling these the AI whispers which to my ears sounds a little bit mystical

How much of this is is you know magic as opposed to science?

Simon: The comparison to magic is really interesting because when you're working with these it really can feel like you're a sort of magician you sort of cast spells at it you don't fully understand what they're going to do and and it reacts sometimes well and sometimes it reacts poorly

And I've talked to AI practitioners who kind of talk about collecting spells for their spell book

But it's also a very dangerous comparison to make because magic is, by its nature, impossible for people to comprehend and can do anything

And these AI models are absolutely not that

See Is the AI spell-casting metaphor harmful or helpful? for more on why magic is a dangerous comparison to make!

Fundamentally, they're mathematics

And you can understand how they work and what they're capable of if you put the work in

Peter: I have to admit, when I first heard about this, I thought it was a kind of a made up job or a bit of a scam to just get people involved

But the more I've read on it, the more I've understood that this is a real skill

But I do think back to, it wasn't all that long ago that we had Google search specialists that helped you figure out how to search for something on Google

Now we all take for granted because we can do it

I wonder if you think, do prompt engineers have a future or are we all just going to eventually be able catch up with them and use this AI more effectively?

Simon: I think a lot of prompt engineering will become a skill that people develop

Many people in their professional and personal lives are going to learn to use these tools, but I also think there's going to be space for expertise

There will always be a level at which it's worth investing sort of full-time experience in in solving some of these problems, especially for companies that are building entire product around these AI engines under the hood

Peter: You know, this is a really exciting time

I mean, it's a really exciting week

We're getting all this new stuff

It's amazing to watch people use it and see what they can do with it

And I feel like my brain is split

On the one hand, I'm really excited about it

On the other hand, I'm really worried about it

Are you in that same place?

And what are the things you're excited about versus the things that you're worried about?

Simon: I'm absolutely in the same place as you there

This is both the most exciting and the most terrifying technology I've ever encountered in my career

Something I'm personally really excited about right now is developments in being able to run these AIs on your own personal devices

I have a series of posts about this now, starting with Large language models are having their Stable Diffusion moment where I talk about first running a useful large language model on my own laptop.

Right now, if you want to use these things, you have to use them against cloud services run by these large companies

But there are increasing efforts to get them to scale down to run on your own personal laptops or even on your own personal phone

I ran a large language model that Facebook Research released just at the weekend on my laptop for the first time, and it started spitting out useful results

And that felt like a huge moment in terms of sort of the democratization of this technology, putting it into people's hands and meaning that things where you're concerned about your own privacy and so forth suddenly become feasible because you're not talking to the cloud, you're talking to the sort of local model

Peter: You know, if I typed into one of these chat bots, you know, should I be worried about the rise of AI

It would absolutely tell me not to be

If I ask you the same question, should we be worried and should we be spending more time figuring out how this is going to seep its way into various corners of our lives?

Simon: I think we should absolutely be worried because this is going to have a major impact on society in all sorts of ways that we don't predict and some ways that we can predict

I'm not worried about the sort of science fiction scenario where the AI breaks out of my laptop and takes over the world

But there are many very harmful things you can do with a machine that can imitate human beings and that can produce realistic human text

My thinking on this was deeply affected by Emily M. Bender, who observed that "applications that aim to believably mimic humans bring risk of extreme harms" as highlighted in this fascinating profile in New York Magazine.

The fact that anyone can churn out very convincing but completely made up text right now will have a major impact in terms of how much can you trust the things that you're reading online

If you read a review of a restaurant, was it written by a human being or did somebody fire up an AI model and generate 100 positive reviews all in one go?

So there are all sorts of different applications to this

Some are definitely bad, some are definitely good

And seeing how this all plays out is something that I think society will have to come to terms with over the next few months and the next few years

Peter: Simon, really appreciate your insight and just thanks for coming with us on the show today

Simon: Thanks very much for having me

For more related content, take a look at the prompt engineering and generative AI tags on my blog.

Tags: media, ai, prompt-engineering, generative-ai, llms

Weeknotes: NICAR, and an appearance on KQED Forum

2023-03-07T22:46:28+00:00

I spent most of this week at NICAR 2023, the data journalism conference hosted this year in Nashville, Tennessee.

This was my third in-person NICAR and it's an absolute delight: NICAR is one of my favourite conferences to go to. It brings together around a thousand journalists who work with data, from all over the country and quite a few from the rest of the world.

People have very different backgrounds and experiences, but everyone has one thing in common: a nerdy obsession with using data to find and tell stories.

I came away with at least a year's worth of new ideas for things I want to build.

I also presented a session: an hour long workshop titled "Datasette: An ecosystem of tools for exploring data and collaborating on data projects".

I demonstrated the scope of the project, took people through some hands-on exercises derived from the Datasette tutorials Cleaning data with sqlite-utils and Datasette and Using Datasette in GitHub Codespaces and invited everyone in the room to join the Datastte Cloud preview and try using datasette-socrata to import and explore some data from the San Francisco open data portal.

My goal for this year's NICAR was to setup some direct collaborations with working newsrooms. Datasette is ready for this now, and I'm willing to invest significant time and effort in onboarding newsrooms, helping them start using the tools and learning what I need to do to help them be more effective in that environment.

If your newsroom is interested in that, please drop me an email at swillison@ Google's email service.

KQED Forum

My post about Bing attracted attention from the production team at KQED Forum, a long-running and influential Bay Area news discussion radio show.

They invited me to join a live panel discussion on Thursday morning with science-fiction author Ted Chiang and Claire Leibowitz from Partnership on AI.

I've never done live radio before, so this was an opportunity that was too exciting to miss. I ducked out of the conference for an hour to join the conversation via Zoom.

Aside from a call with a producer a few days earlier I didn't have much of an idea what to expect (similar to my shorter live TV appearance). You really have to be able to think on your feet!

A recording is available on the KQED site, and on Apple Podcasts.

I'm happy with most of it, but I did have one offensive and embarassing slip-up. I was talking about the Kevin Roose ChatGPT conversation from the New York Times, where Bing declared its love for him. I said (05:30):

So I love this particular example because it actually accidentally illustrates exactly how these things work.

All of these chatbots, all of these language models they're called, all they can do is predict sentences.

They predict the next word that statistically makes sense given what's come before.

And if you look at the way it talks to Kevin Roose, I've got a quote.

It says, "You're married, but you're not happy. You're married, but you're not satisfied. You're married, but you're not in love."

No human being would talk like that. That's practically a kind of weird poetry, right?

But if you're thinking about in terms of, OK, what sentence should logically come after this sentence?

"You're not happy, and then you're not satisfied", and then "you're not in love" - those just work. So Kevin managed to get himself into the situation where this bot was way off the reservation.

This is one of the most monumental software bugs of all time.

This was Microsoft's Bing search engine. They had a bug in their search engine where it would try and get a user to break up with their wife!

That's absolutely absurd.

But really, all it's doing is it had got itself to a point in the conversation where it's like, Okay, well, I'm in the mode of trying to talk about how why a marriage isn't working?

What comes next? What comes next? What comes next?

In talking about Bing's behaviour I've been trying to avoid words like "crazy" and "psycho", because those stigmatize mental illness. I try to use terms like "wild" and "inappropriate" and "absurd" instead.

But saying something is "off the reservation" is much worse!

The term is deeply offensive, based on a dark history of forced relocation of Native Americans. I used it here thoughtlessly. If you asked me to think for a moment about whether it was an appropriate phrase I would have identified that it wasn't. I'm really sorry to have said this, and I will be avoiding this language in the future.

I'll share a few more annotated highlights from the transcript, thankfully without any more offensive language.

Here's my response to a question about how I've developed my own understanding of how these models actually work (19:47):

I'm a software engineer. So I've played around with training my own models on my laptop. I found an example where you can train one just on the complete works of Shakespeare and then have it spit out garbage Shakespeare, which has "thee" and "thus" and so forth.

And it looks like Shakespeare until you read a whole sentence and you realize it's total nonsense.

I did the same thing with my blog. I've got like 20 years of writing that I piped into it and it started producing sentences which were clearly in my tone even though they meant nothing.

It's so interesting seeing it generate these sequences of words in kind of a style but with no actual meaning to them.

And really that's exactly the same thing as ChatGPT. It's just that ChatGPT was fed terabytes of data and trained for months and months and months, whereas I fed in a few megabytes of data and trained it for 15 minutes.

So that really helps me start to get a feel for how these things work. The most interesting thing about these models is it turns out there's this sort of inflection point in size where you train them and they don't really get better up until a certain point where suddenly they start gaining these capabilities.

They start being able to summarize text and generate poems and extract things into bullet pointed lists. And the impression I've got from the AI research community is people aren't entirely sure that they understand why that happens at a certain point.

A lot of AI research these days is just, let's build it bigger and bigger and bigger and play around with it. And oh look, now it can do this thing. I just saw this morning that someone's got it playing chess. It shouldn't be able to play chess, but it turns out the Bing one can play chess and like nine out of ten of the moves it generates are valid moves and one out of ten are rubbish because it doesn't have a chess model baked into it.

So this is one of the great mysteries of these things, is that as you train them more, they gain these capabilities that no one was quite expecting them to gain.

Another example of that: these models are really good at writing code, like writing actual code for software, and nobody really expected that to be the case, right? They weren't designed as things that would replace programmers, but actually the results you can get out of them if you know how how to use them in terms of generating code can be really sophisticated.

One of the most important lessons I think is that these things are actually deceptively difficult to use, right? It's a chatbot. How hard can it be? You just type things and it says things back to you.

But if you want to use it effectively, you have to understand pretty deeply what its capabilities and limitations are. If you try and give it mathematical puzzles, it will fail miserably because despite being a computer - and computers should be good at maths! - that's not something that language models are designed to handle.

And it'll make things up left, right, and center, which is something you need to figure out pretty quickly. Otherwise, you're gonna start believing just garbage that it throws out at you.

So there's actually a lot of depth to this. I think it's worth investing a lot of time just playing games with these things and trying out different stuff, because it's very easy to use them incorrectly. And there's very little guidance out there about what they're good at and what they're bad at. It takes a lot of learning.

I was happy with my comparison of writing cliches to programming. A caller had mentioned that they had seen it produce an answer to a coding question that invented an API that didn't exist, causing them to lose trust in it as a programming tool (23:11):

I can push back slightly on this example. That's absolutely right. It will often invent API methods that don't exist. But as somebody who creates APIs, I find that really useful because sometimes it invents an API that doesn't exist, and I'll be like, well, that's actually a good idea.

Because the thing it's really good at is consistency. And when you're designing APIs, consistency is what you're aiming for. So, you know, in writing, you want to avoid cliches. In programming, cliches are your friend. So, yeah, I actually use it as a design assistant where it'll invent something that doesn't exist. And I'll be like, okay, well, maybe that's the thing that I should build next.

A caller asked "Are human beings not also statistically created language models?". My answer to that (at 35:40):

So I'm not a neurologist, so I'm not qualified to answer this question in depth, but this does come up a lot in AI circles. In the discourse, yeah.

Yes, so my personal feeling on this is there is a very small part of our brain that kind of maybe works a little bit like a language model. You know, when you're talking, it's pretty natural to think what word's going to come next in that sentence.

But I'm very confident that that's only a small fraction of how our brains actually work. When you look at these language models like ChatGPT today, it's very clear that if you want to reach this mythical AGI, this general intelligence, it's going to have to be a heck of a lot more than just a language model, right?

You need to tack on models that can tell truth from fiction and that can do sophisticated planning and do logical analysis and so forth. So yeah, my take on this is, sure, there might be a very small part of how our brains work that looks a little bit like a language model if you squint at it, but I think there's a huge amount more to cognition than just the tricks that these language models are doing.

These transcripts were all edited together from an initial attempt created using OpenAI Whisper, running directly on my Mac using MacWhisper.

Releases this week

datasette-simple-html: 0.1 - 2023-03-01
Datasette SQL functions for very simple HTML operations
datasette-app: 0.2.3 - (5 releases total) - 2023-02-27
The Datasette macOS application

TIL this week

A simple Python wrapper for the ChatGPT API

Tags: bing, data-journalism, media, radio, ai, weeknotes, generative-ai, chatgpt, whisper, llms, nicar, podcast-appearances, macwhisper

HTML5 Media Support in WebKit

2007-11-12T23:21:40+00:00

HTML5 Media Support in WebKit

WebKit continues to lead the pack when it comes to trying out new HTML5 proposals. The new audio and video elements make embedding media easy, and provide a neat listener API for hooking in to “playback ended” events.

Tags: audio, events, html5, javascript, macos, media, safari, video, webkit