The Bitter Lesson

Rich Sutton, one of the founders of reinforcement learning and winner of the Turing Award in 2024, wrote this essay in March 2019 on his own website. It is barely two pages long and has become one of the most quoted texts about AI, because it explains in a few paragraphs why the field keeps moving the way it does.

The argument

Sutton starts from an observation that runs through the whole history of AI. Researchers try again and again to build what they know about a problem into their systems: how a chess master thinks, how words are made of phonemes, what edges and shapes make up an object in a picture. This helps at first and is satisfying for the researchers, but sooner or later a method that knows much less overtakes it, because it can make use of the growing power of computers. Two kinds of methods scale in this way: search, which tries out far more possibilities than a person could, and learning, which finds the patterns in data by itself.

His examples are the familiar milestones:

• Chess: Deep Blue beat Garry Kasparov in 1997 with massive search, to the dismay of researchers who had modelled the human understanding of the game.
• Go: Twenty years later the same happened in Go, this time through learning from games against itself.
• Speech recognition: statistical methods won against systems built on human knowledge of words and sounds, and later deep learning won again with even less of it.
• Computer vision: hand-designed features such as edges gave way to neural networks that learn their own.

The lesson is bitter because it means that the knowledge researchers are proud of matters less in the long run than the computation they can throw at a problem. Sutton's conclusion is that we should build methods that can discover the complexity of the world themselves, instead of building in our own picture of it.

Why it fits here

The essay explains why language models keep growing, and why a model that was simply trained on more data with more computation often beats a more cleverly designed one. It also helps with agents: a harness full of hand-written rules and step-by-step instructions may work today, while the bitter lesson predicts that a model trained on good results will catch up with it. Whether the lesson holds everywhere, for instance in messy organisations where nobody can say what a good result is, is an open question.

Ethan Mollick's music video

Ethan Mollick explains the bitter lesson in an unusual way in his post The Dot and the Swarm on his blog One Useful Thing: he asked AI models to turn it into a music video, with lyrics and music generated by AI and the video assembled in code without any image generation. The video is in the post and also on YouTube. That a machine can now make such a video from one prompt is itself a small example of the lesson.

In an earlier post, The Bitter Lesson versus The Garbage Can, Mollick asks what the lesson means for companies whose processes are anything but tidy.