Nobody set out to build AI: how we got here anyway

The usual story gives AI a precise birthday: 30 November 2022, the afternoon ChatGPT went public. Before it, nothing much; after it, everything. It's a clean story, and it's wrong in an interesting way. What went public that afternoon was the far end of a chain of ideas that runs back to the 1930s, and almost nobody who forged a link in that chain was trying to build anything like it. They were settling a question in logic, or measuring a telephone line, or making a computer's memory writable by ordinary people. Intelligence was never the goal. It's what was left over.

This is a short walk along that chain. It leaves out whole fields that deserve their own essay, so take it as partial by design. But the through-line is real, and I think it's the part worth holding onto: the machine that now runs AI was built, brick by brick, by people aiming at something else.

Building the machine

Start in 1936, with a young mathematician trying to kill a problem. David Hilbert had asked whether every mathematical statement could, in principle, be settled by a mechanical procedure: a fixed set of steps you follow without insight, the way you do long division. To answer it, Alan Turing first had to say precisely what a "mechanical procedure" even was, and the definition he reached for was a small imaginary device: a head moving along an endless tape, reading and writing symbols according to a table of rules. He proved that no such procedure could decide every statement; some questions are formally undecidable. But the device he invented to settle the point mattered far more than the answer did. It was the universal machine, one mechanism that, given the right instructions, could carry out any computation at all. Every computer you have ever used is a physical version of it. Turing was ending an argument about the limits of logic, and the computer fell out of the proof.

Twelve years later, at Bell Labs, Claude Shannon was working on something far more practical: how much information you can push down a noisy wire. His 1948 paper gave the answer and, almost in passing, rebuilt the foundations of the whole subject. He defined the bit, the irreducible unit of information. He showed that any message (text, sound, an image) could be reduced to a stream of ones and zeros. And he borrowed a word from physics, entropy, to measure how much surprise a message carries: how hard it is to guess the next part from the parts you've already seen. To make the idea concrete he ran a small experiment a few years later, published in 1951. Take an ordinary English sentence, hide the end, and ask someone to guess the next letter, then the next. People are good at it, because language is predictable, and exactly how predictable it is was the thing Shannon was measuring. Hold on to that experiment. It comes back.

From there the story is about making the universal machine usable by humans rather than only by mathematicians. The earliest computers were programmed in raw binary, then in assembly, a thin shorthand for the machine's own instructions. The leap that counted was the claim (radical at the time, and pushed hardest by Grace Hopper) that a machine could translate something close to English into its own code automatically. That idea became the compiler, and the compiler made high-level programming languages possible. The point isn't the parade of languages that followed; it's the principle under them. Each new layer of abstraction let people stop thinking about the machine and start thinking about the problem, and that is what let software grow from a few hundred instructions into systems of millions.

In the early 1970s, Dennis Ritchie and Ken Thompson at Bell Labs laid down the substrate the rest would be built on: the C language, and the Unix operating system written in it, organised around small programs that each did one thing and could be piped into each other. Unix and its descendants still sit underneath most of the computing on the planet. By then the machine was no longer a curiosity. It was something you could build large, durable systems on.

The last brick was the network. Tim Berners-Lee's web, at the end of the 1980s, turned a heap of disconnected computers into a single readable space of linked documents. Ten years on, two students at Stanford built a way to rank those documents by how many others pointed to them. They were trying to make search work. What they also did, without anyone framing it that way, was begin assembling and ordering the largest body of human writing that had ever existed. Nobody called it a training set. There was nothing yet to train.

Add it up, and by around the year 2000 we had the whole thing without having meant to: a universal symbol-manipulating machine, cheap and everywhere, wired into a global network, sitting on top of an unprecedented and still-growing pile of human language. Every piece had been laid by someone solving a different problem.

Teaching it to learn

That pile of text, and the cheap computing power stacking up next to it, were exactly what a much older idea had been waiting for: an idea that had already been pronounced dead at least once.

Go back to 1958. Frank Rosenblatt, a psychologist at Cornell, built a machine called the perceptron, loosely inspired by how neurons fire. Its trick was that it learned: shown examples and told when it was wrong, it nudged its own internal weights until it did better. The hype was immediate and enormous. The Navy funded it, and the New York Times wrote it up as the embryo of a computer that would one day walk, talk, and be conscious of its own existence.

Then, in 1969, Marvin Minsky and Seymour Papert published a careful book setting out what a single-layer perceptron simply could not do, including problems as basic as exclusive-or, telling you whether two inputs differ. The limitation was real for that simple design, and the field's enthusiasm, which had run well ahead of the evidence, collapsed along with it. The money went elsewhere. For most of the next two decades the whole approach was treated as a dead end. The "AI winter" is partly a story the field tells about itself, neater in the retelling than it felt at the time, but the loss of funding and attention was real enough.

What revived it, in 1986, was a way to train networks with layers stacked between input and output. Rumelhart, Hinton and Williams showed how an error at the output could be pushed backwards through those hidden layers, each one adjusting its weights a little, so the network could discover useful intermediate features for itself instead of having them hand-designed. They popularised the method more than invented it; the same mathematics had been worked out earlier by others, which is the usual way of these things. It worked, and it still wasn't enough. The networks were small, the data was thin, and the computers were too slow for anything ambitious. The idea was sound, and it stuck around.

What unstuck it wasn't a cleverer idea. It was the arrival of the two things the 1986 method had been missing, both of them by-products of the first half of this story. In 2012 a network called AlexNet, built by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton, entered an image-recognition contest and won by an embarrassing margin: a top-five error rate of 15.3 per cent against the next-best entry's 26.2. The gap wasn't a new theory. It was a large labelled dataset to learn from, and a pair of consumer graphics cards (gaming hardware) fast enough to do the training. That was the moment the field stopped doubting that deep learning worked.

Five years later, a team at Google changed the shape of the network itself. Their 2017 paper, titled with some confidence "Attention Is All You Need", dropped the assumption that a model had to read a sentence in order, one word after another. Instead it let every word weigh every other word at once, working out how much each one mattered to the rest. This was the transformer, and its great virtue was that it scaled: you could make it enormous and feed it enormous amounts of text, and the training stayed efficient. Google published the design in the open. It is the "T" in GPT.

The last step was a bet. In 2020 OpenAI took the transformer and simply made it vast (175 billion internal parameters, trained on a large slice of the public internet) to see what would happen. What happened was that the model began doing things nobody had explicitly taught it: translating, summarising, answering questions, writing code that ran, all as side effects of a single relentless task. And the task is the one to keep in mind. Underneath all of it (the scale, the hardware, the billions of parameters) the model is doing precisely what Shannon asked a person to do in 1951. It is looking at a stretch of text and guessing what comes next. Two years after that bet, wrapped in a chat window, it became the thing that looked, to everyone watching, like it had appeared out of nowhere.

What I take from it

Two things, really.

The first is how little of this was inevitable. Laid out as a clean sequence it reads like a march, each step leading sensibly to the next, but lived forwards it was nothing of the kind. The central idea was abandoned for the better part of twenty years on the strength of a book that was, narrowly, correct. The breakthrough that revived it then waited decades for hardware that arrived mostly because people wanted better video games. The architecture under every current AI system was given away for free by a company that had no obligation to. Pull out any one of those accidents and the timeline looks completely different. This was contingent, lumpy, full of dead ends and luck, and it's worth assuming the next stretch will be too.

The second is the bookend. The deepest idea in the most modern technology we have is seventy-five years old, and it came from a man working out how much you could push down a phone line. Shannon's guessing game isn't a metaphor for what these systems do. It is, mechanically, the thing they do: run at a scale he couldn't have pictured, but the same move. I find that more impressive than any of the marketing, and quietly humbling about how much of the future is already lying around in old work, waiting for the rest of us to catch up to it.

Two easy stories, and why I trust neither

Which brings me back to the two ways this usually gets told. One says AI appeared in late 2022, a clean break with everything before. The other says it was the smooth and inevitable output of progress, always coming, right on schedule. Both are tidy, both are wrong, and they fail in opposite directions: the first forgets the ninety years of groundwork, the second forgets how nearly none of it had to happen the way it did.

Holding both of those errors in view is, I think, most of what AI literacy actually is. If you know the lineage is long, you don't mistake a new release for magic. If you know it was contingent and largely accidental, you don't mistake a confident roadmap for destiny, and you read the hype and the doom with the same measured scepticism. If you want one place to start, start with Shannon's 1948 paper. It's harder going than this post and far more rewarding, and most of what came after is a footnote to it.

---

Sources

• Alan Turing, On Computable Numbers, with an Application to the Entscheidungsproblem (1936)
• Claude Shannon, A Mathematical Theory of Communication (1948)
• Claude Shannon, Prediction and Entropy of Printed English (1951)
• Marvin Minsky & Seymour Papert, Perceptrons: An Introduction to Computational Geometry (1969)
• David Rumelhart, Geoffrey Hinton & Ronald Williams, Learning representations by back-propagating errors (1986)
• Alex Krizhevsky, Ilya Sutskever & Geoffrey Hinton, ImageNet Classification with Deep Convolutional Neural Networks (2012)
• Ashish Vaswani et al., Attention Is All You Need (2017)
• Tom Brown et al., Language Models are Few-Shot Learners (2020)