Claude Shannon

The Man Who Invented Information

Born: 30 April 1916, Petoskey, Michigan, USA Died: 24 February 2001, Medford, Massachusetts, USA Age at death: 84

In the summer of 1948, the Bell System Technical Journal published a paper that most of its readers would have found difficult to place in any existing category of science. It was not physics. It was not mathematics in the pure sense. It was not engineering, though the author worked for a telephone company. It opened with the statement that “the fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point,” and it proceeded, over sixty-six pages, to solve that problem completely.

The paper was called “A Mathematical Theory of Communication.” Its author was Claude Elwood Shannon, a thirty-two-year-old research mathematician at Bell Laboratories in New Jersey. What he had done, though neither he nor anyone else fully grasped it immediately, was invent the concept of information itself — to give it a rigorous mathematical definition, measure it in bits, and derive the fundamental laws governing its transmission, compression, and noise.

The implications took decades to fully unfold. Shannon’s theory turned out to be the mathematical foundation not just for telecommunications but for every digital technology that followed: computers, the internet, mobile phones, data storage, cryptography, and — at several removes but with genuine directness — artificial intelligence. He had not set out to do any of this. He had been trying to figure out how many telephone calls you could cram down a wire.

Claude Shannon is not a household name. He is, in the community of scientists and engineers who actually understand what he did, routinely described as one of the greatest minds of the twentieth century. He is also one of the most underappreciated figures in popular accounts of how the modern world was built, partly because information theory is genuinely difficult to explain and partly because Shannon himself, a quietly unusual man who preferred juggling and unicycling to public intellectual life, did almost nothing to promote his own reputation.

Michigan to MIT

Shannon was born in 1916 in Petoskey, Michigan, a small city on the shore of Lake Michigan, and grew up in Gaylord, an even smaller town about an hour south. His father was a probate judge and businessman; his mother was the local high school principal. It was a modestly prosperous, conventionally midwestern upbringing that gave little outward indication of what was coming.

He was good at mathematics and tinkering — he built a telegraph line to communicate with a friend half a mile away, using barbed wire fencing as the conductor — and he attended the University of Michigan, where he took a double degree in mathematics and electrical engineering. It was an unusual combination, and it turned out to be exactly the right one. The two disciplines had not, at that point, been seriously connected. Shannon would spend his career connecting them.

In 1936 he went to MIT for graduate work and took a part-time job operating the differential analyser — a large mechanical analogue computer used to solve differential equations — built by Vannevar Bush, the engineer who would later run American science during the Second World War. The analyser was built from shafts, gears, and motors. To operate it you had to physically reconfigure the machine for each new problem. Shannon, sitting with this contraption day after day, began to notice something.

The machine’s logic was purely mechanical — physical connections routing signals through switching elements. But the behaviour of these switches corresponded exactly to the algebra of logic developed by George Boole in the 1850s, a formal system for reasoning about true-and-false propositions using symbols. Nobody had previously made this connection explicit. Shannon made it the subject of his master’s thesis, submitted in 1937.

The thesis is titled “A Symbolic Analysis of Relay and Switching Circuits.” It is, by wide consensus, the most important master’s thesis in the history of electrical engineering. It showed that any logical or arithmetic operation could be performed by a circuit of electrical switches, and that the design of such circuits could be reduced to Boolean algebra — making it a mathematical problem rather than an engineering art form. It was the theoretical basis for every digital circuit ever built. Shannon was twenty-one years old.

Bell Labs and the Idea That Changed Everything

Shannon joined Bell Laboratories in 1941, after completing a PhD in mathematics at MIT on population genetics — a subject he had chosen almost at random, to demonstrate that he could work in a completely different field. Bell Labs was, at that time, the most productive research institution in the world. Its scientists and engineers would eventually accumulate over thirty thousand patents and produce eight Nobel Prizes. It operated on the principle, unusual for a corporate laboratory, that fundamental research should be pursued without immediate practical application in mind. Shannon fit this culture perfectly.

During the war he worked on cryptography and fire control systems, and he met Alan Turing, who visited Bell Labs in 1943 while working on a secure voice communication system for Churchill and Roosevelt. The two men had lunch together regularly. They talked about computing, about thinking machines, about the theoretical problems both of them were independently working toward. Neither published anything about these conversations, and the details are lost. But both men later described them as significant.

It was during these years that Shannon was developing the ideas he would publish in 1948. The core problem he was addressing was one that engineers at Bell and other telecommunications companies had been struggling with since the advent of the telephone: how much information can you send through a communication channel, and what limits how much you can send?

The question sounds practical. Shannon’s answer was not. He realised that to make progress, you first had to define information precisely — not in terms of meaning, not in terms of the content of messages, but in terms of their statistical structure. A message conveys information, Shannon argued, to the extent that it reduces uncertainty. If you already know what someone is going to say, their saying it conveys no information. If what they say is completely unpredictable, it conveys maximum information.

This insight allowed him to define information mathematically, using the concept of entropy borrowed from thermodynamics. The entropy of a message source — what Shannon called H — measured the average uncertainty of its outputs. The unit of this measure was the binary digit: the bit. The bit was Shannon’s invention. He chose the name in a memo, credited the word to his colleague John Tukey, and moved on to more important things.

The 1948 Paper

“A Mathematical Theory of Communication” is divided into four parts. The first establishes the mathematical framework: the model of a communication system (source, encoder, channel, decoder, receiver), the definition of entropy, and the first of what Shannon called his fundamental theorems.

The first theorem — the source coding theorem — says that any message can be compressed to a size approaching its entropy, but not smaller. This is the mathematical limit of data compression. Every compression algorithm ever written — zip files, MP3 audio, JPEG images, streaming video codecs — is working within the limits Shannon identified. None can do better than his theorem allows.

The second part addresses the channel — the physical medium through which the message travels. Shannon introduces the concept of channel capacity: the maximum rate at which information can be reliably transmitted through a channel with a given bandwidth and noise level. This is a fixed property of the channel, independent of the message being sent.

The second theorem — the channel coding theorem, the most surprising result in the paper — says that reliable communication is possible at any rate below channel capacity, regardless of how much noise the channel contains. This was startling. The conventional engineering wisdom was that noise was the enemy and that reducing errors meant reducing the transmission rate. Shannon showed this was wrong: you could transmit at the theoretical maximum rate and still have arbitrarily low error rates, as long as you encoded your message cleverly enough. The proof was non-constructive — it showed that good codes existed without saying how to find them. Constructing codes that actually approached the Shannon limit kept information theorists busy for the next fifty years.

The paper also contains Shannon’s famous formula for channel capacity:

C = B log₂(1 + S/N)

where B is the bandwidth of the channel, S is the signal power, and N is the noise power. This equation determines the maximum data rate of every communication system ever built — every phone call, every Wi-Fi connection, every satellite link. It is sometimes called the Shannon-Hartley theorem, and it is engraved on a plaque at Bell Labs.

The response to the paper was immediate and disproportionate. Word spread through the scientific community faster than the paper itself. Warren Weaver, a mathematician at the Rockefeller Foundation, wrote a popular exposition of Shannon’s ideas that was published alongside the original in a 1949 book. The combination of Shannon’s technical paper and Weaver’s accessible commentary made information theory known almost overnight.

What followed was a period of, as Shannon himself later called it, “bandwagon” enthusiasm that he found somewhat alarming. Researchers in biology, psychology, linguistics, economics, and the social sciences began applying information-theoretic concepts to their own fields, often with more enthusiasm than rigour. Shannon published a short note in 1956 titled “The Bandwagon” — three paragraphs, barely a page — gently suggesting that the application of information theory to fields outside communications and computing required more care than was being exercised. It was one of his rare public interventions in scientific discourse, and it was characteristically mild.

The Chess-Playing Machine

In 1950 — the same year Turing published his paper on machine intelligence — Shannon published “Programming a Computer for Playing Chess.” It was the first serious technical analysis of how a computer might be made to play chess, a game that had long been considered a test of human intelligence.

Shannon’s paper made two foundational contributions. The first was a formal description of the minimax algorithm — the method by which a chess-playing program evaluates positions by looking ahead several moves and assuming that both players play optimally. This is still the basis of all chess engines. The second was the recognition that exhaustive search — looking at every possible position to some depth — was computationally impossible, and that any practical program would need a heuristic evaluation function to estimate the value of positions it couldn’t fully analyse.

He also introduced the distinction, now standard in game-playing AI, between “Type A” programs (which search exhaustively to a fixed depth) and “Type B” programs (which selectively explore promising lines more deeply, like human players). Modern chess engines use variants of this selective approach.

The paper is not primarily about artificial intelligence in the general sense. It is a very focused engineering analysis of a specific problem. But it established the vocabulary and framework for a huge portion of what would become AI research, and it was the direct ancestor of the programs that eventually beat Garry Kasparov and conquered Go.

The Curious Private Life

Shannon was, by all accounts, a man of pronounced and cheerful eccentricity. He rode a unicycle through the corridors of Bell Labs, sometimes while juggling. He built a flame-throwing trumpet. He constructed a motorised pogo stick. He designed a machine — built in collaboration with Marvin Minsky, who appears later in this series — that did nothing whatsoever: you flipped a switch to turn it on, a mechanical hand emerged from a box and turned it off again. He called it the Ultimate Machine.

He was also a serious juggler and wrote a mathematical paper on the mechanics of juggling — developing a formula, later known as the Shannon juggling theorem, that relates the number of balls, the time each ball is in the air, and the time each hand is empty. It has been experimentally verified.

These were not affectations. They were the natural expression of a mind that was genuinely curious about how things worked — any things — and that found the boundary between work and play essentially meaningless. His colleagues at Bell Labs described him as someone who pursued ideas wherever they led, without apparent concern for whether the destination was respectable or publishable or sensible.

He was also, in later life, an enthusiastic stock picker who thought he could apply information theory to financial markets. By the accounts of those who knew him, he was quite good at it.

His personal life was quiet and, by all appearances, happy. He married Mary Elizabeth Moore in 1949 — she was an MIT researcher he had met during the war — and they had three children. They lived in a large house in Winchester, Massachusetts, that Shannon filled with mechanical toys, musical instruments, and an eventually world-class collection of juggling equipment. He and Mary remained married until his death.

The Silence and the Decline

After the late 1950s, Shannon largely withdrew from academic life. He continued working — on automata theory, on artificial intelligence, on the mathematics of games — but published less and less. He attended fewer conferences. He declined most invitations to speak. When information theory became an established field with its own journals and its own conferences, he appeared at them rarely and sometimes unexpectedly, sitting in the back of lecture halls and asking quiet questions.

He was not reclusive in the way that suggests unhappiness. He was simply someone for whom intellectual work was private, whose curiosity was self-sustaining, and who had no particular interest in the ceremonies of scientific recognition. He received most of the major awards available to him — the National Medal of Science, the Kyoto Prize, the Claude E. Shannon Award, given by the Information Theory Society he had effectively founded — and accepted them with grace and without apparent excitement.

In his final years, he developed Alzheimer’s disease. By the late 1990s, he no longer recognised his colleagues or remembered his work. He died in a nursing home in Medford, Massachusetts, on 24 February 2001. He was eighty-four.

The Inheritance

Shannon’s contribution to artificial intelligence is not always made explicit in popular histories of the field, which tend to centre on figures like Turing, McCarthy, and the neural network researchers. This is a mistake.

Information theory is not peripheral to AI. It is structural. The concept of entropy underlies decision trees, random forests, and many other machine learning algorithms. The mathematics of compression underlies representation learning — the problem of finding compact, useful descriptions of complex data. The channel capacity theorem underlies the analysis of what neural networks can and cannot learn from a given amount of data. The probabilistic framework Shannon introduced for reasoning about messages is, at a deep level, the same framework used in Bayesian approaches to machine learning, in language models, in virtually every system that reasons about uncertainty.

More concretely: the bit — Shannon’s unit — is the atom of all digital computation. Every neural network weight, every training example, every model parameter is, at the hardware level, a collection of bits. The question of how much information a model contains, how much it can learn from a given dataset, how much it can compress the patterns in the data — these are Shannon questions, asked in Shannon’s language, governed by Shannon’s theorems.

He invented the mathematical language in which modern AI is written. He did it in 1948, before there was a digital computer capable of running anything recognisable as an AI program. He did it while building a flame-throwing trumpet and riding a unicycle down a laboratory corridor, which is either an irrelevant biographical detail or a perfect encapsulation of a mind that found the universe genuinely, inexhaustibly interesting.

Probably both.

Key Works & Further Reading

Primary sources:

“A Mathematical Theory of Communication.” Bell System Technical Journal, Vol. 27, July and October 1948. Reprinted with Warren Weaver’s exposition in The Mathematical Theory of Communication, University of Illinois Press, 1949.
“Programming a Computer for Playing Chess.” Philosophical Magazine, Series 7, Vol. 41, March 1950.
“Prediction and Entropy of Printed English.” Bell System Technical Journal, Vol. 30, January 1951.

Recommended reading:

A Mind at Play: How Claude Shannon Invented the Information Age — Jimmy Soni and Rob Goodman (2017). The definitive biography. Thoroughly researched, vividly written, and appropriately awed without being hagiographic.
The Information — James Gleick (2011). Places Shannon in the broader history of information and communication. Essential context.
The Mathematical Theory of Communication — Shannon and Weaver (1949). The original paper plus Weaver’s accessible introduction. Still worth reading.
Genius at Play: The Curious Mind of John Horton Conway — Siobhan Roberts (2015). Not about Shannon, but about a similar type of mathematical mind; useful comparative reading.