A black stain in the lab, two gaming cards, and a billionaire feud that led to the AI revolution
From a birthday party argument involving Elon Musk to modern AI models: the history of deep learning is a journey from laboratory experiments with silver salts to the computing power that changed the world.
At one of Elon Musk's birthday parties, an argument broke out that sounds almost prophetic today. Sitting across from Musk was Larry Page, one of the founders of Google. Musk warned that powerful artificial intelligence could endanger humanity. Page saw things differently: if one day machines smarter than humans appeared, they would represent the next stage in the evolution of intelligence. Humans are the most intelligent creatures on Earth; in the future, Page thought, those would be the machines. According to Musk's biography by Walter Isaacson, Musk was furious, and in response, Page called Musk a "speciesist," meaning someone who gives the human species priority over other life forms. Musk shot back: "I am obviously pro-human."
The argument did not stay at the birthday party. Musk was convinced by the conversation that the future of intelligence should not be left in the hands of one company, especially if that company is Google. This argument led to the founding of OpenAI. In December 2015, Elon Musk, Sam Altman, Ilya Sutskever, and others announced the establishment of OpenAI as a non-profit research lab. The founders and supporters presented funding commitments of an unusual scale of $1 billion. The original goal was as ambitious as the technology itself: to develop artificial intelligence that would benefit all of humanity and not be controlled by a single corporation.
But to understand how a philosophical argument at a birthday party turned into an industry burning billions of dollars on chips and data centers, one must go back about 140 years, to a black stain in an Italian lab.
From neurons to computation
In 1873, Italian physician Camillo Golgi developed a staining method based on silver salts, called the "black reaction." It stained nerve cells, or neurons. The method revealed the neuron's body and its branches. For the first time, it was possible to see the brain as a vast forest. Spanish scientist Santiago Ramón y Cajal improved the method and reached a revolutionary conclusion: the brain is indeed built of a forest of neurons, as Golgi believed, but each neuron is a separate unit, and they communicate with each other. In 1906, they shared a Nobel Prize, even though they held opposing interpretations.
Seventy years later, in 1943, neurophysiologist Warren McCulloch and young mathematician Walter Pitts took the biological neuron and turned it into a mathematical model. They showed that it is possible to represent a neuron as a simple computing unit, and that when many such units are connected into a network, they are capable of performing logical operations. The first artificial neuron was not a chip. It was born on paper.
The path to deep learning
In 1958, Frank Rosenblatt introduced the "perceptron" to the world: a machine that could learn from examples by changing the strength of the connections. In an early demonstration, they assembled an IBM 704 computer that contained several perceptrons. The machine weighed over 5 tons. Rosenblatt declared that this was "the first machine capable of learning." However, in 1969, Marvin Minsky and others revealed a big hole in the promise: a single-layer perceptron could not even learn XOR, a basic logical operation. The hype turned into embarrassment, and funding for follow-up research began to disappear.
In 1986, Geoffrey Hinton and his colleagues presented a practical way to solve the problem: Backpropagation, a method that calculates how to distribute the responsibility for an error among the various connections in the network and update each one according to its share. In 1989, Yann LeCun and his colleagues at Bell Labs used backpropagation to train a network that recognized handwritten digits, using a method called "convolution." LeCun called the improved network LeNet. By the late 90s, LeNet's descendants were already reading millions of bank account numbers from checks.
The era of GPUs and Transformers
In 2006, NVIDIA introduced CUDA, which allowed developers to use the GPU for general calculations. In 2012, Alex Krizhevsky, Ilya Sutskever, and Hinton trained a deep network on ImageNet, dividing it between two GeForce GTX 580 gaming cards. AlexNet finished the global ImageNet competition with an error of 15.3%, compared to 26.2% for the second place. It was the declaration of a new era.
In 2017, eight researchers from Google published the paper "Attention Is All You Need." In the paper, the Transformer idea was born for the translation task. At its heart is the attention mechanism, which allows each word to "look" at the other words and decide which of them are important for its understanding. In 2020, OpenAI introduced GPT-3, a model with 175 billion parameters. On November 30, 2022, OpenAI launched ChatGPT, and the technology left the lab and entered the lives of all of us.
The bitter lesson and the future
In 2019, Rich Sutton published an essay titled "The Bitter Lesson," postulating that general methods that utilize more computation and data tend to bypass systems where experts manually encoded their knowledge. In 2020, OpenAI gave this a formula: the performance of models improves in a predictable way when increasing parameters, data, and computation.
Yann LeCun, one of the pioneers of deep networks, is a prominent critic of the belief that LLMs will surpass human intelligence. His bet is on "world models" that will learn from video and interaction. In March 2026, AMI Labs, the new company LeCun founded, raised $1 billion. The argument about whether text is enough for AGI or if the experience of the physical world is needed continues. It is likely that LLMs will one day be just a layer in a much larger and stranger machine of the future.
