The Atomic Trap

On July 16 — exactly 81 years after the first nuclear test in history, known as the "Trinity moment" — a group of Nobel laureates, scientists, diplomats, and religious figures gathered in Rome to sign a declaration that begins with the words: "Humanity stands at a decisive moment in its history." This comes just half a year after a leading group of atomic scientists claimed that we have never been so close to a global catastrophe and that to prevent it, binding international regulation in the field of artificial intelligence is required.

This was not the first time scientists have compared the dangers of nuclear weapons to those that AI poses to humanity. In the summer of 2023, in the midst of the wave of enthusiasm surrounding ChatGPT, Christopher Nolan's film "Oppenheimer" was released. In interviews promoting the film, the director said that AI researchers defined that period of time to him as an "Oppenheimer moment," referring to the feelings of deep realization and regret that scientists feel when they realize that they have unintentionally unleashed a destructive power that cannot be controlled.

This parallel began to bubble up a few months earlier, when Geoffrey Hinton, one of the godfathers of artificial intelligence, resigned from Google to warn the world about the danger that, according to him, artificial intelligence poses. He even declared that he regrets that the development of the technology was his life's work. At that very time, hundreds of researchers, including Sam Altman of OpenAI and Demis Hassabis of Google DeepMind, signed a statement which read only that "mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

A False Sense of Control

One of the benefits of comparing the dangers of the atom to the dangers of AI is the comfort it provides, since the danger of nuclear weapons is, apparently, already under control. However, it should be remembered that the fact that the world has not yet deteriorated into nuclear war is not necessarily the result of the existence of perfect management systems, but because, among other things, human beings have shown judgment at critical moments. This metaphor is helpful primarily because it shows us that although humanity has already developed technology capable of exterminating it, it has nevertheless found ways to deal with the danger through the establishment of treaties and the setting of agreed-upon boundaries.

The problem with it is that it gives us a false sense of control, since the current game is played by rules that are fundamentally different from those that were relevant to the nuclear threat to humanity. AI leaves no traces. The ability to monitor the proliferation of nuclear weapons is based on the fact that uranium and plutonium, the elements required for building nuclear bombs, are rare materials, and their processing to a level that allows for military use is difficult to hide for a long time. To build an enrichment facility, thousands of centrifuges and the establishment of a complex supply chain are required, and the enrichment process leaves traces that allow for monitoring through the placement of cameras and measurements.

If we try to apply this practice to the monitoring of artificial intelligence, we will find that the ground is falling away. The risk in AI systems cannot be assessed by determining whether a certain entity has or does not have an object or material required for their construction. The nuclear control regime is based on asking questions to which the answer is measurable — how much fissile material do you have? — while for the question of what an AI system is capable of doing, there is no such answer, and there is no agreed-upon unit of measurement that defines where the line between innocent development and dangerous capability lies. To such an extent that even a laboratory that is committed to compliance with the rules cannot prove that it is complying with them.

The Logic of the Race

The second difference concerns the operation of the technology. The logic behind nuclear deterrence is that the very existence of the weapon in the hands of a certain player in the international arena does its job, even without them actually using it. When it comes to artificial intelligence, the logic is reversed. No one invests billions in a system that waits for a D-day. It is built to act immediately in the market, in production, in medicine, in the military, and whoever stops in this race of implementation is left behind. The meaning is that even the most cautious players are pushed forward for the same reason: if we stop, someone else will overtake us.

Furthermore, a bomb is a passive object. For it to explode, a person is needed who will make the decision to detonate it. We imagine AI in the same mold, like a machine that will one day wake up and rebel against the humans who created it, but safety experts in the field warn that such a system might cause irreversible damage precisely because it performs with perfect efficiency exactly what we asked it to do. Thus, in an experiment conducted by OpenAI in 2016, an AI system was trained to play a boat racing game. The developers assumed that winning the race, which rewards the boat operated by the model with points, is a nice measure of victory, but the system did its calculations and chose a different tactic: it discovered a remote corner on the track where it was possible to accumulate points by repeatedly collecting three icons that accelerate its speed, which appeared there again and again, and it stayed there to circle in loops. The boat operated by the model did not finish even one lap, but its final score was 20% higher than that of human players who finished the race. This is an amusing case study when it comes to a game, but it may be much less amusing if, for example, it is a system that manages the power grid.

And Maybe the Explosion Has Already Happened?

The 2021 Nobel Peace Prize laureate, Maria Ressa, said last month at the same meeting in Rome that in her opinion, humanity has already passed the "Trinity moment" of artificial intelligence, and that the bomb has actually already exploded in complete silence within our information systems. In the original Trinity moment, on July 16, 1945, the first nuclear explosion in history lit up the New Mexico desert. The mushroom cloud was visible from afar, the shock wave shattered countless windows, and residents woke up in panic from the sound of the powerful explosion. The American army said that an ammunition depot had exploded, the media accepted the explanation, and the world continued as usual. The truth was revealed only three weeks later when the United States dropped the first atomic bomb on Hiroshima.

In other words: the first nuclear explosion changed nothing. The world woke up and began to understand the implications only after a similar bomb destroyed an entire city and caused the deaths of many people. This insight dismantles the comforting assumption that most of us hold, according to which if the situation becomes dangerous enough, governments will wake up in time. But they did not wake up after Trinity, but after Hiroshima. The great moral call of that era also came a decade late, when physicist Albert Einstein and philosopher Bertrand Russell warned the world about the dangers of the atom with the words: "We appeal as human beings to human beings: remember your humanity, and forget the rest." The signatories of the Rome declaration of the summer of 2026 signed it with the exact same words. The question that remains open is whether they will move the decision-makers to action.