In the 2023 film Oppenheimer, during the Manhattan project to develop the nuclear bomb, one of the concerns was whether the nuclear explosion created during a test might create such high temperatures that it leads to the nuclei of nitrogen atoms in the atmosphere fusing together and triggering a chain reaction that essentially sets the atmosphere on fire, frying the entire planet. Oppenheimer tells general Leslie Groves, the director of the project, that the calculations of Arthur Compton showed that the chance of such a thing happening was less that three in a million, and thus acceptable. When Groves said that he was hoping that the answer would be zero, Oppenheimer replied that you could not expect such an answer from theory alone..

While the idea that theory can never give you absolute certainty about anything is correct, the actual story is more complicated. It turns out that the Oppenheimer-Compton story is based on an article written by Pearl S. Buck, based on an interview she had with Compton, and some of the details are apocryphal. Hans Bethe, head of the theoretical program at Los Alamos, who had shown how fusion reactions lay behind the energy production of stars, had concluded early on that the idea of a runaway fusion reaction igniting the air was so small as to not be worth worrying about.

But the idea of a three-in-a-million is an acceptable level risk for deciding to go ahead when something undesirable might happen (now referred to as the Compton number), has taken root and was recently invoked by computer scientist Max Tegmark in relation to whether the current AI efforts could escape from human control and lead to a runaway catastrophe, similar to the fears about nuclear consequences. He argues that calculations analogous to Compton’s should be done for AI before that work is taken further.

Artificial intelligence companies have been urged to replicate the safety calculations that underpinned Robert Oppenheimer’s first nuclear test before they release all-powerful systems.

Max Tegmark, a leading voice in AI safety, said he had carried out calculations akin to those of the US physicist Arthur Compton before the Trinity test and had found a 90% probability that a highly advanced AI would pose an existential threat. 

The US government went ahead with Trinity in 1945, after being reassured there was a vanishingly small chance of an atomic bomb igniting the atmosphere and endangering humanity.

In a paper published by Tegmark and three of his students at the Massachusetts Institute of Technology (MIT), they recommend calculating the “Compton constant” – defined in the paper as the probability that an all-powerful AI escapes human control. In a 1959 interview with the US writer Pearl Buck, Compton said he had approved the test after calculating the odds of a runaway fusion reaction to be “slightly less” than one in three million.

Tegmark said that AI firms should take responsibility for rigorously calculating whether Artificial Super Intelligence (ASI) – a term for a theoretical system that is superior to human intelligence in all aspects – will evade human control.

“The companies building super-intelligence need to also calculate the Compton constant, the probability that we will lose control over it,” he said. “It’s not enough to say ‘we feel good about it’. They have to calculate the percentage.”

Tegmark said a Compton constant consensus calculated by multiple companies would create the “political will” to agree global safety regimes for AIs.

[Note that the newspaper article erroneously refers to the threshold as one in three million and not three in one million as Tegmark’s paper correctly says.]

As for the film Oppenheimer, it was a huge critical and popular hit, winning seven Academy Awards, including for best picture, best director, best lead actor (Cilian Murphy as Oppenheimer) and best supporting actor (Robert Downey Jr. as Lewis Strauss). It is long (around three hours) and I found the first half to drag. It had a lot of characters and complicated and interweaving story lines. Although I knew the general story and the names of the principals, and the science involved were all familiar to me (the only scientist named in the film whom I actually met in person was Hans Bethe), I found that the way the story was told, with its frequent jumping back and forth in time, to be confusing. I wondered how viewers without the benefit of a scientific background in that area made sense of it. I also found the ending of this part of the film, showing the triumphalism of the US and the cheering that accompanied the news of bombing of Hiroshima and Nagasaki, to be distasteful. The deaths of all those ordinary Japanese people will, or at least should, remain forever part of the collective guilt that all physicists have to live with.

The second half of the film was much better. It dealt with the political intrigues that led to the revocation of Oppenheimer’s security clearance and the way that Lewis Strauss, while pretending to be a disinterested person and even a supporter of Oppenheimer, schemed to take him down during the dark days of the McCarthy era.

The film was successful in showing the complex character of Oppenheimer. His sterling scientific credentials and his abilities as an administrator shepherding the work of so many arrogant scientists were never in doubt and he was highly respected by his peers, which is why he was able to recruit so many of the most eminent physicists to come to Los Alamos, as well as work elsewhere, on the Manhattan project. His political views and his willingness from time to time to compromise his principles, as well as his seeming naivete about what might be the consequences of his work, were well displayed in the film

Here’s the trailer.