The Center for AI Safety recently posted this simple statement, which was signed by a variety of AI scientists and other notables.
“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
I find it ludicrous to think that a model that predicts the next word should be considered as the same level of threat as nuclear war or pandemics! Words do indeed have power, but statements like this are more appropriate to cults than to serious science or technology.
Pandemics and nuclear Armageddon are both clear and present dangers. They actually exist. The means by which they lead to death are well understood and obvious. None of that is true about artificial intelligence. Current artificial intelligence methods are not capable of causing such events and future developments are unlikely to be any more capable.
Today’s state of the art in artificial intelligence methods, large language models, are based on a transformer model. They operate by learning to guess the next word, given a context of other words. Their parameters capture the statistical properties of word usage in the language. Some models are further trained by humans to try to adjust their parameters to give preference to certain patterns and deflate others, but their sole function is to predict the next word. Some people ascribe greater cognitive abilities to these models, but this assertion is based on a logical fallacy of affirming the consequent. Here is an example of that fallacy that is so outrageous as to make the fallacy obvious.
If the robot shot Abraham Lincoln, then Lincoln is dead. Lincoln is dead, therefore, he was killed by a robot.
We all know that Abraham Lincoln was killed by John Wilkes Booth and not a robot. Lincoln being dead does not imply that a robot killed him. In current AI, the argument is the same:
If the computer reasons, then it will describe a solution to this reasoning problem. It describes a solution to this reasoning problem, therefore, it can reason.
A description of a reasoned solution no more implies the presence of reasoning than the death of Lincoln implies that a robot killed him. The fallacy of affirming the consequent fails to consider that other events could have caused the observed outcome. Booth could have killed Lincoln, language models could have produced the language that we observed simply by predicting subsequent words. In the case of Lincoln we know that there was no robot that could have killed Lincoln. In the case of AI, despite claims to the contrary, we know that just predicting the next word is not reasoning. The model produces patterns that are consistent with its learned statistics, and has difficulty with tasks that require other patterns. As Miceli-Barone (2023) has shown:
“LLMs still lack a deep, abstract understanding of the content they manipulate, making them unsuitable for tasks that statistically deviate from their training data.”
Among the large volume of text on which they have been trained, there are some examples of the kinds of reasoning being evaluated. Predicting words according to the statistical models derived from these texts would also produce a description of a solution to the problem without having to actually reason.
At best, large language models might be said to exhibit a limited kind of “crystallized intelligence.” Some psychologists distinguish between “fluid intelligence,” which is roughly the ability to reason through a problem, and “crystallized intelligence,” which is the ability to use known information to solve it. Large language models use the statistical patterns of text that they have read to solve the problem, but there is no evidence that they actually “reason” about anything.
As with other machine learning models, large language models do one thing and do it well—they predict the next word. They are not manifestations of general intelligence. They are capable of producing a wide variety of language, but all they know (I use the word “know” here loosely) is a statistical representation of the text on which they have been trained. If those statistical patterns are consistent with a certain task, then they can perform that task, if they are not, they fail.
Furthermore, the benchmarks on which many models are evaluated are also widely described among the training text. The large language models have had the opportunity to memorize and model the response patterns appropriate to these benchmark problems, so passing them does not imply any underlying knowledge. They can give the right answer to bar exams without knowing anything about the law. Lawyers Steven Schwartz and Peter LoDuca are facing court sanctions for submitting a legal brief written by ChatGPT, which cited six cases fabricated by the LLM. It is easier to think of these models as intelligent if we ignore such failures.
Large language models, though useful for many things, thus, are just not capable of presenting a direct existential threat to humanity. Guessing the next word is just not sufficient to take over the world. Even a radical increase in the size of the models will not change the fundamental characteristic that they are statistical word predictors. Larger models may be more fluent, they may take advantage of larger contexts and simulate more cognitive processes, but they still are just language models capturing the (complex) patterns of word usage.
Language models present no immediate threat to human existence, but people can speculate that other AI models might. Three scenarios have been described.
1- According to the singularity scenario, machines become so intelligent that they leave humans behind. If we are lucky, the machines will keep us as pets.
2- According to the Terminator trope, machines become so intelligent that they become sentient and see humans as a threat to their own existence.
3- According to the paperclip perspective, machines become intelligent enough to improve their own function and single-mindedly execute their mission of making paperclips or something else, even at the expense of human resources.
None of these scenarios is likely; each one falls on a logical flaw.
The singularity scenario is the idea that a system will become so intelligent that it will improve itself exponentially, resulting in a singularity analogous to the event horizon of a black hole. (the word “singularity” has been used to refer to a scenario where humans upload their consciousness to a machine, but that is not the concern here).
Vernor Vinge, in 1983, popularized the idea based on some earlier discussions by John von Neumann and I.J. Good. Vinge predicted that soon there would be an invention that would be able to improve its own programming and therefore improve its own intelligence with ever-increasing speed. We now know that the main limitation to artificial intelligence is not the quality of the programming but the quality, scale, and scope of the data. Better programming might lead to better matrix multiplication, for example, but that does not lead directly to better intelligence any more than having access to more electrical power leads to intelligence.
The two other scenarios are variations on the genie schema. A fictional genie will grant wishes, but there is always a cost when the genie interprets the wishes too literally. These scenarios are thought experiments, which, in order to succeed, make contradictory assumptions. They assume that the machine is intelligent enough to create new methods to address its problem, but dumb enough to obsessively pursue narrow goals. Like the genie, they are intelligent enough to be successful but thoughtless enough to be dangerous. They have to be both bright and dumb at the same time. If they could improve their capabilities, why are they presumed to be stuck with an unintelligent situational analysis?
All three scenarios also depend on some kind of “miracle,” where, by accumulating enough complexity, they suddenly because uncontrollably intelligent. They rest on the assumption that computational complexity is sufficient to produce uncontrollable intelligence. This is another logical fallacy, mistaking necessity (intelligent models must be complex) for sufficiency (complex models must be intelligent). The sufficiency conclusion simply does not follow. Complexity does not guarantee intelligence.
The large language models do not represent any breakthrough in artificial intelligence algorithms. They are still just transformers. They differ from their predecessors in the amount of text on which they are trained and on the number of parameters used to model that text. More data and more parameters allow the models to represent more patterns. Additional data might lead to the appearance that the model had more intelligence, but even then, there is diminishing value in increasing size.
Large language models should lead us to reexamine even what we would mean by increasing intelligence. One definition of intelligence might be the ability to solve multiple problems. So, an increase in intelligence would be indicated by an increase in the number of problems that can be solved. Large language models rely on a different approach in that they collapse multiple problems to a single one of guessing the next word. The variety of texts on which they have been trained allow them to solve what appear to be multiple problems, but they do it all by solving one kind of problem, statistically predicting the next word. And it is people, not the machine who had the insight to mimic multiple problem solvers with a word-guesser.
If a large language model solves only one problem and our definition of a problem is arbitrary, then it is not clear just what an increase in intelligence means in the context of large language models. So far, machine learning consists solely of parameter adjustments and so far, only humans can reconceptualize problems to map them against those parameters. General intelligence requires both capabilities, but there is essentially no research on how a machine might reconceptualize problems to make unsolvable ones solvable.
From a practical perspective, large language models enable a range of tasks that were difficult to achieve with other methods. From an engineering perspective they exhibit a level of scaling progress that was difficult to imagine even a few years ago. From a theoretical perspective, they do little that is innovative beyond scaling an approach that was described in 1948. Unprecedented scale for memory and for computing power are necessary to achieve high levels of performance, but they are not sufficient. There is no evidence that large language models do anything that cannot be explained by their core capability of predicting the next word based on the statistics that they have learned from ingestion of massive amounts of text.
Language models do not represent a breakthrough in artificial general intelligence; they are just as focused on a narrow task as is any other machine learning model. It is humans who attribute cognitive properties to that singular task in a kind of collective delusion. There is no way that these models could exceed human intelligence because everything that they “know” is derived from written records created by fallible humans or from human feedback. For all they get correct, they also get a substantial amount wrong. As a result, there is no way that they could cause an extinction-level event.
One could imagine that a future AI might gain some physical capabilities and might be dangerous, but that is just unjustified speculation, unconstrained by our current level of knowledge. Pandemics and the threat of nuclear war are real. An asteroid striking the earth is a real possibility. Even the sun going supernova is less speculative than current artificial intelligence models causing the extinction of humans. It is not that it cannot happen. Any of these events could happen, but we need to assess their likelihood against more evidence than just our imagination of their possibility.
We do not have to speculate about the harmful effects of nuclear war or pandemics, but any claims about the lethality of artificial intelligence are based purely on unjustified speculation. On the other hand, it does not take much speculation to predict that someone with a fear of artificial intelligence might take it on himself to mitigate that danger by attacking AI researchers. Like pandemics and threats of nuclear war, we have seen how fake news can lead to real world violence. Let us not engage in unbridled speculation and thereby put targets on researchers’ backs.