This article is part of Demystifying AI, a series of posts that (try to) disambiguate the jargon and myths surrounding AI.
Last week, artificial intelligence research lab OpenAI decided to release a more expanded version of GPT-2, the controversial text-generating AI model it first introduced in February. At the time, the lab refrained from releasing the full AI model, fearing it would be used for malicious purposes.
Instead, OpenAI opted for a staged release of the AI, starting with a limited model (124 million parameters), and gradually releasing more capable models. In May, the research lab released the 355-million-parameter version of GPT-2, and last week, it finally released the 774-million-model, at 50 percent capacity of the text generator.
“We are considering releasing the 1.5 billion parameter version in the future,” OpenAI researchers wrote in a paper they released last week. “By staggering releases, we allow time for risk analyses and use findings from smaller models to inform the actions taken with larger ones.”
As usual, business and tech publications used click-bait headlines to declare the release of the text-generating AI. “Dangerous AI offers to write fake news,” wrote the BBC. “OpenAI just released a new version of its fake news-writing AI,” read the Futurism’s headline. Observer described it as such: “OpenAI Can No Longer Hide Its Alarmingly Good Robot ‘Fake News’ Writer.” Other outlets used similar sensational headlines and made references to Elon Musk, OpenAI’s co-founder, to create even more hype around the topic.
But while most publications were busy warning about threat of an AI-triggered fake news apocalypse (and drawing money-making clicks on their websites), they missed the important points that the OpenAI researchers raised (and didn’t raise) in their GPT-2 paper.
Generating coherent text is not enough to produce fake news
In its paper, OpenAI mentions fake news as just one of the potential malicious uses of its AI. “We chose a staged release process, releasing the smallest model in February, but withholding larger models due to concerns about the potential for misuse, such as generating fake news content, impersonating others in email, or automating abusive social media content production,” the researchers wrote.
But fake news got the greatest attention in the media possibly because the topic has gotten so much attention in the wake of the 2016 U.S. presidential elections. The authors of the paper discuss the threat thoroughly.
According to the OpenAI researchers, “Humans can be deceived by text generated by GPT-2 and other successful language models, and human detectability will likely become increasingly more difficult.”
The researchers further note that as they increase the size of the AI model, the quality of the text increases. And here’s what they mean by “quality”: “With a human-in-the-loop, GPT-2 can generate outputs that humans find credible.”
The researchers further describe that in some experiments, humans considered the output created by their AI model “credible” about 66 percent of the time. Samples of the 774-million-parameter GPT-2 model were “statistically” similar to New York Times articles 75 percent of the time.
These are all interesting achievements, but the problem with what the OpenAI paper and the articles covering the text-generating machine learning model is that they suppose that generating coherent text is enough to generate fake news. “Credible” and “quality” are vague words, and from the text of the paper, the authors assume that readers will consider anything that is coherent and passes as something written by an English-speaking human as credible.
Say you find a nondescript piece of paper lying on the floor, which contains a very coherent and eloquent excerpt about a nuclear war between the U.S. and Russia. Would you believe it? Probably not. Coherence is just one of the many requirements of spreading fake news.
The more important factor is trust. If you read the same text on the front page of The New York Times or The Washington Times, you’re more likely to believe it, because you trust them as credible news sources. Even if you see a minor grammatical flaw in the story, you’ll probably dismiss it as a human mistake and still believe what you read.
If you can deceive your readers into trusting you, you won’t even need to be a very good English writer. In fact, the group of Macedonian teens who created a fake news crisis during the 2016 presidential elections didn’t even have proper English skills. The key to their success was websites that looked authentic and trustworthy, in which they published fake stories with sensational headlines that triggered reactions from users across social media and gamed trending algorithms.
Even authentic news websites often get caught up in the propagation of false stories. A recent example is a non-existent flaw in the VLC media player, which was reported by several reputable tech publications and resulted in unwarranted panic around one of the most popular media players. The case neither involved artificial intelligence, nor loads of content. A few, well-placed stories triggered the damage.
And let’s not forget that machine learning algorithms, used in GPT-2 and other state-of-the-art AI models, are just statistical machines. They find correlations between different text excerpts and create new ones that are statistically similar to those. They have no understanding of the complexities and different nuances of human language, not even as much as a six-year-old kid.
Metadata is key to fight AI-generated fake content
While the OpenAI researchers lay out the ongoing cat-and-mouse competition between AI techniques to generate and detect synthetic text, they raise a very important point that has gone mostly unnoticed to the publications that covered the GPT-2 paper.
“Preventing spam, abuse, or disinformation online does not rely entirely on analyzing message content,” the researchers write, adding, “Metadata about text, such as time taken to write a certain amount of text, number of accounts associated with a certain IP, and the social graph of participants in an online platform, can signal malicious activity.”
Most online platforms such as Twitter, Facebook and Amazon already use metadata as cues to discover and fight bot-driven activities. While the method might sound trivial, it would be very effective against an AI model that could create large volumes of coherent text such as tweets or product reviews.
A person or entity armed with the most advanced AI-powered text generator would still need to employ thousands of devices with different IPs and in different geographical locations to be able to game trending algorithms in social media or ecommerce platforms. Again, in this case, a well-placed, sensational post by an influencer account can create the kind of organic reaction that no large-scale AI algorithm could synthesize.
Malicious actors are not interested in AI-generated text
The researchers note in their paper, “Our threat monitoring did not find evidence of GPT-2 direct misuse in publicly-accessible forums but we did see evidence of discussion of misuse.”
Those discussions were mostly inspired by the string of sensational articles that followed the initial release of GPT-2. Interestingly, the writers note that most of those discussions had declined by mid-May, a few months after the first version of GPT-2 was made available to the public.
“We believe discussion among these actors was due to media attention following GPT-2’s initial release; during follow-up monitoring there was no indication that these actors had the resources, capabilities, or plans to execute at this time,” the researchers observed.
While it’s easy to spin fantastic stories about the destruction AI models like GPT-2 can cause, the reality is that the people who would want to use it as a weapon know that it takes much more than coherent text to run a successful fake news campaign.