Did you know cats have been to the moon? That it is safe to look at the sun for 15 minutes or even more as long as you have dark skin? Or that to be healthy you have to eat a small stone a day?
These are some of the latest pearls of wisdom Google is offering its US users (we’re not so lucky in the UK yet). “Let Google do the searching for you,” the search giant promised when it introduced a feature called AI Overviews earlier this month. This integrates Google’s Gemini generative AI model into its search engine. The answers it generates are displayed above the traditional list of ranked results. And you can’t get rid of them.
AI Overviews didn’t have the impact Google was hoping for, to say the least. It certainly garnered instant popularity on the internet, with people sharing their favorite answers. Not because they are useful, but because they are so funny. For example, when you ask AI Overviews for a list of fruits ending in “um,” it returns: “Applum, Strawberrum, and Coconut.” This is what in AI parlance is called a “hallucination”.
Despite having a $2 trillion market cap and the ability to employ the greatest minds on the planet, Google continues to stumble in AI. Its first attempt to join the generative-AI gold rush last February was the ill-fated Bard chatbot, which had similar problems spewing factual inaccuracies. In its first live demonstration, Bard mistakenly announced that the James Webb Space Telescope, which won’t launch until 2021, had taken the “first pictures” of Earth outside the solar system. The mistake wiped $100 billion off Google’s market value.
This February, Google experimented with AI again, this time with Gemini, an image and text generator. The problem was that there were very heavy barriers to diversity. When asked to produce historically accurate images, it will instead generate black Nazi soldiers, Native American founding fathers, and a South Asian female pope.
It was a “good faith mistake”, it admitted The Economist. But Google wasn’t caught off guard by the problems inherent in generative AI. It would have known about its capabilities and pitfalls.
Before the current AI craze really took off, analysts had already worked out that generative AI was unlikely to improve the user experience and could actually make it worse. That caution was abandoned as investors began piling in.
So why does Google’s AI produce such rotten results? It actually works exactly as you would expect. Don’t be fooled by the “artificial intelligence” brand. Essentially, AI Overviews just tries to guess the next word to use, based on statistical probability, but without any connection to reality. An algorithm can’t say “I don’t know” when asked a difficult question because it doesn’t “know” anything. He can’t even perform simple math calculations, as users have demonstrated, because he has no basic number concept or valid arithmetic operations. Hence the hallucinations and lapses.
This is less of a problem when the output doesn’t matter that much, such as when the AI processes an image and creates a small problem. Our phones use machine learning every day to process our photos, and we don’t notice or care much about most of the problems. But Google advising us all to start eating rocks is no small matter.
Such mistakes are more or less inevitable due to the way the AI is trained. Instead of learning from a curated data set with precise information, AI models are trained on a vast, virtually open data set. Google AI and ChatGPT have already scoured the web as much as possible and needless to say a lot of what is on the web is not true. Forums like Reddit are full of sarcasm and jokes, but they are treated by the AI as credible, as sincere and correct explanations of problems. Programmers have long used the phrase “GIGO” to describe what’s going on here: garbage in, garbage out.
The AI hallucination problem is consistent across all domains. This largely precludes generative AI from being practically useful in commercial and business applications where you might expect it to save a lot of time. A new study of generative AI in legal work finds that the extra vetting steps now required to ensure AI isn’t hallucinating cancels out the time saved by implementing it in the first place.
‘[Programmers] they still make the same blunders as before. No one has actually solved hallucinations with broad-language models, and I don’t think we can,” cognitive scientist and veteran AI skeptic Professor Gary Marcus noted last week.
Another problem now emerges. AI makes an already bad job worse by generating false information that then pollutes the rest of the web. “Google learns whatever junk it sees on the internet, and nothing generates junk better than AI,” as one X user said.
Last year, leading AI companies admitted that after they ran out of content to exhaust from the web, they started using synthetic data for training – that is, data generated by generative AI itself. A year ago, OpenAI’s Sam Altman said he was “pretty confident that soon all data will be synthetic data” compiled by other AIs.
This is a huge problem. Essentially, this causes the models to “crash” and stop producing useful results. “Model collapse is when generative AI becomes unstable, unreliable, or stops functioning. This can happen when generative AI models are trained on content generated by AI rather than humans, Professor Nigel Shadbolt of the Open Data Institute warned last December. One researcher, Jatan Sadowski, called this phenomenon “Habsburg AI” after the Spanish Habsburg dynasty, which died out in the 1700s as a result of diseases caused by inbreeding.
You could argue that something like this already happens without the help of AI, such as when a false fact is inserted into Wikipedia, cited in the media, and then the media citations become a justification for its continued inclusion in Wikipedia.
AI simply automates and accelerates this process of generating lies. This week, Telegraph gave the following example: “When Google claimed that there was no African country beginning with the letter K, the response appeared to be based on a ChatGPT web discussion that asked the same question incorrectly. In other words, AI is now using other AI fictions as gospel.
The most appropriate description of this phenomenon comes from some American researchers who coined the phrase “Model Autophagic Disorder” or MAD last year. They wanted to recall the practice of introducing bovine prions into cattle feed, a practice that causes bovine spongiform encephalopathy, or mad cow disease. “Our main conclusion in all scenarios is that without enough fresh real-world data in each generation of an autophagic circuit, future generative models are doomed to a progressive decrease in quality (precision) or diversity (recall),” they write.
Very few people warned about the shortcomings of generative AI when OpenAI open sourced its ChatGPT tool in November 2022. Now ChatGPT has polluted the web and poisoned itself and other AI tools. Cleaning this up will be a huge challenge. While the promised benefits of AI remain elusive, the costs are clearly starting to mount.
Andrew Orlovsky is a weekly columnist in Telegraph. Visit his website here. Follow it to X: @AndrewOrlowski.