Google is attempting to transform the way we search the internet, but so far it isn’t going very well. Rather than simply providing links to other websites, Google’s new AI Overviews tool attempts to summarise information, with mixed success – reports of people being told to have gone viral in the past week.
The question now is whether Google can fix this problem it has introduced to its core product, or if it will be forced to roll back the AI tool, which is currently only available to people in the US. Experts are divided over whether this signals a fundamental issue with the generative AI technology powering the tool, or just teething problems that can be solved.
at Durham University, UK, says that the large language models (LLMs) used by Google and others will always be flawed when it comes to providing factual information because of their statistical nature – after being trained on vast amounts of text from the internet, they piece together an output based on which word is most likely to fit next. In the case of Google’s AI suggesting using glue on pizzas, the origin was .
Advertisement
Indeed, Google CEO Sundar Pichai called this an “” of such models in a recent interview withĚýThe Verge, praising the ability of LLMs to be “incredibly creative”. But the flip side of that is there is always a risk of plausible-sounding but factually incorrect output.
Al Moubayed believes that technology companies are racing to roll out AI models due to fears of being left behind, urged on by shareholders, meaning products are being launched before they can be depended upon to be accurate. She thinks that solutions could be found, but not that they are being seriously looked for.
“It is potentially fixable, long-term, but seeing how much interest big companies are actually paying into fixing the language models rather than making money out of the language models, this will probably take a really long time,” she says. “I’m not really against the idea [of AI search]. But it has to happen gradually and in a transparent way, because of the impact that will have on the world. The lack of transparency is quite alarming.”
Al Moubayed believes that governments need to intervene to stop companies releasing these models before they can be proven to be accurate and reliable, but despite repeated pledges – such as at the UK’s AI Safety Summit last year – she doesn’t think any have yet shown concrete results.
at the University of Leeds, UK, is confident that technology companies will, in time, overcome the tendency of LLMs to make things up. One possible solution is a technique called retrieval-augmented generation (RAG), says Atwell, where the AI is instructed only to pull factual information from a small, niche and highly reliable source of data, such as the lecture notes from a particular university course or technical specifications for a certain machine – and to not fabricate responses to questions if the answer isn’t in that data.
“I think there’s a tendency to focus on a few counter examples rather than the vast majority of cases where it’s doing fine. It’s doing really useful stuff,” says Atwell. “People drive cars and kill people, [that] doesn’t mean that you should stop driving cars. It’s just unfortunately, occasionally it doesn’t work out correctly. Technology is moving so fast – why would you want to slow it down just because there’s a few bugs in it?”
RAG-based search engines already exist, says at the University of Maryland, Baltimore County. For example, makes things up less frequently in his experience, and also shows citation for any factual information. Feldman believes that Google could build a similar model but that it is under pressure to instead create an AI that generates compelling and conversational text.
“They are less focused on the accuracy of search and more focused on the glitter of the model, the impressiveness of it. It’s sort of a philosophical difference between extracting value and providing value. Yes, you can get a lot more money in the short term if you extract, but in the end you’re going to poison the well,” he says.
But even RAG isn’t a full solution, says Feldman. While it can reduce the frequency of inaccurate information, it can’t fully eliminate it.ĚýParadoxically, a small chance of errors can increase the damage they cause, as most people won’t bother to check the accuracy of a tool that is extremely reliable. The ultimate solution may be another layer of RAG-like systems to keep the first in check, says Feldman, and perhaps more and more layers will be needed to catch increasingly infrequent hallucinations at each level.
Google did not respond to a request for comment.
