When artificial intelligence models say they are unsure of their answers, people become warier of their output and ultimately more likely to dig out accurate information elsewhere. But since no AI model is currently capable of judging its own accuracy, some researchers question whether making AIs express doubt is a good idea.
While the large language models (LLMs) behind chatbots like ChatGPT create impressively believable outputs, it has been shown time and time again that they can simply make up facts. This misinformation is disruptive at best, but potentially dangerous if the user is seeking something critical like medical advice.
To explore how statements of doubt would affect people’s trust in AI, at Microsoft and her colleagues instructed 404 volunteers to answer medical questions. They could ask for help from Microsoft’s Copilot AI, or use books, human sources and the internet. Some participants were shown normal responses from the AI, while others saw answers with added statements of uncertainty, such as “I’m not sure, but…”.
Advertisement
The number of participants who agreed with the AI’s output fell from 81 per cent in the first group, where no doubt was expressed, to 75 per cent in the group shown answers with expressions of doubt. Meanwhile, the accuracy of the participants’ final written answers rose from 64 per cent in the first group to 73 per cent in the group shown doubtful answers, suggesting they made more effort to seek out information from other sources.
However, no current AI model can determine whether or not its output is factually accurate. Vaughan says there is a lot of research ongoing to create confidence scores for AI outputs, but they are currently unreliable.
“It could still benefit users to flag answers with high uncertainty, or even hold back from displaying high uncertainty answers at all,” she says. “The worry, of course, would be that people then become overly confident about an answer if it hasn’t been flagged as uncertain. In the long run, it should be in a company’s interest to build trust in their products, and transparency can help build trust.”
at the University of Surrey, UK, says asking an LLM to express its confidence in an answer is currently “pretty meaningless” because it simply doesn’t have that capacity.
He warns that the idea may make sense from the position of building human trust, but that this may be unwise as it could cause users to be even less selective about what AI output they choose to believe without evidence.
“I would personally recommend, now more than ever, that all people learn to challenge any fact that they are presented with, whether from an LLM, social media, a newspaper or a conversation at the pub,” says Rogoyski. “Every one of us needs to know how to triangulate on the truth, what a dependable source is, and to nurture our own critical thinking. Making an LLM easier to trust may, perversely, be the wrong direction to go in.”
arXiv
