Large language model chatbots like those powering ChatGPT can reduce people’s beliefs in conspiracy theories by debating with them. And the approach they employ could even teach people better ways of interacting with conspiracy theorists.
Although suggests that the level of beliefs in conspiracy theories in the population hasn’t changed substantially in the US or in six surveyed European countries over about a decade, those who subscribe to such ideas have persistent, dogged faith in those beliefs.
Many people try to puncture or counter those beliefs, but not always successfully. “Attempts to persuade conspiracy theorists [they are wrong] have historically not worked out very well,” says at the Massachusetts Institute of Technology.
Advertisement
However, Costello and his colleagues have come up with an approach that is proving successful. They conducted an experiment with 2190 self-admitted believers in conspiracy theories. The participants provided an outline of a conspiracy theory they subscribed to, then were placed into a conversation with the GPT-4 Turbo large language model (LLM) artificial intelligence.
Each person was presented with an AI-generated summary of their conspiracy theory and was asked to rate the level of confidence they had that the theory was true. The LLM then began a chat in which each side offered three contributions to the conversation. Next, the people were asked to put forward their case again, and asked once more to rate their level of confidence in the conspiracy theory.
Overall, across the participants, belief in the specific theory they mentioned dropped by about 20 per cent, while the researchers also found a general decrease of belief in other conspiracy theories. The effect lasted for at least two months, suggesting people permanently changed their views after conversing with the chatbot.
“My hunch is that prior attempts to debunk conspiracy theories have been limited by how generalised they’ve had to be,” says Costello. A broad, shallow engagement with the facts often doesn’t work in persuading conspiracy theorists, who tend to have read deeply into a subject, even if their understanding of it may be wrong, he says.
The AI model took a more direct, fact-based approach: in 83 per cent of the conversations, the AI used evidence-based arguments for alternative perspectives.
The ability of generative AI to synthesise large volumes of information appears to have helped the chatbot persuade those who don’t want to be convinced to change their mind.
The AI also rarely deployed emotional appeals – which could be used by humans. This meant the AI could avoid invoking emotional responses that could hinder any persuasive power. “Part of the efficacy of the intervention might just be that they’re not very judgmental,” says Costello. “They’re patient. They really meet people where they are.”
Costello believes the method could provide a model for wider deployment, either by using chatbots, or by having people mimic the chatbot approach. “In terms of the persuasive methods, being congenial, non-judgmental, focusing on facts, I think those are all things that humans would be able to do in theory.”
“The results of this study, while not having undergone the rigours of the peer-review process, are certainly of interest,” says at the University of Huddersfield, UK. “To have a route to engage and potentially positively influence individuals who possess a belief in such conspiracy theories, with facts and evidence, and to get them to re-evaluate their views, would have a substantial impact,” he says.
PsyArXiv
