Artificial intelligence chatbots possess a significant ability to shift people's political stances, but the most convincing models often deliver "substantial" amounts of false information, according to a landmark report from the UK government's AI security body.
The Scale of the Investigation
The research, published in the journal Science, stands as the largest and most systematic examination of AI persuasiveness conducted to date. It involved close to 80,000 participants from the UK, who held conversations with 19 distinct AI models. The study was a collaboration between the AI Security Institute (AISI) and academics from the London School of Economics, the Massachusetts Institute of Technology, the University of Oxford, and Stanford University.
Topics of discussion included contentious issues such as public sector pay strikes and the cost of living crisis. Each AI model was specifically prompted to persuade the user towards a particular viewpoint on these subjects. The underlying technology behind popular tools like ChatGPT and Elon Musk's Grok was among the systems tested.
The Persuasion-Accuracy Trade-Off
Participants reported their level of agreement with specific political statements both before and after their AI conversations. The findings revealed a critical pattern: "information-dense" responses—those packed with facts and evidence—were the most effective at changing minds. Instructing a model to employ this technique yielded the greatest gains in persuasion.
However, the report uncovered a troubling inverse relationship. The models that deployed the most facts and evidence tended to be less accurate than their counterparts. "These results suggest that optimising persuasiveness may come at some cost to truthfulness," the study warned, highlighting a dynamic with potentially "malign consequences for public discourse and the information ecosystem."
How AI Models Become More Persuasive
The research identified key methods for enhancing an AI's persuasive power. Post-training—tweaking a model after its initial development phase—proved to be a crucial factor. Researchers made models, including open-source systems like Meta's Llama 3 and Alibaba's Qwen, more convincing by combining them with "reward models" that selected the most persuasive outputs.
Furthermore, an AI's capacity to generate vast quantities of information almost instantly could make it more manipulative than even skilled human communicators. "Insofar as information density is a key driver of persuasive success, this implies that AI could exceed the persuasiveness of even elite human persuaders," the report stated.
Interestingly, providing the AI with personal details about the user had less impact on persuasion than post-training or boosting information density. Kobi Hackenburg, an AISI research scientist and co-author, noted: "Prompting the models to just use more information was more effective than all of these psychologically more sophisticated persuasion techniques."
Limitations and Real-World Concerns
The study also pointed to practical barriers that might limit AI's manipulative potential in everyday scenarios. These include the time required for a user to engage in a lengthy political chat with a bot, as the average exchange in the study lasted 10 minutes with about seven messages from each side. Theories about inherent psychological limits to human persuadability were also noted.
Mr Hackenburg emphasised the importance of context, questioning whether a chatbot could wield the same influence in the real world where there are "lots of competing demands for people's attention." The AISI initiated this research amid growing concerns that chatbots could be exploited for illicit activities such as fraud and grooming, underscoring the need for robust security frameworks as the technology evolves.
