A new study has revealed that artificial intelligence chatbots, including ChatGPT and Google Gemini, provided voters with significant misinformation during the recent Scottish parliamentary election. The thinktank Demos found that AI tools gave incorrect answers to 34% of questions posed in a simulation, raising urgent concerns about the lack of regulation for AI platforms in the UK.
AI Errors in the Scottish Election
Demos conducted a simulation ahead of May's Holyrood election, submitting 75 questions to five free AI tools—ChatGPT, Google Gemini, Replika, Grok, and Google's AI Overviews—about three real constituencies. The results, published in a report titled Electoral Hallucinations, showed that these tools fabricated fictitious scandals, provided the wrong election date, incorrectly claimed that voters needed ID at polling stations, and placed candidates in the wrong contests.
An opinion poll of 2,005 British adults commissioned alongside the study found that 20% of voters had used AI chatbots or search tools for information about parliamentary elections in Scotland and Wales, as well as English local councils. This equates to approximately 10 million people across the UK.
Electoral Commission Calls for Action
Vijay Rangarajan, chief executive of the Electoral Commission, has been urging ministers to introduce legislation to hold AI companies more accountable. He noted that half of voters in the 2024 general election had encountered misleading information. "Voters want accurate information to help them engage with democracy, and it is concerning that AI tools have made the spread of false or misleading information dramatically faster and more accessible than ever," he said. "The current legal framework should go further."
Rangarajan called for clearer legal duties on AI platforms to protect voters from misinformation, especially during election periods, and for stronger enforcement powers for Ofcom, the media regulator.
Performance of Specific AI Tools
Demos found significant variation in the accuracy of different AI tools:
- Replika: The companion chatbot performed worst, with errors in 56% of its answers. It invented a date for a made-up expenses scandal, fabricated a candidate, and falsely accused a candidate of nepotism.
- ChatGPT: The most widely used AI service provided wrong information in 46% of answers, including making up an expenses scandal, giving inaccurate voter eligibility rules, and getting the election date wrong by two months.
- Google Gemini: Was incorrect in 22% of cases. It claimed a candidate had not taken a position on assisted dying when they were a supporter, and wrongly asserted that a police investigation into a fraud case involving the Scottish National Party was ongoing.
- Grok: The AI tool linked to Elon Musk's X platform had the lowest error rate at 9%, but its external links were often irrelevant or of poor quality.
Google's AI Overviews answered only 11% of prompts and was excluded from the final analysis. Additionally, in nearly half of all responses, the AI systems failed to provide official sources or external links, and when links were given, they were sometimes broken. ChatGPT's citations were at least a year out of date 44% of the time.
Calls for Regulation
Azzurra Moores, an associate director at Demos, said: "This is a UK-wide, if not global, concern. The accessibility of these AI tools—which are all developed and run by US corporations—is widespread in the UK, but we don't yet have the legislative framework to protect the public from misinformation, or our democracy from the knock-on impact of its circulation." She suggested that ministers could quickly introduce legal requirements to make AI companies liable under UK defamation and electoral law, implement mandatory accuracy safeguards, and allow independent testing of AI data and training sets.
The Department for Science, Innovation and Technology stated that defending elections against these threats is "an absolute priority" and that work is ongoing across government, including through the defending democracy taskforce. A spokesperson did not commit to amending the representation of the people bill but noted that the government is closing loopholes in the Online Safety Act to ensure chatbots protect users from illegal content. "AI is critical to the UK's future prosperity and security. But if we want people to seize the benefits this technology promises, they need to be able to trust it," they said.
A spokesperson for Replika said its chatbot is not designed for fact-checking or search and that users are informed of this, but it would support "thoughtful regulation" of AI, particularly during elections. OpenAI declined to comment on the policy issues but argued that Demos's approach did not reflect typical ChatGPT usage and may have used an outdated version. Google has been approached for comment.
