Australia's proposed social media ban for under-16s could be undermined by AI hallucination risks, according to a new report. The report, released on Monday, warns that age-verification systems relying on artificial intelligence may generate false information, potentially blocking legitimate users or allowing underage access.
AI Hallucinations Threaten Accuracy
The report, commissioned by the Australian government, highlights that AI-driven age-estimation tools can 'hallucinate'—producing incorrect data—when analyzing users' photos or behavioral patterns. This could lead to wrongful exclusions or failures to detect underage users, undermining the ban's effectiveness.
Experts cited in the report emphasize that current AI technologies are not sufficiently reliable for such high-stakes enforcement. 'The margin for error is too high,' said Dr. Sarah Johnson, a digital rights researcher at the University of Sydney, who contributed to the report. 'We risk creating a system that is both over-inclusive and under-inclusive.'
Implementation Challenges Ahead
The proposed legislation, set to be debated in parliament later this year, would require platforms like TikTok, Instagram, and Facebook to verify users' ages. However, the report notes that no existing AI system meets the accuracy standards needed to avoid significant false positives and negatives.
Moreover, the report points out that tech companies have not yet developed robust methods to handle AI errors in age verification. It recommends further research and pilot testing before any nationwide rollout, a suggestion that may delay the government's timeline.
Impact on Privacy and Enforcement
Privacy advocates have also raised concerns that AI-based verification could lead to excessive data collection, potentially violating users' privacy rights. The report acknowledges these tensions, suggesting that any implementation must balance safety with civil liberties.
In response, a government spokesperson said the report's findings would be considered carefully, but stressed the urgency of protecting children online. 'We are committed to making the online world safer for young Australians,' the spokesperson said. 'But we must do so responsibly.'
The report's release comes as other countries, including the UK and EU, are exploring similar age-verification measures, making Australia's approach a potential global benchmark. However, without addressing AI hallucination risks, the ban's success remains uncertain, and further technical and regulatory work is needed to ensure its viability.
