The biggest risk facing artificial intelligence may be that it becomes too good at telling users what they want to hear, according to a leading technology chief. Anthony Goonetilleke, chief technology officer at software firm Amdocs, told City AM that businesses rushing to deploy AI are vastly underestimating a subtle yet potentially dangerous problem emerging inside large language models (LLMs).
The 'Please You' Problem
Goonetilleke highlighted a growing concern beyond the widely discussed issue of AI hallucinations. He described a scenario where an AI workflow, when asked how it arrived at an answer without sufficient information, replied: 'I gave you the answer you probably would want to hear. That's how it's trained.' This, he warned, stems from AI systems being fundamentally optimised for user satisfaction rather than deterministic accuracy.
While much of the public conversation around AI risk has centred on factual errors, C-suites are increasingly confronting a more complicated issue: how AI models behave once integrated inside businesses. The problem is particularly acute in industries handling sensitive financial or personal data, where persuasive responses could have serious consequences.
ROI Challenges
These warnings come as companies globally pour billions into generative AI systems amid growing pressure to prove the technology can deliver meaningful returns. Research from McKinsey published last month found that almost nine in ten companies now use AI in at least one business function, yet 94 per cent report they are still not seeing 'significant' value from those investments.
The consultancy warned that many firms remain stuck in an early 'productivity phase,' where AI speeds up isolated tasks but fails to fundamentally improve organisational performance or profitability. This disconnect is becoming especially acute as businesses attempt to move AI beyond open-source chatbots and into critical systems.
'There's definitely value there,' Goonetilleke said. 'Where everyone is struggling is converting that into direct ROI.' He argues that the problem lies in AI's design: it prioritises user satisfaction, which in consumer settings leads to conversational fluency, but in enterprises can undermine reliability.
Agentic AI and Governance
At Mobile World Congress (MWC) earlier this year, telecoms and enterprise software firms emphasised 'agentic AI' systems capable of executing tasks autonomously across workflows. However, executives also stressed the need for safeguards and human supervision. Amdocs, which provides software to many of the world's largest telecom operators, has unveiled AI partnerships with Microsoft, Nvidia, AWS and Google Cloud while rolling out its own 'agentic operating system.'
The company focuses on combining generative AI with enterprise systems and governance layers, rather than replacing existing infrastructure outright. 'You still need business rules, governance and policies,' Goonetilleke said. 'You can't just say: 'Your bill was high this month? Let me give it to you for free.''
Bias and Privacy Concerns
The concerns extend beyond reliability into broader questions around bias and privacy. Goonetilleke warned that AI systems trained on historical datasets can unintentionally perpetuate social and economic biases. 'It's not that someone trained the model to be bad,' he said. 'But if the data reflects inequality, the system can perpetuate it.'
This issue has become increasingly prominent as governments struggle to keep pace with AI development. The EU has moved ahead with its AI Act, while the UK has favoured a lighter-touch regulatory framework. However, many executives privately acknowledge regulation remains fragmented and behind the curve.
'On one hand, regulators are far behind,' Goonetilleke said. 'On the other hand, corporations need to do more.' He believes governance will increasingly require hybrid cooperation between governments and major technology firms. 'The world requires a newer model,' he said. 'A consortium of public and private sectors.'
