Relax, Accountants. Gen-AI Still Can’t Count

At present, there seems to be a new AI-related threat to worry about almost every day. However, based on some recent work using Gen-AI, accountants at least have some good news.

In a piece of research I was doing on oil-rich Guyana, the subject was the total cost of a government windfall cash transfer of G$100,000 to every adult citizen. Somewhere in the discussion, ChatGPT quietly converted the G into a US and began treating the payment as US$100,000 per adult. At roughly G$208–209 to US$1, the real payment was around US$480 the actual programme cost was about G$60 billion, or US$288 million. ChatGPT's version had somehow presented the Guyanese treasury — admittedly quite flush — with a bill of US$60 billion.

What stood out as the numbers reached stratospheric levels was that all the surrounding prose remained fluent, confident and entirely plausible. Nothing difficult had happened mathematically. The currency unit had simply been lost, and no order-of-magnitude sanity check had intervened.

What happened? The weakness is particularly associated with general-purpose generative AI built around large language models — ChatGPT, Claude, Gemini and similar systems — used without additional commercial controls. These forms of AI produce extraordinarily fluent language, but, as the example shows, fluency can outrun numerical discipline. Gen-AI can count; the problem is that nobody can be entirely sure when it has decided not to, while dazzling everyone with perfect prose. AI jamming, perhaps.

For future reference the recurring traps are familiar: currencies, units, percentages versus percentage points, monthly versus annual figures, thousands versus millions or billions, gross versus net values, FX direction and denominators. The deeper problem is not arithmetic but failure to recognise when a numerical check is required.

All of which suggests one simple rule when working with numbers and Gen-AI: no material number should enter a decision document without its unit, source and scale being validated. Hardly glamorous AI governance, but it could prevent some horribly embarrassing presentation interruptions.

And accountants can probably relax for a while yet.