GLM 5.2 is (nearly) as accurate as a human book-keeper at less than 1% of the cost

Vineyard Finance ran an experiment that sounds almost like a stunt until you look at the numbers: they handed GLM 5.2, an open-weights model served via Fireworks AI, the job of preparing a real UK VAT return for Q1 2026. Fifty-nine transactions, a bash-only tool harness, no vision needed since every receipt was a text PDF. The model logged into the accounting software through a CLI, worked through three months of bank feeds one by one, and came out the other end with a return that missed the human-prepared net VAT figure (Box 5) by just seven pence. Total cost: $2.73. Total time: 68 minutes across three separate monthly sessions.

That's the headline. The detail is where it gets interesting. Out of 354 individual checks — six criteria across 59 transactions — the model failed 20, and only one of those failures actually matters. It booked a founder's 10,000 GBP capital contribution into a generic "Capital Account" instead of the legally distinct "Unpaid Shares" line, which in UK company law carries real creditor-protection implications and needs specific disclosure at year-end. Wrong VAT impact: zero. Wrong in a way a real accountant would never be: yes.

The rest of the mistakes are smaller and stranger. Fourteen times the model mixed up "zero-rated" and "tax-exempt" VAT categories — two buckets that both mean no VAT changes hands but are treated differently by tax authorities for subtler reasons. Curiously, it made this exact error every single time in January and February, then got every instance right in March, a flip that suggests something closer to inconsistent reasoning than a fixed knowledge gap. A handful of remaining errors came from Wise's habit of splitting a single card payment across currency balances, which confused the model into slightly double-counting VAT on a couple of split transactions — again immaterial to the bottom line, but the kind of thing a sharp bookkeeper would spot immediately.

What the model never got wrong is arguably the more important story. It correctly untangled duplicate same-day, same-vendor, same-amount transactions. It correctly identified a transfer dressed up as a card purchase. It attached the right receipt to the right transaction every single time. These are the exact judgment calls that separate a competent bookkeeper from a cheap, unreliable one, and until very recently only frontier proprietary models or paid humans could reliably clear that bar.

Vineyard Finance's own read is blunt: bookkeeping is becoming a solved problem, and the interesting work now is building the scaffolding — the tools, the checks, the human oversight layer — that lets SMEs actually trust and deploy something like this instead of hand-waving at a black box.