Earlier this year a client sent us a website audit they had generated with an AI tool. It was long, confident, and formatted like a real report, with severity levels and a numbered list of findings. Before responding, we did the boring thing and checked every claim against the live site. Roughly six out of ten held up. The rest were wrong.
The pattern in the misses was the interesting part. The tool reported a pricing inconsistency that did not exist. Every product on the site was the same price everywhere. It flagged a product page as broken when it loaded fine. It listed a page as live that had been unpublished. None of those were close calls. They were things you could disprove by clicking.
What the tool got right was also telling. It correctly caught a sitewide metadata problem, an outdated claim in old page copy, and an external image host that the site depended on. Those are pattern level findings, the kind of thing a language model is good at spotting because they look like known categories of problem. Where it failed was on observed values. A price. A status code. Whether a URL resolves. A model can only guess at those, and it will guess in the same confident tone it uses when it is right.
That is the discovery, and it is one idea: an AI audit is a hypothesis generator, not an inspector. It is useful for producing a list of things worth checking. It is not evidence that any of them are true.
Why it matters for anyone running a business site is that these reports are going to keep landing in inboxes, and they carry weight because they look thorough. If you act on one without verification, you will spend time fixing things that are not broken and you may miss the real problems that the tool happened to describe vaguely. If you dismiss one entirely, you throw away the forty percent of it that was worth reading.
The practical takeaway is to separate the two jobs. Let an AI tool propose. Let a crawler verify. A crawler like Screaming Frog reports what it actually observed on each URL, the status code, the title, the canonical, the price in the structured data. It cannot invent a number because it never guesses. When we moved a client from AI generated audits to a crawler based one, the conversation changed from arguing about whether findings were real to deciding which ones to fix first.
At Burgama we now run audits that way by default. If a finding cannot be reproduced on the live site, it does not go in the report.