← All writing

Musing · · 2 min read

Lazy geniuses: why AI gives bad tax answers

These models know an absurd amount, but they'll do the minimum to produce an answer. A search isn't research, and the model isn't the research system.

I first posted this on X on September 14, 2026. It's here as posted.

I explain these coding agents as lazy geniuses to a lot of people I talk to.

They know an absurd amount, but they’ll often do the minimum amount of work necessary to produce an answer. If retrieval isn’t part of the workflow, you can’t assume the model actually checked anything. It may just be answering from its training data.

And even when it does search, a search isn’t the same thing as research. It might find a result, read a snippet or part of a page, and decide it has enough. That can be a major problem in tax, where the answer may depend on one sentence buried three paragraphs down in the actual Code section, regulation, notice, or revenue procedure.

That’s where a lot of bad tax answers come from.

When a new tax law is enacted, the model may simply not have it in its training data yet. If you don’t give it access to current authority and explicitly tell it to go find and verify the law, you can’t reliably expect the right answer.

A lot of the “look how dumb AI is” screenshots I see aren’t necessarily model failures. They’re workflow failures. The model was asked a question, gave an answer from whatever context it had available, and nobody made it prove that answer against the actual source.

So stop treating tax research like a chat and instead treat it like a research system.

I do most of mine in Claude Code now because I can control the environment around the model. My global CLAUDE.md basically says: if I ask a factual question, go verify it. Don’t rely on your training when current or authoritative information is available.

Then my tax research folder adds another layer. It tells the agent which sources to prioritize, when to pull primary authority, to read the relevant section rather than just a summary, how far around the cited language to read for context, and how I want the final answer cited.

And every time it gets something wrong, I don’t just correct the answer.

I correct the system.

I update the instructions so the same failure is less likely to happen again.

That’s been the bigger shift for me: the model isn’t the research system.

The model + tools + sources + instructions + verification process is the research system.

Where to next

Follow along: @cbarmorecpa on X ↗ · The newsletter ↗ · More writing →