A $1 AI code review that found 11 blocking bugs behind a green CI

Tomas Vykruta@tvykruta

I've spent years in C and C++ at Microsoft and Google, and I've reviewed thousands of changes. I've watched good engineers ship bad code because the reviewer skimmed it, approved it, and went to lunch.

Now AI writes code 200× faster. Who reviews it?

If you're not enforcing strict 3-engineer code reviews on every change before it ships, you have no business building software.

Kidding. That was true yesterday.

Today I'm going to answer whether AI can review AI-generated code. Then I'll show you how to get an outstanding, senior-level review that doesn't miss details, for under a dollar a review.

Bookmark this. The full REVIEW.md is in the first reply, ready to paste.

The results surprised me

1/ I asked the AI session that wrote the code to review its own work. It found only 3 of 23 issues and recommended merging. Then I started a brand-new agent on the same model, with no memory of writing the code, and gave it only the diff. It found 17 of 23.

2/ Reviewers that only read the code found zero of the 8 logic bugs in the feature PR. Every logic bug was found by a reviewer that ran the code.

3/ One rule added to the reviewer's instructions took it from 4 of 8 logic bugs to 7 of 8, plus one nobody had found. Same model, $0.72 → $0.74, 27% faster.

4/ The most expensive model wasn't the best deal. Opus matched or beat Fable on both PRs at about a third of the price: under $1 a review vs. over $2.

5/ The next day, a brand-new reviewer agent found 4 blocking bugs in a PR with 272 passing tests and green CI.

6/ "Always get a code review" was written into the agent's instructions, and the agent skipped it on the very first PR.

It's an incredible unlock. No more bugs sailing through on a green checkmark.

Here's how I got there.

The setup

• 2 real PRs from a production codebase. One was a 75-file refactor, the other a 23-file feature.
• 12 reviews in total: 4 by an AI session that already had context on the work, 8 by brand-new agents that had only the diff, the repo and a rules file.
• 3 models: Fable 5.1, Opus 5.5, Sonnet 5.5.
• I re-ran every reported defect myself, on the PR and on today's head, to sort live bugs from fixed ones.
• Costs are estimated from each reviewer's token count at API list prices, with no caching discount, so they lean high.

Then I put it to work on a real feature. That case study is below.

1. Context is the blind spot, not authorship.

On the refactor:
• The session that wrote the code, reviewing its own work (Fable): found only 3 of 23 issues
• A brand-new Fable agent, with no memory of the work: found 17 of 23
• A brand-new Opus agent: found 18 of 23

On the feature PR, written by a different session three days earlier:
• A Fable session that hadn't written it but had spent a day working in this repo: found 0 of 8 logic bugs
• Brand-new Fable, Opus and Sonnet agents: found 2 to 4 of 8 each

The feature-PR reviewer didn't write the code, and it still found nothing. A reviewer that shares the writer's mental model inherits the writer's blind spots, whoever wrote the code.

2. Reading finds style problems. Running the code finds bugs.

Every logic bug came from a reviewer that built a throwaway database, called the changed code with inputs it picked itself, and watched what came back. The session that already had context ran nothing. It read the code and found no logic bugs.

None of the 8 logic bugs on the feature PR are visible from the diff text alone. Every one of them is obvious once you run the code.

3. A rules file did almost nothing for the reviewer with context.

Giving it a written checklist added one finding per PR. Logic bugs found: still zero. With the same file, brand-new Fable and Opus agents found 13 to 18 issues per PR. A checklist can't fix a reviewer that already believes the code is correct.

4. The failure was in the order the reviewer worked.

The best brand-new reviewer still missed half the logic bugs, so I pulled its full tool-call trace and went through it step by step.

It had opened the functions behind all three bugs it missed, and it understood each one. But it wrote its test inputs after reading the implementation, so every input followed the implementation's own assumptions. Its edge cases were variations on the cases the code already handled.

The code had effectively written its own exam, and it passed.

The fix was one rule: "Write your test inputs before you read the implementation. Then run them." Same model, same PR: recall went from 4 of 8 to 7 of 8, plus a bug nobody had found, for two cents more.

5. The model matters, but less than the process.

Opus 5.5: $0.72 to $0.93 per review, 8 to 11 minutes. It found the most issues on both PRs: the best balance of thoroughness and cost.

Fable 5.1: $2.11 to $2.73, about 3× Opus. Found a bug nobody else caught on each PR, but skipped the base-branch comparison when the tests were green.

Sonnet 5.5: $0.14 to $0.21. The cheapest per finding, but it found 2 of 8, then 4 of 8 on the identical input. That run-to-run variance is the real cost.

The two strongest reviewers overlapped on only 10 of 20 findings.

A surprising case study: one real feature, reviewed end to end

An agent built a real feature: 37 files, 5,000+ lines of code, 272 tests passing, CI green. It pushed without a review.

I made it get one.

11 major bugs. A brand-new Opus reviewer found 11 blocking bugs, plus dozens of smaller issues. Every one of them was behind a green checkmark.

8 reviews, 6 rounds of fixes. Each cycle was review → fix → review again:
• The feature: 4 blockers → fix → approved, but that fix broke a filter → fix → approved. That's 3 reviews.
• The tool that enforces review: 4 blockers → fix → 2 more → fix → 1 more → fix → approved → small fix → approved. That's 5 reviews.

It all took under 40 minutes, at under a dollar per review.

What would have shipped without review:
• Every page of the new feature would crash on Feb 29.
• Users would see records labelled with a premium category they didn't belong to.
• A list would show stale data whenever a user changed a filter.
• A crafted link could slip hidden recipients into an email the user sends.
• The review gate wouldn't have enforced review: it checked the wrong repo and let real pushes through.
• Worse, installing it would have silently switched off every safety check that already ran on each commit.
• A database file would have been committed to the repo.

What this case taught me:

1/ 272 passing tests said nothing about these bugs. None of them had a test.

2/ The first approval wasn't the end. A fix for one bug created another, and only the second review caught it.

3/ The tool that enforces review needed 3 rounds of changes before it worked. Review the gate, not just the code.

Best practices for your AI code reviewer

1/ Always start a brand-new agent with no memory of writing the code. Give it only the diff, the repo and REVIEW.md. On the same model, this took recall from 3 of 23 to 17 of 23.

2/ Make it run the code, not just read it. Reading found zero logic bugs; every logic bug came from running the code.

3/ Have it write its test inputs before it reads the implementation. This one rule took recall from 4 of 8 to 7 of 8 for two cents more.

4/ Compare test results against the base branch every time, even when everything is green.

5/ Re-review every fix commit. Fixes create bugs.

6/ Use Opus by default. Add a second reviewer on a different model for business rules, thresholds and gates. The two best reviewers overlapped on only half their findings.

7/ Enforce review with a gate, not a prompt. Use a git pre-push hook: no approved review for that exact commit, no push. Edits to the rules themselves get reviewed too.

8/ Make it report on every design principle by name: separation of concerns, programming by intention, encapsulation, high cohesion, low coupling. A principle left unmentioned means the review is incomplete.

9/ Check honesty, not just style. Unknown stays unknown. The UI never invents data.

10/ Demand a clear verdict (APPROVE or CHANGES) and report findings as "N of the K found so far." The bug pool is never complete.

The full REVIEW.md is in the reply below Paste it into your repo and point your review agent at it.