Standard vs Thorough: what the extra pass finds, and what it costs

Auditor's deep review comes in two depths, Standard and Thorough. Both use the same models and both end with a verifier that checks every finding before it's posted. The difference is how much code the review reads, and that turns out to decide which bugs it can find at all.

This article explains what each depth does, what our eval showed about the gap between them, and how to choose. The method behind the numbers, the case list and every result are in How we measure an AI code reviewer.

What Standard does

Standard runs three find passes over the same change. Each pass gets the diff plus as much of the changed files as fits, and reads the files in a different order, so one pass's blind spot is less likely to be all three's.

The passes then vote. A finding that at least two passes agree on goes to the verifier, as does any finding a single pass rates High. Everything else is dropped. The verifier makes one call that sees each finding with the code around it, and confirms or rejects it.

Standard never sees a file the pull request didn't change. It has no tools to open one.

What Thorough does

Thorough runs four passes instead of three, and each looks for one kind of problem:

• logic and edge cases
• security
• data and concurrency
• API and contract breaks

Each pass can read, search and list the repository at the pull request's head. The contracts pass is told to search for every caller of what the diff changes and check each one. Since each pass has its own focus, there's no vote; duplicates are merged and every finding goes to the verifier, which can explore the repository too.

In real reviews, Thorough also runs static analysis first: the repository's own type checker and linter where it's set up for them, Semgrep's default rules, and Theme Check on a Shopify theme. Their results are handed to the passes so they don't repeat them.

What the eval showed

Our eval runs 22 pull requests with 38 bugs we planted, plus five clean pull requests where any finding is noise. It measures the deep review alone, without static analysis. These are the Oct 11, 2026 results:

The usual caveats apply, and they matter here. The bugs are seeded and the answer key is ours. The repositories are mostly two or three files. One bug is 2.6 points of recall, and runs vary: on one real-file case, three Thorough runs in a row found seven, eight and seven of its eight bugs, while two Standard runs found all eight.

So the headline gap of four bugs isn't the useful part. Which four is.

The bugs Standard missed

Two cross-file contract bugs. In one case a lookup function stops throwing and returns null, and a caller in another file still reads a field off the result. In the other, a serializer renames createdAt to created_at, and an untouched feed still sorts on createdAt. In both, the changed code looks correct on its own. The breakage is in a file outside the diff.

Standard can't see that file, and none of its passes proposed anything on either case, in this run or the one on Oct 6. On Thorough, all four passes searched for the callers and flagged both bugs. This is the clearest difference between the depths, and it isn't noise: it's a matter of what each depth can read.

A bug the verifier talked itself out of. A pull request titled "Drop the zero default from balanceOf" makes an unknown account throw instead of returning 0. All three Standard passes found it, and it went to the verifier, which rejected it because the title said the default was dropped on purpose. Thorough's verifier opened the caller, saw nothing guards against the throw, and confirmed it. Reading the caller turned an intent question into a fact.

One of 24 bugs in the real-file cases. Standard found 23, Thorough all 24. At this sample size that's within run-to-run variance.

Standard's one false positive was a fair point the answer key doesn't include: a new retry loop retries card declines with no backoff, alongside the planted bug of a dropped idempotency key, which it also found.

What the extra pass costs

Thorough made about three times as many model calls (250 against 81) and cost about three times as much: $2.46 for all 22 cases against $0.82. The ratio depends on the change.

On the small cases, Thorough's exploring is most of the cost, so it runs about four times Standard. On the larger real files the diff and file context dominate both, and Thorough runs about twice. These are estimates from token usage and catalog prices. In a real repository there's more to search, so expect Thorough to cost more than the eval shows, and Standard about the same.

Either way, every request a review makes is billed to the Auditor key and listed in Logs, so the cost of each review is visible, and spend caps apply to both depths.

Which to pick

Thorough is the default. Keep it where a change in one file can break code in another, which is most application code: shared types, API handlers and their clients, serializers, anything with callers. Those are the bugs Standard can't see, and they're often the ones that get past a human reviewer too, because the diff looks fine.

Standard is a reasonable choice when:

• the repository sees many small, self-contained changes, and cost per review matters more than catching cross-file breaks
• most changes are content, configuration or isolated files, where there are few callers to check
• the repository already has strong type checking and tests across module boundaries, so contract breaks tend to fail CI before review

Depth is set for the team in the Deep review section of the Auditor page, and each repository can override it in its settings. So a team can keep Thorough on its main application and use Standard on a docs site or a repository of small scripts.