An open, practical comparison of code-review models in one real review harness. Quality first, then completion, speed, and cost.
Current recommendation
Claude Opus 5 led known-issue recall at 73.3%. GPT-5.6 Sol was the practical winner for most bot runs: 56.7% recall, full completion, the fastest median review, and a $7.13 run cost.
anthropic/claude-opus-5 · Amazon Bedrock
openai/gpt-5.6-sol · Azure
x-ai/grok-4.6 · xAI
A reviewer earns credit for identifying an evidence-backed defect once. Rephrasing it, guessing loudly, or padding the review does not help.
Defect recall, precision, clean-review restraint, severity judgment, and consistency across repeated attempts.
One-to-one matching, proof-backed gold findings, human adjudication, and visible uncertainty for incomplete labels.
Latency is reported after review quality, with failures and budget exhaustion kept separate from successful reviews.
Observed token and cost data where available. Unknown cost is null—never quietly treated as free.
Hold prompt, tools, budgets, context, scorer, and task release still. Change the model or provider endpoint.
Measure the whole review system: its model, prompt, tools, parsing, and orchestration. OpenRouter Review Bot and Grok Code Review Bot are first.
Current state
The current board is a small, three-attempt finalist run—not a universal measure of coding ability. It is useful for choosing models in the OpenRouter Review Bot, and future runs can add harder cases and newly released models.