Technique · Feedback and evaluation

Reviewer evaluation

Benchmarks reviewers on labelled changes: did they find the known bugs, and how many findings were wrong.

Sub-techniques

Labelled benchmark V1067.001

PRs with known bugs and known clean changes.
Model as judge V1067.002

A judge model matches reviewer findings to the labelled bugs.
Side-by-side comparison V1067.003

Run two reviewer versions on the same set and compare.