Awesome Reviewers

Verification · Feedback and evaluation

Missed bug analysis

For each bug that escaped review, finds which technique should have caught it and why it did not, then clusters the causes.

Sub-techniques

  • Root cause trace V1066.001

    Replay the review trace for the PR that introduced the bug.

  • Cause clustering V1066.002

    Cluster root causes to find the systematic gaps.

  • Eval set growth V1066.003

    Add each missed bug to an evaluation set for the reviewers.

How to build it

  1. Link every bug fix to the PR that introduced the bug.
  2. Trace what each check and reviewer saw on that PR.
  3. Add the case to the eval set and re-run reviewers on it.

Signals in a repository

  • Postmortems that name the introducing PR, and tracing for review agents.