Awesome Reviewers

Verification · Code review

Reviewers learned from past reviews

Mines a repository's review discussions for recurring feedback and turns each cluster into a reusable review instruction.

Sub-techniques

  • Discussion mining V1037.001

    Collect resolved review threads with the code they were on.

  • Topic clustering V1037.002

    Cluster comments into themes with a minimum number of supporting discussions.

  • Instruction distillation V1037.003

    Write one instruction per theme with examples from the threads.

  • Memory of facts and rules V1037.004

    Extract facts from new discussions and match them to existing rules to update or add them.

How to build it

  1. Export review threads with their file paths and diffs.
  2. Embed and cluster comments; keep clusters with enough independent threads.
  3. Distill each cluster into an instruction and evaluate it on held-out PRs.

Signals in a repository

  • The instructions on this site are this technique applied to each source repository.