Reviewer optimization
Tunes reviewer prompts and instructions against the evaluation set instead of by hand.
Sub-techniques
2
| ID | Name | Description |
|---|---|---|
V1068.001 | Prompt optimization | Search prompt variants against a metric. |
V1068.002 | Instruction updates from feedback | Update reviewer instructions from accepted and rejected findings. |
Craft it
- Define the metric first (from reviewer evaluation).
- Optimize on a train split; report on a held-out split.
- Review every prompt change like code.
Reverse engineer it
Signs that a codebase already runs this technique, and where to read how it is configured.
- Optimizer scripts and versioned prompt files in the repository.