Verification · Feedback and evaluation · Reviewer optimization
Prompt optimization
Search prompt variants against a metric.
How to build it
- Define the metric from the reviewer evaluation set first.
- Split the evaluation set into train and held-out parts.
- Keep a prompt variant only if it improves the held-out score.
Signals in a repository
- An optimizer script that imports dspy or textgrad.
- Versioned prompt files in the repository with their held-out scores.
- An evaluation set file with train and held-out splits.
In practice
No scanned repository runs a tool for this check. The steps above are the only guidance.