Feedback and evaluation
Closes the loop: production errors and missed bugs become new checks, and the reviewers themselves are measured and tuned.
When: Continuously, once checks and reviewers are in place. Every escaped bug is a missing technique somewhere to the left.
VA10 tactic
7 techniques
16 sub-techniques
not detectable from CI configuration; no usage counts
| ID | Technique | Description | Sub |
|---|---|---|---|
V1064 |
Production error investigation | Investigates production errors from observability data, finds the responsible code and proposes a fix with a regression test. | 2 |
V1065 |
Alert deduplication and triage | Groups related production alerts and errors and ranks them so investigation starts with the most impactful. | 2 |
V1066 |
Missed bug analysis | For each bug that escaped review, finds which technique should have caught it and why it did not, then clusters the causes. | 3 |
V1067 |
Reviewer evaluation | Benchmarks reviewers on labelled changes: did they find the known bugs, and how many findings were wrong. | 3 |
V1068 |
Reviewer optimization | Tunes reviewer prompts and instructions against the evaluation set instead of by hand. | 2 |
V1069 |
Developer feedback capture | Records how developers respond to findings (fixed, dismissed, argued) and the sentiment of replies, as the ground truth for every other loop. | 2 |
V1070 |
CI failure analysis | Reads failing CI runs, separates real failures from flaky ones, and maps each to the job, step and change that caused it. | 2 |