Verification · Feedback and evaluation · CI failure analysis
Flake detection
Re-run and compare to tell flaky tests from real failures.
How to build it
- Re-run each failing test a fixed number of times on the same commit.
- Label a test flaky when it both passes and fails on that commit.
- Open an issue for each flaky test and quarantine it until fixed.
Signals in a repository
- A quarantine list or skip marker with an issue link for each flaky test.
- CI retry settings for specific tests or jobs.
- Test result uploads that record pass and fail per run.
In practice
No scanned repository runs a tool for this check. The steps above are the only guidance.