When tests exhibit known false positives or false negatives, preserve these test cases with clear documentation rather than removing them. This practice prevents accidental "fixes" that mask underlying problems and helps track when issues are genuinely resolved.
When tests exhibit known false positives or false negatives, preserve these test cases with clear documentation rather than removing them. This practice prevents accidental “fixes” that mask underlying problems and helps track when issues are genuinely resolved.
As noted in code review discussions, removing tests for known issues can lead to situations where “someone could make a refactor/speedup patch of the check and accidentally fix FP. Without this test, developer wouldn’t know he fixed a FP with an NFC patch.”
When modifying test expectations:
Example approach:
// FIXME: This triggers a false positive: the "passed to same function" heuristic
// can't map the parameter index 1 to A and B because myprint() has no
// parameters.
// warning: 2 adjacent parameters of 'passedToSameKNRFunction' of similar type ('int')
#if 0
int myprint();
// Test case preserved to track when this FP is actually fixed
#endif
This ensures test suite integrity and prevents regression in issue tracking.
Enter the URL of a public GitHub repository