Non-brittle Contract Tests

When writing tests, assert the real behavioral contract (the thing that could regress), not incidental formatting or brittle implementation details.

Practical rules:

• Prefer contract/behavior checks that remain valid under real constraints (e.g., if output is truncated, assert required directives appear within the kept region and in the right envelope).
• Make harness-style tests deterministic and replay-only: freeze time, block real network calls, and use recorded fixtures.
• Avoid vacuous or misleading metrics: if the test expects structural outcomes (e.g., multi-member clusters), fail or score appropriately when the structure doesn’t form.
• Remove tautological assertions that cannot fail given earlier passing assertions.
• Avoid brittleness from ordering or cardinality: search by key (e.g., plugin name) instead of assuming single-entry lists or stable ordering.
• Avoid “security theatre”/non-functional checks that only verify text literals in config without meaningful behavioral coverage.

Example (truncation-survival contract):

def kept_region(text: str) -> str:
    return text[: text.index("## Ranked Evidence Clusters")]

text = render.render_compact(sample_report())
head = kept_region(text)

assert "SYNTHESIS CONTRACT" in head
assert "<!-- END EVIDENCE FOR SYNTHESIS -->" not in head  # tail-only checks don’t help

Apply these principles so test failures pinpoint the true regression and don’t break due to incidental changes (ordering, redundant guards, or irrelevant text).