Property-based tests
Generates inputs and checks invariants that must hold for all of them, then shrinks failures to a minimal case.
Sub-techniques
3
| ID | Name | Description |
|---|---|---|
V1021.001 | Round-trip properties | decode(encode(x)) == x for serializers and parsers. |
V1021.002 | Model-based properties | Compare the implementation against a simple reference model. |
V1021.003 | Stateful testing | Generate sequences of operations and check invariants after each. |
Craft it
- Start with round-trip and idempotence properties; they need no oracle.
- Persist the failure database so found cases are replayed.
- Promote shrunk failures to example tests.
Reverse engineer it
Signs that a codebase already runs this technique, and where to read how it is configured.
- @given decorators, fc.assert or proptest! macros in tests.
- .hypothesis/ in .gitignore.
Where the scanned repositories configure it:
Cargo.tomlin 11 repositoriespyproject.tomlin 10 repositoriespackage.jsonin 3 repositories
Detected by the scanner as: Property-based tests (edge-case inputs nobody wrote a test for; invariants over generated data.)
Open-source packages
3
- Hypothesis HypothesisWorks/hypothesis
- fast-check dubzzz/fast-check
- proptest proptest-rs/proptest
Tools seen in scans
by repositories that run them
Procedure examples
repositories that run it, most-starred first
| Repository | Tools | Runs in | Evidence |
|---|---|---|---|
| langgenius/dify | hypothesis |
api/pyproject.toml [dependency-groups] dev: hypothesis |
|
| langflow-ai/langflow | hypothesis |
pyproject.toml [dependency-groups] dev: hypothesis |
|
| denoland/deno | proptest |
Cargo.toml [workspace.dependencies]: proptest |
|
| pytorch/pytorch | hypothesis |
pyproject.toml [dependency-groups] dev: hypothesis |
|
| infiniflow/ragflow | hypothesis |
pyproject.toml [dependency-groups] test: hypothesis |
|
| zed-industries/zed | proptest |
Cargo.toml [workspace.dependencies]: proptest |
|
| nuxt/nuxt | fast-check |
package.json devDependencies: fast-check |
|
| apache/spark | hypothesis |
pyproject.toml [dependency-groups] internal_test: hypothesis |
|
| astral-sh/ruff | proptest |
Cargo.toml [workspace.dependencies]: quickcheck |
|
| helix-editor/helix | proptest |
Cargo.toml [workspace.dependencies]: quickcheck |
|
| pola-rs/polars | proptest |
Cargo.toml [workspace.dependencies]: proptest |
|
| diegosouzapw/OmniRoute | fast-check |
package.json devDependencies: fast-check |
|
| servo/servo | proptest |
Cargo.toml [workspace.dependencies]: quickcheck |
|
| influxdata/influxdb | proptest |
Cargo.toml [workspace.dependencies]: proptest |
|
| tokio-rs/tokio | proptest |
tokio/Cargo.toml [target.'cfg(not(target_family = "wasm"))'.dev-dependencies]... |
|
| PostHog/posthog | hypothesis |
pyproject.toml [dependency-groups] dev: hypothesis |
|
| BerriAI/litellm | hypothesis, proptest |
pyproject.toml [dependency-groups] dev: hypothesis |
|
| pydantic/pydantic | hypothesis |
pydantic-core/pyproject.toml [dependency-groups] testing-extra: hypothesis |
|
| tokio-rs/axum | proptest |
axum/Cargo.toml [dev-dependencies]: quickcheck |
|
| jj-vcs/jj | proptest |
Cargo.toml [workspace.dependencies]: proptest |
|
| bytedance/deer-flow | hypothesis |
backend/pyproject.toml [dependency-groups] dev: hypothesis |
|
| langfuse/langfuse | fast-check |
worker/package.json devDependencies: fast-check |
|
| anthropics/anthropic-sdk-python | hypothesis |
pyproject.toml [dependency-groups] dev: hypothesis |