methods / testing
Testing
Runs the code: unit and integration tests, browser tests, property-based tests, fuzzing, mutation testing.
When: On every change that alters behaviour. Add generated inputs (property tests, fuzzing) when the code parses, transforms or computes, and mutation testing when the tests themselves change.
Unit and integration tests
called for by: any change; public apis, signatures, error paths, nullability; service and framework code; network calls, retries, timeouts, protocols; databases, migrations, caching; model, prompt and llm code
Catches: Behaviour regressions on the paths the tests exercise.
| Tool | Languages | Repos | Command seen | Fix | Docs |
|---|---|---|---|---|---|
vitest |
javascript, typescript | 91 | vitest run |
docs | |
pytest |
python | 77 | pytest |
docs | |
cargo-test |
rust | 44 | cargo test --locked |
docs | |
jest |
javascript, typescript | 38 | jest |
docs | |
node-test |
javascript, typescript | 35 | node --test |
docs | |
go-test |
go | 29 | go test ./... |
docs | |
unittest |
python | 20 | unittest |
docs | |
mocha |
javascript, typescript | 10 | mocha -- $npm_package_config_test_args |
docs |
End-to-end browser tests
called for by: ui components and pages
Catches: User flows broken across components, routing and the network.
| Tool | Languages | Repos | Command seen | Fix | Docs |
|---|---|---|---|---|---|
playwright |
javascript, typescript | 55 | playwright test |
docs |
Property-based tests
called for by: public apis, signatures, error paths, nullability; algorithms, parsers, data transforms; threads, async code, locks, shared state
Catches: Edge-case inputs nobody wrote a test for; invariants over generated data.
| Tool | Languages | Repos | Command seen | Fix | Docs |
|---|---|---|---|---|---|
proptest |
rust | 11 | docs | ||
hypothesis |
python | 10 | docs | ||
fast-check |
javascript, typescript | 3 | docs |
Fuzzing
called for by: algorithms, parsers, data transforms; network calls, retries, timeouts, protocols
Catches: Crashes, panics and hangs on malformed input, especially in parsers and decoders.
Mutation testing
called for by: algorithms, parsers, data transforms; the tests themselves
Catches: Tests that keep passing whatever the code does.
| Tool | Languages | Repos | Command seen | Fix | Docs |
|---|---|---|---|---|---|
stryker |
javascript, typescript | 3 | stryker run |
docs | |
mutmut |
python | 1 | mutmut run 2>&1 |
docs |
Version and platform matrix
called for by: service and framework code; databases, migrations, caching; dependencies and lockfiles; the tests themselves
Catches: Breakage on a runtime version, OS or feature combination you did not run locally.