Verifying Python changes

• Security scanning Injection, unsafe APIs and tainted data reaching a sink.
  codeql 28% · 18 of 64 repos

  runs via GitHub Action github/codeql-action/analyze

  exit codes, docs, alternatives, where it runs

  • Runs inci
  • Alternativesbandit 9%, semgrep 8%
  • Seen inlangflow-ai/langflow, home-assistant/core, getsentry/sentry, apache/airflow, crewaiinc/crewai, openai/openai-python
  • Docshttps://github.com/github/codeql
• Secret scanning Credentials and tokens committed to the repository.
  gitleaks 2% · 1 of 64 repos

  runs via pre-commit hook gitleaks

  exit codes, docs, alternatives, where it runs

  • Runs inpre-commit, ci
  • Seen inawslabs/mcp
  • Docshttps://github.com/gitleaks/gitleaks
• Dependency audit Known-vulnerable or disallowed dependencies and licenses.
  dependency-review 8% · 5 of 64 repos

  runs via GitHub Action actions/dependency-review-action

  exit codes, docs, alternatives, where it runs

  • Runs inci
  • Alternativespip-audit 3%
  • Seen inhome-assistant/core, mvanhorn/last30days-skill, apache/airflow, awslabs/mcp, strands-agents/sdk-python
  • Docshttps://github.com/actions/dependency-review-action

• Prevent sensitive data exposureNever log or store sensitive information (passwords, tokens, secrets) in clear text. This common security vulnerability can lead to crede...

  bridgecrewio/checkov Security Python security 2025-02-05

  copy raw open
• Protect sensitive dataNever log, display, or store sensitive information like passwords, tokens, or secrets in clear text. This creates significant security vu...

  bridgecrewio/checkov Security Python security 2025-02-05

  copy raw open
• Boundary Hardening RulesTreat every transition from “untrusted / scoped data” into a sensitive sink (shell execution, subprocess credentials, HTTP fetch, filesys...

  nousresearch/hermes-agent Security Python ai-agents 2026-07-28

  copy raw open
• Fail-Closed Input SanitizationAll externally supplied values must be validated/escaped for the *exact* security-sensitive context they’re interpolated into, and securi...

  awslabs/mcp Security Python ai-agents 2026-07-28

  copy raw open
• Secure Untrusted BoundariesWhen code crosses security boundaries (subprocess/PowerShell execution, network URLs, filesystem writes, or auth/TLS), treat user/environ...

  Azure/azure-cli Security Python cloud-infra 2026-07-26

  copy raw open

43 more in Security — the playbook lists the top 12.

• Type checking Wrong types, unhandled null or None, call signatures broken across modules.
  mypy 42% · 27 of 64 repos

  mypycopy

  exit codes, docs, alternatives, where it runs

  • Exit codes0 no type errors
    1 type errors found
    2 crash or invalid invocation
  • Commands seenmypy
mypy --version
mypy --cache-dir=/dev/null --config-file pyproject.toml .'
  • Runs inpre-commit, ci, local
  • Alternativespyright 17%
  • Seen inlangflow-ai/langflow, pytorch/pytorch, fastapi/fastapi, home-assistant/core, python/cpython, pallets/flask
  • Docshttps://mypy.readthedocs.io/en/stable/command_line.html
• Unit and integration tests Behaviour regressions on the paths the tests exercise.
  pytest 86% · 55 of 64 repos

  pytestcopy

  exit codes, docs, alternatives, where it runs

  • Exit codes0 all tests passed
    1 some tests failed
    2 interrupted by the user
    3 internal error
    4 command line usage error
    5 no tests were collected
  • Commands seenpytest
pytest -q
pytest -v
  • Runs inpre-commit, ci, local
  • Alternativesunittest 12%
  • Seen innousresearch/hermes-agent, TheAlgorithms/Python, langchain-ai/langchain, langgenius/dify, langflow-ai/langflow, pytorch/pytorch
  • Docshttps://docs.pytest.org/en/stable/reference/exit-codes.html
• Property-based tests Edge-case inputs nobody wrote a test for; invariants over generated data.
  hypothesis 14% · 9 of 64 repos

  exit codes, docs, alternatives, where it runs

  • Seen inlanggenius/dify, langflow-ai/langflow, pytorch/pytorch, infiniflow/ragflow, PostHog/posthog, BerriAI/litellm
  • Docshttps://github.com/HypothesisWorks/hypothesis

• Actionable Error PropagationWhen handling failures, ensure your code (a) raises the right exception to control exit code/automation, (b) doesn’t silently ignore user...

  Azure/azure-cli Error Handling Python cloud-infra 2026-07-26

  copy raw open
• Avoid silent error maskingEstablish a team-wide error-handling contract: 1) Never swallow exceptions in production paths - Avoid bare `except: pass` / broad except...

  awslabs/mcp Error Handling Python ai-agents 2026-07-30

  copy raw open
• Exception Handling PolicyAdopt an exception-handling standard that distinguishes **critical control-flow** from **non-critical side effects**, while enforcing saf...

  the-pr-agent/pr-agent Error Handling Python ai-agents 2026-06-12

  copy raw open
• Validate Inputs ExplicitlyEnforce each function’s documented input contract with early, explicit checks; raise specific exceptions with accurate messages (avoid as...

  TheAlgorithms/Python Error Handling Python docs 2026-05-05

  copy raw open
• Surface errors clearlyRule: Fail fast, validate assumptions, and distinguish user cancellation. Motivation - Hidden or silently handled errors lead to incorrec...

  p-e-w/heretic Error Handling Python ml-systems 2026-01-20

  copy raw open

162 more in API, Error Handling, Null Handling — the playbook lists the top 12.

• Unit and integration tests Behaviour regressions on the paths the tests exercise.
  pytest 86% · 55 of 64 repos

  pytestcopy

  exit codes, docs, alternatives, where it runs

  • Exit codes0 all tests passed
    1 some tests failed
    2 interrupted by the user
    3 internal error
    4 command line usage error
    5 no tests were collected
  • Commands seenpytest
pytest -q
pytest -v
  • Runs inpre-commit, ci, local
  • Alternativesunittest 12%
  • Seen innousresearch/hermes-agent, TheAlgorithms/Python, langchain-ai/langchain, langgenius/dify, langflow-ai/langflow, pytorch/pytorch
  • Docshttps://docs.pytest.org/en/stable/reference/exit-codes.html
• Version and platform matrix Breakage on a runtime version, OS or feature combination you did not run locally.
  ci-matrix 56% · 36 of 64 repos

  analyze (include: 2)copy

  exit codes, docs, alternatives, where it runs

  • Commands seenanalyze (include: 2)
analyze (include: 3)
analyze (language: 1)
  • Runs inci
  • Seen inlanggenius/dify, langflow-ai/langflow, comfyanonymous/ComfyUI, python/cpython, microsoft/markitdown, pallets/flask
  • Docshttps://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/run-job-variations

• Test Canvas Edge CasesWhen changing Celery task dispatch semantics (especially Canvas: chains/groups/chords, unroll/freeze, and composed workflows), add regres...

  celery/celery Celery Python orchestration 2026-04-06

  copy raw open
• Ensure deterministic resume behaviorWhen implementing persistence and resume functionality in durable execution systems, ensure that state is persisted at consistent event b...

  strands-agents/sdk-python Temporal Python ai-agents 2025-11-10

  copy raw open
• Offload heavy operationsMove heavy operations, especially those involving large data processing or bulk deletions, from synchronous request handlers to asynchron...

  langgenius/dify Celery Python llm-infra 2025-08-28

  copy raw open
• leverage framework capabilitiesStructure workflows and activities to take full advantage of the orchestration framework's built-in capabilities for fault tolerance, ret...

  PostHog/posthog Temporal Python data-systems 2025-08-19

  copy raw open
• Task signature methodsWhen chaining Celery tasks, choose the correct signature method: - Use `.s()` when a task needs the output from the previous task in the ...

  prowler-cloud/prowler Celery Python security 2025-06-23

  copy raw open

1 more in Fastify, NestJS, Spring, Axum, Celery, Temporal — the playbook lists the top 6.

• Property-based tests Edge-case inputs nobody wrote a test for; invariants over generated data.
  hypothesis 14% · 9 of 64 repos

  exit codes, docs, alternatives, where it runs

  • Seen inlanggenius/dify, langflow-ai/langflow, pytorch/pytorch, infiniflow/ragflow, PostHog/posthog, BerriAI/litellm
  • Docshttps://github.com/HypothesisWorks/hypothesis
• Fuzzing Crashes, panics and hangs on malformed input, especially in parsers and decoders.

  No scanned repository in this language runs a tool for this. Lean on the review prompts.

• Mutation testing Tests that keep passing whatever the code does.
  mutmut 2% · 1 of 64 repos

  mutmut run 2>&1copy

  exit codes, docs, alternatives, where it runs

  • Runs inci
  • Seen inBerriAI/litellm
  • Docshttps://github.com/boxed/mutmut
• Proofs and specifications Design-level bugs in protocols and algorithms, checked against a specification.
  lean 2% · 1 of 64 repos

  exit codes, docs, alternatives, where it runs

  • Seen inoraios/serena
  • Docshttps://github.com/leanprover/lean4

• Choose optimal algorithmsAlways select appropriate data structures and algorithms based on their performance characteristics and the specific operations your code...

  bridgecrewio/checkov Algorithms Python security 2024-12-04

  copy raw open
• Deterministic Algorithm CorrectnessWhen implementing non-trivial algorithms (graph ordering, batching under constraints, multi-pass transformations), apply four requirement...

  langchain-ai/langchain Algorithms Python ai-agents 2026-04-26

  copy raw open
• optimize algorithmic efficiencyChoose efficient algorithms and data structures to avoid unnecessary computational complexity. Look for opportunities to replace expensiv...

  python-poetry/poetry Algorithms Python devtools 2024-11-11

  copy raw open
• Prevent algorithmic pitfallsAvoid common algorithmic errors that lead to incorrect results or inefficient code by following these practices: 1. **Never modify a coll...

  boto/boto3 Algorithms Python cloud-infra 2023-11-29

  copy raw open
• Precise Matching RulesWhen implementing parsing/classification logic, treat the input grammar as a contract: (1) constrain pattern matches to the intended enve...

  nousresearch/hermes-agent Algorithms Python ai-agents 2026-07-18

  copy raw open

42 more in Algorithms — the playbook lists the top 12.

• Concurrency model checking Races and orderings that fail only under specific interleavings.

  No scanned repository in this language runs a tool for this. Lean on the review prompts.

• Undefined-behaviour detection Memory-safety and aliasing violations in unsafe code.

  No scanned repository in this language runs a tool for this. Lean on the review prompts.

• Property-based tests Edge-case inputs nobody wrote a test for; invariants over generated data.
  hypothesis 14% · 9 of 64 repos

  exit codes, docs, alternatives, where it runs

  • Seen inlanggenius/dify, langflow-ai/langflow, pytorch/pytorch, infiniflow/ragflow, PostHog/posthog, BerriAI/litellm
  • Docshttps://github.com/HypothesisWorks/hypothesis

• Nonblocking Concurrency RulesEstablish a single concurrency standard: in async code, never block the event loop; in multi-thread/state-machine code, make ownership an...

  nousresearch/hermes-agent Concurrency Python ai-agents 2026-07-28

  copy raw open
• Avoid shared mutable stateWhen working with concurrency (threads/async), ensure code is safe under parallel invocations and does not block or re-enter running even...

  langchain-ai/langchain Concurrency Python ai-agents 2026-04-13

  copy raw open
• Thread-safe lifecycle isolationWhen changing concurrency/background execution, ensure (a) state is isolated per thread/consumer/instance, and (b) shutdown/cleanup and e...

  celery/celery Concurrency Python orchestration 2026-03-12

  copy raw open
• Safe Async Resource HandlingShared resources (connections, clients, caches) must be concurrency-safe across async tool invocations. **Standard** 1. **Validate before...

  awslabs/mcp Concurrency Python ai-agents 2026-07-30

  copy raw open
• Avoid event-loop blockingDo not perform blocking I/O or other long-running synchronous work on asyncio’s event loop (especially in async client construction). Off...

  openai/openai-python Concurrency Python ai-agents 2026-05-01

  copy raw open

25 more in Concurrency — the playbook lists the top 12.

• Unit and integration tests Behaviour regressions on the paths the tests exercise.
  pytest 86% · 55 of 64 repos

  pytestcopy

  exit codes, docs, alternatives, where it runs

  • Exit codes0 all tests passed
    1 some tests failed
    2 interrupted by the user
    3 internal error
    4 command line usage error
    5 no tests were collected
  • Commands seenpytest
pytest -q
pytest -v
  • Runs inpre-commit, ci, local
  • Alternativesunittest 12%
  • Seen innousresearch/hermes-agent, TheAlgorithms/Python, langchain-ai/langchain, langgenius/dify, langflow-ai/langflow, pytorch/pytorch
  • Docshttps://docs.pytest.org/en/stable/reference/exit-codes.html
• Fuzzing Crashes, panics and hangs on malformed input, especially in parsers and decoders.

  No scanned repository in this language runs a tool for this. Lean on the review prompts.

• Restrict public accessCloud resources should be configured to restrict public network access by default to minimize potential attack surfaces. Always explicitl...

  bridgecrewio/checkov Networking Python security 2025-05-13

  copy raw open
• Restrict public networkAlways configure cloud resources to restrict or disable public network access unless explicitly required for the application's functional...

  bridgecrewio/checkov Networking Python security 2025-05-13

  copy raw open
• Configure HTTP requests properlyWhen making HTTP requests, use standard library enums for status codes, set appropriate timeouts, and handle redirects efficiently. Avoid...

  python-poetry/poetry Networking Python devtools 2024-02-26

  copy raw open
• Preserve protocol semanticsNetwork/HTTP code must strictly respect protocol contracts and avoid silent behavioral changes. When parsing values governed by an RFC gr...

  python/cpython Networking Python cloud-infra 2026-10-23

  copy raw open
• Robust aiohttp NetworkingWhen implementing HTTP/multipart requests and streaming (SSE/chunked) parsing with aiohttp, follow these rules: - Prefer aiohttp’s built-...

  openai/openai-python Networking Python ai-agents 2023-01-12

  copy raw open

5 more in Networking, Axios — the playbook lists the top 10.

• End-to-end browser tests User flows broken across components, routing and the network.

  No scanned repository in this language runs a tool for this. Lean on the review prompts.

• Linting Bug-prone patterns, unused code and conventions the formatter does not cover.
  ruff-check 72% · 46 of 64 repos autofix

  ruff check .copy

  exit codes, docs, alternatives, where it runs

  • Exit codes0 no violations
    1 violations found
    2 invalid configuration, CLI options or internal error
  • Commands seenruff check .
ruff check
ruff check . --fix
  • Runs inpre-commit, ci, local
  • Alternativesflake8 20%, pylint 12%
  • Seen innousresearch/hermes-agent, TheAlgorithms/Python, langgenius/dify, langflow-ai/langflow, pytorch/pytorch, fastapi/fastapi
  • Docshttps://docs.astral.sh/ruff/linter/#exit-codes

• Unit and integration tests Behaviour regressions on the paths the tests exercise.
  pytest 86% · 55 of 64 repos

  pytestcopy

  exit codes, docs, alternatives, where it runs

  • Exit codes0 all tests passed
    1 some tests failed
    2 interrupted by the user
    3 internal error
    4 command line usage error
    5 no tests were collected
  • Commands seenpytest
pytest -q
pytest -v
  • Runs inpre-commit, ci, local
  • Alternativesunittest 12%
  • Seen innousresearch/hermes-agent, TheAlgorithms/Python, langchain-ai/langchain, langgenius/dify, langflow-ai/langflow, pytorch/pytorch
  • Docshttps://docs.pytest.org/en/stable/reference/exit-codes.html
• Version and platform matrix Breakage on a runtime version, OS or feature combination you did not run locally.
  ci-matrix 56% · 36 of 64 repos

  analyze (include: 2)copy

  exit codes, docs, alternatives, where it runs

  • Commands seenanalyze (include: 2)
analyze (include: 3)
analyze (language: 1)
  • Runs inci
  • Seen inlanggenius/dify, langflow-ai/langflow, comfyanonymous/ComfyUI, python/cpython, microsoft/markitdown, pallets/flask
  • Docshttps://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/run-job-variations

• SQLAlchemy 2.0 patternsMigrate from legacy SQLAlchemy 1.x query patterns to modern 2.0 style for better performance, clarity, and future compatibility. This inv...

  langgenius/dify Database Python llm-infra 2025-08-31

  copy raw open
• Cache Scope and KeysWhen adding or modifying caching/deduplication, ensure the cache identity (routing + key + scope) matches the exact contract where reuse ...

  nousresearch/hermes-agent Caching Python ai-agents 2026-07-28

  copy raw open
• Safe caching practicesApply caching only when it is safe, bounded, and correct: 1) Avoid unbounded method memoization - Don’t use `functools.lru_cache/cache` d...

  langchain-ai/langchain Caching Python ai-agents 2026-05-01

  copy raw open
• Credential-Aware CachingWhen caching heavyweight clients or connections, treat authentication identity (tokens/secret_arn/derived ARN) as part of cache correctne...

  awslabs/mcp Caching Python ai-agents 2026-07-22

  copy raw open
• Cache invalidation consistencyEnsure comprehensive and consistent cache invalidation patterns across all models that affect cached data. Every model that can impact ca...

  PostHog/posthog Caching Python data-systems 2025-08-18

  copy raw open

29 more in Database, Migrations, Caching — the playbook lists the top 12.

No tool-based check applies. The review prompts below are the verification.

• Avoid Unnecessary WorkPrefer eliminating work that can’t affect observable behavior and prevent resource retention in hot paths. Apply this as a checklist: 1) ...

  celery/celery Performance Optimization Python orchestration 2026-04-08

  copy raw open
• Bounded Hot-Path WorkIn performance-sensitive hot paths, ensure you (1) don’t repeat expensive work, (2) keep async/event-loop paths non-blocking, (3) bound e...

  nousresearch/hermes-agent Performance Optimization Python ai-agents 2026-07-27

  copy raw open
• Bound Work and QueriesWhen improving performance, make sure code neither (a) fetches too much data, nor (b) does the same expensive work repeatedly, nor (c) ru...

  infiniflow/ragflow Performance Optimization Python llm-infra 2026-05-08

  copy raw open
• Manage tensors proactivelyMotivation: Large models and matrix ops can produce big ephemeral tensors, cross-device copies, and temporary allocations that lead to ex...

  p-e-w/heretic Performance Optimization Python ml-systems 2026-01-30

  copy raw open
• Eliminate redundant operationsAvoid duplicate function calls, repeated tensor operations, and redundant computations that can significantly impact performance. Impleme...

  sgl-project/sglang Performance Optimization Python llm-infra 2025-08-22

  copy raw open

31 more in Performance Optimization — the playbook lists the top 12.

• Unit and integration tests Behaviour regressions on the paths the tests exercise.
  pytest 86% · 55 of 64 repos

  pytestcopy

  exit codes, docs, alternatives, where it runs

  • Exit codes0 all tests passed
    1 some tests failed
    2 interrupted by the user
    3 internal error
    4 command line usage error
    5 no tests were collected
  • Commands seenpytest
pytest -q
pytest -v
  • Runs inpre-commit, ci, local
  • Alternativesunittest 12%
  • Seen innousresearch/hermes-agent, TheAlgorithms/Python, langchain-ai/langchain, langgenius/dify, langflow-ai/langflow, pytorch/pytorch
  • Docshttps://docs.pytest.org/en/stable/reference/exit-codes.html

• Model integration rulesWhen adding or changing model loading, quantization, LoRA/abliteration logic, or merging behavior, follow these explicit rules to ensure ...

  p-e-w/heretic AI Python ml-systems 2026-02-01

  copy raw open
• prefer OpenAI compatibilityWhen integrating new LLM providers or AI services, prioritize OpenAI-compatible patterns and reuse existing utilities rather than creatin...

  BerriAI/litellm AI Python llm-infra 2025-06-06

  copy raw open
• Improve code organizationPrioritize clean code structure and readability in AI/ML codebases by avoiding inline conditionals, using descriptive expressions, and ma...

  sgl-project/sglang AI Python llm-infra 2025-08-25

  copy raw open
• Device-agnostic acceleration codeAvoid hardcoding specific device types like 'cuda' in AI code. Instead, use device-agnostic approaches such as device type variables or a...

  pytorch/pytorch AI Python ml-systems 2025-07-07

  copy raw open
• Avoid magic numbersReplace hardcoded values and magic numbers in AI model code with named constants or configuration parameters. This improves maintainabili...

  vllm-project/vllm AI Python llm-infra 2025-07-07

  copy raw open

35 more in AI, Pytorch — the playbook lists the top 12.

No tool-based check applies. The review prompts below are the verification.

• Scoped, Accurate Warning LogsWhen adding warnings/logs, treat them as part of the user experience: only emit warnings when they are actionable, accurate, and non-nois...

  Azure/azure-cli Logging Python cloud-infra 2026-07-26

  copy raw open
• Logging Correctness RulesWhen adding or modifying logging: 1) **Match logging intent to behavior (level + visibility)** - If a user-supplied config or runtime opt...

  celery/celery Logging Python orchestration 2026-04-09

  copy raw open
• Follow logging best practicesMaintain high-quality logging throughout the codebase by following these best practices: 1. **Use appropriate log levels**: Reserve `debu...

  vllm-project/vllm Logging Python llm-infra 2025-07-08

  copy raw open
• Logging consistency and safetyWhen writing/adjusting code, treat logging as part of error handling: - Use the right level: log request- or operation-failure conditions...

  the-pr-agent/pr-agent Logging Python ai-agents 2025-04-23

  copy raw open
• Use appropriate log levelsChoose the correct logging level based on the importance and purpose of the message. Use DEBUG for debugging information that's only usef...

  langgenius/dify Logging Python llm-infra 2025-08-14

  copy raw open

21 more in Logging, Observability — the playbook lists the top 12.

• CI workflow linting Broken GitHub Actions syntax, script injection, over-broad token permissions.

  No scanned repository in this language runs a tool for this. Lean on the review prompts.

• File hygiene Invalid JSON, YAML or TOML, merge markers, trailing whitespace, oversized files.
  pre-commit-hooks 39% · 25 of 64 repos autofix

  runs via pre-commit hook end-of-file-fixer

  exit codes, docs, alternatives, where it runs

  • Commands seenpre-commit hook end-of-file-fixer
pre-commit hook check-yaml
pre-commit hook check-merge-conflict
  • Runs inpre-commit, ci, local
  • Seen inTheAlgorithms/Python, langchain-ai/langchain, langflow-ai/langflow, fastapi/fastapi, home-assistant/core, python/cpython
  • Docshttps://github.com/pre-commit/pre-commit-hooks

• Configuration ownership and determinismSummary - Ensure configuration is explicit, owned by the correct component (application vs plugin), and deterministic so behavior is pred...

  p-e-w/heretic Configurations Python ml-systems 2026-02-01

  copy raw open
• Centralized Config StandardsWhen implementing new behavior behind feature flags or configuration (paths, env vars, index dimensions, toggles like text2gql), use cent...

  eosphoros-ai/DB-GPT Configurations Python ai-agents 2025-10-30

  copy raw open
• Centralize environment variablesAll environment variables should be defined in a centralized location (`checkov/common/util/env_vars_config.py`) rather than scattered th...

  bridgecrewio/checkov Configurations Python security 2025-06-23

  copy raw open
• validate configuration defaultsConfiguration parameters should be properly validated with appropriate defaults and clear handling of edge cases. Avoid using arbitrary d...

  Unstructured-IO/unstructured Configurations Python llm-infra 2025-01-10

  copy raw open
• Use configurable default valuesMake configuration values flexible and robust by avoiding hardcoded values and providing sensible defaults. This ensures your application...

  prowler-cloud/prowler Configurations Python security 2025-06-25

  copy raw open

63 more in CI/CD, Configurations — the playbook lists the top 12.

• Dependency audit Known-vulnerable or disallowed dependencies and licenses.
  dependency-review 8% · 5 of 64 repos

  runs via GitHub Action actions/dependency-review-action

  exit codes, docs, alternatives, where it runs

  • Runs inci
  • Alternativespip-audit 3%
  • Seen inhome-assistant/core, mvanhorn/last30days-skill, apache/airflow, awslabs/mcp, strands-agents/sdk-python
  • Docshttps://github.com/actions/dependency-review-action
• Lockfile consistency A lockfile that no longer matches its manifest.
  uv-lock 20% · 13 of 64 repos autofix

  uv lock --checkcopy

  exit codes, docs, alternatives, where it runs

  • Commands seenuv lock --check
uv lock --check --project cli
pre-commit hook uv-lock
  • Runs inpre-commit, ci, local
  • Seen inTheAlgorithms/Python, langgenius/dify, pallets/flask, apache/airflow, crewaiinc/crewai, openai/openai-python
  • Docshttps://github.com/astral-sh/uv
• Version and platform matrix Breakage on a runtime version, OS or feature combination you did not run locally.
  ci-matrix 56% · 36 of 64 repos

  analyze (include: 2)copy

  exit codes, docs, alternatives, where it runs

  • Commands seenanalyze (include: 2)
analyze (include: 3)
analyze (language: 1)
  • Runs inci
  • Seen inlanggenius/dify, langflow-ai/langflow, comfyanonymous/ComfyUI, python/cpython, microsoft/markitdown, pallets/flask
  • Docshttps://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/run-job-variations

• Mutation testing Tests that keep passing whatever the code does.
  mutmut 2% · 1 of 64 repos

  mutmut run 2>&1copy

  exit codes, docs, alternatives, where it runs

  • Runs inci
  • Seen inBerriAI/litellm
  • Docshttps://github.com/boxed/mutmut
• Version and platform matrix Breakage on a runtime version, OS or feature combination you did not run locally.
  ci-matrix 56% · 36 of 64 repos

  analyze (include: 2)copy

  exit codes, docs, alternatives, where it runs

  • Commands seenanalyze (include: 2)
analyze (include: 3)
analyze (language: 1)
  • Runs inci
  • Seen inlanggenius/dify, langflow-ai/langflow, comfyanonymous/ComfyUI, python/cpython, microsoft/markitdown, pallets/flask
  • Docshttps://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/run-job-variations

• Thorough edge-case verificationTests should fully exercise the behavior being verified, not just a single happy-path value that happens to pass even without the fix. Be...

  python/cpython Testing Python app-frameworks 2026-10-28

  copy raw open
• Mock tests in tests/litellmAll new functionality must include mock tests placed in the `tests/litellm/` directory structure. Mock external API calls and dependencie...

  BerriAI/litellm Testing Python llm-infra 2025-08-30

  copy raw open
• Parameterize similar testsUse `pytest.mark.parametrize` to consolidate similar test cases instead of duplicating test code. This approach improves maintainability,...

  python-poetry/poetry Testing Python devtools 2025-06-15

  copy raw open
• Mock external dependencies onlyWhen writing tests for Home Assistant integrations, mock external library dependencies rather than Home Assistant internals like coordina...

  home-assistant/core Testing Python app-frameworks 2025-07-28

  copy raw open
• Automate All Behavior TestsFor every algorithmic change (and every public function/class), require automated tests that lock in both correctness and failure modes. ...

  TheAlgorithms/Python Testing Python docs 2026-05-02

  copy raw open

47 more in Testing — the playbook lists the top 12.

• Docs build Broken cross-references, links and markup in documentation.
  sphinx 12% · 8 of 64 repos

  sphinx-build -j2 -b configcheck -d {envtmpdir}/doctrees docs docs/_build/configcheckcopy

  exit codes, docs, alternatives, where it runs

  • Commands seensphinx-build -E -W -b dirhtml docs docs/_build/dirhtml
sphinx-build -E -a -W -b html -T . _build/html
sphinx-build -b gettext docs/source docs/build/gettext
  • Runs inci, local
  • Alternativesmkdocs 6%
  • Seen inTheAlgorithms/Python, pallets/flask, apache/airflow, celery/celery, oraios/serena, strands-agents/sdk-python
  • Docshttps://github.com/sphinx-doc/sphinx
• Spelling Typos in identifiers, comments and docs.
  codespell 25% · 16 of 64 repos

  codespell --toml pyproject.toml --writecopy

  exit codes, docs, alternatives, where it runs

  • Commands seencodespell
codespell --toml pyproject.toml
codespell --toml pyproject.toml --write
  • Runs inpre-commit, ci, local
  • Seen inTheAlgorithms/Python, langflow-ai/langflow, pytorch/pytorch, home-assistant/core, pallets/flask, browser-use/browser-use
  • Docshttps://github.com/codespell-project/codespell

• Docstring StandardsApply a single documentation rule set across the codebase: - **Module/file docstring:** Each file should have a clear module-level docstr...

  TheAlgorithms/Python Documentation Python docs 2026-05-05

  copy raw open
• Docstrings With IntentAdopt a single documentation standard: every docstring/comment should (1) explain intent and observable behavior, (2) make responsibility...

  langchain-ai/langchain Documentation Python ai-agents 2026-03-06

  copy raw open
• PEP 257 docstring complianceAll public modules, classes, methods, and functions must have docstrings that follow PEP 257 formatting standards. This ensures consisten...

  Unstructured-IO/unstructured Documentation Python llm-infra 2025-01-23

  copy raw open
• Document code thoroughlyAlways include comprehensive documentation for your code through both\ \ docstrings and explanatory comments. \n\nFor classes, functions,...

  vllm-project/vllm Documentation Python llm-infra 2025-07-07

  copy raw open
• Concise, Correct CLI HelpCLI help (per-flag short summaries, long summaries, and examples) must be readable, consistent, and executable. Apply these rules: - **Ke...

  Azure/azure-cli Documentation Python cloud-infra 2026-06-02

  copy raw open

27 more in Documentation — the playbook lists the top 12.