sources / promptfoo/promptfoo

promptfoo/promptfoo

The checks this repository runs, scanned at f2b012e, and the review prompts derived from its discussions.

12 recognised tools

17 prompts

26565 stars

Static analysis

4 tools

Tool	Kind	Command	Runs in	Found in
`codeql`	Security scanning	`uses github/codeql-action/analyze`	ci	.github/workflows/codeql.yml
`ruff-check`	Linting	`ruff check --select F401,F841,I --fix`	ci	.github/workflows/main.yml .ruff.toml
`tsc`	Type checking	`tsc` `tsc --noEmit` `tsc --noEmit --project tsconfig.tests.json`	ci, local	.github/workflows/main.yml site/package.json code-scan-action/package.json .github/workflows/release-please.yml
`biome` configured, not run	Linting			package.json biome.jsonc

5 tools

Tool	Kind	Command	Runs in	Found in
`ci-matrix`	Version and platform matrix	`gradio-demo (python-version: 2)` `analyze (language: 5)` `package-build (node: 1)`	ci	.github/workflows/browser-example-python.yml .github/workflows/codeql.yml .github/workflows/main.yml .github/workflows/mcp-example-python.yml
`go-test`	Unit and integration tests	`go test -v wrapper.go wrapper_test.go`	ci	.github/workflows/main.yml
`unittest`	Unit and integration tests	`unittest discover -s examples/integration-browser/headless -p test_.py -v` `unittest discover -s .github/scripts -p test_examples.py -v` `unittest discover -s src/python -p _test.py`	ci	.github/workflows/browser-example-python.yml .github/workflows/examples.yml .github/workflows/main.yml .github/workflows/mcp-example-python.yml
`vitest`	Unit and integration tests	`vitest run` `vitest run --config vitest.smoke.config.ts` `vitest run --coverage`	ci, local	.github/workflows/main.yml package.json site/package.json
`jest` configured, not run	Unit and integration tests			examples/integration-jest/jest.config.js

2 tools

Tool	Kind	Command	Runs in	Found in
`prettier`	Formatting	`prettier --check */.{css,scss,html,md,mdc,mdx,yaml,yml}` `prettier --no-config --no-editorconfig --write ${changelog_files[@]}` `prettier --write */.{css,scss,html,md,mdc,mdx,yaml,yml}`	ci, local	.github/workflows/main.yml package.json .github/workflows/release-please-format.yml site/package.json
`ruff-format`	Formatting	`ruff format`	ci	.github/workflows/main.yml

1 tool

Tool	Kind	Command	Runs in	Found in
`npm-audit`	Dependency audit	`npm audit fix` `npm audit fix --prefix src/app` `npm audit fix --prefix site`	local	package.json

28 other commands the scanner could not classify

Commands from CI steps and task runners that look like checks but match no known tool. Some are checks, some are not.

buildx imagetools inspect ${IMAGE_REF})
cut -d = -f2-)
def test_function(a, b)
diff --cached --name-only --diff-filter=ACMRTUXB
else
html
md
mdc
mdx
node --import tsx ./scripts/checkCoverageRatchets.ts
node scripts/preparePackageArtifactTest.mjs --artifact-directory $ARTIFACT_DIRECTORY
node scripts/testContractsIsolation.ts
nothing to check"
python scripts/generate_modelaudit_schema.py
raise RuntimeError(MCP server exited during the client test)
ruby src/ruby/wrapper.rb /tmp/test_script.rb test_function /tmp/input.json /tmp/output.json
scss
sha256sum --check
true
ts
tsx scripts/checkArchitectureBoundaries.ts
tsx scripts/checkTypeScriptCoverage.ts
tsx scripts/generateOpenApi.ts --check
tsx scripts/testPackageArtifact.ts
xargs git add
xargs npx @biomejs/biome check --write
xargs npx prettier --write
yaml

most discussed first