Awesome Reviewers

Verification · Intent and spec conformance · Functional spec verification

Evidence capture

Screenshot and annotate each verdict.

How to build it

  1. Take a screenshot at each assertion, including the failing one.
  2. Store the verdict and the criterion text next to each screenshot.
  3. Attach the screenshots to the pull request or to the job artifacts.

Signals in a repository

  • screenshot: 'only-on-failure' or trace settings in playwright.config.
  • A CI step that uploads test-results or screenshot folders as artifacts.

In practice

55 repositories run a tool for this check in the scanned set.

  • playwright · 55 repos

Examples

  • freeCodeCamp/freeCodeCamp · playwright

    playwright test --ui-port=0
  • nousresearch/hermes-agent · playwright

    playwright test -c e2e/core/playwright.config.ts local-gateway-attach.spec.ts
  • microsoft/vscode · playwright

    playwright test
  • n8n-io/n8n · playwright

    playwright test --project=e2e tests/e2e/building-blocks/ --workers=1 --retries=2 --reporter=list
  • langflow-ai/langflow · playwright

    playwright test $SCAN_SPECS --project=chromium --shard=$EXPR/$EXPR --workers=2 --retries=1