Awesome Reviewers

Verification · Code review · LLM application security review

Runtime input and output scanning

Scan prompts and responses in production for injection and leakage.

How to build it

  1. Place a scanner in front of the prompt path and behind the response path.
  2. Block or redact the content the scanner flags, and log the flag without the content.
  3. Track the rate of flagged inputs and outputs per route.

Signals in a repository

  • Scanner calls that wrap the request and response functions in the service.
  • A guardrails or rails config directory in the repository.
  • Logs or metrics that count scanner flags per route.

In practice

No scanned repository runs a tool for this check. The steps above are the only guidance.