Efficient hot-path discipline

Adopt a performance-safety standard: prevent unbounded state growth in monitoring, and keep critical-path CPU/memory work linear and avoidable.

Apply these rules: 1) Treat performance instrumentation as state with lifecycle

• If you create marks/measures per request/stream, ensure they don’t accumulate indefinitely.
• If downstream consumers rely on marks being observable right after return, document and test that behavior; otherwise prefer “clear after creation/consumption” to bound memory.

2) Keep streaming/decoding parsers linear

• Avoid re-concatenating/rehashing the entire buffer on every data event.
• Use a rolling buffer + cursor offset / slicing so each byte is processed once.

3) Precompute repeated work in bounded loops

• Tokenization/scoring: compute invariant sets once per request (e.g., context tokens) and reuse for each candidate.
• Policy lookups: precompute maps at module load for O(1) repeated checks.

4) Add cheap guards before expensive transforms

• Before regex/whitespace stripping/base64 decode, do a fast size check to fail early and avoid CPU burn.

5) Gate unnecessary upstream calls

• Only do quota/usage preflights when at least one enforcement condition is active; otherwise skip the upstream fetch to reduce latency and load.

Example patterns (self-contained):

// 1) Bounded performance marks performance.mark(“omni-request-body-size”, { detail: bodySize }); // … read/observe in your monitoring pipeline … performance.clearMarks(“omni-request-body-size”);

// 2) Rolling buffer (linear-time scanning) const chunks: Buffer[] = []; let processed = 0; req.on(“data”, (chunk) => { chunks.push(Buffer.from(chunk)); const buffer = Buffer.concat(chunks); // process from processed.. and advance processed cursor // then drop processed prefix to keep future work bounded });

// 3) Precomputed lookup const unsupportedParamsByModel = new Map<string, readonly string[]>(); // module init fills map function getUnsupportedParams(modelId: string) { return unsupportedParamsByModel.get(modelId) ?? []; }

// 4) Cheap guard before expensive work if (payload.length > MAX_BYTES * 2) throw new Error(“too large”); // only then do regex/strip/decode

// 5) Latency gate if (!hasAnyEnforcementCondition) return; // skip upstream preflight