Back to all reviewers

Benchmark performance assumptions

semgrep/semgrep
Based on 4 comments
Other

Always validate performance concerns with actual measurements before making optimization decisions. Avoid making assumptions about what is "too slow" without benchmarking evidence.

Performance Optimization Other

Reviewer Prompt

Always validate performance concerns with actual measurements before making optimization decisions. Avoid making assumptions about what is “too slow” without benchmarking evidence.

When performance issues are confirmed through benchmarks, optimize strategically by:

  • Minimizing expensive operations (e.g., avoid repeated calls to costly functions like Fpath.v)
  • Eliminating unnecessary allocations (e.g., avoid constructing intermediate lists when direct iteration is possible)
  • Caching expensive computations at parse-time rather than recomputing them repeatedly

Example of evidence-based decision making:

(* Initial assumption: "We cannot afford to call Unix.realpath for each path because it's too slow" *)
(* After benchmarking: "running Semgrep with all free and Pro rules on 38 repos, and seeing no difference in perf wrt baseline" *)
(* Conclusion: "clearly we can afford Unix.realpath" *)

Example of strategic optimization:

(* Inefficient: constructing unnecessary intermediate lists *)
let trace_tokens = trace |> List.concat_map tokens_of_trace_item in
let filenames = trace_tokens |> List.map get_filename in

(* Better: direct iteration without intermediate allocations *)
let filenames = trace |> List.fold_left (fun acc item -> 
  get_filename_from_item item :: acc) [] |> List.rev in

This approach prevents both premature optimization and performance regressions by ensuring decisions are grounded in measurable data.

4
Comments Analyzed
Other
Primary Language
Performance Optimization
Category

Source Discussions