Awesome Reviewers expert instructions

domains / / microsoft/VSCode

Resilient Efficient CI

CI/CD workflows should be resilient, deterministic, and efficient: 1) Required checks should be resilient and scope-correct - Use a single “PR gate” (or equivalent) that passes when all relevant test jobs succeeded or were skipped.

raw .md CI/CD Yaml

CI/CD workflows should be resilient, deterministic, and efficient:

1) Required checks should be resilient and scope-correct

  • Use a single “PR gate” (or equivalent) that passes when all relevant test jobs succeeded or were skipped.
  • Prefer a fails-open approach when upstream planning/change detection is unavailable, so outages don’t block PR merges.
  • Use always() so you still report job outcomes when failures happen.

Example (pattern):

pr-gate:
  name: PR Gate
  needs: [changes, linux-electron-unit-tests, ...]
  if: $
  runs-on: ubuntu-24.04
  steps:
    - name: Check job results
      env:
        NEEDS: $
      run: |
        set -euo pipefail
        detection=$(jq -r '.changes.result' <<< "$NEEDS")
        # If detection/planner failed, do not fail the gate; let PR proceed.
        status=0
        # ...set status based on required job results...
        exit $status

2) Optional artifacts/selection inputs must not accidentally expand the test scope

  • If a “test plan”/selection artifact might be absent, handle the missing artifact explicitly (e.g., download step continues on error and then remove the plan so tests run with the intended fallback).
  • Ensure the behavior is deterministic: missing plan => clearly defined “run all” (or “skip”), but not an accidental mix.

Example (pattern):

- name: Download unit test plan
  if: $
  continue-on-error: true
  uses: actions/download-artifact@<sha>
  with:
    pattern: unit-test-plan
    path: .build/unit-test-plan

# On download failure, delete the directory so tests see "no plan".
- name: Remove plan if missing
  if: $
  run: rm -rf .build/unit-test-plan

3) Don’t multiply heavy CI preparation for small additional test subsets

  • Avoid adding separate jobs/workflows that repeat full repo checkout/build/dependency setup just to run a small extra shard.
  • Instead: reuse prepared outputs (artifacts, caches, restore/transpile/Electron setup) or run the additional test subset concurrently on the same prepared runner.
  • Only keep isolated jobs when overlap would be unsafe (e.g., tests that mutate shared system/network state).

4) Add retries only to network-dependent steps, with targeted env knobs

  • Set retryCountOnTaskFailure on the specific step types that are network/caching dependent.
  • For cargo/crate fetching, scope env/network tuning to the crate-fetching steps (e.g., CARGO_NET_RETRY / disable HTTP multiplexing) rather than changing unrelated build/lint behavior.

Example (pattern):

- script: cargo build
  retryCountOnTaskFailure: 2
  env:
    CARGO_HTTP_MULTIPLEXING: "false"
    CARGO_NET_RETRY: "5"

Adopt this checklist in PR reviews for CI/CD changes to prevent merge-blocking outages, avoid accidental test-scope changes, reduce duplicated CI cost, and improve robustness against transient network failures.