CI/CD Pipeline Optimization

Prerequisite: Read the CI/CD Pipelines page to understand the basic workflow structure.

Overview

Our CI/CD pipelines avoid wasted builds, tests, and deployments through a centralized affected-path gate plus build caching. Instead of every workflow re-implementing its own change detection, automatic deployments share the same TypeScript decision function while avoiding a separate configuration runner when the decision can be made inside an already-required job. This page documents how that gate works, how the per-app Vercel workflows are wired, and how to debug skip decisions.

The platform is mid-migration from apps/web (Next.js, port 7803) to apps/tanstack-web (TanStack Start) plus apps/backend (Rust, port 7820). See TanStack + Rust migration. The optimization model below applies to both stacks: every Vercel app target is gated the same way, regardless of framework.

The Problem

Before centralized gating, workflows ran on every push regardless of:

Whether files relevant to that specific app or package actually changed
Whether the change only touched an unrelated app
Whether build artifacts were already cached

With a large monorepo of apps and shared packages, this meant a single commit to one app could trigger dozens of unrelated build and deploy workflows, slowing feedback and congesting the Actions queue.

How Gating Works Today

Vercel affected-app gating does not rely on GitHub’s native on.push.paths filters. Path filters are coarse (they cannot understand workspace dependency graphs), so the Vercel logic stays centralized. A narrow exception is the external-app compatibility build: its contract is simply “apps/external or any internal package changed,” so native trigger paths can avoid creating a runner at all for unrelated app-only commits. The shared logic is organized in four pieces:

tuturuuu.ts — the source of truth. It exports a ci toggle map (each workflow can be globally enabled/disabled), the vercelWorkflowTargets table that maps each app to its preview/production workflow and package name, and the getWorkflowDecision() function that decides whether a given workflow is affected by a set of changed files.
scripts/ci/resolve-changed-files.ts — resolves the changed-file set for the current event (push payload, pull-request base/head, or git diff against the last successful deployment marker).
.github/workflows/ci-check.yml — the reusable gate for non-Vercel workflows that still need an individual should_run output.
.github/workflows/vercel-production.yaml — the single production planner. It resolves every app against that app’s last successful deployment marker in one runner, then calls only affected production workflows as reusable jobs in the same commit-associated push run.

Reusable and inline gates

Non-Vercel gated workflows can declare a check-ci job that calls ci-check.yml, then gate their real job on the result. Platform preview is different: it must complete on every protected main push so supabase-staging.yaml receives its workflow-run signal. Its deploy job checks configuration and affected paths inline, then skips install/build/deploy steps when the app is unaffected. This keeps the workflow signal while using one runner instead of a configuration runner followed by a deploy runner.

jobs:
  Deploy-Preview:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
        with:
          fetch-depth: 0
      - uses: actions/setup-node@v7
        with:
          node-version: 24
      - name: Compute changed files
        id: changed_files
        run: node --experimental-strip-types scripts/ci/resolve-changed-files.ts
      - name: Check preview configuration
        id: check_config
        run: node --experimental-strip-types scripts/ci/check-workflow-config.ts

Internally, ci-check.yml uses a sparse checkout (only the manifests and scripts it needs), computes the changed files, and runs the decision script:

- name: Compute changed files
  id: changed_files
  env:
    GITHUB_TOKEN: ${{ github.token }}
    WORKFLOW_NAME: ${{ inputs.workflow_name }}
  run: node --experimental-strip-types scripts/ci/resolve-changed-files.ts

- name: Check Configuration
  id: check_config
  env:
    CHANGED_FILES_FILE: ${{ steps.changed_files.outputs.changed_files_path }}
    WORKFLOW_NAME: ${{ inputs.workflow_name }}
  run: node --experimental-strip-types scripts/ci/check-workflow-config.ts

The scripts run with Node’s --experimental-strip-types, so the .ts sources execute directly without a separate build step. Trusted manual satellite preview workflows launch their guarded deploy job directly: manual dispatch already bypasses affected-path gating, so a reusable preflight would only add a second queued runner. These workflows remain protected by the main dispatch ref, TRUSTED_PREVIEW_DEPLOY_ACTORS, and their GitHub Environment. Preview concurrency is keyed by workflow and preview_ref; a repeated request for the same preview cancels stale work, while different preview refs remain independent.

The production planner

A production push starts one plan job instead of starting one check-ci runner per app. resolve-production-vercel-targets.ts reads the workspace graph once, memoizes dependency closures, resolves each target from its own successful deployment marker, and emits a reviewable Actions summary. Static reusable jobs call only the selected vercel-production-<app>.yaml workflows, so deployments stay attached to the original push SHA instead of appearing as detached workflow_dispatch runs. Per-app production workflows remain independently protected by their GitHub Environment and a static per-app concurrency group. The group must include the app identifier because reusable workflows inherit caller context; a shared caller-derived key would cancel sibling deployments and show red X statuses instead of clean skips. They accept workflow_call for commit-driven planner jobs and retain workflow_dispatch for deliberate operator reruns. Their cancel-in-progress predicate is enabled only for a production push, so a main commit or manual recovery run cannot cancel an active production deploy. A missing or untrusted marker fails open for that app without forcing every other app’s gate open.

The decision function

getWorkflowDecision() in tuturuuu.ts resolves to should_run using this order of precedence:

Global toggle. If the workflow is false in the ci map, it never runs.
workflow_dispatch bypass. Manual runs always proceed (affected-path gating is skipped), so you can force a deploy from the Actions UI.
Non-Vercel workflows. Workflows without an entry in vercelWorkflowTargets fall back to their static ci toggle.
Unavailable change data. If the changed-file set cannot be resolved, the gate stays open (should_run = true) to fail safe.
Affected-path matching (Vercel workflows). The workflow runs only if at least one changed file matches one of:
- a global affecting path (bun.lock, package.json, turbo.json, tuturuuu.ts, .github/workflows/ci-check.yml, the shared Bun/remote-cache actions, or the deterministic metadata generator);
- the workflow’s own workflow file;
- the target app’s directory (e.g. apps/web/** for the platform target);
- any package inside the target app’s workspace dependency closure. The closure is built from workspace:* dependencies, so editing a shared package only triggers the apps that actually depend on it.

If nothing matches, the decision is should_run = false and the heavy job is skipped.

Vercel target table

Every Vercel app is registered once in vercelWorkflowTargets. Adding a new app means adding a single entry there (app slug, app path, package name, and the preview/production workflow filenames) — the gating logic and changed-file matching then apply automatically.

{
  app: 'platform',
  appPath: 'apps/web',
  packageName: '@tuturuuu/web',
  previewWorkflow: 'vercel-preview-platform.yaml',
  productionWorkflow: 'vercel-production-platform.yaml',
}

Applied to: the 24-target production planner, the inline platform preview gate, and other gated CI workflows wired through ci-check.yml. The other 23 preview workflows are trusted manual dispatches and therefore do not allocate a standalone configuration runner.

Build Caching

Once a workflow decides it must run, cacheable JavaScript work is invoked from the repository root through .github/actions/run-with-turbo-remote-cache/action.yml:

- name: Type check
  uses: ./.github/actions/run-with-turbo-remote-cache
  with:
    command: bun turbo:local run type-check --concurrency=4
    token: ${{ secrets.TURBO_TOKEN }}
    team: ${{ vars.TURBO_TEAM || secrets.TURBO_TEAM }}

The composite action exposes credentials only to the wrapped command step. It runs locally when both token and team are absent, rejects a token without a team, and keeps Turbo’s local rebuild behavior available if the remote service is unavailable. TURBO_TOKEN must never be passed to pull-request or Dependabot jobs. Those jobs restore a task-family-scoped .turbo fallback from GitHub’s cache without writing shared entries; trusted default-branch jobs seed that fallback. The shared Bun runtime and package-download caches follow the same ownership rule: only trusted main jobs save entries. Production, pull-request, Dependabot, and other branch jobs restore the default-branch cache without creating branch-scoped duplicates. This is especially important for Bun’s package cache, which can approach 1 GB for the full monorepo lockfile. Turborepo hashes inputs and reuses cached task outputs, so unchanged builds, type checks, and tests resolve to cache hits instead of recompiling. The task graph includes transit-only dependency nodes for tests: dependency source changes invalidate downstream tests without running every dependency’s test suite. Build outputs include framework output, coverage, and TypeScript incremental state where those artifacts are deterministic. Output-affecting environment values are hashed, while CPU, heap, and concurrency controls are pass-through values. Every active Vercel project has a checked-in buildCommand that invokes its full workspace build through root Turbo. The 48 deployment workflows generate deterministic source metadata and then wrap vercel build; they do not run a separate dependency prebuild. This lets an unchanged full app build restore as one task while .vercel/output remains an in-job handoff to vercel deploy --prebuilt, not an uploaded artifact or generic cache. Docker and Rust verification use the same principle at their native cache layers. Each shared GitHub BuildKit scope has one trusted main writer: Docker setup owns web, TanStack, and storage scopes; Rust CI owns the backend scope; and E2E shard 1 owns the small leaf-service scopes. Production, other shards, and migration E2E restore without exporting. Expensive web, TanStack, and backend images use mode=max; validation-only development and small leaf images use mode=min or restore-only caching. Dockerized E2E jobs explicitly select a docker-container Buildx builder so Compose can import and export the GitHub cache backend. Turbo values enter image builds only as BuildKit secrets. Rust verify and deploy jobs share one pinned, toolchain-aware Cargo cache, with writes restricted to protected branches. Flutter workflows cache dependencies and native tool state (pub, Gradle, and CocoaPods), never final APK, AAB, IPA, archive, or application-bundle outputs. Dependency resolution and the platform build always execute so a cache hit cannot skip validation. Managed uv and CodeQL caches remain owned by their upstream actions rather than being duplicated. Automatic CodeQL is owned by GitHub’s organization-managed dynamic/github-code-scanning/codeql workflow, which scans JavaScript/TypeScript and Python. The checked-in .github/workflows/codeql.yml is manual-only: it preserves an explicit fallback and avoids the Security UI’s missing-workflow warning without adding push, pull-request, or cron duplicates. bun git-sync mirrors the already-scanned main SHA to production. The full Dockerized E2E graph is created for E2E specs, Playwright and Docker configuration, database fixtures, package manifests, lockfiles, and its own runner scripts. Ordinary application source commits do not create six expensive E2E consumers. There is no cron schedule: automatic E2E runs are commit-driven and path-scoped, while manual dispatch remains available for deliberate validation. E2E uses native push paths and launches its image producer directly, without an extra static switchboard job. Supabase staging and production migrations use the last successful migration deployment marker as their change-range baseline. Each workflow resolves the entire pending range, fails open when no trustworthy marker exists, and runs supabase db push --include-all only when apps/database/** or migration control files changed. Evaluation and deployment share one serialized job per environment, so no-op workflow-run signals finish quickly while database migrations cannot race. Production still requires the successful same-SHA platform deployment marker and staging migration before applying changes.

Runner And Storage Policy

Tuturuuu is a public repository in a GitHub Free organization. Standard hosted Linux and Windows jobs are budgeted for 4 CPUs and 16 GB RAM; the organization can run 20 jobs concurrently, including at most 5 macOS jobs. CPU-bound Turbo checks therefore default to concurrency 4. Memory-heavy Next.js and Docker stages limit inner build concurrency to 2 and retain an 8 GB Node heap. Keep independent jobs parallel, keep macOS matrices below the platform limit, and do not switch to billable larger runners as a cache optimization. The live repository cache configuration, queried through GitHub’s API, is a 10 GB maximum with 7-day retention. The weekly Actions resource report groups cache usage by key prefix and reports informational status at 80%, warning at 90%, and critical status at 100%. It also reports artifact count, bytes, age, and largest workflow families without enforcing a fixed artifact byte cap. Steady-state cache usage should stay below 9 GB to leave eviction headroom. The July 10 audit observed active usage temporarily above the configured limit; that is eviction lag and cache churn, not an additional open-source allowance. Artifacts are retained only as long as their handoff or diagnostic value requires:

E2E and Playwright diagnostics: one failure-only artifact per shard or migration mode, 7 days.
Package-release tarballs: 1 day with compression disabled.
Mobile development deliverables: 7 days; production store deliverables: 14 days. Pre-compressed bundles are archived first and uploaded without redundant compression.
Discord JUnit and Cloudflare smoke diagnostics: failure-only or explicit 7-day retention as applicable.

Every artifact upload must declare both retention and missing-file behavior. Required release outputs fail when absent; optional diagnostics may warn. Closed trusted pull requests delete cache entries scoped to their merge ref. Legacy broad Turbo, run-ID Supabase, final Flutter-output, and old Rust cache prefixes are removed manually only after their replacements demonstrate hits; default-branch CodeQL caches are never part of automated cleanup.

Earlier versions of these workflows ran an inline Turborepo --dry-run=json step to decide whether to skip a build. That per-workflow skip step has been removed in favor of the centralized ci-check gate above. Turborepo is now used for caching the build itself, not for the run/skip decision.

Race-Condition Handling

Production deployments serialize each app and branch with GitHub Actions concurrency:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: ${{ github.event_name == 'push' && github.ref == 'refs/heads/production' }}

When another production commit arrives, GitHub cancels the superseded run and keeps the newest run deployable. Older runs no longer finish green after skipping every deployment step, which makes queued and deployed SHAs observable without allowing stale artifacts to replace newer production code. Commits that exist only on main and manual recovery dispatches leave an active production deployment running. Affected-app gating always diffs the newest SHA against the last successful deployment marker. If no trustworthy marker is available, the gate fails open and runs the deployment instead of considering only the newest push payload; this preserves app changes from canceled intermediate commits.

Edge Case Handling

resolve-changed-files.ts and the gate are written to fail safe.

Missing or unresolvable change data

If the changed-file set cannot be computed (e.g. an unusual event payload or a missing base commit), getWorkflowDecision() returns should_run = true. The guiding principle: build when uncertain rather than skip incorrectly.

Initial commits and shallow clones

When a parent commit is unavailable (the very first commit, or a shallow clone without history), the changed-file resolver cannot diff against a base, so the gate opens and a full build runs. The workflow that needs deeper history checks out with an appropriate fetch-depth, and ci-check.yml uses fetch-depth: 0 with a sparse checkout so it always has enough history to diff.

Last-successful-deployment markers

For push events, the resolver can diff against the SHA of the last successful deployment (recorded as a GitHub deployment marker) rather than only the parent commit. This catches the case where intermediate commits never deployed, so the full diff since the last live deploy is considered.

Monitoring & Debugging

Inspect the gate decision

check-workflow-config.ts logs its decision and the matched paths. Preview check-ci logs show the individual decision; production uses the planner’s Production Vercel deployment plan job summary, including the per-app baseline source and deploy/skip result. Individual logs still look like:

Workflow: vercel-production-platform.yaml
Should run: false
Reason: vercel-production-platform.yaml is unaffected by the changed paths

or, when it does run:

Should run: true
Reason: vercel-production-platform.yaml is affected by 3 changed path(s)
Matched paths:
- apps/web/src/app/page.tsx
- packages/ui/src/button.tsx

Reproduce a decision locally

You can run the decision script directly to debug why a workflow did or did not run:

CHANGED_FILES="apps/web/src/app/page.tsx" \
WORKFLOW_NAME="vercel-production-platform.yaml" \
node --experimental-strip-types scripts/ci/check-workflow-config.ts

There are also unit tests covering the gating logic:

node --test scripts/ci/check-workflow-config.test.js
node --test scripts/ci/resolve-changed-files.test.js
node --test scripts/ci/resolve-production-vercel-targets.test.js

Common Issues

Issue: A workflow never runs for legitimate changes

Likely causes:

The app is missing (or misconfigured) in vercelWorkflowTargets.
The changed package is not in the target app’s workspace dependency closure (the app does not actually workspace:*-depend on it).
The workflow is toggled off in the ci map in tuturuuu.ts.

Fix: Verify the target entry and the dependency edges, then re-run the local reproduction above to confirm the decision.

Issue: A workflow always runs

Likely causes:

A global affecting path (bun.lock, package.json, turbo.json, tuturuuu.ts) changed — these intentionally trigger every Vercel target.
The change-data resolver failed, so the gate fell open by design.

This is usually correct behavior; only investigate if it persists for changes that clearly do not touch shared roots.

Issue: Need to force a run

Use the app workflow’s manual trigger. A direct workflow_dispatch starts that app without re-running affected-path gating:

on:
  workflow_dispatch:

Performance Impact

The exact savings depend on the size of the app set and how localized each commit is, so treat the following as illustrative rather than a fixed figure. The monorepo currently registers 24 active Vercel app targets, each with a preview and a production workflow (48 Vercel deploy workflows), out of 91 workflow files. Production planning prevents a single-app change from creating 23 unrelated production runners. Preview workflows create a runner only for the inline platform signal or an explicitly requested satellite preview. For a change that touches only one app:

1 production planner workflow
+ only the affected app deployment workflows
+ one inline platform preview gate on protected main
+ direct satellite preview jobs only when explicitly dispatched

Each skipped production app now consumes no runner at all; only the shared planner and selected reusable deployment jobs run under the production push. Changes to shared roots (package.json, bun.lock, turbo.json, tuturuuu.ts) intentionally fan out to all targets, so headline skip rates will vary with how often those roots change.

To measure real-world impact, inspect the production planner’s job summary and compare its selected workflow count with the 24 registered targets. Platform preview reports its inline decision in the deploy job; manual satellite previews have no separate gate job.

Best Practices

1. Keep `tuturuuu.ts` as the source of truth

When adding an app or workflow, register Vercel app metadata in vercelWorkflowTargets and register automatic switchboard-controlled workflows in the ci map. Trusted manual satellite previews and intentionally path-triggered compatibility smokes do not keep unused ci toggles. The gate and the tests key off these boundaries.

2. Trust the fail-safe

When change data is missing, the gate opens and the workflow runs. This is intentional — it is safer to build unnecessarily than to skip a real change. Don’t add ad-hoc skip logic that defeats it.

3. Model dependencies with `workspace:*`

The dependency-closure matching only works because shared packages are wired as workspace:* dependencies. If an app consumes shared code without declaring the dependency, the gate will not know to run it when that package changes.

4. Use `workflow_dispatch` to debug

To bypass gating for a one-off investigation, trigger the workflow manually from the Actions UI rather than touching the gate logic.

5. Update tests when changing the gate

getWorkflowDecision() and the resolver are covered by unit tests under scripts/ci/. Update them alongside any change to the matching rules so the behavior stays pinned.

Summary

Our CI/CD optimization centralizes change detection so each workflow stays thin:

Reusable ci-check.yml calls remain available where a separate gate is necessary; production planning and platform preview avoid per-app preflight fan-out.
tuturuuu.ts decides, per workflow, whether the change is relevant — including workspace dependency-closure awareness for shared packages.
The remote-cache wrapper keeps Turbo credentials step-scoped and gives secretless jobs a GitHub-backed local fallback.
Turborepo caches full app builds, type checks, tests, and release preparation once a run is warranted.
The gate fails safe (builds on uncertainty), and trusted manual satellite previews start their guarded deploy job directly.

The result: a one-app change selects only the necessary production deploys in the commit-associated push run, platform preview retains its required staging signal without a second runner, manual previews never queue a redundant preflight, and superseded previews are canceled before they occupy the queue indefinitely.

Overview

Platform

Build

Learn

Reference

CI/CD Pipeline Optimization

Overview

The Problem

How Gating Works Today

Reusable and inline gates

The production planner

The decision function

Vercel target table

Build Caching

Runner And Storage Policy

Race-Condition Handling

Edge Case Handling

Missing or unresolvable change data

Initial commits and shallow clones

Last-successful-deployment markers

Monitoring & Debugging

Inspect the gate decision

Reproduce a decision locally

Common Issues

Issue: A workflow never runs for legitimate changes

Issue: A workflow always runs

Issue: Need to force a run

Performance Impact

Best Practices

1. Keep `tuturuuu.ts` as the source of truth

2. Trust the fail-safe

3. Model dependencies with `workspace:*`

4. Use `workflow_dispatch` to debug

5. Update tests when changing the gate

Further Reading

Summary

​Overview

​The Problem

​How Gating Works Today

​Reusable and inline gates

​The production planner

​The decision function

​Vercel target table

​Build Caching

​Runner And Storage Policy

​Race-Condition Handling

​Edge Case Handling

​Missing or unresolvable change data

​Initial commits and shallow clones

​Last-successful-deployment markers

​Monitoring & Debugging

​Inspect the gate decision

​Reproduce a decision locally

​Common Issues

​Issue: A workflow never runs for legitimate changes

​Issue: A workflow always runs

​Issue: Need to force a run

​Performance Impact

​Best Practices

​1. Keep tuturuuu.ts as the source of truth

​2. Trust the fail-safe

​3. Model dependencies with workspace:*

​4. Use workflow_dispatch to debug

​5. Update tests when changing the gate

​Further Reading

​Summary

Overview

The Problem

How Gating Works Today

Reusable and inline gates

The production planner

The decision function

Vercel target table

Build Caching

Runner And Storage Policy

Race-Condition Handling

Edge Case Handling

Missing or unresolvable change data

Initial commits and shallow clones

Last-successful-deployment markers

Monitoring & Debugging

Inspect the gate decision

Reproduce a decision locally

Common Issues

Issue: A workflow never runs for legitimate changes

Issue: A workflow always runs

Issue: Need to force a run

Performance Impact

Best Practices

1. Keep `tuturuuu.ts` as the source of truth

2. Trust the fail-safe

3. Model dependencies with `workspace:*`

4. Use `workflow_dispatch` to debug

5. Update tests when changing the gate

Further Reading

Summary