Skip to main content
This section is the runbook for shipping and operating Tuturuuu. It documents the deployment surfaces that exist in this repository today, the workflows that own them, and the commands the team should use when something needs to be built, released, or recovered.

Deployment Surfaces

Read This In Order

  1. Active runtime and retired services
  2. Develop without Docker
  3. Environments & Release Flow
  4. GitHub Actions Runbook
  5. Secrets & Configuration
Docker and Rust/TanStack runbooks are retained historical references. Their commands and CI are inactive until explicitly resumed.

Core Principles

  • GitHub Actions is the canonical automation layer for hosted deployments, platform build validation, and database migrations.
  • tuturuuu.ts can disable individual workflows; ci-check.yml enforces that toggle before a job does real work.
  • bun validate includes path-sensitive Discord Python validation when the local diff touches apps/discord/** or .github/workflows/discord-python-ci.yml. That path runs the same blocking checks as Discord Python CI through scripts/check-discord-python.js.
  • Vercel handles hosted satellite web deployments, platform preview build validation, and the hosted platform production deploy. Automatic TanStack production builds are disabled while that migration is paused. Supabase migrations run as separate workflows; main drives staging schema promotion and production drives production schema promotion.
  • Docker setup and its CI/E2E image provisioning are inactive. The Docker cron runner is retired; use the owning hosted scheduler and processor runbooks.
  • Rust/TanStack Start are paused and not in use. Do not build, test, port, refresh manifests, or update their dependencies until further notice. Docker-backed E2E jobs are paused; direct Next Playwright requires an explicitly prepared safe fixture and does not provision containers.
  • Secrets live in GitHub Actions secrets/variables or local env files such as apps/web/.env.local. They do not belong in the repo.

Vercel Cost Controls

Start with measured usage instead of applying cache directives globally. The following commands identify the expensive projects and inspect a bounded sample of production traffic without enabling a paid observability add-on:
vercel metrics requires Observability Plus. A payment_required response is an expected limitation when the add-on is disabled; do not enable it solely for routine cost diagnosis. Apply these controls in order:
  1. Reduce request count and response size before adding cache writes. Prefer direct-to-storage signed uploads for large payloads and avoid routing public assets through application functions when a stable CDN URL is available.
  2. Cache only public or safely scoped data. Never apply shared CDN caching to authenticated, user-specific, or workspace-private responses.
  3. Shared Satellite and Nova React Query clients use a 30-second stale window and one retry. Override this only when a surface has a documented freshness or reliability requirement. Realtime-backed notification counts use a 15-minute fallback poll and a 5-minute stale window. Active timers render elapsed time locally and use 1-minute running / 5-minute idle reconciliation. Keep mutations and realtime events invalidating their query keys immediately instead of shortening these safety-net intervals. Versioned external-project asset reads return early from the global API proxy and rely on the asset route’s publication/access checks, avoiding the global guard’s cookie validation, suspicious-request checks, and rate-limit work per cache-busted media redirect. Mutations, unversioned reads, and WebGL delivery remain behind the global guard.
  4. Dashboard sidebar links disable viewport prefetch and prefetch on hover or keyboard focus. This preserves responsive navigation without invoking every visible destination merely because the sidebar rendered.
  5. After deployment, compare the same billing window and project grouping. Confirm error rates and latency remain healthy before keeping a cost change.

Operational Flow

Fleet dry-run consumer

Cron Control owns the Fleet consumer and retention job on its shared 15-minute trigger. The job starts disabled. For an approved activation, wire the FLEET_CONSUMER_TOKEN Worker secret and set its SHA-256 verifier FLEET_CONSUMER_TOKEN_SHA256 on Web. Include the Worker bindings in the publication workflow and parity fixture. Confirm the Web routes and private RPCs are ready, then add fleet-access-consumer to CRON_CONTROL_ENABLED_JOBS and set FLEET_CONSUMER_MODE=dry-run. The mode enables dry-run intent receipts and actual retention deletes. To stop in-flight mutations, use the Fleet Owner database operation private.fleet_set_stopped(true, 'operator_stop') with the verified Owner session. It advances the Fleet generation and waits for database work that already holds the fence. This stops Fleet access as well as the consumer. For a consumer-only stop between API calls, use the authenticated Cron Control operation POST /control/jobs/fleet-access-consumer/stop. Verify the persisted state with GET /control/jobs. Set FLEET_CONSUMER_MODE=off or remove the job from the allowlist to stop new admissions. Removing the Web verifier disables the consumer credential. The three database mutation RPCs check the generation in their own transactions. Each slot makes one attempt and stops on the first unknown result. Five consecutive failures put the job in the persisted ledger stop state. Inspect Cron Control job status, lease states and append-only purge counts before resuming. The next admitted slot reconciles SQL state without renewing expired leases. A Fleet-wide stop requires fresh intent authorization. Resume Cron Control only after the fault is resolved and the appropriate Fleet stop is cleared. The operator should rehearse both stops and review hosted usage and alert delivery before activation.

What Changed Recently

  • TanStack/Rust migration maintenance is paused. rust-backend.yml and automatic TanStack production builds are switchboard-disabled; the retained source and runbooks are for local or explicitly approved future resumption work.
  • apps/web now supports both in-place Docker production deploys and blue/green deploys.
  • docker-setup-check.yaml is disabled, including manual dispatch; retained image parity steps do not execute.
  • Production Redis in Docker now requires a token, but scripts/docker-web.js satisfies that automatically by generating and injecting the value unless you explicitly opt out with --without-redis. Watcher-managed Infrastructure projects do not inherit that platform Redis token; they start with Redis disabled and require project-scoped MANAGED_PROJECT_<PROJECT_ID>_UPSTASH_* credentials when Redis is intentionally enabled.
If you are changing deployment behavior, update this section and add any new page to apps/docs/docs.json.