Observability vs Monitoring: Monitoring tells you when something is wrong.
Observability tells you why it’s wrong and how to fix it. Tuturuuu leans on a
self-hosted, Postgres-backed log drain rather than a third-party observability
vendor.
Architecture in Transition
The legacyapps/web Next.js runtime (port 7803) is being replaced by
apps/tanstack-web (TanStack Start) plus apps/backend (Rust, port 7820).
The observability surface described below lives mostly in apps/web today and
is migrating alongside the rest of the platform. See
TanStack Start And Rust Migration
for the contract, runtime shape, and ports.
1. Request-Scoped Correlation (Not Distributed Tracing)
The pattern
Distributed tracing assigns each request a trace ID that flows through every service, with each service recording spans (units of work) linked by that trace ID. This is invaluable in a deep microservice mesh where one user action fans out across dozens of services.What Tuturuuu actually does
Tuturuuu does not run OpenTelemetry or@vercel/otel, and there is no
instrumentation.ts that registers an OTel exporter. The real
apps/web/src/instrumentation.ts does not install a console drain:
AsyncLocalStorage context that carries a generated
requestId. API and cron handlers run inside withRequestLogDrain /
withCronLogDrain, and every log emitted during that request is stamped with
the same requestId, route, deployment color/stamp, and (once known)
userId/userEmail:
log_events are written
to Postgres in one transaction, so you can later query every log line for a
single requestId. This gives per-request correlation and timing without an OTel
backend—but it does not propagate a trace across the apps/web → Rust
backend boundary. True cross-service tracing remains aspirational.
Takeaway: Correlate by
requestId, not by OTel spans. There is no
getActiveTrace(), trace.span(), or io.getSpan() API in this codebase.2. Centralized Logging via the Log Drain
The pattern
In a distributed system, scattered per-instance log files are impractical. Centralized logging streams structured logs to a unified store where they can be searched, filtered, and correlated across boundaries.What Tuturuuu actually does
This pattern is partially implemented.apps/web/src/lib/infrastructure/log-drain.ts
is a self-hosted Postgres store for retained request, cron, deployment, and
usage events. Server runtime logs use the native console method that matches
severity; do not add serverLogger runtime imports or automatic console
log-drain installation.
Captured retained events are persisted to dedicated tables:
| Table | Holds |
|---|---|
requests | One row per request/cron run: method, path, status, duration |
log_events | Individual log lines, linked to a request_id |
cron_runs | Cron execution outcomes (success/failed, http status) |
usage_events | Numeric usage/metric samples (metric, value, unit) |
deployments | Blue/green deployment metadata used to stamp logs |
level, message, route, requestId, deployment
color/stamp, optional userId/userEmail, and a JSON metadata bag. The drain
redacts bearer tokens and token/key/secret/password query params before
persisting, and is designed to never throw into the caller—logging failures
are swallowed.
PLATFORM_LOG_DRAIN_ENABLED— set to0/false/offto disable.PLATFORM_LOG_DRAIN_DATABASE_URL— Postgres connection for the drain.PLATFORM_LOG_DRAIN_RAW_RETENTION_DAYS— rawlog_eventsretention (default 30 days).PLATFORM_LOG_DRAIN_SUMMARY_RETENTION_DAYS— summary table retention (default 90 days).PLATFORM_BLUE_GREEN_COLOR,PLATFORM_DEPLOYMENT_STAMP,PLATFORM_PROJECT_ID— deployment stamping.
Querying logs
Logs are read back through the internal observability API underapps/infrastructure/src/app/api/v1/infrastructure/observability/* (for example logs,
requests, cron-runs, deployments, overview, resources, analytics),
which power the in-app infrastructure/observability dashboard. There is no
third-party log platform (no Datadog/ELK) in the loop.
3. Metrics and Usage Telemetry
The pattern
Per-service metrics—request rates, error rates, latency percentiles, resource utilization—let you detect and localize failures quickly and scale individual services on their own load.What Tuturuuu actually does
Tuturuuu does not run Prometheus, ametrics.counter()/histogram()/gauge()
client, or a @tuturuuu/observability package. Instead, metrics are derived
from the same Postgres-backed drain:
- The
requeststable already encodes per-request status and duration_ms, so request rate, error rate, and latency are computed by aggregating that table over a time window. - The
usage_eventstable stores arbitrary numeric samples (metric,value,unit)—the place to record product/business telemetry such as tokens consumed or workspaces created. apps/web/src/lib/infrastructure/observability.tsaggregates these (and container resource metrics like CPU percent / memory bytes) for the observabilityoverview,analytics, andresourcesendpoints.
usage_events rows through the infrastructure layer
rather than reaching for a Prometheus client that does not exist.
4. Health Checks
The pattern
Services expose standardized health endpoints. Orchestrators probe them and stop routing to—or replace—instances that fail, enabling self-healing deployments. A common refinement splits liveness (“is the process running?”) from readiness (“is it ready to serve traffic?”).What Tuturuuu actually does
Theapps/web health endpoint is intentionally trivial—it does not probe
the database, Supabase, or Trigger.dev, and there are no liveness/readiness
sub-routes:
apps/backend) goes further and does separate liveness
from readiness:
GET /api/health→{ "status": "ok" }(parity with the web app,no-store).GET /healthz→ liveness, returning service name, runtime ("rust"), and environment.GET /readyz→ readiness; returns200only when the backend is fully configured (for example,BACKEND_INTERNAL_TOKENis present), otherwise503.
livenessProbe/readinessProbe
YAML shown in earlier revisions of this page is illustrative only—Tuturuuu’s
deployment model is the blue/green container watcher, not raw Kubernetes pods.
5. Scheduled Background Work (Not Event Sourcing)
The pattern
Event sourcing persists an immutable, ordered history of domain events and rebuilds state by replaying them—enabling audit trails, time-travel debugging, and projection rebuilds.What Tuturuuu actually does
Tuturuuu does not implement event sourcing. There is no event store, noeventStore.append/query, and no rebuildProjection. Background work runs on
Trigger.dev v4 (@trigger.dev/sdk ^4.4.5) using the task() API—not the
removed v2 client.defineJob / eventTrigger / io.runTask APIs.
requests, cron_runs, and log_events tables: cron and
request history is queryable for a retention window, and cron outcomes are
recorded as success/failure with duration and HTTP status. That is operational
history, not a replayable domain-event log.
What Is Implemented vs. Conceptual
| Capability | Status in Tuturuuu | Reality |
|---|---|---|
| Request-scoped correlation | Implemented | AsyncLocalStorage requestId in the log drain |
| Distributed tracing (OTel) | Aspirational | No OTel / @vercel/otel; no cross-service spans |
| Centralized logging | Partial | Native console.* plus retained Postgres request/cron/deployment data |
| Per-service metrics | Partial | SQL aggregation over requests/usage_events; no Prometheus |
| Health checks | Implemented | Web /api/health; backend /healthz + /readyz |
| Self-healing orchestration | Partial | Blue/green watcher, not Kubernetes pods |
| Event sourcing / replay | Not implemented | Trigger.dev v4 task() scheduled jobs + cron history |
Observability Best Practices Here
- Use native server logs. In API, cron, and infrastructure runtime code, log diagnostics with the native console method that matches severity. Include structured metadata in the same call so retained platform views and deployment logs stay readable.
- Correlate by
requestId. When debugging, find the request row, then pull itslog_events—that is the unit of correlation, not a trace ID. - Record metrics as
usage_events. New product/business metrics belong in the usage table via the infrastructure layer, surfaced through the observability API. - Never let logging break the caller. The drain swallows its own errors by design; preserve that property in any extension.
- Reference config by name. Tune retention and enablement through the
PLATFORM_LOG_DRAIN_*environment variables; never commit connection values.
Related Documentation
- TanStack Start And Rust Migration - Frontend/backend runtime split and ports
- Microservices Patterns - Service design and deployment
- Event-Driven Architecture - Event/queue patterns
- Extensibility, Resilience & Scalability - System quality attributes
- Security Architecture - Security observability and audit trails