Skip to main content

Build and qualify the exact image

Build the image in CI, never on a workstation. Dispatch the Playground execution image workflow (playground-execution-image.yaml) on main at the reviewed commit. It holds no Cloudflare credentials. The job builds the Dockerfile in apps/playground-execution/container for linux/amd64 and then runs two qualifications:
  • verify-image.mjs runs every stock template under Docker’s own isolation.
  • qualify-launcher.mjs runs the real launcher (run.py) as the slot does, with only the capabilities it needs to build its sandbox. It checks every stock template; UID, GID, capability sets and no-new-privileges; denied egress to public, metadata and private addresses and DNS on a bridged network, so the launcher’s network namespace is the barrier; writes outside the workspace; the command timeout; the output limit; and the process limit.
The job uploads image.tar.gz and receipt.json (source SHA, image ID, tag, image and archive sizes, archive SHA-256, run ID) as the playground-execution-image-<sha> artifact for 14 days. The registry upload is a separately approved operation on that archive.

Upload the image to the registry

Dispatch Playground execution registry upload (playground-execution-registry.yaml) on main with the image run ID and its full source SHA. The job accepts only a successful image run of that commit on main. It verifies the archive SHA-256 and image ID against the receipt, then pushes with the lockfile-pinned cf containers push --tag, retrying a failed push at most three times. It checks out the dispatched main tip, and the image commit must be an ancestor of it. It runs in the reviewer-gated playground-registry environment. Its only credential is the environment secret PLAYGROUND_REGISTRY_CLOUDFLARE_API_TOKEN: a short-lived token scoped to the Tuturuuu account and Containers registry write. The account comes from the CLOUDFLARE_ACCOUNT_ID variable. The job uploads a receipt with the registry reference registry.cloudflare.com/<account>/tuturuuu-playground@sha256:<digest> that the Worker image configuration uses. Revoke the token after the upload. The fixture uses the same stock commands that Meet and Learn load. It checks Python, JavaScript, TypeScript, C, C++, Java, Rust, Go, Ruby, PHP, and Shell. Each template runs in a fresh local container with UID and GID 65534, all capabilities removed, no new privileges, no network, a read-only root, and bounded writable directories. The Dockerfile uses a pinned amd64 base manifest and signed, dated Debian snapshots. Refresh the base and snapshots together when updating toolchains. The image contains no application credentials. Record the source SHA, build invocation, result for every language, and image ID. After an approved registry upload, also record the Cloudflare registry digest. Use the digest in the Worker image configuration. The local image ID and the registry manifest digest identify different objects. Retain the previous digest while it is a rollback target. Cloudflare image management

Deploy to staging and qualify the sandbox

Dispatch Playground execution staging deploy (playground-execution-deploy.yaml) on main with a registry reference pinned by digest. The build job has no credentials: it runs the Worker tests, builds the staging Worker with cf build, and checks that the emitted output names only that image and every Durable Object class. The deploy job runs in the reviewer-gated playground-staging environment. It deploys the prebuilt output with cf deploy --prebuilt --no-provision, uploading the ingress and operator secrets from the environment through a temporary secrets file, then runs hosted qualification and confirms admission is still paused. The environment holds PLAYGROUND_STAGING_CLOUDFLARE_API_TOKEN (a short-lived token minted for the run), PLAYGROUND_STAGING_INGRESS_SECRET and PLAYGROUND_STAGING_OPERATOR_SECRET, and uses the CLOUDFLARE_ACCOUNT_ID variable. Staging is reachable only on its workers.dev hostname, so it needs no zone or route change. Only the staging configuration contains the qualifier: the PLAYGROUND_QUALIFY flag, the PLAYGROUND_QUALIFIER binding and its one-guest container class. Production has none of them, which a configuration test enforces, and /qualify there answers like any unknown path. POST /qualify takes the operator secret. It starts one fresh guest and runs fixed fixture jobs, never request input, through the production exec, launcher and sandbox. The cases are every stock template, standard input, the dropped identity and capability sets, denied egress to public, metadata and private addresses and DNS, workspace write bounds, the command timeout, the output and process limits, and a compile-only check per language. It also records the guest’s own view as root (identity, capabilities, the kernel’s security modules and version) and the cold start time, then destroys the guest. The job uploads the report as playground-execution-qualification-<sha> and fails unless every case passed and the guest stopped. Check cases need an image built from a commit that includes check mode.

Hosted qualification

Run the fixture again against the digest on the actual Cloudflare execution path before enabling product traffic. Prove the guest has no external DNS, direct IP, or private network egress. Check effective UID and GID, all Linux capability sets, no new privileges, filesystem write bounds, resource limits, command timeout, output truncation, cancellation, and descendant cleanup. Confirm the container stops after idle. Measure cold latency and, if the per-run isolation boundary supports reuse, measure warm latency with a clean workspace. Include a second participant’s result from a real Meet room for each language. A Docker fixture verifies the image and template contracts. Cloudflare applies its own runtime policy. In particular, Durable Object scheduling does not remove root capabilities merely because an exec process sets a numeric user. Require hosted isolation evidence before enabling an execution route. Cloudflare Container exec

Cloudflare admission and rollback

The Cloudflare path uses a dedicated Worker, a SQLite coordination Durable Object, and at most 20 slot Durable Objects that can start guests. Meet keeps its existing authorization and passes a short-lived capability for the exact saved revision. The slot uses that capability to download source from Web. The source-signing key stays in the application servers. The guest receives only the saved files, command, and stdin. The Meet adapter is selected only when MEET_PLAYGROUND_EXECUTION_PROVIDER=cloudflare. The Durable Object starts with admission paused. The ingress and operator secrets are separate. Production deployment, registry upload, secret installation and admission changes each require operator authorization for the exact operation. Set the values lower for an early deployment. The defaults are also the approved ceilings. A duplicate request uses the original receipt and consumes no further start allowance. A failed or ambiguous dispatch consumes its original allowance. The active project limit protects shared rooms where two participants can ask to run the same revision. The current Meet adapter handles Run, Stop and Console results. File changes use the existing collaborative merge and revision-checked Drive checkpoint. Preview and Learn retain their current provider. Qualify the writeback and concurrent editor behavior before production activation. Verify that the legacy runner pool has no eligible Playground capacity while Meet uses Cloudflare and Learn still uses the legacy provider. The operator calls POST /control with the operator credential and {"enabled":false}. The response records the new generation and the number of active reservations. Admission then refuses new work. A draining response uses HTTP 202. A claimed Drive writeback keeps the reservation active until the coordinator receives the checkpoint confirmation, or until 60 seconds after the job deadline. That grace covers one room checkpoint and its settlement calls; after it the Run fails without a saved revision and releases its slot, user and project, so a lost settlement cannot wedge capacity or block re-enabling. Stop fences new writes and lets an in-flight export finish its bounded receipt wait. Confirm there are zero active reservations and zero running instances before rolling back the application route or rotating service secrets. The same control request is idempotent. Re-enable only after a scoped approval and verified teardown. Keep the result endpoint available for existing cloud run receipts while switching the Meet adapter back to its previous provider. An accepted reservation schedules finite work. The normal path makes one assignment RPC, one source fetch with an eight-second timeout, one guest start, one command, one finalization, and at most one file export. The slot journals its start before fetching source. Interrupted execution is reconciled without launching the command again. Cleanup makes at most three attempts per reservation. If cleanup stays unconfirmed, the slot remains occupied for operator reconciliation. Clients stop polling at the run deadline. Terminal jobs clear their deadline work. Result polling uses only the journal. A new admission prunes up to 64 expired receipts. The Node tests in apps/playground-execution/src pin these counts. admission.test.mjs covers the ledger: statements per reservation, restart deduplication, daily and rolling limits, the pause fence and the cleanup budget. slot.test.mjs covers the slot: the normal path’s fetch, start, command and storage writes, no replay after a restart in the started phase, an unconfirmed teardown that keeps the slot occupied, and Stop or pause before the guest starts. container-run.test.mjs covers the guest: egress disabled, output bounds and confirmed destruction. Run them with bun run --cwd apps/playground-execution test.

Code checks

POST /check returns compile-only diagnostics for unsaved room files in C, C++ and Java. No program runs. The coordinator charges a separate check budget in its own table: 60 checks per requester and 300 per room in a rolling 10 minutes, and PLAYGROUND_MAX_CHECKS_PER_UTC_DAY (default and ceiling 20,000) per UTC day. Every attempt is charged, including a retry with the same request ID, and an ID already used by another person or room is refused. The durable admission gate that pauses Runs pauses checks too, and a pause also stops the warm guests of rooms with checks in the last 35 minutes; /control answers 202 until each confirms it stopped. The pause is a durable fence: a check admitted before it is refused even if it reaches the room’s checker afterwards. Each room with recent checks has one PlaygroundChecker Durable Object and one warm guest on the basic instance (1/4 vCPU, 1 GiB) with Internet disabled. The guest stops after 90 seconds without checks, and an alarm replaces it 30 minutes after it starts, even under continuous checks. The checker container class allows at most 50 live guests. A cold room starts its guest and answers check_warming; the client retries. Each person has at most one check in flight, and a newer check kills the older one (check_superseded). A room runs at most two checks at once (check_busy). Every check is a separate exec with its own mount namespace, a fresh /tmp workspace, PID and network namespaces, the dropped identity and Landlock, so nothing a check writes survives it or reaches another check. The launcher runs a fixed command, never one from the request: gcc or g++ with -fsyntax-only, or javac -proc:none -implicit:none with a throwaway class directory. Sources are passed as ./path so no file name is read as an option or argument file. gcc and g++ get 4 seconds of wall and CPU time and 512 MiB of address space; javac gets 8 seconds, because a cold JVM on the smallest instance needs more, and a 192 MiB heap. The Worker abandons a check 4 seconds after the guest’s own limit, and the sandbox dies with its launcher, so a killed or superseded check leaves no compiler running. The Worker reports diagnostics only for request paths; a problem located in any other file (a toolchain header or an absolute include) becomes the fixed text “Problem in an included file” at the include site. A message located in a request file can still quote a value the compiler derived from an included file, and an include can test whether a path exists. The guest holds only the public image and the request’s own files, so this reveals nothing a Run could not read. A warm room guest costs about $0.00000278 per live second plus the CPU of its checks, about $0.012 per active room-hour at the published overage rates.

Launcher behaviour

The trusted launcher (container/run.py) runs as root only to mount a fresh /tmp and build the sandbox. It first runs the same namespace, privilege drop and Landlock chain around /bin/true; if the kernel or runtime cannot provide any part of it, the Run fails as sandbox_failed before user code runs. The program then runs as UID and GID 65534 with no capabilities, no new privileges, an empty network namespace, its own PID namespace and writes limited to /tmp. After a normal exit the launcher exports the workspace as a complete inventory. Files the program created that cannot be saved (binaries, names outside the path rules, files over 256 KiB, non-UTF-8 text) are left out. If a project file still exists but cannot be saved, or the workspace exceeds the inventory or size limits, nothing is exported and the Run keeps its output. An export problem never replaces the program’s output.

Usage and cost

The initial instance size is 0.5 vCPU, 4 GiB of memory, and an 8 GB disk. The command has a 30-second timeout within a 50-second job deadline. Output is capped at 64 KiB. Fresh instances are destroyed after execution, with a 15-second inactivity timeout as a fallback. Warm reuse requires independent hosted qualification before use. At published overage rates, the estimated Containers cost is $0.00001056 × live seconds + $0.000020 × active vCPU-seconds. A 15-second idle tail costs about $0.0001584 per instance plus actual CPU. Sleeping container compute is $0. The estimates exclude the Workers Paid base charge, Workers and Durable Object requests, duration, SQLite storage, and logs, and are before shared included allowances. The stress scenario is an estimate. Provider shutdown latency can change billed usage. Record real cold-start and shutdown times, journal operation counts, and actual usage during qualification. Cloudflare Containers pricing · Workers and Durable Objects pricing