Build and qualify the exact image
Build the image in CI, never on a workstation. Dispatch thePlayground execution image workflow (playground-execution-image.yaml) on
main at the reviewed commit. It holds no Cloudflare credentials. The job
builds the Dockerfile in apps/playground-execution/container for
linux/amd64 and then runs two qualifications:
verify-image.mjsruns every stock template under Docker’s own isolation.qualify-launcher.mjsruns the real launcher (run.py) as the slot does, with only the capabilities it needs to build its sandbox. It checks every stock template; UID, GID, capability sets and no-new-privileges; denied egress to public, metadata and private addresses and DNS on a bridged network, so the launcher’s network namespace is the barrier; writes outside the workspace; the command timeout; the output limit; and the process limit.
image.tar.gz and receipt.json (source SHA, image ID, tag, image and archive sizes,
archive SHA-256, run ID) as the playground-execution-image-<sha> artifact for
14 days. The registry upload is a separately approved operation on that
archive.
Upload the image to the registry
DispatchPlayground execution registry upload
(playground-execution-registry.yaml) on main with the image run ID and its
full source SHA. The job accepts only a successful image run of that commit on
main. It verifies the archive SHA-256 and image ID against the receipt, then
pushes with the lockfile-pinned cf containers push --tag, retrying a failed
push at most three times. It checks out the dispatched main tip, and the image
commit must be an ancestor of it. It runs in the reviewer-gated playground-registry environment. Its
only credential is the environment secret
PLAYGROUND_REGISTRY_CLOUDFLARE_API_TOKEN: a short-lived token scoped to the
Tuturuuu account and Containers registry write. The account comes from the
CLOUDFLARE_ACCOUNT_ID variable. The job uploads a receipt with the registry
reference registry.cloudflare.com/<account>/tuturuuu-playground@sha256:<digest>
that the Worker image configuration uses. Revoke the token after the upload.
The fixture uses the same stock commands that Meet and Learn load. It checks
Python, JavaScript, TypeScript, C, C++, Java, Rust, Go, Ruby, PHP, and Shell.
Each template runs in a fresh local container with UID and GID 65534, all
capabilities removed, no new privileges, no network, a read-only root, and
bounded writable directories. The Dockerfile uses a pinned amd64 base manifest and signed, dated Debian
snapshots. Refresh the base and snapshots together when updating toolchains.
The image contains no application credentials.
Record the source SHA, build invocation, result for every language, and image
ID. After an approved registry upload, also record the Cloudflare registry
digest. Use the digest in the Worker image configuration. The local image ID
and the registry manifest digest identify different objects. Retain the
previous digest while it is a rollback target. Cloudflare image management
Deploy to staging and qualify the sandbox
DispatchPlayground execution staging deploy
(playground-execution-deploy.yaml) on main with a registry reference
pinned by digest. The build job has no credentials: it runs the Worker tests,
builds the staging Worker with cf build, and checks that the emitted
output names only that image and every Durable Object class. The deploy job
runs in the reviewer-gated playground-staging environment. It deploys the
prebuilt output with cf deploy --prebuilt --no-provision, uploading the
ingress and operator secrets from the environment through a temporary
secrets file, then runs hosted qualification and confirms admission is still
paused. The environment holds PLAYGROUND_STAGING_CLOUDFLARE_API_TOKEN (a
short-lived token minted for the run), PLAYGROUND_STAGING_INGRESS_SECRET
and PLAYGROUND_STAGING_OPERATOR_SECRET, and uses the CLOUDFLARE_ACCOUNT_ID
variable.
Staging is reachable only on its workers.dev hostname, so it needs no zone
or route change. Only the staging configuration contains the qualifier: the
PLAYGROUND_QUALIFY flag, the PLAYGROUND_QUALIFIER binding and its one-guest
container class. Production has none of them, which a configuration test
enforces, and /qualify there answers like any unknown path.
POST /qualify takes the operator secret. It starts one fresh guest and runs
fixed fixture jobs, never request input, through the production exec,
launcher and sandbox. The cases are every stock template, standard input, the
dropped identity and capability sets, denied egress to public, metadata and
private addresses and DNS, workspace write bounds, the command timeout, the
output and process limits, and a compile-only check per language. It also
records the guest’s own view as root (identity, capabilities, the kernel’s
security modules and version) and the cold start time, then destroys the
guest. The job uploads the report as playground-execution-qualification-<sha>
and fails unless every case passed and the guest stopped. Check cases need an
image built from a commit that includes check mode.
Hosted qualification
Run the fixture again against the digest on the actual Cloudflare execution path before enabling product traffic. Prove the guest has no external DNS, direct IP, or private network egress. Check effective UID and GID, all Linux capability sets, no new privileges, filesystem write bounds, resource limits, command timeout, output truncation, cancellation, and descendant cleanup. Confirm the container stops after idle. Measure cold latency and, if the per-run isolation boundary supports reuse, measure warm latency with a clean workspace. Include a second participant’s result from a real Meet room for each language. A Docker fixture verifies the image and template contracts. Cloudflare applies its own runtime policy. In particular, Durable Object scheduling does not remove root capabilities merely because an exec process sets a numeric user. Require hosted isolation evidence before enabling an execution route. Cloudflare Container execCloudflare admission and rollback
The Cloudflare path uses a dedicated Worker, a SQLite coordination Durable Object, and at most 20 slot Durable Objects that can start guests. Meet keeps its existing authorization and passes a short-lived capability for the exact saved revision. The slot uses that capability to download source from Web. The source-signing key stays in the application servers. The guest receives only the saved files, command, and stdin. The Meet adapter is selected only whenMEET_PLAYGROUND_EXECUTION_PROVIDER=cloudflare. The Durable Object starts
with admission paused. The ingress and operator secrets are separate.
Production deployment, registry upload, secret installation and
admission changes each require operator authorization for the exact operation.
Set the values lower for an early deployment. The defaults are also the approved
ceilings. A duplicate request uses the original receipt and consumes no further
start allowance. A failed or ambiguous dispatch consumes its original allowance.
The active project limit protects shared rooms where two participants can ask to
run the same revision. The current Meet adapter handles Run, Stop and Console
results. File changes use the existing collaborative merge and revision-checked
Drive checkpoint. Preview and Learn retain their current provider. Qualify the
writeback and concurrent editor behavior before production activation. Verify
that the legacy runner pool has no eligible Playground capacity while Meet uses
Cloudflare and Learn still uses the legacy provider.
The operator calls
POST /control with the operator credential and
{"enabled":false}. The response records the new generation and the number of
active reservations. Admission then refuses new work. A draining response uses
HTTP 202. A claimed Drive writeback keeps the reservation active until the
coordinator receives the checkpoint confirmation, or until 60 seconds after the
job deadline. That grace covers one room checkpoint and its settlement calls;
after it the Run fails without a saved revision and releases its slot, user and
project, so a lost settlement cannot wedge capacity or block re-enabling. Stop fences new writes
and lets an in-flight export finish its bounded receipt wait. Confirm there are zero
active reservations and zero running instances before rolling back the
application route or rotating service secrets. The same control request is
idempotent. Re-enable only after a scoped approval and verified teardown.
Keep the result endpoint available for existing cloud run receipts while
switching the Meet adapter back to its previous provider.
An accepted reservation schedules finite work. The normal path makes one
assignment RPC, one source fetch with an eight-second timeout, one guest
start, one command, one finalization, and at most one file export. The slot journals its start before
fetching source. Interrupted execution is reconciled without launching the
command again. Cleanup makes at most three attempts per reservation. If
cleanup stays unconfirmed, the slot remains occupied for operator
reconciliation. Clients stop polling at the run deadline. Terminal jobs clear their deadline work. Result polling
uses only the journal. A new admission prunes up to 64 expired receipts.
The Node tests in apps/playground-execution/src pin these counts.
admission.test.mjs covers the ledger: statements per reservation, restart
deduplication, daily and rolling limits, the pause fence and the cleanup
budget. slot.test.mjs covers the slot: the normal path’s fetch, start,
command and storage writes, no replay after a restart in the started phase,
an unconfirmed teardown that keeps the slot occupied, and Stop or pause before
the guest starts. container-run.test.mjs covers the guest: egress disabled,
output bounds and confirmed destruction. Run them with
bun run --cwd apps/playground-execution test.
Code checks
POST /check returns compile-only diagnostics for unsaved room files in C,
C++ and Java. No program runs. The coordinator charges a separate check
budget in its own table: 60 checks per requester and 300 per room in a
rolling 10 minutes, and PLAYGROUND_MAX_CHECKS_PER_UTC_DAY (default and
ceiling 20,000) per UTC day. Every attempt is charged, including a retry
with the same request ID, and an ID already used by another person or room
is refused. The durable admission gate that pauses Runs pauses checks too,
and a pause also stops the warm guests of rooms with checks in the last 35
minutes; /control answers 202 until each confirms it stopped. The pause is a
durable fence: a check admitted before it is refused even if it reaches the
room’s checker afterwards.
Each room with recent checks has one PlaygroundChecker Durable Object and
one warm guest on the basic instance (1/4 vCPU, 1 GiB) with Internet
disabled. The guest stops after 90 seconds without checks, and an alarm
replaces it 30 minutes after it starts, even under continuous checks. The checker container class allows at most 50 live
guests. A cold room starts its guest and answers check_warming; the client
retries. Each person has at most one check in flight, and a newer check kills
the older one (check_superseded). A room runs at most two checks at once
(check_busy).
Every check is a separate exec with its own mount namespace, a fresh /tmp
workspace, PID and network namespaces, the dropped identity and Landlock, so
nothing a check writes survives it or reaches another check. The launcher
runs a fixed command, never one from the request: gcc or g++ with
-fsyntax-only, or javac -proc:none -implicit:none with a throwaway class
directory. Sources are passed as ./path so no file name is read as an
option or argument file. gcc and g++ get 4 seconds of wall and CPU time
and 512 MiB of address space; javac gets 8 seconds, because a cold JVM on the
smallest instance needs more, and a 192 MiB heap. The Worker abandons a check
4 seconds after the guest’s own limit, and the sandbox dies with its
launcher, so a killed or superseded check leaves no compiler running.
The Worker reports diagnostics only for request paths; a problem located in
any other file (a toolchain header or an absolute include) becomes the fixed
text “Problem in an included file” at the include site. A message located in
a request file can still quote a value the compiler derived from an included
file, and an include can test whether a path exists. The guest holds only the
public image and the request’s own files, so this reveals nothing a Run could
not read.
A warm room guest costs about $0.00000278 per live second plus the CPU of
its checks, about $0.012 per active room-hour at the published overage rates.
Launcher behaviour
The trusted launcher (container/run.py) runs as root only to mount a fresh
/tmp and build the sandbox. It first runs the same namespace, privilege drop
and Landlock chain around /bin/true; if the kernel or runtime cannot provide
any part of it, the Run fails as sandbox_failed before user code runs. The
program then runs as UID and GID 65534 with no capabilities, no new
privileges, an empty network namespace, its own PID namespace and writes limited to
/tmp.
After a normal exit the launcher exports the workspace as a complete
inventory. Files the program created that cannot be saved (binaries, names
outside the path rules, files over 256 KiB, non-UTF-8 text) are left out. If a
project file still exists but cannot be saved, or the workspace exceeds the
inventory or size limits, nothing is exported and the Run keeps its output.
An export problem never replaces the program’s output.
Usage and cost
The initial instance size is 0.5 vCPU, 4 GiB of memory, and an 8 GB disk. The command has a 30-second timeout within a 50-second job deadline. Output is capped at 64 KiB. Fresh instances are destroyed after execution, with a 15-second inactivity timeout as a fallback. Warm reuse requires independent hosted qualification before use. At published overage rates, the estimated Containers cost is$0.00001056 × live seconds + $0.000020 × active vCPU-seconds.
A 15-second idle tail costs about $0.0001584 per instance plus actual CPU.
Sleeping container compute is $0. The estimates exclude the Workers Paid
base charge, Workers and Durable Object requests, duration, SQLite storage,
and logs, and are before shared included allowances. The stress scenario
is an estimate. Provider shutdown latency can change billed usage. Record
real cold-start and shutdown times, journal operation counts, and actual
usage during qualification. Cloudflare Containers pricing · Workers and Durable Objects pricing