le-app/docs/le-app-database-migration/phase2b-stage3-namespace-activation

Phase 2B Stage 3 — namespace activation and inert acceptance

Status: prepared for review only, 2026-07-13. Do not execute the activation or rollback scripts from this uncommitted tree. Stage 3 does not authorize the systemd user manager, lingering, rootless Docker, Codex, source/data/secret copying, application startup, service enablement, or production changes.

Entry points

tests/50-activate-stage3.sh is the sole activation entry point. tests/51-rollback-stage3.sh is the separate Stage 3-only rollback/recovery entry point. Both require root, a clean committed worktree, a validated inherited or newly acquired exclusive lock on the existing root-owned runtime directory, exact Stage 1/2/3 manifests, the pinned Stage 1 state and all 16 installed files plus all 20 directory records, and unchanged Stage 1/2 trees since Stage 2 commit 909f0a8200c79efc205d92618be47e06ebf764fc.

Successful Stage 2 execution is an external operator prerequisite confirmed for TITANSERVER. Git provenance can prove which Stage 2 code is trusted; it cannot by itself prove that the prior operator run occurred.

Activation behavior

Before any mutation, activation requires:

• Stage 1 source df6b53e63916880bec2c77219b04b010b8b453f1, artifact-manifest digest 5aaa91b1e6846eda033961dcee392b9657476ad6fd149420979e9309b484445f, exact COMPLETE state, and 16 matching installed files;
• Stage 2 commit and package-manifest provenance, plus a clean committed Stage 3 package;
• installed policy digest 4289b705dbd786281daccb6d5aa660c27b83441f1a34d0cd18590666124be386 and a root nft --check that does not load it;
• current public/NAT IPv4 74.67.173.56, the committed 41-network rootful Docker configuration, the expected LAN/WireGuard identities, and an already-enabled host IPv4 forward setting that Stage 3 never changes; every Docker read scrubs context/TLS selector variables and uses an explicit /run/docker.sock host plus the committed empty client configuration, so it cannot inherit another context or read operator credentials;
• fresh immediate fingerprints of nonproject nftables, stable nonproject host network state, rootful Docker networks/runtime/ports, host listeners, resolver metadata, and public IPv4; and
• no marker, namespace, project veth/table, user manager, linger, UID-1200 process, or rootless socket.

Immediately before publication and start, the script revalidates the loaded manager state: systemd-analyze verify; exact systemctl cat fragment/drop-in surfaces for both units; exact namespace fragment path with no drop-ins; current manager cache; no pending job; exact single ExecStart/ExecStop; oneshot/RemainAfterExit; static inactive state; the approval-marker condition; Before=user@1200.service; and the reciprocal user-manager Requires/After, namespace path, fragment, and exact two-drop-in set.

The script renders and fsyncs a root-owned mode 0600, single-link version-2 record before atomically renaming it into the approval-marker path and syncing the parent directory. No hard-link publication window exists, and an interrupted write cannot replace the prior authoritative record. Parsing requires the exact canonical one-delimiter, final-newline-complete byte representation, so trailing fields, truncated final writes, and embedded NUL bytes are rejected. The immutable fields include an activation UUID, machine-ID digest, approved Quad9 pair, Stage 1 source and artifact digest, Stage 2 verification commit, Stage 3 activation commit, UTC timestamp, hostname, boot ID, public IPv4, installed-policy digest, and baseline fingerprints. It starts le-app-codex-netns.service, then atomically replaces PREPARED with an ACTIVE record bound to the service invocation, namespace inode, host-veth ifindex/iflink/MAC, both nft table handles, independent counter-normalized table digests, and the combined project-policy digest.

Post-start validation requires a non-symlink nsfs namespace containing exactly loopback and the namespace veth, a typed host inventory with no host-resident peer, reciprocal host/peer ifindex/iflink identity, exact /30 addresses/routes, no namespace IPv6 address/route, IPv6-disable sysctls, an empty namespace, no listener/published port, exactly the intended two nftables tables, three sets, two filter chains with 4 and 10 rules, and one postrouting masquerade rule. Rule expressions and order are checked against strict nftables JSON envelopes rather than presentation-dependent text; IPv4 set elements receive an additional text cross-check. Each provenance capture derives both independent table hashes and the combined policy hash from the same two raw canonical tables, brackets them with typed handle inventories, then requires a second complete capture to match. This stable exact snapshot is repeated immediately before publishing ACTIVE, after network probes, and as the final activation policy gate. It rejects extra nftables object types, DNAT, redirect, and prerouting publication. Counter-delta tests—not connection failure alone—prove the private/LAN/WireGuard/all-39-Docker-gateway/metadata/reserved/public-IP/SMTP/nonapproved-DNS/other-port denials. Both Quad9 servers must answer classic DNS over UDP and TCP. A Quad9-resolved public address must work over TCP 80 and 443 using curl --resolve with every uppercase and lowercase proxy variable removed.

The installed Stage 1 10-inert-containment.sh is executed unchanged through tests/52-run-stage1-inert.sh. That reviewed wrapper reproduces the installed drop-in's service containment settings, including the resolver bind and inaccessible paths, and points the transient directly at the already-active namespace instead of duplicating the user-manager unit's [Unit] dependency declarations. It removes every uppercase/lowercase proxy variable from the transient process environment. The old test's missing /srv/le-app-codex/tmp write probe is narrowly redirected by two read-only bound wrappers to the existing approved build-artifacts directory and is identity-checked and removed on every success, error, and handled-signal path.

The existing test can exercise the systemd mount, device, UID, cgroup, and namespace properties only as a transient unit. Stage 3 therefore treats one activation-UUID-named systemd-run --service-type=exec --wait --collect service—and only its short-lived UID-1200 test process tree—as the narrow acceptance-test exception to “start only the namespace service” and the otherwise-inert UID. It is never installed, enabled, or persistent; its name must be unclaimed, and the committed command statically fixes every containment property. At runtime an additional random token binds the exact transient fragment, user/group, namespace path, activation-specific description, environment, single ExecStart, and captured invocation to this launch. HUP/INT/TERM are deferred only across client PID/start-time/ownership capture and replayed immediately afterward; cleanup will signal that client only while its /proc start time still matches. Success requires captured ownership, while error/signal cleanup stops only that token-bound identity and requires three consecutive successful not-found observations before accepting collection. It revalidates the activation process's inherited exclusive lock, has a 120-second runtime ceiling, and must leave no UID-1200 or codex-named process before activation can succeed or rollback can begin. No user manager is started.

Every unrelated baseline is compared after start and after testing. Every same-boot automatic or explicit runtime cleanup compares against the recorded pre-activation baselines before mutation and again after cleanup, in addition to a fresh immediate rollback before/after comparison. Changed-boot recovery also compares the recorded baselines before and after and normally fails closed if reboot changed a runtime identity. Any network, policy, containment, provenance, or baseline failure exits nonzero without starting the user manager.

DNS boundary

The nftables policy restricts classic DNS on UDP/TCP port 53 to 9.9.9.9 and 149.112.112.112, and the contained resolver file contains only those two servers. Because public TCP 443 is intentionally allowed, this L3/L4 policy cannot distinguish ordinary HTTPS from DNS-over-HTTPS to another public provider. Stage 3 therefore proves the enforceable classic-DNS boundary; it does not claim application-layer DoH prevention.

Explicit exclusions

Activation never enables a unit, starts user@1200.service, enables lingering, starts rootless Docker or Codex, authenticates Codex, invokes Docker mutation APIs, loads an arbitrary nftables file, creates a published port, accesses container environments or secrets, clones/copies application material, changes host sysctls, or repairs/restarts production. Rootful Docker access is read-only and explicitly uses unix:///run/docker.sock; safe fingerprints exclude container environments.

During review, run only tests/53-stage3-static-tests.sh and tests/54-stage3-failure-path-tests.sh. The latter uses isolated /tmp fixtures and source guards; it never invokes the activation, rollback, or inert operational entry points, nor any systemd, networking, nftables, or Docker runtime command.

Future operator invocations

Only after this package has been reviewed, committed, and checked out as a clean worktree on TITANSERVER, run from the repository root:

sudo /bin/bash docs/le-app-database-migration/phase2b-stage3-namespace-activation/tests/50-activate-stage3.sh

If activation fails or Stage 3 must be removed, use the separately reviewed retryable rollback entry point:

sudo /bin/bash docs/le-app-database-migration/phase2b-stage3-namespace-activation/tests/51-rollback-stage3.sh