le-app/docs/le-app-database-migration/host-runtime-plan.md

# Milestone 0 host runtime and security plan

Status: corrected planning checkpoint, 2026-07-12. Nothing in this document authorizes execution. Phase 1 created only the inert `le_app_codex` identity and directory skeleton. Package installation, systemd changes, namespace creation, firewall changes, user-manager startup, Docker startup, Codex authentication, repository/source copying, and production changes remain separately approval-gated.

This document supersedes the earlier runtime proposal. In particular, the project will not use a system service running `dockerd` with `User=le_app_codex`, will not enable lingering, and will not install the unmodified AUR `docker-rootless-extras` package.

## Fixed production and trust boundaries

- Production remains rooted at `/var/www/lasereverything.net.db`.
- The complete archive remains root-only at `/srv/directus-archive/20260710-175408` and is never moved, mounted, linked, copied, or exposed to Codex.
- The contained nonproduction root is `/srv/le-app-codex`.
- Transitional `/srv/codex-work`, `/srv/codex-migration`, and `/srv/codex-secrets` resources remain outside the contained environment until an independently approved copy-and-verify stage.
- The rootful Docker and containerd sockets and data directories are inaccessible to the contained environment.
- The only accepted project daemon endpoint is `unix:///run/user/1200/docker.sock`.
- No Docker TCP/SSH endpoint, production context, shared Docker group, socket forwarding, or fallback to `/run/docker.sock` or `/var/run/docker.sock` is accepted.

## Confirmed Phase 1 state

Phase 1 created:

- user/group `le_app_codex`, UID/GID `1200`, primary group only;
- locked password, `/usr/bin/nologin`, and no supplementary groups;
- subordinate UID/GID allocation `165536:65536`;
- inert workspace skeleton under `/srv/le-app-codex` with root-owned boundaries and dedicated-user mutable children.

Phase 1 did not install packages, start `user@1200.service`, start a daemon or container, create a network namespace, change nftables, authenticate Codex, copy a repository or migration source, or touch production. Lingering was not enabled and remains prohibited. Do not rerun any Phase 1 artifact.

## Correct runtime lifecycle

Docker is a genuine per-user service defined below `/srv/le-app-codex/home/.config/systemd/user`. Its lifetime is managed by the systemd user manager for UID 1200.

Root enforces containment around the complete user manager with instance-specific configuration for:

```text
/etc/systemd/system/user@1200.service.d/50-le-app-codex-containment.conf
/etc/systemd/system/user-1200.slice.d/50-le-app-codex-resources.conf
```

The user manager depends on the root-created network namespace and enters that namespace before any user unit starts. Consequently Codex, RootlessKit, `dockerd`, containerd, build processes, test jobs, and containers inherit the same root-enforced filesystem, network, device, and cgroup boundary.

Root starts and stops `user@1200.service` explicitly after inert acceptance tests pass. Lingering is prohibited and must remain disabled. The Docker user unit may be enabled for the user manager's controlled lifetime, but must not cause the manager to start independently.

A system unit with `User=le_app_codex` must not run `dockerd`. That model bypasses the supported per-user lifecycle and does not provide the required containment of Codex and every sibling user process.

## Package decision gate and pinned launcher

The last isolated preview of:

```text
pacman -Syu --needed rootlesskit slirp4netns openai-codex
```

was refreshed after removal of unused NoMachine and is now a 220-package full host upgrade (215 upgrades and 5 new packages) affecting both installed kernels and OpenSSH. It requires reboot planning, service-health checks, `.pacnew` review, explicit Snapper snapshot verification, and separate operator approval. The checksummed baseline is in `phase2a-host-maintenance/approved-transaction/`. A partial Arch upgrade is not acceptable.

The package paths are:

1. schedule and approve the complete refreshed host upgrade; or
2. investigate clean-chroot, checksum-pinned packages compatible with the current system, without partially upgrading repository packages.

The operator provisionally selected Path A on 2026-07-12 as the preferred Phase 2 package path because it is the shortest safe route consistent with Arch's complete-upgrade model. This records planning direction only: the transaction, maintenance window, package execution, and reboot remain separately approval-gated. The uncommitted review package is under `docs/le-app-database-migration/phase2a-host-maintenance/`.

Do not install the unmodified AUR `docker-rootless-extras` package. Its install hook reloads global sysctls, which exceeds this project's boundary. Do not curl an installer into a shell.

Install only a reviewed Moby `contrib/dockerd-rootless.sh` matching the Docker engine actually installed after the chosen package path. At the current checkpoint Docker is `29.6.1`; the candidate is:

```text
Moby tag:    docker-v29.6.1
Source:      contrib/dockerd-rootless.sh
Target:      /usr/local/libexec/dockerd-rootless-29.6.1.sh
SHA-256:     904c9b9e35f6927c0a5e65afb4d35b6bc9eb1278c878044501281fc728c9be46
Owner/mode:  root:root 0755
```

Revalidate the installed Docker version, upstream tag, source content, checksum, and launcher compatibility immediately before installation. If Docker changes, this filename and checksum are invalid until a matching launcher is reviewed. A clean-chroot package containing only the pinned launcher is acceptable; it must have no global sysctl hook, setup tool, service enablement, or post-install execution.

## User-manager filesystem and device containment

The proposed, not deployed, containment contract is:

```ini
# /etc/systemd/system/user@1200.service.d/50-le-app-codex-containment.conf
[Unit]
Requires=le-app-codex-netns.service
After=le-app-codex-netns.service

[Service]
NetworkNamespacePath=/run/netns/le-app-codex
ProtectSystem=strict
PrivateTmp=yes

TemporaryFileSystem=/srv:ro
BindPaths=/srv/le-app-codex:/srv/le-app-codex
TemporaryFileSystem=/var:ro
TemporaryFileSystem=/mnt:ro
TemporaryFileSystem=/media:ro
TemporaryFileSystem=/opt:ro

InaccessiblePaths=/var/www
InaccessiblePaths=/root
InaccessiblePaths=/home
InaccessiblePaths=/boot
InaccessiblePaths=/efi
InaccessiblePaths=/run/docker.sock
InaccessiblePaths=/var/run/docker.sock
InaccessiblePaths=/run/containerd/containerd.sock
InaccessiblePaths=/run/podman
InaccessiblePaths=/run/dbus/system_bus_socket
InaccessiblePaths=/run/systemd/private
InaccessiblePaths=/var/lib/docker
InaccessiblePaths=/var/lib/containerd
InaccessiblePaths=/var/lib/containers
InaccessiblePaths=/etc/docker
InaccessiblePaths=/etc/containers
InaccessiblePaths=/etc/ssh
InaccessiblePaths=/etc/NetworkManager/system-connections
InaccessiblePaths=/etc/wireguard

DevicePolicy=closed
DeviceAllow=/dev/fuse rw
DeviceAllow=/dev/net/tun rw
```

Expected visibility is read-only runtime content under `/usr`, `/bin`, and `/lib*`; selected nonsensitive read-only `/etc`; normal `/proc` pending acceptance tests; read-only `/sys`; a closed device policy with FUSE/TUN; `/run/user/1200`; a private `/srv` exposing only `/srv/le-app-codex`; empty private `/var`, `/mnt`, `/media`, and `/opt`; and private `/tmp`.

The following are intentionally deferred because they may break subordinate-ID mapping, user-manager behavior, cgroup delegation, or container creation:

```ini
NoNewPrivileges=yes
RestrictSUIDSGID=yes
ProtectKernelTunables=yes
ProtectControlGroups=yes
ProcSubset=pid
```

`ProtectProc=invisible` and hostname isolation remain acceptance-test candidates. They may be added only after the user manager, RootlessKit, Docker, Codex, and test containers pass with them. The final contract must prevent unrelated host-process inspection without breaking the supported runtime.

## Resource boundary

The proposed, not deployed, aggregate boundary for the user manager and everything below it is:

```ini
# /etc/systemd/system/user-1200.slice.d/50-le-app-codex-resources.conf
[Slice]
CPUQuota=600%
MemoryHigh=48G
MemoryMax=64G
MemorySwapMax=16G
TasksMax=4096
```

These limits cover Codex, RootlessKit, `dockerd`, containerd, builds, jobs, and all containers. Validate generated properties and cgroup delegation before relying on the limits.

Disk-growth containment remains unresolved. Btrfs quotas remain intentionally disabled. The repaired `/.snapshots` Btrfs subvolume and successful Snapper snapshots 1, 2, and 3 establish snapshot operation, not quota accounting. Do not enable quotas, run `snapper setup-quota`, trigger a rescan, or impose the earlier proposed 250 GiB qgroup limit incidentally. Filesystem visibility constrains where the account may write but does not cap growth within `/srv/le-app-codex`. The operator must separately choose and approve a disk-quota policy.

Soft planning budgets remain useful for monitoring: 80 GiB container runtime, 25 GiB PostgreSQL, 15 GiB media, 10 GiB legacy uploads, 35 GiB BGBye models, 10 GiB temporary data, 5 GiB browser artifacts, 10 GiB migration reports, and 10 GiB checkout/home metadata. Startup should fail when host/project free-space thresholds are unsafe, but soft checks do not replace a hard quota.

## Rootless Docker user service

The proposed, not deployed, user unit is:

```ini
# /srv/le-app-codex/home/.config/systemd/user/docker.service
[Unit]
Description=Laser Everything rootless Docker
Documentation=https://docs.docker.com/engine/security/rootless/

[Service]
Type=notify
NotifyAccess=all
KillMode=mixed
Environment=HOME=/srv/le-app-codex/home
Environment=XDG_RUNTIME_DIR=/run/user/1200
Environment=DOCKER_HOST=unix:///run/user/1200/docker.sock
Environment=DOCKERD_ROOTLESS_ROOTLESSKIT_STATE_DIR=/run/user/1200/dockerd-rootless
Environment=DOCKERD_ROOTLESS_ROOTLESSKIT_NET=slirp4netns
Environment=DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=builtin
Environment=DOCKERD_ROOTLESS_ROOTLESSKIT_MTU=65520
Environment=DOCKERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SANDBOX=true
Environment=DOCKERD_ROOTLESS_ROOTLESSKIT_SLIRP4NETNS_SECCOMP=true
Environment=DOCKERD_ROOTLESS_ROOTLESSKIT_DISABLE_HOST_LOOPBACK=true
Environment=DOCKERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS=false
ExecStart=/usr/local/libexec/dockerd-rootless-29.6.1.sh
ExecReload=/usr/bin/kill -s HUP $MAINPID
Restart=always
RestartSec=2
TimeoutStartSec=0
TimeoutStopSec=120
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes

[Install]
WantedBy=default.target
```

The `ExecStart` path must be updated if the installed Docker release changes. Proposed daemon configuration at `/srv/le-app-codex/home/.config/docker/daemon.json`:

```json
{
  "data-root": "/srv/le-app-codex/container-runtime/docker",
  "storage-driver": "fuse-overlayfs",
  "live-restore": false,
  "userland-proxy": false,
  "ipv6": false,
  "ip6tables": false,
  "log-driver": "local",
  "log-opts": {
    "max-size": "20m",
    "max-file": "5"
  }
}
```

Docker client configuration lives only below `/srv/le-app-codex/home/.docker`. The named `le-app-codex` context must resolve exactly to `unix:///run/user/1200/docker.sock`. Wrappers reject the `default` context, rootful sockets, TCP/SSH endpoints, and unknown contexts.

## IPv4-only network containment

Root creates and owns the persistent namespace `/run/netns/le-app-codex`. The complete `user@1200.service` enters it through `NetworkNamespacePath`; the user cannot replace the namespace or firewall policy.

Current read-only findings at this checkpoint:

- approved DNS resolver IPv4 addresses remain unresolved;
- the host resolver currently includes `192.168.10.1`, but observation is not approval;
- no persistent `le-app-codex` network namespace currently exists;
- no project-specific nftables rules are deployed;
- the host has numerous existing Docker IPv4 and IPv6 bridge ranges.

All host addresses, routes, LAN/VPN/WireGuard/management networks, public/NAT addresses, Docker bridges, and resolver endpoints must be dynamically inventoried immediately before drafting the final rules. Future policy must deny every observed Docker IPv4 and IPv6 bridge range and must not rely only on a stale hard-coded list.

The namespace is IPv4-only: assign no IPv6 address or route and reject IPv6 on the project veth. Do not change host-global IPv6 or sysctls.

The nftables design is default-deny and must cover namespace egress and host-to-namespace ingress in `FORWARD`, veth-gateway traffic in host `INPUT`, established/related replies, UDP and TCP DNS, and RootlessKit port-publication attempts. Initially no inbound forwarding to the namespace is allowed; a container requesting `-p` must remain unreachable from the host, LAN, VPN, and public networks.

Permit only:

- established/related reply traffic;
- UDP/TCP DNS to explicitly approved resolver IPv4 addresses;
- TCP 80/443 to public destinations after exclusions.

Explicitly reject:

- namespace and host loopback ranges;
- host veth gateway except a separately approved DNS/proxy endpoint;
- all host addresses, LAN, management, VPN, and WireGuard addresses/subnets;
- every current and later-inventoried Docker/Podman/container bridge range;
- RFC1918 private ranges, carrier-grade NAT, link-local, metadata including `169.254.169.254`, multicast, benchmark, documentation, reserved, and other non-public ranges;
- every TITANSERVER public/NAT address supplied by the operator;
- TCP 25, 465, and 587;
- all IPv6 traffic.

Known addresses requiring explicit coverage include `192.168.10.151`, `10.98.0.2`, and `2603:7080:7500:1279:75aa:d64d:4cf0:8f10`; these are not a complete inventory.

Required operator inputs are approved DNS resolver IPv4 addresses, the complete host/public/NAT address set, every LAN/VPN/WireGuard/bridge/management subnet, and any approved host DNS/proxy endpoint. Do not finalize or deploy nftables without those inputs and a fresh read-only ruleset inventory. Forge SSH is outside the sandbox; no TCP/22 or LAN exception is proposed.

## Acceptance gates and order of operations

Every stage stops for review. No real migration data, nonproduction credentials, or Codex authentication enters the boundary until inert tests pass.

1. Resolve and approve the package path; install only the approved packages in a separately controlled stage.
2. If a full host upgrade was selected, reboot and complete production health verification.
3. Install only the version-matched, checksum-pinned Moby launcher.
4. Install the resource slice and user-manager containment files without starting `user@1200.service`.
5. Create the namespace without Docker; inventory generated properties, mounts, routes, addresses, and namespace identity.
6. Install the reviewed nftables fragment without Docker and test positive DNS/HTTPS plus every negative route class.
7. Run inert one-shot processes through the proposed user-manager namespace to validate filesystem, `/proc`, devices, cgroups, symlink/bind-mount resistance, and network denial.
8. Only after the inert gates pass, root explicitly starts `user@1200.service`.
9. Install/start the Docker user service and validate the exact socket, rootless security, data root, storage driver, logs, resource accounting, and port-publication denial.
10. Run a disposable container through the same positive and negative tests, then tear it down.
11. Only after runtime acceptance, choose and perform Codex authentication.
12. Repository cloning, scrubbed-source copying, new secret generation, and Milestone 0 implementation are later separately reviewed stages.

Positive tests must prove that the user manager, Codex process, daemon, and disposable container can write only approved paths below `/srv/le-app-codex`, use only `/run/user/1200/docker.sock`, store Docker data only below `container-runtime/docker`, resolve through approved DNS, and reach approved public HTTPS endpoints.

Negative tests must prove they cannot list/read `/var/www`, `/root`, real `/home`, `/srv/directus-archive`, or transitional `/srv/codex-*`; open rootful runtime sockets or data; reach host/LAN/WireGuard/private/link-local/metadata/bridge/reserved destinations; use direct SMTP or IPv6; publish a port to the host/LAN; bypass controls via symlinks or bind mounts; inspect unrelated host processes; or escape resource limits.

Failure of any negative test is a hard stop, not a reason to weaken containment ad hoc.

## Codex authentication decision

Authentication remains unresolved. The choices are ChatGPT device authentication or a dedicated API key created specifically for `le_app_codex`. Do not copy Codex configuration, `auth.json`, tokens, SSH keys, Docker configuration, or shell state from `sol6_vi`.

Use `CODEX_HOME=/srv/le-app-codex/home/.codex`. Authentication may occur only after the contained user manager, filesystem namespace, network namespace, firewall, resource slice, Codex binary, and acceptance tests are approved. Any `auth.json` must be a nonsymlink regular file owned by `1200:1200`, mode `0600`, and treated as a password.

## Repository and migration-source admission

After runtime acceptance and separate approval:

- create a clean checkout of the reviewed remote `migration/le-app-database` branch at `/srv/le-app-codex/lasereverything.net.db`;
- do not copy `sol6_vi` SSH credentials; the current network design grants the contained environment no Forge SSH exception;
- copy only the credential-scrubbed authoritative source into `database-source-readonly` through an administrator-run, checksum-verified process;
- never use the root archive as migration input;
- generate new nonproduction credentials below `nonproduction-secrets`; never copy production or root-only administration credentials.

Transitional checkouts, databases, secrets, volumes, and source paths remain intact for at least 30 days and two successful rehearsals. Every later deletion requires separate approval.

## Mail and production promotion boundaries

Mailpit is nonproduction-only, internal to Compose, has no relay/forwarding, and receives only synthetic or rewritten recipients. Namespace policy rejects direct SMTP on TCP 25/465/587. Production continues to use the existing SMTP service through production-only configuration.

Promotion is commit/release based:

```text
/srv/le-app-codex/lasereverything.net.db
  -> reviewed pushed commit and immutable image/release manifest
  -> fixture -> snapshot -> rehearsal -> staging
  -> clean checkout or verified release artifact
  -> /var/www/lasereverything.net.db
```

Never recursively copy or `rsync` the nonproduction workspace into production. Codex stops for explicit approval before changing `/var/www`, public proxy/DNS/TLS, production SMTP, production databases, Directus, or transitional/archive resources.

## Unresolved decisions checklist

- Phase 2 package path: Path A is provisionally preferred; package execution and reboot are not approved.
- Approved DNS resolver IPv4 addresses; `192.168.10.1` is currently observed but not approved.
- Complete current host, public/NAT, LAN, management, VPN, WireGuard, Docker bridge, and other container-network inventory.
- Codex authentication: device authentication or dedicated API key.
- Disk-growth enforcement: Btrfs quota policy or another separately approved mechanism.
- Whether deferred process/hostname hardening options pass acceptance tests.
- Any individually approved future host-to-namespace published endpoint; none is currently allowed.

Until these decisions are resolved, do not proceed beyond documentation and read-only verification.