define canonical migration deployment paths
This commit is contained in:
parent
ed1c84eb76
commit
27fd4f4ae3
4 changed files with 238 additions and 21 deletions
|
|
@ -39,13 +39,39 @@ Proposed fixed identity:
|
|||
- shell/home: `/bin/bash`, `/home/le_payload_dev`;
|
||||
- no supplementary production groups, no `docker` group, no sudo/wheel membership;
|
||||
- subordinate UID and GID range: `165536:65536`, non-overlapping with the observed `sol6_ii:100000:65536` allocation;
|
||||
- dedicated project root: Btrfs subvolume `/srv/le-payload-dev`, owner `1200:1200`, mode `0700`;
|
||||
- workspace: `/srv/le-payload-dev/workspace/lasereverything.net.db`;
|
||||
- dedicated project root: Btrfs subvolume `/srv/le-payload-dev`, owner `root:le_payload_dev`, mode `0750`;
|
||||
- sole checkout: `/srv/le-payload-dev/lasereverything.net.db`;
|
||||
- authoritative scrubbed input: `/srv/le-payload-dev/source`, root-owned and read-only to the development group;
|
||||
- nonproduction credentials only: `/srv/le-payload-dev/secrets`;
|
||||
- daemon data root: `/srv/le-payload-dev/runtime/docker`;
|
||||
- runtime socket: `/run/user/1200/docker.sock`;
|
||||
- Docker client/config: `/home/le_payload_dev/.docker` and `/home/le_payload_dev/.config/docker`;
|
||||
- Compose project prefix: `le_payload_dev`, enforced by wrapper preflight.
|
||||
|
||||
Canonical nonproduction tree:
|
||||
|
||||
```text
|
||||
/srv/le-payload-dev/
|
||||
├── lasereverything.net.db/ # sole persistent nonproduction Git checkout
|
||||
├── source/ # root-owned scrubbed authoritative input
|
||||
├── secrets/ # nonproduction credentials only
|
||||
├── runtime/ # rootless Docker data/configuration support
|
||||
├── data/ # disposable PostgreSQL, media, clones, caches, temp
|
||||
└── artifacts/ # migration reports, logs, browser evidence
|
||||
```
|
||||
|
||||
Ownership and permissions:
|
||||
|
||||
| Path | Owner:group | Mode/policy |
|
||||
|---|---|---|
|
||||
| `/srv/le-payload-dev` | `root:le_payload_dev` | `0750`; prevents unrelated users and prevents project-root restructuring by the development account |
|
||||
| `lasereverything.net.db` | `le_payload_dev:le_payload_dev` | directories `0750`, normal Git files subject to repository modes; sole checkout |
|
||||
| `source` | `root:le_payload_dev` | directories `0550`, files `0440`; account and daemon may read but cannot alter; no writable symlink target |
|
||||
| `secrets` | `root:le_payload_dev` | directory `0750`, provisioned files `0640`; nonproduction values only; optional `generated/` is `0700 le_payload_dev` for ephemeral fixture secrets |
|
||||
| `runtime` | `le_payload_dev:le_payload_dev` | `0700`; rootless Docker layers/configuration only |
|
||||
| `data` | `le_payload_dev:le_payload_dev` | `0700`; child ownership may use rootless subordinate IDs |
|
||||
| `artifacts` | `le_payload_dev:le_payload_dev` | `0700`; TTL cleanup; snapshot artifacts treated as sensitive |
|
||||
|
||||
Only the dedicated account receives:
|
||||
|
||||
- read/write access to its workspace and runtime/storage subtree;
|
||||
|
|
@ -82,23 +108,32 @@ sudo btrfs subvolume create /srv/le-payload-dev
|
|||
sudo chown 1200:1200 /srv/le-payload-dev
|
||||
sudo chmod 0700 /srv/le-payload-dev
|
||||
sudo -u le_payload_dev mkdir -p \
|
||||
/srv/le-payload-dev/workspace \
|
||||
/srv/le-payload-dev/lasereverything.net.db \
|
||||
/srv/le-payload-dev/source \
|
||||
/srv/le-payload-dev/secrets/generated \
|
||||
/srv/le-payload-dev/runtime/docker \
|
||||
/srv/le-payload-dev/data/{postgres,media,legacy-uploads,bgbye-models,tmp} \
|
||||
/srv/le-payload-dev/artifacts/{browser,migration-reports} \
|
||||
/srv/le-payload-dev/secrets
|
||||
/srv/le-payload-dev/artifacts/{browser,migration-reports}
|
||||
|
||||
# Root owns the boundary and authoritative source; development owns mutable areas.
|
||||
sudo chown root:le_payload_dev /srv/le-payload-dev
|
||||
sudo chmod 0750 /srv/le-payload-dev
|
||||
sudo chown -R le_payload_dev:le_payload_dev \
|
||||
/srv/le-payload-dev/lasereverything.net.db \
|
||||
/srv/le-payload-dev/runtime /srv/le-payload-dev/data /srv/le-payload-dev/artifacts \
|
||||
/srv/le-payload-dev/secrets/generated
|
||||
sudo chmod 0700 /srv/le-payload-dev/runtime /srv/le-payload-dev/data \
|
||||
/srv/le-payload-dev/artifacts /srv/le-payload-dev/secrets/generated
|
||||
sudo chown root:le_payload_dev /srv/le-payload-dev/source /srv/le-payload-dev/secrets
|
||||
sudo chmod 0550 /srv/le-payload-dev/source
|
||||
sudo chmod 0750 /srv/le-payload-dev/secrets
|
||||
|
||||
# 5. Apply a 250 GiB ceiling to the project subvolume.
|
||||
sudo btrfs quota enable /srv
|
||||
sudo btrfs qgroup limit 250G /srv/le-payload-dev
|
||||
|
||||
# 6. Grant only source traversal/read and one migration secret file.
|
||||
# Recheck the resolved `current` target before applying ACLs.
|
||||
readlink -f /srv/codex-migration/current
|
||||
sudo setfacl -m u:le_payload_dev:--x /srv/codex-migration
|
||||
sudo setfacl -R -m u:le_payload_dev:r-X /srv/codex-migration/source-20260710-175408
|
||||
sudo setfacl -m u:le_payload_dev:--x /srv/codex-secrets
|
||||
sudo setfacl -m u:le_payload_dev:r-- /srv/codex-secrets/le-payload-migration.env
|
||||
# 6. Transitional source access is used only by the administrator-run copy and
|
||||
# verification procedure below. The final account needs no ACL on /srv/codex-*.
|
||||
|
||||
# 7. Remediate production secret readability separately.
|
||||
# Root-owned Compose can still read mode 0600 files. Confirm the production
|
||||
|
|
@ -141,10 +176,10 @@ test ! -x /var/lib/docker
|
|||
test ! -x /var/lib/mysql
|
||||
test ! -r /var/run/docker.sock
|
||||
test ! -x /srv/directus-archive
|
||||
test -r /srv/codex-migration/source-20260710-175408/README.txt
|
||||
test ! -w /srv/codex-migration/source-20260710-175408/README.txt
|
||||
test -r /srv/codex-secrets/le-payload-migration.env
|
||||
test ! -w /srv/codex-secrets/le-payload-migration.env
|
||||
test -r /srv/le-payload-dev/source/README.txt
|
||||
test ! -w /srv/le-payload-dev/source/README.txt
|
||||
test -r /srv/le-payload-dev/secrets/migration.env
|
||||
test ! -w /srv/le-payload-dev/secrets/migration.env
|
||||
DOCKER_HOST=unix:///run/user/1200/docker.sock docker info --format '{{json .SecurityOptions}}'
|
||||
```
|
||||
|
||||
|
|
@ -158,6 +193,7 @@ Every stack/job wrapper will fail before Compose evaluation if:
|
|||
- Docker endpoint is not exactly the approved rootless Unix socket;
|
||||
- Compose project name lacks the fixed nonproduction prefix;
|
||||
- any resolved bind source is `/var/www`, `/var/lib/docker`, `/var/lib/mysql`, `/srv/directus-archive`, `/var/run`, `/run/docker.sock`, or a descendant;
|
||||
- the Git checkout is not exactly `/srv/le-payload-dev/lasereverything.net.db` in nonproduction or `/var/www/lasereverything.net.db` in production;
|
||||
- any non-finite application service receives `.migration.env`, `.migration-source`, a MariaDB source URL, or migration credentials;
|
||||
- destination database host/port is not the approved nonproduction target/service;
|
||||
- production-named secret scopes, SMTP hosts, OAuth variables, Ko-fi secrets, or webhook credentials appear in fixture/snapshot/rehearsal rendered Compose;
|
||||
|
|
@ -175,6 +211,8 @@ Tests render each Compose mode and inspect mounts, secrets, environment variable
|
|||
- Reset/purge operates only on labeled nonproduction destinations. It does not follow source symlinks and refuses all external/bind volumes.
|
||||
- `/srv/directus-archive` remains inaccessible and is never mentioned as a mount source in executable Compose configuration.
|
||||
|
||||
The complete root-only archive remains permanently outside the development boundary at `/srv/directus-archive/20260710-175408`. It is never moved, mounted, linked, or copied into `/srv/le-payload-dev`.
|
||||
|
||||
## Nonproduction storage plan
|
||||
|
||||
The host currently has approximately 650 GiB available on the Btrfs filesystem containing `/srv` and `/home`. Proposed aggregate project quota: 250 GiB.
|
||||
|
|
@ -193,6 +231,19 @@ The host currently has approximately 650 GiB available on the Btrfs filesystem c
|
|||
|
||||
The listed soft budgets total 200 GiB. The 250 GiB hard qgroup ceiling reserves 50 GiB emergency headroom. Startup refuses when host free space is below 100 GiB, project usage exceeds 200 GiB, or required free space for a rehearsal cannot be reserved. The production filesystem is not used for spillover.
|
||||
|
||||
## Resources intentionally outside `/srv/le-payload-dev`
|
||||
|
||||
| Resource | Permanent location | Why it remains outside |
|
||||
|---|---|---|
|
||||
| Complete untouched Directus archive | `/srv/directus-archive/20260710-175408` | Root-only disaster-recovery evidence; must not enter the Codex trust boundary |
|
||||
| Production checkout/deployment root | `/var/www/lasereverything.net.db` | Canonical production policy and existing shared-server operations boundary |
|
||||
| Production Docker image/layer storage | Host runtime-managed location | Operational daemon state, declared by the canonical production deployment but not Git content |
|
||||
| Production PostgreSQL/media storage | Approved production volume/storage paths | Persistent production data requires independent ownership, capacity, and backup controls |
|
||||
| Shared public reverse-proxy configuration/certificates | Existing TITANSERVER ingress paths | Server-wide infrastructure serves multiple applications and is never container-controlled |
|
||||
| Existing self-hosted SMTP infrastructure | Existing mail-server deployment | Production email remains independent; Mailpit is nonproduction-only |
|
||||
|
||||
Transitional `/srv/codex-*` paths remain temporarily outside during the rollback period only. They are not permanent alternate project roots and become cleanup candidates solely through the explicit process below.
|
||||
|
||||
## Production ingress boundary
|
||||
|
||||
Production will use an **internal application-gateway container behind TITANSERVER's existing shared host reverse proxy**.
|
||||
|
|
@ -206,6 +257,156 @@ Production will use an **internal application-gateway container behind TITANSERV
|
|||
|
||||
Local development uses its own gateway/TLS because it does not share production ingress. This design preserves server-wide ownership while keeping application routing rules testable and versioned.
|
||||
|
||||
## Transitional resource inventory
|
||||
|
||||
Metadata-only audit results at the planning checkpoint:
|
||||
|
||||
| Resource | Current state |
|
||||
|---|---|
|
||||
| `/srv/codex-work/lasereverything.net.db` | Primary Git checkout, branch `temporary/utilities-only`, commit `305ea41`; owns the current local Git database |
|
||||
| nested `.worktrees/payload-migration` | Migration worktree, branch `migration/payload`; planning commit begins at `ed1c84e` |
|
||||
| `.migration-source` | Symlink to `/srv/codex-migration/current` |
|
||||
| `.migration.env` | Symlink to `/srv/codex-secrets/le-payload-migration.env` |
|
||||
| `/srv/codex-migration/current` | Symlink to `source-20260710-175408` |
|
||||
| `/srv/codex-migration/source-20260710-175408` | Scrubbed source snapshot, mode `0750`, approximately 150 MB aggregate at audit time |
|
||||
| `/srv/codex-secrets/le-payload-migration.env` | Current readable nonproduction migration environment, mode `0640`; values not inspected |
|
||||
| `/srv/codex-secrets/le-payload-migration-admin.env` | Root-only transitional administration environment, mode `0600`; values not inspected and not copied into the development environment |
|
||||
| `/srv/codex-work` | Approximately 2.0 GB aggregate, including repository history/worktrees and application files |
|
||||
| recorded rootful MariaDB | container `le-directus-source`, volume `le_directus_source_data_20260710-175408` |
|
||||
| recorded rootful PostgreSQL | container `le-payload-pg`, volume `le_payload_pg_data_20260710-175408` |
|
||||
| retired Directus | container `directus`; not a disposable migration resource and not removed by consolidation |
|
||||
|
||||
The current user cannot query the rootful production Docker daemon. Before consolidation, an administrator must produce a metadata-only inventory without printing container environments, labels that contain secrets, database contents, or mount-file contents:
|
||||
|
||||
```bash
|
||||
sudo docker ps -a --filter name=^/le-directus-source$ \
|
||||
--filter name=^/le-payload-pg$ --format '{{.Names}} {{.Image}} {{.Status}}'
|
||||
sudo docker inspect le-directus-source le-payload-pg \
|
||||
--format '{{.Name}} image={{.Image}} mounts={{range .Mounts}}{{.Type}}:{{.Name}}:{{.Destination}};{{end}}'
|
||||
sudo docker volume inspect \
|
||||
le_directus_source_data_20260710-175408 \
|
||||
le_payload_pg_data_20260710-175408 \
|
||||
--format '{{.Name}} driver={{.Driver}} mountpoint={{.Mountpoint}} labels={{json .Labels}}'
|
||||
```
|
||||
|
||||
If names, image IDs, volumes, or mounts differ from the recorded manifest, stop and revise the plan. Do not inspect `.Config.Env` or print secret values.
|
||||
|
||||
## Reversible consolidation procedure
|
||||
|
||||
No step deletes or mutates a transitional resource. Consolidation is copy/recreate/verify, followed by a rollback observation period.
|
||||
|
||||
### Phase A — manifests and Git preservation
|
||||
|
||||
1. Push `migration/payload`, including planning commits, to `origin` and verify the remote commit.
|
||||
2. Record structural manifests for every transitional path, symlink, owner/mode, aggregate size, Git branch/commit/remote, source checksum manifest, and the administrator container/volume inventory above.
|
||||
3. Confirm the full root-only archive remains independently present and inaccessible; do not traverse or copy it.
|
||||
4. Record the exact existing rootful container/volume names and leave them stopped/running exactly as found unless a later approved step says otherwise.
|
||||
|
||||
### Phase B — canonical checkout
|
||||
|
||||
After account/runtime approval, create one clean checkout as the dedicated user:
|
||||
|
||||
```bash
|
||||
sudo -iu le_payload_dev git clone --branch migration/payload --single-branch \
|
||||
ssh://FORGE/makearmy/le-app.git \
|
||||
/srv/le-payload-dev/lasereverything.net.db
|
||||
sudo -iu le_payload_dev git -C /srv/le-payload-dev/lasereverything.net.db \
|
||||
fetch --prune origin
|
||||
sudo -iu le_payload_dev git -C /srv/le-payload-dev/lasereverything.net.db \
|
||||
status --short --branch
|
||||
sudo -iu le_payload_dev git -C /srv/le-payload-dev/lasereverything.net.db \
|
||||
rev-parse HEAD
|
||||
```
|
||||
|
||||
The HEAD must equal the reviewed remote `migration/payload` commit, the worktree must be clean, and no nested permanent worktree may exist. Provision a dedicated repository-scoped Git credential for `le_payload_dev`; do not copy `sol6_vi`'s private key.
|
||||
|
||||
### Phase C — scrubbed source copy
|
||||
|
||||
The administrator copies from the resolved source directory, never from the root-only archive:
|
||||
|
||||
```bash
|
||||
source_old=/srv/codex-migration/source-20260710-175408
|
||||
source_new=/srv/le-payload-dev/source
|
||||
|
||||
sudo rsync -aHAX --numeric-ids --delete-delay "$source_old/" "$source_new/"
|
||||
|
||||
# Verify supplied manifests without emitting filenames or record contents.
|
||||
sudo sh -c "cd '$source_new' && sha256sum --check --status CHECKSUMS.sha256"
|
||||
sudo sh -c "cd '$source_new' && sha256sum --check --status uploads.sha256"
|
||||
|
||||
# Freeze the canonical input against development writes.
|
||||
sudo chown -R root:le_payload_dev "$source_new"
|
||||
sudo find "$source_new" -type d -exec chmod 0550 {} +
|
||||
sudo find "$source_new" -type f -exec chmod 0440 {} +
|
||||
```
|
||||
|
||||
Before using `--delete-delay`, assert `source_new` resolves exactly beneath `/srv/le-payload-dev/source` and is not a symlink. The source checksum files, row-count manifests, schema, dump, extensions, and all uploads must validate. The old copy remains untouched.
|
||||
|
||||
### Phase D — nonproduction secrets
|
||||
|
||||
- Preserve both old credential files in place during rollback.
|
||||
- Do not copy `le-payload-migration-admin.env`.
|
||||
- Generate new fixture/snapshot/rootless database, Payload, Mailpit, and local gateway credentials directly into root-owned `/srv/le-payload-dev/secrets` with mode `0640`.
|
||||
- If the MariaDB read-only credential from the scrubbed environment is temporarily required to compare the old rootful source, transfer only the minimum required variable set through an administrator-reviewed, non-logging process; replace it once the rootless source clone is verified.
|
||||
- No production SMTP, OAuth, Ko-fi, Directus token, webhook, or server credential enters the new directory.
|
||||
|
||||
### Phase E — rootless database recreation
|
||||
|
||||
1. Restore the scrubbed SQL from canonical read-only `source` into a new project-scoped rootless MariaDB volume.
|
||||
2. Create a distinct import account with `SELECT` only; application/migration jobs never receive the MariaDB administrative credential.
|
||||
3. Mount/copy uploads only through finite jobs; the MariaDB and legacy Directus services never receive writable access to canonical `source`.
|
||||
4. Create fresh disposable rootless PostgreSQL and Payload media volumes.
|
||||
5. Verify source table counts/checksum summaries against supplied manifests and verify PostgreSQL health/reset isolation.
|
||||
6. Record new rootless image digests, volume names, Compose project labels, and service health without recording secrets.
|
||||
|
||||
### Phase F — rollback observation period
|
||||
|
||||
Keep `/srv/codex-work`, `/srv/codex-migration`, `/srv/codex-secrets`, rootful `le-directus-source`, rootful `le-payload-pg`, and their recorded volumes intact for at least 30 days **and** through at least two complete successful rootless migration rehearsals, whichever is later. During this period:
|
||||
|
||||
- new work occurs only in the canonical checkout;
|
||||
- old resources are read-only/frozen where possible;
|
||||
- checksum, Git, service, and validation comparisons remain reproducible;
|
||||
- rollback means stop the rootless stack and resume analysis from the untouched transitional resources, not copy changes backward.
|
||||
|
||||
## Later cleanup eligibility — explicit approval required
|
||||
|
||||
Only after the rollback criteria and a fresh inventory may the following become eligible for removal:
|
||||
|
||||
- `/srv/codex-work/lasereverything.net.db/.worktrees/payload-migration` after Git confirms it is clean, pushed, and removable with `git worktree remove`;
|
||||
- `/srv/codex-work/lasereverything.net.db/.migration-source` and `.migration.env` symlinks;
|
||||
- `/srv/codex-work/lasereverything.net.db`, then `/srv/codex-work` only if no other entries exist;
|
||||
- `/srv/codex-migration/current`, `source-20260710-175408`, then `/srv/codex-migration` after canonical checksums pass and rollback expires;
|
||||
- `/srv/codex-secrets/le-payload-migration.env` and `le-payload-migration-admin.env`, then `/srv/codex-secrets`, after credentials are revoked/obsolete and rollback expires;
|
||||
- rootful containers `le-directus-source` and `le-payload-pg`;
|
||||
- rootful volumes `le_directus_source_data_20260710-175408` and `le_payload_pg_data_20260710-175408` after verifying their exact attachment and backup status.
|
||||
|
||||
Every deletion requires a separate explicit approval showing the resolved path/object, before/after checksum or Git proof, attachment status, and rollback consequence. Never include the retired `directus` container, production volumes, `/var/www`, or `/srv/directus-archive/20260710-175408` in this cleanup set.
|
||||
|
||||
## Canonical production promotion
|
||||
|
||||
Promotion path:
|
||||
|
||||
```text
|
||||
/srv/le-payload-dev/lasereverything.net.db
|
||||
-> reviewed Git commit + signed/recorded release manifest
|
||||
-> CI-built immutable image digests tested in fixture/snapshot/rehearsal/staging
|
||||
-> clean checkout or verified release artifact at /var/www/lasereverything.net.db
|
||||
-> production deployment invoked only from /var/www/lasereverything.net.db
|
||||
```
|
||||
|
||||
Required workflow:
|
||||
|
||||
1. The canonical nonproduction checkout is clean and all accepted commits are pushed.
|
||||
2. CI builds images once, records digests/SBOM/provenance, and runs the accepted test gates.
|
||||
3. Staging deploys those exact digests from the same Compose core and an approved staging overlay.
|
||||
4. A reviewed release manifest pins Git commit, image digests, migration set, configuration schema, health contract, and rollback compatibility.
|
||||
5. On TITANSERVER, an operator updates a clean Git checkout at `/var/www/lasereverything.net.db` to the reviewed commit (or atomically unpacks a verified release artifact into that exact root), verifies a clean tree, and verifies the release manifest.
|
||||
6. From that directory only, deployment scripts render/validate production Compose and the host-ingress include, run the gated schema job, pull pinned digests, and roll services with health checks.
|
||||
|
||||
The production checkout contains all application source/configuration needed to reproduce the deployment contract. Physical image layers, declared production PostgreSQL/media volumes, and shared host-proxy configuration may remain outside it, but their names/locations/interfaces are declared in the production overlay and release manifest rooted there.
|
||||
|
||||
Forbidden promotion inputs include `/srv/le-payload-dev/runtime`, `source`, `secrets`, `data`, `artifacts`, local CA material, Mailpit state, test traces, rootless Docker state, disposable databases/volumes, caches, and development `.env` files. No `rsync` or recursive copy from `/srv/le-payload-dev` to `/var/www` is permitted.
|
||||
|
||||
## Local HTTPS and CA trust
|
||||
|
||||
The local gateway uses a repository-pinned proxy image and an ignored, generated development CA to serve:
|
||||
|
|
@ -286,6 +487,6 @@ sudo groupdel le_payload_dev 2>/dev/null || true
|
|||
|
||||
The dedicated Docker context lives only in the dedicated user's home and disappears with that home. If another operator created a client context pointing to the user socket, remove that context explicitly from that operator's client configuration.
|
||||
|
||||
Production secret permission hardening is not rolled back merely because development is removed; it is an independent security correction. Source ACL entries can be removed with `setfacl -x u:le_payload_dev` before account deletion. Removal does not touch `/var/run/docker.sock`, production contexts, `/var/lib/docker`, `/var/lib/mysql`, `/var/www` application content, production volumes/networks, the frozen source, or `/srv/directus-archive`.
|
||||
Production secret permission hardening is not rolled back merely because development is removed; it is an independent security correction. Any temporary transitional ACLs, if an administrator added them contrary to the final copy-based design, must be inventoried and removed before account deletion. Removal does not touch `/var/run/docker.sock`, production contexts, `/var/lib/docker`, `/var/lib/mysql`, `/var/www` application content, production volumes/networks, transitional source during its rollback retention, or `/srv/directus-archive`.
|
||||
|
||||
Final proof compares before/after permission metadata and confirms: production socket remains denied, production service state is unchanged, no project mount referenced forbidden paths, the dedicated UID/subordinate mappings are gone, and only `/srv/le-payload-dev` resources were deleted.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue