Skip to content

[Fix] Preserve operator Compose overrides via an explicit registry owned by the host CLI - #1584

Open
mrubens wants to merge 5 commits into
developfrom
feat/compose-override-registry
Open

[Fix] Preserve operator Compose overrides via an explicit registry owned by the host CLI#1584
mrubens wants to merge 5 commits into
developfrom
feat/compose-override-registry

Conversation

@mrubens

@mrubens mrubens commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Operator Compose overrides currently survive only some code paths, and fixes to the host CLI or systemd unit never reach existing deployments. This PR makes the override mechanism explicit, persisted state and makes roomote upgrade self-delivering, instead of patching individual call sites.

It is an alternative to #1496, implementing the same goal one level deeper. The core ideas:

  1. Explicit registry instead of glob discovery. The deployment's Compose file list lives in COMPOSE_FILE in /opt/roomote/.env (the variable docker compose itself honors), managed by a new roomote override add|remove|list command. Ordering is operator-controlled and visible, stray files in /opt/roomote can't silently join production config, override add validates the merged config before recording anything, and a registered-but-missing file fails loudly with the fix spelled out.
  2. One owner for the Compose invocation. The host CLI's compose() builds the -f list from the registry, and every other surface routes through the CLI: the installer (roomote sync-unit + roomote compose config/pull + roomote up), the systemd unit (ExecStart=/usr/local/bin/roomote up), and the managed deploy/scripts/deploy.sh / upgrade.sh remote blocks (which now ship the CLI and call roomote compose ...). No path can drop overrides because no path constructs its own compose command anymore. A roomote compose passthrough covers arbitrary operations.
  3. Upgrades deliver the fix. roomote upgrade now refreshes /usr/local/bin/roomote from the target release (atomic replace, bash -n guard against truncated downloads) and rewrites the systemd unit via the new CLI's sync-unit. Both are staged into the rollback dir and restored if the upgrade fails. Existing installs need one last installer rerun (or managed redeploy) to receive this release; from here on, upgrades carry CLI and unit fixes themselves.
  4. Backups cover overrides. Bundles stage registered override files under config/compose-overrides/ and roomote restore puts them back before the first compose call against the restored .env. Image-identity collection deliberately stays base-file-only so a locally built override image (the motivating Caddy DNS-plugin case) cannot block backups or upgrades; override images are documented as operator-managed for recovery.
  5. Nonstandard Docker everywhere. The Docker binary path is persisted as ROOMOTE_DOCKER_BIN in .env and every invocation (compose, docker run/pull/inspect/ps/stop/tag) goes through it, not just the wrapper — so a snap Docker outside systemd's or sudo's PATH works for backup/restore/upgrade too, not only for up.
  6. EXIT-trap state is global by construction. cleanup_backup/cleanup_restore are top-level functions reading only backup_*/restore_* globals (bash pops function locals before EXIT traps run under set -e), with a single cleanup body shared by the success and failure paths. This fixes the interrupted-backup restart bug and the same latent leak in restore (decrypted postgres.sql left on disk after a failed restore).

Tests

  • deploy/host/tests/compose-overrides.sh — registry parsing, ordering, stray-file exclusion, override add/remove/list, loud failure on missing registered files
  • deploy/host/tests/compose-docker-path.shROOMOTE_DOCKER_BIN honored from the environment and from .env
  • deploy/host/tests/backup-failed-restart.sh — interrupted backup restarts services (with overrides) from the EXIT trap, no unbound variable, override staged in the bundle
  • deploy/host/tests/restore-failed-cleanup.sh — failed restore cleans its staging dir and passphrase temp file
  • deploy/host/tests/upgrade-refresh.sh — upgrade installs the new CLI, rewrites the unit, seeds COMPOSE_FILE/ROOMOTE_DOCKER_BIN on pre-registry installs, and preserves overrides
  • deploy/host/tests/upgrade-failed-pull.sh — extended: a failed upgrade also restores the previous CLI and unit
  • deploy/host/tests/backup-restore.integration.sh — full real-Docker backup/restore cycle passes unchanged
  • deploy/ci/validate-deployment-artifacts.mjs — new asserts that the installer and both managed scripts route through the host CLI and contain no raw docker compose --env-file invocations

Docs updated in SELF_HOSTING.md and apps/docs/self-hosting.mdx.

…s survive every path

Replaces glob discovery with an explicit COMPOSE_FILE registry in .env
managed by `roomote override add|remove|list`; routes the installer,
systemd unit, and managed deploy/upgrade scripts through the CLI; makes
`roomote upgrade` refresh the CLI and systemd unit; includes overrides
in backup bundles; threads ROOMOTE_DOCKER_BIN through every docker
invocation; and moves backup/restore EXIT-trap state to globals.
@roomote-community

roomote-community Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

No code issues found. See task

  • Managed deploys persist ROOMOTE_DOCKER_BIN before creating the systemd unit.
  • Managed deploys run the worker-image pull through the configured Docker binary.
  • Managed deploy.sh preserves a host's registered COMPOSE_FILE during redeploys.
  • Restore now rewrites ROOMOTE_DOCKER_BIN for the recovery host, avoiding a stale source-host Docker path (deploy/host/roomote:1265).

Reviewed 2c667bc

Comment thread deploy/scripts/deploy.sh
@mrubens

mrubens commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

@zarnivoop Fable came up with this as an alternative to #1496, curious to hear what you think

Comment thread deploy/scripts/deploy.sh
Comment thread deploy/scripts/deploy.sh
Comment thread deploy/host/roomote
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant