{"repo":"NousResearch/hermes-agent","public_id":"33597","display_label":"Docker restart gateway state persistence bug and fixes","summary":"A cluster focused on a bug where signal-initiated container shutdowns incorrectly persisted a stopped gateway state, preventing auto-start on restart. Members include the original s6-overlay feature, the bug report, and multiple fix attempts addressing how gateway desired state is preserved across container restarts.","subsystem":"gateway","counts":{"members":6,"open_issues":0,"open_issues_distinct":0,"open_prs":0,"closed_issues":2,"merged_prs":1,"attempt_prs":3},"landed_fix":{"status":"merged_to_main","basis":"structural","pr":43236},"resolution":{"state":"unresolved","by":null},"freshness":{"state":"verified","last_verified_at":"2026-06-10T04:01:35+00:00","is_stale":false},"grouping":{"backbone":"STRUCTURAL_SPANNING","link_density":0.3573333333333333},"fix_edges":[{"issue_number":33597,"pr_number":43236,"tier":"verified","basis":"superseded_ref","confidence":0.9,"recoverable_from_timeline":false},{"issue_number":42675,"pr_number":43236,"tier":"verified","basis":"superseded_ref","confidence":0.9,"recoverable_from_timeline":false}],"members":[{"kind":"issue","number":33597,"title":"[Bug]: Docker restart persists gateway as stopped, preventing s6 auto-start on next boot","url":null,"state":"closed"},{"kind":"issue","number":42675,"title":"Gateway does not auto-start after container restart/upgrade — signal-initiated shutdown persists `gateway_state=stopped`","url":null,"state":"closed"},{"kind":"pr","number":33619,"title":"fix(docker): preserve running gateways across restart","url":null,"state":"closed","close_reason":{"refs":[{"kind":"pr","value":"#43236","number":43236},{"kind":"pr","value":"#43236","number":43236},{"kind":"pr","value":"#42675","number":42675},{"kind":"pr","value":"#33597","number":33597},{"kind":"pr","value":"#43236","number":43236}],"category":"implemented_elsewhere","implication":"superseded"},"merged":false},{"kind":"pr","number":42740,"title":"fix(gateway): preserve running state on signal shutdown","url":null,"state":"closed","close_reason":{"refs":[{"kind":"pr","value":"#43236","number":43236},{"kind":"pr","value":"#42675","number":42675},{"kind":"pr","value":"#42517","number":42517},{"kind":"pr","value":"#43236","number":43236}],"category":"implemented_elsewhere","implication":"superseded"},"merged":false},{"kind":"pr","number":42789,"title":"fix(gateway): skip persisting gateway_state=stopped on signal-initiated shutdown (#42675)","url":null,"state":"closed","close_reason":{"refs":[{"kind":"pr","value":"#43236","number":43236},{"kind":"pr","value":"#42675","number":42675},{"kind":"pr","value":"#42517","number":42517},{"kind":"pr","value":"#43236","number":43236}],"category":"superseded_by_pr","implication":"superseded"},"merged":false},{"kind":"pr","number":43236,"title":"fix(gateway): auto-start after container restart via planned-stop marker (#42675)","url":null,"state":"merged","close_reason":{"category":"merged","implication":"superseded"},"merged":true,"is_fix_pr":true,"fix_tier":"verified","files_changed":["gateway/run.py","hermes_cli/service_manager.py","tests/docker/test_container_restart.py","tests/gateway/restart_test_helpers.py","tests/gateway/test_gateway_shutdown.py","tests/hermes_cli/test_service_manager.py","website/docs/user-guide/docker.md"],"fix_hunks":[{"file":"gateway/run.py","hunks":["@@ -1953,6 +1953,16 @@ def __init__(self, config: Optional[GatewayConfig] = None):\n         self._exit_code: Optional[int] = None\n         self._draining = False\n         self._restart_requested = False\n+        # Set by shutdown_signal_handler when a SIGTERM/SIGINT arrived\n+        # WITHOUT a planned-stop / takeover marker — i.e. an unexpected\n+        # external signal (container/s6 SIGTERM on `docker restart` or\n+        # image upgrade, OOM-killer, bare `kill`). Distinct from an\n+        # operator-requested stop, which writes a marker first. Used by\n+        # _stop_impl to decide whether to persist gateway_state=stopped\n+        # (see issue #42675): an unexpected signal must NOT persist\n+        # \"stopped\", or container_boot refuses to auto-start the gateway\n+        # on the next boot.\n+        self._signal_initiated_shutdown = False\n         self._restart_task_started = False\n         self._restart_detached = False\n         self._restart_via_service = False","@@ -5952,7 +5962,36 @@ def _phase_elapsed() -> float:\n                 self._exit_reason = self._exit_reason or \"Gateway restart requested\"\n \n             self._draining = False\n-            self._update_runtime_status(\"stopped\", self._exit_reason)\n+            # Persist the terminal gateway_state. The default is \"stopped\",\n+            # but when this teardown was triggered by an UNEXPECTED external\n+            # signal (container/s6 SIGTERM on `docker restart` or image\n+            # upgrade, OOM-killer, bare `kill`) we instead persist \"running\"\n+            # to preserve the operator's run-intent across the restart.\n+            #\n+            # On Docker (s6-overlay), container_boot.py reads gateway_state\n+            # on the next boot and only auto-starts gateways whose last\n+            # state was \"running\" (_AUTOSTART_STATES). Persisting \"stopped\"\n+            # — or leaving the mid-shutdown \"draining\" marker in place — for\n+            # a routine `docker compose up --force-recreate` permanently\n+            # suppresses auto-start, so the messaging channels silently stay\n+            # dark until the operator manually restarts (issue #42675).\n+            #\n+            # An operator-initiated stop (`hermes gateway stop`,\n+            # systemd/launchd ExecStop, the s6 stop path, Ctrl+C) writes a\n+            # planned-stop marker BEFORE signalling, so it is classified as\n+            # a planned stop (not signal-initiated) and correctly persists\n+            # \"stopped\" — respecting the explicit intent. A restart also\n+            # persists \"stopped\" here; the restarting process brings the\n+            # gateway back up itself.\n+            if getattr(self, \"_signal_initiated_shutdown\", False) and not self._restart_requested:\n+                logger.info(\n+                    \"Gateway stopped by an unexpected signal — persisting \"\n+                    \"gateway_state=running so container_boot auto-starts on \"\n+                    \"the next boot (issue #42675)\"\n+                )\n+                self._update_runtime_status(\"running\", self._exit_reason)\n+            else:\n+                self._update_runtime_status(\"stopped\", self._exit_reason)\n             logger.info(\"Gateway stopped (total teardown %.2fs)\", _phase_elapsed())\n \n         self._stop_task = asyncio.create_task(_stop_impl())","@@ -15711,6 +15750,13 @@ def shutdown_signal_handler(received_signal=None):\n             )\n         else:\n             _signal_initiated_shutdown = True\n+            # Mirror onto the runner so _stop_impl can suppress the\n+            # gateway_state=stopped persist for unexpected signals\n+            # (container/s6 SIGTERM on restart, OOM, bare kill) — see\n+            # issue #42675. Operator-initiated stops set a planned-stop\n+            # marker first, land in the `planned_stop` branch above, and\n+            # leave this flag False so they DO persist \"stopped\".\n+            runner._signal_initiated_shutdown = True\n             logger.info(\n                 \"Received %s — initiating shutdown\",\n                 _shutdown_ctx[\"signal\"] if _shutdown_ctx else \"SIGTERM/SIGINT\","]},{"file":"hermes_cli/service_manager.py","hunks":["@@ -739,13 +739,55 @@ def start(self, name: str) -> None:\n         \"\"\"\n         self._run_svc(\"-u\", \"start\", name)\n \n+    def _supervised_pid(self, name: str) -> int | None:\n+        \"\"\"Return the PID of the supervised gateway process, or None.\n+\n+        Parses ``s6-svstat`` output (``up (pid NNNN) ...``). Used to\n+        mark an operator-initiated stop with the planned-stop marker so\n+        the gateway's shutdown handler classifies the incoming SIGTERM\n+        as intentional rather than an unexpected kill (issue #42675).\n+        Best-effort: any parse/exec failure returns None.\n+        \"\"\"\n+        import subprocess\n+\n+        try:\n+            result = subprocess.run(\n+                [f\"{_S6_BIN_DIR}/s6-svstat\", str(self.scandir / name)],\n+                capture_output=True, text=True, timeout=5,\n+            )\n+        except (OSError, subprocess.SubprocessError):\n+            return None\n+        if result.returncode != 0:\n+            return None\n+        m = re.search(r\"\\(pid (\\d+)\\)\", result.stdout)\n+        return int(m.group(1)) if m else None\n+\n     def stop(self, name: str) -> None:\n         \"\"\"Bring down a registered service (``s6-svc -d``).\n \n+        Writes a planned-stop marker naming the supervised gateway PID\n+        BEFORE sending the down command, so the gateway's shutdown\n+        handler recognises this SIGTERM as an operator-initiated stop\n+        and persists ``gateway_state=stopped`` (respecting the explicit\n+        intent). Without the marker, an intentional ``hermes gateway\n+        stop`` is indistinguishable from the container/s6 SIGTERM sent on\n+        ``docker restart``; the latter must NOT persist ``stopped`` or\n+        container_boot refuses to auto-start on the next boot (#42675).\n+        The marker write is best-effort — a failure only means the stop\n+        is treated as signal-initiated, which is the safe fallback.\n+\n         Raises:\n             GatewayNotRegisteredError: no service directory for ``name``.\n             S6CommandError: s6-svc exited non-zero for any other reason.\n         \"\"\"\n+        pid = self._supervised_pid(name)\n+        if pid is not None:\n+            try:\n+                from gateway.status import write_planned_stop_marker\n+\n+                write_planned_stop_marker(pid)\n+            except Exception:\n+                pass\n         self._run_svc(\"-d\", \"stop\", name)\n \n     def restart(self, name: str) -> None:"]},{"file":"tests/docker/test_container_restart.py","hunks":["@@ -250,3 +250,79 @@ def test_stale_gateway_pid_cleaned_up_on_restart(restart_container: str) -> None\n     assert r.returncode != 0, \"stale gateway.pid survived restart\"\n     r = _sh(container, \"test -f /opt/data/profiles/ghost/processes.json\")\n     assert r.returncode != 0, \"stale processes.json survived restart\"\n+\n+\n+def test_live_gateway_autostarts_after_real_restart_without_manual_state_stamp(\n+    restart_container: str,\n+) -> None:\n+    \"\"\"End-to-end guard for issue #42675.\n+\n+    The other tests in this module stamp gateway_state.json directly to\n+    exercise the reconciler's READ side. This one exercises the WRITE\n+    side: a real, live gateway is killed by the container/s6 SIGTERM that\n+    `docker restart` sends — no manual state stamp — and must come back up\n+    on the next boot.\n+\n+    Before the fix, the shutdown handler unconditionally persisted\n+    gateway_state=stopped on that SIGTERM, so the reconciler saw 'stopped'\n+    and registered the slot DOWN — the gateway silently stayed dark after\n+    every container restart. The fix classifies an unmarked SIGTERM as\n+    signal-initiated and persists 'running' instead, so auto-start works.\n+    \"\"\"\n+    container = restart_container\n+\n+    _exec(container, \"hermes\", \"profile\", \"create\", \"live\").check_returncode()\n+    r = _exec(container, \"hermes\", \"-p\", \"live\", \"gateway\", \"start\", timeout=60)\n+    assert r.returncode == 0, f\"gateway start failed: {r.stderr}\"\n+\n+    # Wait for the gateway to actually come up under supervision AND write\n+    # its own gateway_state=running (we do NOT stamp it ourselves).\n+    deadline = time.monotonic() + 20.0\n+    while time.monotonic() < deadline:\n+        r = _sh(container, \"/command/s6-svstat /run/service/gateway-live\")\n+        if r.returncode == 0 and \"up \" in r.stdout:\n+            break\n+        time.sleep(0.5)\n+    assert \"up \" in r.stdout, f\"gateway never came up pre-restart: {r.stdout!r}\"\n+\n+    # Confirm the gateway persisted its own 'running' state (sanity: we're\n+    # testing the real write path, not a stamped fixture).\n+    deadline = time.monotonic() + 15.0\n+    state = \"\"\n+    while time.monotonic() < deadline:\n+        r = _sh(\n+            container,\n+            \"cat /opt/data/profiles/live/gateway_state.json 2>/dev/null\",\n+        )\n+        if r.returncode == 0 and '\"gateway_state\"' in r.stdout:\n+            state = r.stdout\n+            break\n+        time.sleep(0.5)\n+    assert '\"running\"' in state, (\n+        f\"gateway never persisted running state pre-restart: {state!r}\"\n+    )\n+\n+    # Real restart — Docker sends SIGTERM to PID 1; s6 propagates it to the\n+    # supervised gateway. No planned-stop marker is written (this is not an\n+    # operator `hermes gateway stop`), so the shutdown is signal-initiated.\n+    _docker(\"restart\", container, timeout=60).check_returncode()\n+\n+    log = _wait_for_reconcile_log_mention(container, \"live\", deadline_s=30.0)\n+    assert \"profile=live\" in log, (\n+        f\"reconciler never logged live after restart: {log!r}\"\n+    )\n+    # The crux: the reconciler must AUTO-START it, not register it down.\n+    assert \"action=started\" in log, (\n+        f\"gateway did NOT auto-start after a real restart (issue #42675 \"\n+        f\"regression): {log!r}\"\n+    )\n+\n+    # Slot recreated, and NO down marker (we expect auto-start).\n+    assert _wait_for_path(\n+        container, \"/run/service/gateway-live\", kind=\"d\", deadline_s=10.0,\n+    ), \"slot not recreated after restart\"\n+    r = _sh(container, \"test -f /run/service/gateway-live/down\")\n+    assert r.returncode != 0, (\n+        \"down marker present despite a live gateway being restarted — \"\n+        \"the signal-initiated shutdown wrongly persisted 'stopped' (#42675)\"\n+    )"]},{"file":"tests/gateway/restart_test_helpers.py","hunks":["@@ -66,6 +66,7 @@ def make_restart_runner(\n     runner._background_tasks = set()\n     runner._draining = False\n     runner._restart_requested = False\n+    runner._signal_initiated_shutdown = False\n     runner._restart_task_started = False\n     runner._restart_detached = False\n     runner._restart_via_service = False"]},{"file":"tests/gateway/test_gateway_shutdown.py","hunks":["@@ -358,3 +358,90 @@ def _fake_kill_all(task_id=None):\n \n     # Only the final catch-all fires on the graceful path.\n     assert kill_count == 1\n+\n+\n+# ---------------------------------------------------------------------------\n+# gateway_state persistence on shutdown (issue #42675)\n+#\n+# On Docker/s6, container_boot.py only auto-starts gateways whose last\n+# persisted gateway_state was \"running\". An unexpected external signal\n+# (the SIGTERM s6/Docker sends on `docker compose up --force-recreate`,\n+# OOM, bare kill) must NOT persist \"stopped\" — otherwise the gateway\n+# stays down after every container restart. An operator-initiated stop\n+# writes a planned-stop marker first, so it is NOT signal-initiated and\n+# DOES persist \"stopped\", respecting the explicit intent.\n+# ---------------------------------------------------------------------------\n+\n+\n+def _persisted_states(runner) -> list:\n+    \"\"\"All gateway_state values passed to _update_runtime_status, in order.\"\"\"\n+    states = []\n+    for call in runner._update_runtime_status.call_args_list:\n+        args, kwargs = call\n+        state = kwargs.get(\"gateway_state\", args[0] if args else None)\n+        states.append(state)\n+    return states\n+\n+\n+def _stopped_state_persisted(runner) -> bool:\n+    \"\"\"True iff _update_runtime_status was called with gateway_state='stopped'.\"\"\"\n+    return \"stopped\" in _persisted_states(runner)\n+\n+\n[redacted:email]\n+async def test_signal_initiated_shutdown_persists_running_not_stopped(tmp_path, monkeypatch):\n+    \"\"\"Unexpected SIGTERM (container restart / OOM / kill) must persist\n+    gateway_state=running — NOT stopped, and NOT leave the mid-shutdown\n+    'draining' marker — so container_boot auto-starts on next boot (#42675).\"\"\"\n+    monkeypatch.setattr(gateway_run, \"_hermes_home\", tmp_path)\n+    runner, adapter = make_restart_runner()\n+    adapter.disconnect = AsyncMock()\n+    runner._signal_initiated_shutdown = True  # set by handler on unmarked signal\n+\n+    with patch(\"gateway.status.remove_pid_file\"), patch(\"gateway.status.write_runtime_status\"):\n+        await runner.stop()\n+\n+    assert not _stopped_state_persisted(runner), (\n+        \"signal-initiated shutdown must NOT persist gateway_state=stopped\"\n+    )\n+    # The FINAL terminal write must be 'running' so container_boot's\n+    # _AUTOSTART_STATES check passes (it only auto-starts 'running').\n+    assert _persisted_states(runner)[-1] == \"running\", (\n+        f\"final state must be 'running', got: {_persisted_states(runner)}\"\n+    )\n+\n+\n[redacted:email]\n+async def test_operator_initiated_stop_persists_stopped(tmp_path, monkeypatch):\n+    \"\"\"A planned stop (marker written → not signal-initiated) must persist\n+    gateway_state=stopped so an explicit `hermes gateway stop` stays down.\"\"\"\n+    monkeypatch.setattr(gateway_run, \"_hermes_home\", tmp_path)\n+    runner, adapter = make_restart_runner()\n+    adapter.disconnect = AsyncMock()\n+    runner._signal_initiated_shutdown = False  # planned stop classification\n+\n+    with patch(\"gateway.status.remove_pid_file\"), patch(\"gateway.status.write_runtime_status\"):\n+        await runner.stop()\n+\n+    assert _stopped_state_persisted(runner), (\n+        \"operator-initiated stop must persist gateway_state=stopped\"\n+    )\n+\n+\n[redacted:email]\n+async def test_signal_initiated_restart_still_persists_stopped(tmp_path, monkeypatch):\n+    \"\"\"A restart is not a 'stay down' — it persists normally (the new\n+    process/container brings the gateway back up itself). The suppression\n+    only applies to a terminal signal-initiated stop, not a restart.\"\"\"\n+    monkeypatch.setattr(gateway_run, \"_hermes_home\", tmp_path)\n+    runner, adapter = make_restart_runner()\n+    adapter.disconnect = AsyncMock()\n+    runner._signal_initiated_shutdown = True\n+    runner._launch_systemd_restart_shortcut = MagicMock()\n+\n+    with patch(\"gateway.status.remove_pid_file\"), patch(\"gateway.status.write_runtime_status\"):\n+        await runner.stop(restart=True, service_restart=True)\n+\n+    assert _stopped_state_persisted(runner), (\n+        \"a restart must persist gateway_state=stopped via the normal path\"\n+    )"]},{"file":"tests/hermes_cli/test_service_manager.py","hunks":["@@ -799,3 +799,111 @@ def _svstat_down(cmd, **kw):\n         return _sp.CompletedProcess(cmd, 0, \"\", \"\")\n     monkeypatch.setattr(\"subprocess.run\", _svstat_down)\n     assert S6ServiceManager(scandir=s6_scandir).is_running(\"gateway-coder\") is False\n+\n+\n+# ---------------------------------------------------------------------------\n+# S6 stop writes a planned-stop marker (issue #42675)\n+#\n+# `hermes gateway stop` inside a container dispatches through\n+# S6ServiceManager.stop() -> `s6-svc -d`, which SIGTERMs the gateway.\n+# That SIGTERM is indistinguishable from the one s6/Docker sends on a\n+# container restart unless we mark the intentional stop first. Without\n+# the marker, the gateway's shutdown handler can't tell an operator\n+# stop from a restart kill, and the gateway_state=stopped suppression\n+# (run.py) would never engage for explicit stops.\n+# ---------------------------------------------------------------------------\n+\n+\n+def test_s6_supervised_pid_parses_svstat(monkeypatch, s6_scandir):\n+    \"\"\"_supervised_pid extracts the PID from `up (pid NNNN) ...`.\"\"\"\n+    import subprocess as _sp\n+\n+    def _fake(cmd, **kw):\n+        return _sp.CompletedProcess(cmd, 0, \"up (pid 4242) 17 seconds\\n\", \"\")\n+\n+    monkeypatch.setattr(\"subprocess.run\", _fake)\n+    mgr = S6ServiceManager(scandir=s6_scandir)\n+    assert mgr._supervised_pid(\"gateway-coder\") == 4242\n+\n+\n+def test_s6_supervised_pid_none_when_down(monkeypatch, s6_scandir):\n+    \"\"\"A down service (`s6-svstat` rc!=0 or no pid) yields None.\"\"\"\n+    import subprocess as _sp\n+\n+    def _fake(cmd, **kw):\n+        return _sp.CompletedProcess(cmd, 0, \"down (exitcode 0) 3 seconds\\n\", \"\")\n+\n+    monkeypatch.setattr(\"subprocess.run\", _fake)\n+    mgr = S6ServiceManager(scandir=s6_scandir)\n+    assert mgr._supervised_pid(\"gateway-coder\") is None\n+\n+\n+def test_s6_stop_writes_planned_stop_marker(monkeypatch, s6_scandir):\n+    \"\"\"stop() must mark the supervised PID before `s6-svc -d` so the\n+    gateway recognises the SIGTERM as an intentional stop (#42675).\"\"\"\n+    import subprocess as _sp\n+\n+    svc_dir = s6_scandir / \"gateway-coder\"\n+    svc_dir.mkdir()  # so _run_svc doesn't raise GatewayNotRegisteredError\n+\n+    svc_calls: list[list[str]] = []\n+\n+    def _fake(cmd, **kw):\n+        seq = list(cmd) if isinstance(cmd, (list, tuple)) else [str(cmd)]\n+        if seq and seq[0].startswith(\"/command/\"):\n+            seq[0] = seq[0][len(\"/command/\"):]\n+        svc_calls.append(seq)\n+        if seq and seq[0] == \"s6-svstat\":\n+            return _sp.CompletedProcess(cmd, 0, \"up (pid 9090) 5 seconds\\n\", \"\")\n+        return _sp.CompletedProcess(cmd, 0, \"\", \"\")\n+\n+    monkeypatch.setattr(\"subprocess.run\", _fake)\n+\n+    marked: list[int] = []\n+    monkeypatch.setattr(\n+        \"gateway.status.write_planned_stop_marker\",\n+        lambda pid: marked.append(pid) or True,\n+    )\n+\n+    mgr = S6ServiceManager(scandir=s6_scandir)\n+    mgr.stop(\"gateway-coder\")\n+\n+    assert marked == [9090], (\n+        f\"stop() must write the planned-stop marker for the supervised PID; \"\n+        f\"marked={marked}\"\n+    )\n+    # And it must still issue the down command.\n+    assert any(\n+        cmd[0] == \"s6-svc\" and \"-d\" in cmd for cmd in svc_calls\n+    ), f\"s6-svc -d not invoked; saw: {svc_calls}\"\n+\n+\n+def test_s6_stop_tolerates_marker_write_failure(monkeypatch, s6_scandir):\n+    \"\"\"A marker-write failure must not block the stop (best-effort).\"\"\"\n+    import subprocess as _sp\n+\n+    svc_dir = s6_scandir / \"gateway-coder\"\n+    svc_dir.mkdir()\n+\n+    svc_calls: list[list[str]] = []\n+\n+    def _fake(cmd, **kw):\n+        seq = list(cmd) if isinstance(cmd, (list, tuple)) else [str(cmd)]\n+        if seq and seq[0].startswith(\"/command/\"):\n+            seq[0] = seq[0][len(\"/command/\"):]\n+        svc_calls.append(seq)\n+        if seq and seq[0] == \"s6-svstat\":\n+            return _sp.CompletedProcess(cmd, 0, \"up (pid 9090) 5 seconds\\n\", \"\")\n+        return _sp.CompletedProcess(cmd, 0, \"\", \"\")\n+\n+    monkeypatch.setattr(\"subprocess.run\", _fake)\n+\n+    def _boom(pid):\n+        raise OSError(\"disk full\")\n+\n+    monkeypatch.setattr(\"gateway.status.write_planned_stop_marker\", _boom)\n+\n+    mgr = S6ServiceManager(scandir=s6_scandir)\n+    mgr.stop(\"gateway-coder\")  # must not raise\n+\n+    assert any(cmd[0] == \"s6-svc\" and \"-d\" in cmd for cmd in svc_calls)"]},{"file":"website/docs/user-guide/docker.md","hunks":["@@ -183,7 +183,7 @@ Each profile created with `hermes profile create <name>` gets:\n - A dedicated s6 service slot at `/run/service/gateway-<name>/`, registered dynamically by the runtime — no container rebuild required.\n - Auto-restart on crash, backoff-managed by `s6-supervise`.\n - Per-profile rotated logs at `${HERMES_HOME}/logs/gateways/<name>/current` (10 archives × 1 MB each).\n-- State persistence across container restarts: the boot-time reconciler reads `gateway_state.json` from each profile directory and brings the slot back up only for profiles whose last recorded state was `running`. Stopped profiles stay stopped.\n+- State persistence across container restarts: the boot-time reconciler reads `gateway_state.json` from each profile directory and brings the slot back up only for profiles whose last recorded state was `running`. Only a gateway you explicitly stopped (`hermes gateway stop`) stays down across a restart — a container restart, image upgrade, or unexpected exit leaves the recorded state as `running`, so the gateway auto-starts on the next boot.\n \n The lifecycle commands you'd run on the host work the same way from inside the container:\n ","@@ -473,7 +473,7 @@ Each profile created with `hermes profile create <name>` automatically gets an s\n \n - Gateway crashes are auto-restarted by `s6-supervise` after a ~1s backoff.\n - Dashboard, when enabled with `HERMES_DASHBOARD=1`, is supervised on the same supervision tree and gets the same auto-restart treatment.\n-- `docker restart` preserves running gateways: the cont-init reconciler reads `$HERMES_HOME/profiles/<name>/gateway_state.json` and brings the slot back up if the last recorded state was `running`. Stopped gateways stay stopped.\n+- `docker restart`, image upgrades (`docker compose up -d --force-recreate`), and unexpected exits preserve running gateways: the cont-init reconciler reads `$HERMES_HOME/profiles/<name>/gateway_state.json` and brings the slot back up if the last recorded state was `running`. Only an explicit `hermes gateway stop` records `stopped` and keeps the gateway down across the restart; the container/s6 SIGTERM sent on a restart or upgrade is treated as \"still running\" and auto-starts.\n - Per-profile gateway logs persist under `$HERMES_HOME/logs/gateways/<profile>/current` (rotated by `s6-log`), and the reconciler's actions are appended to `$HERMES_HOME/logs/container-boot.log` per boot. See [Where the logs go](#where-the-logs-go) for the full routing map.\n \n `hermes status` inside the container reports `Manager: s6 (container supervisor)`. Use `/command/s6-svstat /run/service/gateway-<name>` for the raw supervisor view (note `/command/` is on PATH for supervision-tree processes only; pass the absolute path when calling from `docker exec`).\n"]}]}],"trust":{"default_tier":"verified","includes_suggested":false}}