# NousResearch/hermes-agent · Gateway restart reliability and completion fixes

## Open now

- Nothing open on GitHub.

These issues and pull requests address problems where the gateway restart process returns before the service has fully come back up, with fixes targeting systemd restart sequencing, service revival, and ensuring the restart flow waits for active status.

## TL;DR

- 3 member(s): 0 open issue(s), 1 closed issue(s), 0 open PR(s), 1 merged PR(s), 1 PR(s) closed without merging.
- A fix was merged to main in PR #9945. Verify it addresses your case.
- **Recommended:** read PR #9945 (and any competing PRs below) before filing. Confirm the merged change covers your symptom.

## Maintainer resolution verdict

- Internal-tracker verdict over this cluster's issue members: `resolved` · deciding PR(s): `pr-9945`. This is distinct from the fix status above. Weigh it yourself.

## How this cluster was grouped

- Link evidence (`STRUCTURAL_SPANNING`): every member is connected by structural evidence: shared changed files, or a closing/superseded reference.
- Pairwise link density `0.67`: the share of member pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36. `1.00` means every pair is linked.

## Closed / duplicate issues

- #8260: hermes gateway restart returns before restart completes

## Prior & competing PRs

- **#9945: fix: hermes gateway restart waits for service to come back up (#8260)**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`
  - files: `hermes_cli/gateway.py`, `tests/hermes_cli/test_gateway_service.py`
  - fix hunks:
  - `hermes_cli/gateway.py`

    ```diff
    @@ -1128,7 +1128,62 @@ def systemd_restart(system: bool = False):
     
         pid = get_running_pid()
         if pid is not None and _request_gateway_self_restart(pid):
    -        print(f"✓ {_service_scope_label(system).capitalize()} service restart requested")
    +        # SIGUSR1 sent — the gateway will drain active agents, exit with
    +        # code 75, and systemd will restart it after RestartSec (30s).
    +        # Wait for the old process to die and the new one to become active
    +        # so the CLI doesn't return while the service is still restarting.
    +        import time
    +        scope_label = _service_scope_label(system).capitalize()
    +        svc = get_service_name()
    +        scope_cmd = _systemctl_cmd(system)
    +
    +        # Phase 1: wait for old process to exit (drain + shutdown)
    +        print(f"⏳ {scope_label} service draining active work...")
    +        deadline = time.time() + 90
    +        while time.time() < deadline:
    +            try:
    +                os.kill(pid, 0)
    +                time.sleep(1)
    +            except (ProcessLookupError, PermissionError):
    +                break  # old process is gone
    +        else:
    +            print(f"⚠ Old process (PID {pid}) still alive after 90s")
    +
    +        # Phase 2: wait for systemd to start the new process
    +        print(f"⏳ Waiting for {svc} to restart...")
    +        deadline = time.time() + 60
    +        while time.time() < deadline:
    +            try:
    +                result = subprocess.run(
    +                    scope_cmd + ["is-active", svc],
    +                    capture_output=True, text=True, timeout=5,
    +                )
    +                if result.stdout.strip() == "active":
    +                    # Verify it's a NEW process, not the old one somehow
    +                    new_pid = get_running_pid()
    +                    if new_pid and new_pid != pid:
    +                        print(f"✓ {scope_label} service restarted (PID {new_pid})")
    +                        return
    +            except (subprocess.TimeoutExpired, FileNotFoundError):
    +                pass
    +            time.sleep(2)
    +
    +        # Timed out — check final state
    +        try:
    +            result = subprocess.run(
    +                scope_cmd + ["is-active", svc],
    +                capture_output=True, text=True, timeout=5,
    +            )
    +            if result.stdout.strip() == "active":
    +                print(f"✓ {scope_label} service restarted")
    +                return
    +        except Exception:
    +            pass
    +        print(
    +            f"⚠ {scope_label} service did not become active within 60s.\n"
    +            f"  Check status: {'sudo ' if system else ''}hermes gateway status\n"
    +            f"  Check logs:   journalctl {'--user ' if not system else ''}-u {svc} --since '2 min ago'"
    +        )
             return
         _run_systemctl(["reload-or-restart", get_service_name()], system=system, check=True, timeout=90)
         print(f"✓ {_service_scope_label(system).capitalize()} service restarted")
    ```
  - `tests/hermes_cli/test_gateway_service.py`

    ```diff
    @@ -452,7 +452,7 @@ def fake_run(*args, **kwargs):
     
     
     class TestGatewaySystemServiceRouting:
    -    def test_systemd_restart_self_requests_graceful_restart_without_reload_or_restart(self, monkeypatch, capsys):
    +    def test_systemd_restart_self_requests_graceful_restart_and_waits(self, monkeypatch, capsys):
             calls = []
     
             monkeypatch.setattr(gateway_cli, "_select_systemd_scope", lambda system=False: False)
    ```

    ```diff
    @@ -466,16 +466,37 @@ def test_systemd_restart_self_requests_graceful_restart_without_reload_or_restar
                 "_request_gateway_self_restart",
                 lambda pid: calls.append(("self", pid)) or True,
             )
    -        monkeypatch.setattr(
    -            gateway_cli.subprocess,
    -            "run",
    -            lambda *args, **kwargs: (_ for _ in ()).throw(AssertionError("systemctl should not run")),
    -        )
    +
    +        # Simulate: old process dies immediately, new process becomes active
    +        kill_call_count = [0]
    +        def fake_kill(pid, sig):
    +            kill_call_count[0] += 1
    +            if kill_call_count[0] >= 2:  # first call checks, second = dead
    +                raise ProcessLookupError()
    +        monkeypatch.setattr(os, "kill", fake_kill)
    +
    +        # Simulate systemctl is-active returning "active" with a new PID
    +        new_pid = [None]
    +        def fake_subprocess_run(cmd, **kwargs):
    +            if "is-active" in cmd:
    +                result = SimpleNamespace(stdout="active\n", returncode=0)
    +                new_pid[0] = 999  # new PID
    +                return result
    +            raise AssertionError(f"Unexpected systemctl call: {cmd}")
    +
    +        monkeypatch.setattr(gateway_cli.subprocess, "run", fake_subprocess_run)
    +        # get_running_pid returns new PID after restart
    +        pid_calls = [0]
    +        def fake_get_pid():
    +            pid_calls[0] += 1
    +            return 999 if pid_calls[0] > 1 else 654
    +        monkeypatch.setattr("gateway.status.get_running_pid", fake_get_pid)
     
             gateway_cli.systemd_restart()
     
    -        assert calls == [("refresh", False), ("self", 654)]
    -        assert "restart requested" in capsys.readouterr().out.lower()
    +        assert ("self", 654) in calls
    +        out = capsys.readouterr().out.lower()
    +        assert "restarted" in out
     
         def test_gateway_install_passes_system_flags(self, monkeypatch):
             monkeypatch.setattr(gateway_cli, "supports_systemd_services", lambda: True)
    ```
- **#8674: fix(gateway): wait for systemd restart to become active**. closed without merge
  - maintainer direction: category `superseded_by_pr`; implication `superseded`; refs `2a98098035ca70459570e99b6b26e1a3ca6fbd27`, `#8260`

## Fix status

- Merged to main in PR #9945 (cited above). No release data is tracked. Verify downstream.
- Fix last verified (merged) `2026-04-15`. This confirmation is old and may be out of date (a revert/reopen would not have invalidated it here); re-verify.