Multi-profile gateway isolation and cross-profile interference fixes

NousResearch/hermes-agent · cluster 4402 · gateway

Data as of

4 open items. Fix merged in #19582. Check it covers your case.

The short answer

2open issues
2open PRs
20items grouped
Known?
Yes. 20 related items grouped here, 6 issues already closed.
Fix event
A fix was merged to main in PR #19582. Check it covers your case. No release data is tracked.
Tracker
The maintainers' tracker marks this partially resolved, deciding PR #53930. That is separate from the fix event.
Still open
2 issues and 2 PRs.

These issues and fixes center on adding isolated profile support to Hermes and correcting gateway operations (status, stop, replace, PID resolution, lock release) that incorrectly affect all profiles instead of only the active one. A recurring theme is cross-profile SIGTERM/SIGKILL interference caused by the --replace mechanism ignoring HERMES_PROFILE.

Open issues2

  • Issue#30155`--replace` cross-kills sibling gateways when multiple HERMES_PROFILE values share one HERMES_HOME
  • Issue#60269[Bug]: Multi-profile gateway autostart race condition — --replace causes crash loop when two profiles start simultaneously

Open PRs2

  • PR#19504fix(gateway): filter status PIDs by profile
  • PR#60828fix(gateway): scope replace checks by profile

Fixes on record

Merged to main. A stamp counts the issues a fix is linked to.

  • #19582 fix(gateway): show other profiles in 'gateway status' to prevent confusion

    merged to main

    files: hermes_cli/gateway.py

    Show the diff
    • hermes_cli/gateway.py
      @@ -690,6 +690,32 @@ def _print_gateway_process_mismatch(snapshot: GatewayRuntimeSnapshot) -> None:
           print("  can refuse to start another copy until this process stops.")
       
       
      +def _print_other_profiles_gateway_status() -> None:
      +    """Print a summary of gateway status across all profiles.
      +
      +    Shown at the bottom of ``hermes gateway status`` output so users with
      +    multiple profiles can tell at a glance which gateways are running and
      +    avoid confusing another profile's process with the current one.
      +    """
      +    try:
      +        from hermes_cli.profiles import get_active_profile_name
      +
      +        current = get_active_profile_name()
      +        other_processes = [
      +            p for p in find_profile_gateway_processes()
      +            if p.profile != current
      +        ]
      +        if not other_processes:
      +            return
      +
      +        print()
      +        print("Other profiles:")
      +        for proc in other_processes:
      +            print(f"  ✓ {proc.profile:<16s} — PID {proc.pid}")
      +    except Exception:
      +        pass
      +
      +
       def kill_gateway_processes(force: bool = False, exclude_pids: set | None = None,
                                  all_profiles: bool = False) -> int:
           """Kill any running gateway processes. Returns count killed.
      @@ -4456,6 +4482,9 @@ def _gateway_command_inner(args):
                           print("  hermes gateway install  # Install as user service")
                           print("  sudo hermes gateway install --system  # Install as boot-time system service")
       
      +        # Show other profiles' gateway status for multi-profile awareness
      +        _print_other_profiles_gateway_status()
      +
           elif subcmd == "migrate-legacy":
               # Stop, disable, and remove legacy Hermes gateway unit files from
               # pre-rename installs (e.g. hermes.service). Profile units and
  • #53930 fix(gateway): namespace --replace takeover marker by HERMES_HOME to stop cross-profile flap (#29092)

    merged to main

    files: gateway/status.py, tests/gateway/test_status.py

    Show the diff
    • gateway/status.py
      @@ -1133,6 +1133,22 @@ def _consume_pid_marker_for_self(
                   pass
               return False
       
      +    # Cross-profile guard (#29092): reject markers written by a gateway
      +    # running under a different HERMES_HOME. When two profile gateway
      +    # services share the same default ~/.hermes (HERMES_HOME not set
      +    # distinctly), the marker path resolves to the same file for both. A
      +    # --replace from profile B could land in profile A's marker, match on
      +    # PID + start_time by coincidence of a shared PID namespace, and make
      +    # profile A exit 0 — only to be revived by systemd Restart=always,
      +    # which then races the replacer again, flapping indefinitely. The
      +    # field is absent in markers written by older Hermes versions; treat
      +    # absent as "same home" so old markers and single-profile setups are
      +    # unaffected. Leave a mismatched marker in place so the correct
      +    # profile can still consume it.
      +    replacer_home = record.get("replacer_hermes_home")
      +    if replacer_home is not None and replacer_home != str(get_hermes_home()):
      +        return False
      +
           our_pid = os.getpid()
           our_start_time = _get_process_start_time(our_pid)
           # Start-time is a PID-reuse guard. It is only meaningful when both
      @@ -1179,6 +1195,7 @@ def write_takeover_marker(target_pid: int) -> bool:
                   "target_pid": target_pid,
                   "target_start_time": target_start_time,
                   "replacer_pid": os.getpid(),
      +            "replacer_hermes_home": str(get_hermes_home()),
                   "written_at": _utc_now_iso(),
               }
               _write_json_file(_get_takeover_marker_path(), record)
    • tests/gateway/test_status.py
      @@ -1128,6 +1128,64 @@ def test_consume_ignores_marker_for_different_process_and_prevents_stale_grief(
               # We are not the target — must NOT consume as planned
               assert result is False
       
      +    def test_write_marker_records_replacer_hermes_home(self, tmp_path, monkeypatch):
      +        """The marker stamps the replacer's HERMES_HOME for cross-profile guard (#29092)."""
      +        monkeypatch.setenv("HERMES_HOME", str(tmp_path))
      +        monkeypatch.setattr(status, "_get_process_start_time", lambda pid: 42)
      +
      +        status.write_takeover_marker(target_pid=12345)
      +
      +        payload = json.loads((tmp_path / ".gateway-takeover.json").read_text())
      +        assert payload["replacer_hermes_home"] == str(tmp_path)
      +
      +    def test_consume_rejects_marker_from_different_profile(self, tmp_path, monkeypatch):
      +        """Regression (#29092): a marker written by a gateway under a DIFFERENT
      +        HERMES_HOME must be rejected even when PID + start_time coincidentally
      +        match — otherwise two profile services sharing a default ~/.hermes flap
      +        each other in an infinite SIGTERM/Restart loop. The mismatched marker is
      +        left in place so the profile it was actually meant for can consume it.
      +        """
      +        monkeypatch.setenv("HERMES_HOME", str(tmp_path))
      +        monkeypatch.setattr(status, "_get_process_start_time", lambda pid: 100)
      +        marker_path = tmp_path / ".gateway-takeover.json"
      +        from datetime import datetime, timezone
      +        # Marker names OUR pid + start_time (the coincidental match the bug
      +        # relied on) but was written by a gateway in a different profile.
      +        marker_path.write_text(json.dumps({
      +            "target_pid": os.getpid(),
      +            "target_start_time": 100,
      +            "replacer_pid": 99999,
      +            "replacer_hermes_home": str(tmp_path / "profiles" / "other"),
      +            "written_at": datetime.now(timezone.utc).isoformat(),
      +        }))
      +
      +        result = status.consume_takeover_marker_for_self()
      +
      +        assert result is False
      +        # Left in place for the correct profile, not griefed away.
      +        assert marker_path.exists()
      +
      +    def test_consume_accepts_legacy_marker_without_hermes_home(self, tmp_path, monkeypatch):
      +        """Back-compat (#29092): markers written by older Hermes versions have no
      +        ``replacer_hermes_home`` field; an absent field is treated as same-home so
      +        single-profile setups and mixed old/new deployments keep working.
      +        """
      +        monkeypatch.setenv("HERMES_HOME", str(tmp_path))
      +        monkeypatch.setattr(status, "_get_process_start_time", lambda pid: 100)
      +        marker_path = tmp_path / ".gateway-takeover.json"
      +        from datetime import datetime, timezone
      +        marker_path.write_text(json.dumps({
      +            "target_pid": os.getpid(),
      +            "target_start_time": 100,
      +            "replacer_pid": 99999,
      +            "written_at": datetime.now(timezone.utc).isoformat(),
      +        }))
      +
      +        result = status.consume_takeover_marker_for_self()
      +
      +        assert result is True
      +        assert not marker_path.exists()
      +
       
       class TestPlannedStopMarker:
           """Tests for intentional service/manual gateway stop markers."""

8 PRs closed without merging

From the close comments: merged via another change 2 · closed without comment 1 · duplicate 1 · stale or abandoned 1 · won't fix / by design 1 · other 2

  • #19115 fix(gateway): show other profiles in `gateway status` to prevent confusion · merged via another change → #19582
  • #19134 fix: scope gateway status to current profile · duplicate → #19115, #16375
  • #29111 fix(gateway): keep launchd gateways alive after clean exits · stale or abandoned
  • #29952 fix(cli): honour HERMES_PROFILE env var in _apply_profile_override · other
  • #30043 fix(constants): honor HERMES_PROFILE env var in get_hermes_home() · closed without comment
Show 3 more
  • #30196 fix(gateway): refuse --replace across HERMES_PROFILE mismatch (#30155) · won't fix / by design → #30043
  • #31414 fix(gateway): prevent cross-profile SIGTERM flap loop in multi-profile systemd setups · merged via another change → 2b73dd1, #53930, #34597
  • #60610 fix(local): guard killpg against own process group (#47788) · other → #47788
Closed issues · 6
  • Closed#4402gateway stop kills ALL profile gateways instead of only the current profile
  • Closed#4587Multi-profile gateway is not safe: kill_gateway_processes() and release_all_scoped_locks() are profile-blind
  • Closed#19113[Bug]: `gateway status` reports running when only a different profile's gateway is alive
  • Closed#29092[Bug]: Two profile gateway services enter SIGTERM flap loop — `--replace` + PID file cross-profile mis-routing
  • Closed#29948--replace singleton pidfile resolver ignores HERMES_PROFILE, causing cross-gateway SIGKILL on multi-persona hosts
  • Closed#47788[印证] Gateway SIGTERM leak in multi-profile concurrent startup (--replace mechanism)
How this cluster was grouped
  • Link evidence (partly structural links): some members are connected by structural evidence (shared changed files, or a closing/superseded reference); the rest are joined by an automated relevance judgement over their text.
  • Members outside that structural core may belong to a different underlying problem.
  • Pairwise link density 0.08: the share of item pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36. 1.00 means every pair is linked.

Pick up a Lab task

Check whether a fix covers the open issues here, with your own agent. Bring back evidence for maintainers.

Open the Lab

At a glance

Subsystem
gateway
Merged PRs
#19582, #53930
Closed issues
6
Attempts
8 PRs closed without merging
Fix link checked
28 Jun 2026

For your agent

The same facts, typed.