# NousResearch/hermes-agent · Gateway session_reset zombie slot cleanup fixes

## Open now

- Nothing open on GitHub.

A bug and several fixes addressing zombie agent slots left behind when session_reset races with in-flight runs or credential pool exhaustion, causing subsequent messages to be silently dropped. Related cleanup includes consolidating per-session teardown, closing SessionDB on shutdown, and stripping a stray environment variable from the Windows restart helper.

## TL;DR

- 5 member(s): 0 open issue(s), 1 closed issue(s), 0 open PR(s), 1 merged PR(s), 3 PR(s) closed without merging.
- A fix was merged to main in PR #39060. Verify it addresses your case.
- **Recommended:** read PR #39060 (and any competing PRs below) before filing. Confirm the merged change covers your symptom.

## Maintainer resolution verdict

- Internal-tracker verdict over this cluster's issue members: `unresolved`. This is distinct from the fix status above. Weigh it yourself.

## How this cluster was grouped

- Link evidence (`STRUCTURAL_SPANNING`): every member is connected by structural evidence: shared changed files, or a closing/superseded reference.
- Pairwise link density `0.40`: the share of member pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36. `1.00` means every pair is linked.

## Closed / duplicate issues

- #28686: Bug: session_reset + credential pool exhaustion leaves thread session in zombie state — subsequent messages silently dropped

## Prior & competing PRs

- **#39060: fix(gateway): clear zombie agent slot when session_reset races in-flight run**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`; implication `superseded`
  - files: `gateway/run.py`, `tests/gateway/test_session_state_cleanup.py`
  - fix hunks:
  - `gateway/run.py`

    ```diff
    @@ -8470,18 +8470,14 @@ async def _do_undo():
                     logger.debug("goal continuation hook failed: %s", _goal_exc)
                 return _agent_result
             finally:
    -            # If _run_agent replaced the sentinel with a real agent and
    -            # then cleaned it up, this is a no-op.  If we exited early
    -            # (exception, command fallthrough, etc.) the sentinel must
    -            # not linger or the session would be permanently locked out.
    -            if self._running_agents.get(_quick_key) is _AGENT_PENDING_SENTINEL:
    -                self._release_running_agent_state(_quick_key)
    -            else:
    -                # Agent path already cleaned _running_agents; make sure
    -                # the paired metadata dicts are gone too.
    -                self._running_agents_ts.pop(_quick_key, None)
    -                if hasattr(self, "_busy_ack_ts"):
    -                    self._busy_ack_ts.pop(_quick_key, None)
    +            # Unconditional release covers every exit path. _release_running_agent_state
    +            # is idempotent (pop-on-absent is harmless) and, called without a
    +            # run_generation guard, always clears the slot regardless of which
    +            # generation it holds. This evicts the zombie left when session_reset
    +            # bumps the generation (N -> N+1) mid-flight: gen-N's guarded release
    +            # inside _run_agent returns False, and the old sentinel-only check here
    +            # missed the leftover real agent — locking the session out forever (#28686).
    +            self._release_running_agent_state(_quick_key)
     
         async def _prepare_inbound_message_text(
             self,
    ```

    ```diff
    @@ -9978,6 +9974,12 @@ async def _handle_reset_command(self, event: MessageEvent) -> Union[str, Ephemer
             # Get existing session key
             session_key = self._session_key_for_source(source)
             self._invalidate_session_run_generation(session_key, reason="session_reset")
    +        # Evict the running-agent slot now that the generation is bumped. The
    +        # in-flight run's own guarded release (run_generation=old) will return
    +        # False and leave its dead agent behind; clearing here keeps the slot
    +        # from becoming a zombie that silently drops all later messages (#28686).
    +        # Idempotent, so the run's finally calling it again is harmless.
    +        self._release_running_agent_state(session_key)
     
             # Snapshot the old entry so on_session_finalize can report the
             # expiring session id before reset_session() rotates it.
    ```
  - `tests/gateway/test_session_state_cleanup.py`

    ```diff
    @@ -228,3 +228,54 @@ def test_shutdown_tolerates_close_raising(self):
     
             flaky_db.close.assert_called_once()
             healthy_db.close.assert_called_once()
    +
    +
    +class TestSessionResetZombieRace:
    +    """Regression for #28686 — a session_reset racing the in-flight run's
    +    guarded release must not leave a dead agent locking the slot forever.
    +    """
    +
    +    def test_generation_guard_blocks_then_unconditional_release_evicts(self):
    +        runner = _make_runner()
    +        runner._session_run_generation = {}
    +        key = "agent:main:telegram:private:1"
    +
    +        gen_n = runner._begin_session_run_generation(key)
    +        dead_agent = MagicMock()
    +        runner._running_agents[key] = dead_agent
    +        runner._running_agents_ts[key] = 1.0
    +        runner._busy_ack_ts[key] = 1.0
    +
    +        # session_reset bumps the generation while gen-N is still in flight.
    +        runner._invalidate_session_run_generation(key, reason="session_reset")
    +
    +        # gen-N's own guarded release is correctly blocked — slot would be a
    +        # zombie if nothing else cleared it (the pre-fix behaviour).
    +        assert runner._release_running_agent_state(key, run_generation=gen_n) is False
    +        assert runner._running_agents.get(key) is dead_agent
    +
    +        # The fix: unconditional release (no run_generation) always clears it.
    +        assert runner._release_running_agent_state(key) is True
    +        assert key not in runner._running_agents
    +        assert key not in runner._running_agents_ts
    +        assert key not in runner._busy_ack_ts
    +
    +    def test_normal_completion_is_not_evicted_by_outer_release(self):
    +        """Guarded release with the current generation succeeds; the outer
    +        unconditional release that follows is a harmless no-op.
    +        """
    +        runner = _make_runner()
    +        runner._session_run_generation = {}
    +        key = "agent:main:telegram:private:2"
    +
    +        gen = runner._begin_session_run_generation(key)
    +        runner._running_agents[key] = MagicMock()
    +        runner._running_agents_ts[key] = 1.0
    +        runner._busy_ack_ts[key] = 1.0
    +
    +        assert runner._release_running_agent_state(key, run_generation=gen) is True
    +        assert key not in runner._running_agents
    +        # Outer finally runs the unconditional release after — nothing stranded.
    +        assert runner._release_running_agent_state(key) is True
    +        assert key not in runner._running_agents_ts
    +        assert key not in runner._busy_ack_ts
    ```
- **#28689: fix(gateway): clear stale agent slot after session_reset to prevent zombie thread**. closed without merge
  - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `693f4c7e9ce583ed740daaa5aa8eb738390c643b`, `ae29484f9a643db8328f8fdc672d2d4717318696`, `#39060`, `#39060`, `#28686`
- **#30834: fix(gateway): clear zombie agent slot when session_reset races credential pool exhaustion**. closed without merge
  - maintainer direction: category `merged_via`; implication `superseded`; refs `693f4c7e9`, `#39060`
- **#40059: fix(gateway): strip _HERMES_GATEWAY from Windows detached restart helper env**. closed without merge
  - maintainer direction: category `uncategorized`; implication `needs_llm`

## Fix status

- Merged to main in PR #39060 (cited above). No release data is tracked. Verify downstream.
- Fix last verified (merged) `2026-06-04`.
- Subsystem: `gateway`