NousResearch/hermes-agent

Goal loop pausing on repeated judge errors to prevent spam

Cluster 27585 · agent. Data as of 2026-10-24 20:38 UTC

Nothing open on GitHub. Fix merged in #67997 on 20 Jul 2026. Check it covers your case.

open issues

open PRs

items grouped

Known?: Yes. 4 related items grouped here, 1 issue already closed.
Fix event: A fix was merged to main in PR #67997. Check it covers your case. No release data is tracked.
Tracker: The maintainers' tracker marks this resolved, deciding PR #67997. That is separate from the fix event.
Still open: Nothing open on GitHub.

Next step: read #67997, then check it covers your case.

Read #67997 Pick up a Lab task

These items center on the goals feature, where an issue reports repeated completion messages when goal_judge errors fall back to continuing. Three pull requests propose pausing the goal loop after consecutive judge or API errors, with varied triggers such as transport failures or terminal responses.

Open issues0

No open issues.

Open PRs0

No open pull requests.

Fixes on record

Merged to main. A stamp counts the issues a fix is linked to.

#67997 fix(goals): auto-pause goal loop on consecutive judge transport failures

merged to main

files: hermes_cli/goals.py, hermes_cli/kanban.py, tests/cli/test_cli_goal_interrupt.py, tests/gateway/test_goal_verdict_send.py, tests/hermes_cli/test_goals.py, tests/hermes_cli/test_kanban_goal_mode.py, tests/tools/test_kanban_tools.py, tools/kanban_tools.py

Show the diff

hermes_cli/goals.py

@@ -66,6 +66,11 @@
 # exhausted with every reply shaped like `judge returned empty response` or
 # `judge reply was not JSON`.
 DEFAULT_MAX_CONSECUTIVE_PARSE_FAILURES = 3
+# Transport failures (API auth errors 401, timeouts, DNS, etc.) are also
+# tracked and auto-pause the loop after this many consecutive failures.
+# A broken/invalid API key returns 401 every call — the loop must not
+# run until the turn budget, wasting every turn on an unreachable judge.
+DEFAULT_MAX_CONSECUTIVE_TRANSPORT_FAILURES = 5
 
 
 CONTINUATION_PROMPT_TEMPLATE = (
@@ -399,6 +404,10 @@ class GoalState:
     last_reason: Optional[str] = None
     paused_reason: Optional[str] = None       # why we auto-paused (budget, etc.)
     consecutive_parse_failures: int = 0       # judge-output parse failures in a row
+    # Transport failures are API/auth/network errors.  Broken API keys return
+    # 401 every call — track them separately so the loop auto-pauses instead
+    # of burning every turn budget slot on an unreachable judge.
+    consecutive_transport_failures: int = 0   # judge API/transport errors in a row
     # User-added criteria appended mid-loop via the /subgoal command.
     # When non-empty the judge prompt and continuation prompt both
     # include them so the agent works toward them and the judge factors
@@ -457,6 +466,7 @@ def from_json(cls, raw: str) -> "GoalState":
             last_reason=data.get("last_reason"),
             paused_reason=data.get("paused_reason"),
             consecutive_parse_failures=int(data.get("consecutive_parse_failures", 0) or 0),
+            consecutive_transport_failures=int(data.get("consecutive_transport_failures", 0) or 0),
             subgoals=subgoals,
             waiting_on_pid=(int(data["waiting_on_pid"]) if data.get("waiting_on_pid") else None),
             waiting_on_session=(str(data["waiting_on_session"]) if data.get("waiting_on_session") else None),
@@ -841,17 +851,23 @@ def judge_goal(
     subgoals: Optional[List[str]] = None,
     background_processes: Optional[List[Dict[str, Any]]] = None,
     contract: Optional[GoalContract] = None,
-) -> Tuple[str, str, bool, Optional[Dict[str, Any]]]:
+) -> Tuple[str, str, bool, Optional[Dict[str, Any]], bool]:
     """Ask the auxiliary model whether the goal is satisfied.
 
-    Returns ``(verdict, reason, parse_failed, wait_directive)`` where verdict
+    Returns ``(verdict, reason, parse_failed, wait_directive, transport_failed)`` where verdict
     is ``"done"``, ``"continue"``, ``"wait"``, or ``"skipped"`` (when the
     judge couldn't be reached). ``wait_directive`` is set only for ``"wait"``
     (``{"pid": int}`` or ``{"seconds": int}``); ``None`` otherwise.
 
     ``parse_failed`` is True only when the judge call succeeded but its output
     was unusable (empty or non-JSON). API/transport errors return False — they
-    are transient and should fail-open silently. Callers use this flag to
+    are transient and should fail-open silently.
+
+    ``transport_failed`` is True only when the judge couldn't reach the API at
+    all (auth 401, timeout, DNS, connection error).  Repeated transport
+    failures signal a permanent config problem (e.g. invalid API key).  Callers
+    use this flag to auto-pause after N consecutive transport failures (see
+    ``DEFAULT_MAX_CONSECUTIVE_TRANSPORT_FAILURES``). Callers use this flag to
     auto-pause after N consecutive parse failures (see
     ``DEFAULT_MAX_CONSECUTIVE_PARSE_FAILURES``).
 
@@ -867,21 +883,23 @@ def judge_goal(
     judge prompt; when none are set, behavior is identical to the original
     free-form judge.
 
-    This is deliberately fail-open: any error returns ``("continue", ..., False, None)``
-    so a broken judge doesn't wedge progress — the turn budget and the
-    consecutive-parse-failures auto-pause are the backstops.
+    This is deliberately fail-open: transport errors return ``("continue", ..., ..., None, True)``
+    — the ``transport_failed=True`` flag lets callers track and auto-pause after
+    N consecutive transport failures (see
+    ``DEFAULT_MAX_CONSECUTIVE_TRANSPORT_FAILURES``) so a permanently broken
+    judge doesn't burn the entire turn budget.
     """
     if not goal.strip():
-        return "skipped", "empty goal", False, None
+        return "skipped", "empty goal", False, None, False
     if not last_response.strip():
         # No substantive reply this turn — almost certainly not done yet.
-        return "continue", "empty response (nothing to evaluate)", False, None
+        return "continue", "empty response (nothing to evaluate)", False, None, False
 
     try:
         from agent.auxiliary_client import call_llm
     except Exception as exc:
         logger.debug("goal judge: auxiliary client import failed: %s", exc)
-        return "continue", "auxiliary client unavailable", False, None
+        return "continue", "auxiliary client unavailable", False, None, False
 
     # Build the prompt. Priority: contract > subgoals > plain. When both a
     # contract and subgoals exist, the subgoals are appended into the
@@ -941,7 +959,7 @@ def judge_goal(
         )
     except Exception as exc:
         logger.info("goal judge: API call failed (%s) — falling through to continue", exc)
-        return "continue", f"judge error: {type(exc).__name__}", False, None
+        return "continue", f"judge error: {type(exc).__name__}", False, None, True
 
     try:
         raw = resp.choices[0].message.content or ""
@@ -954,7 +972,7 @@ def judge_goal(
         verdict, _truncate(reason, 120),
         f" wait={wait_directive}" if wait_directive else "",
     )
-    return verdict, reason, parse_failed, wait_directive
+    return verdict, reason, parse_failed, wait_directive, False
 
 
 def gather_background_processes(task_id: Optional[str] = None) -> List[Dict[str, Any]]:
@@ -1425,7 +1443,7 @@ def evaluate_after_turn(
         state.turns_used += 1
         state.last_turn_at = time.time()
 
-        verdict, reason, parse_failed, wait_directive = judge_goal(
+        verdict, reason, parse_failed, wait_directive, transport_failed = judge_goal(
             state.goal,
             last_response,
             subgoals=state.subgoals or None,
@@ -1443,6 +1461,16 @@ def evaluate_after_turn(
         else:
             state.consecutive_parse_failures = 0
 
+        # Track consecutive transport failures separately — persistent API
+        # errors (401 auth, DNS, timeout) signal a broken config, not
+        # transient network flakiness.  Auto-pause after N consecutive
+        # transport failures so a permanently broken judge doesn't burn
+        # every turn budget slot on an unreachable API.
+        if transport_failed:
+            state.consecutive_transport_failures += 1
+        else:
+            state.consecutive_transport_failures = 0
+
         # WAIT verdict: the judge decided the agent is blocked on async work
         # and re-poking now would be busy-work. Set the barrier and park —
         # the turn we just counted stands (the judge call happened), but no
@@ -1480,6 +1508,36 @@ def evaluate_after_turn(
                 "message": f"✓ Goal achieved: {reason}",
             }
 
+        # Auto-pause when the judge cannot reach the API at all N turns in a
+        # row (401 auth, DNS failure, timeout).  Persistent transport failures
+        # signal a broken configuration (e.g. invalid API key), not transient
+        # flakiness.  Without this guard, a permanently broken judge burns
+        # every turn budget slot on an unreachable API.
+        if state.consecutive_transport_failures >= DEFAULT_MAX_CONSECUTIVE_TRANSPORT_FAILURES:
+            state.status = "paused"
+            state.paused_reason = (
+                f"judge API unreachable {state.consecutive_transport_failures} turns in a row "
+                f"(check auxiliary.goal_judge provider/key in config.yaml)"
+            )
+            save_goal(self.session_id, state)
+            return {
+                "status": "paused",
+                "should_continue": False,
+                "continuation_prompt": None,
+                "verdict": "continue",
+                "reason": reason,
+                "message": (
+                    f"⏸ Goal paused — judge API returned errors "
+                    f"({state.consecutive_transport_failures} turns). "
+                    "Check the goal_judge provider/key in ~/.hermes/config.yaml:\n"
+                    "  auxiliary:\n"
+                    "    goal_judge:\n"
+                    "      provider: deepseek\n"
+                    "      model: deepseek-v4-flash\n"
+                    "Then /goal resume to continue."
+                ),
+            }
+
         # Auto-pause when the judge model can't produce the expected JSON
         # verdict N turns in a row. Points the user at the goal_judge config
         # so they can route this side task to a model that follows the
@@ -1679,7 +1737,7 @@ def _log(msg: str) -> None:
         # The kanban worker loop has no wait-barrier concept (workers finish
         # via kanban_complete / kanban_block, not by parking), so a WAIT
         # verdict is treated as CONTINUE here.
-        verdict, reason, _parse_failed, _wait = judge_goal(goal_text, last_response)
+        verdict, reason, _parse_failed, _wait, _transport_failed = judge_goal(goal_text, last_response)
         if verdict == "wait":
             verdict = "continue"
         _log(f"kanban goal loop: turn {turns_used}/{max_turns} verdict={verdict} reason={_truncate(reason, 120)}")

hermes_cli/kanban.py

@@ -2016,10 +2016,11 @@ def _cmd_complete(args: argparse.Namespace) -> int:
                     reason = ""
                     try:
                         # judge_goal returns (verdict, reason, parse_failed,
-                        # wait_directive) — see hermes_cli/goals.py. Unpacking
-                        # fewer raises ValueError into the fail-open handler
-                        # below, silently disabling the gate.
-                        verdict, reason, _, _ = judge_goal(
+                        # wait_directive, transport_failed) — see
+                        # hermes_cli/goals.py. Unpacking fewer raises
+                        # ValueError into the fail-open handler below,
+                        # silently disabling the gate.
+                        verdict, reason, _, _, _ = judge_goal(
                             goal=f"{task.title}\n\n{task.body or ''}".strip(),
                             last_response=(summary or args.result or "").strip(),
                         )

tests/cli/test_cli_goal_interrupt.py

@@ -169,7 +169,7 @@ def test_clean_response_enqueues_continuation_when_judge_says_continue(
         # Force the judge to say "continue" without touching the network.
         with patch(
             "hermes_cli.goals.judge_goal",
-            return_value=("continue", "needs more steps", False, None),
+            return_value=("continue", "needs more steps", False, None, False),
         ):
             cli._maybe_continue_goal_after_turn()
 
@@ -189,7 +189,7 @@ def test_clean_response_marks_done_when_judge_says_done(self, hermes_home):
 
         with patch(
             "hermes_cli.goals.judge_goal",
-            return_value=("done", "goal satisfied", False, None),
+            return_value=("done", "goal satisfied", False, None, False),
         ):
             cli._maybe_continue_goal_after_turn()

tests/gateway/test_goal_verdict_send.py

@@ -107,7 +107,7 @@ async def test_goal_verdict_done_sent_via_adapter_send(hermes_home):
     mgr = GoalManager(session_entry.session_id)
     mgr.set("ship the feature")
 
-    with patch("hermes_cli.goals.judge_goal", return_value=("done", "the feature shipped", False, None)):
+    with patch("hermes_cli.goals.judge_goal", return_value=("done", "the feature shipped", False, None, False)):
         await runner._post_turn_goal_continuation(
             session_entry=session_entry,
             source=src,
@@ -136,7 +136,7 @@ async def test_goal_verdict_continue_enqueues_continuation(hermes_home):
     mgr = GoalManager(session_entry.session_id)
     mgr.set("polish the docs")
 
-    with patch("hermes_cli.goals.judge_goal", return_value=("continue", "still needs work", False, None)):
+    with patch("hermes_cli.goals.judge_goal", return_value=("continue", "still needs work", False, None, False)):
         await runner._post_turn_goal_continuation(
             session_entry=session_entry,
             source=src,
@@ -164,7 +164,7 @@ async def test_goal_verdict_budget_exhausted_sends_pause(hermes_home):
     state.turns_used = 2
     save_goal(session_entry.session_id, state)
 
-    with patch("hermes_cli.goals.judge_goal", return_value=("continue", "keep going", False, None)):
+    with patch("hermes_cli.goals.judge_goal", return_value=("continue", "keep going", False, None, False)):
         await runner._post_turn_goal_continuation(
             session_entry=session_entry,
             source=src,
@@ -211,7 +211,7 @@ def __init__(self):
 
     runner.adapters[Platform.TELEGRAM] = _NoSendAdapter()
 
-    with patch("hermes_cli.goals.judge_goal", return_value=("done", "ok", False, None)):
+    with patch("hermes_cli.goals.judge_goal", return_value=("done", "ok", False, None, False)):
         # must not raise
         await runner._post_turn_goal_continuation(
             session_entry=session_entry,

tests/hermes_cli/test_goals.py

@@ -152,13 +152,13 @@ class TestJudgeGoal:
     def test_empty_goal_skipped(self):
         from hermes_cli.goals import judge_goal
 
-        verdict, _, _, _wd = judge_goal("", "some response")
+        verdict, _, _, _wd, _tf = judge_goal("", "some response")
         assert verdict == "skipped"
 
     def test_empty_response_continues(self):
         from hermes_cli.goals import judge_goal
 
-        verdict, _, _, _wd = judge_goal("ship the thing", "")
+        verdict, _, _, _wd, _tf = judge_goal("ship the thing", "")
         assert verdict == "continue"
 
     def test_no_aux_client_continues(self):
@@ -169,7 +169,7 @@ def test_no_aux_client_continues(self):
             "agent.auxiliary_client.call_llm",
             side_effect=RuntimeError("No LLM provider configured"),
         ):
-            verdict, _, _, _wd = goals.judge_goal("my goal", "my response")
+            verdict, _, _, _wd, _tf = goals.judge_goal("my goal", "my response")
         assert verdict == "continue"
 
     def test_api_error_continues(self):
@@ -180,7 +180,7 @@ def test_api_error_continues(self):
             "agent.auxiliary_client.call_llm",
             side_effect=RuntimeError("boom"),
         ):
-            verdict, reason, _, _wd = goals.judge_goal("goal", "response")
+            verdict, reason, _, _wd, _tf = goals.judge_goal("goal", "response")
         assert verdict == "continue"
         assert "judge error" in reason.lower()
 
@@ -193,7 +193,7 @@ def test_judge_says_done(self):
                 choices=[MagicMock(message=MagicMock(content='{"done": true, "reason": "achieved"}'))]
             ),
         ):
-            verdict, reason, _, _wd = goals.judge_goal("goal", "agent response")
+            verdict, reason, _, _wd, _tf = goals.judge_goal("goal", "agent response")
         assert verdict == "done"
         assert reason == "achieved"
 
@@ -206,7 +206,7 @@ def test_judge_says_continue(self):
                 choices=[MagicMock(message=MagicMock(content='{"done": false, "reason": "not yet"}'))]
             ),
         ):
-            verdict, reason, _, _wd = goals.judge_goal("goal", "agent response")
+            verdict, reason, _, _wd, _tf = goals.judge_goal("goal", "agent response")
         assert verdict == "continue"
         assert reason == "not yet"
 
@@ -295,7 +295,7 @@ def test_evaluate_after_turn_done(self, hermes_home):
         mgr = GoalManager(session_id="eval-sid-1")
         mgr.set("ship it")
 
-        with patch.object(goals, "judge_goal", return_value=("done", "shipped", False, None)):
+        with patch.object(goals, "judge_goal", return_value=("done", "shipped", False, None, False)):
             decision = mgr.evaluate_after_turn("I shipped the feature.")
 
         assert decision["verdict"] == "done"
@@ -311,7 +311,7 @@ def test_evaluate_after_turn_continue_under_budget(self, hermes_home):
         mgr = GoalManager(session_id="eval-sid-2", default_max_turns=5)
         mgr.set("a long goal")
 
-        with patch.object(goals, "judge_goal", return_value=("continue", "more work", False, None)):
+        with patch.object(goals, "judge_goal", return_value=("continue", "more work", False, None, False)):
             decision = mgr.evaluate_after_turn("made some progress")
 
         assert decision["verdict"] == "continue"
@@ -329,7 +329,7 @@ def test_evaluate_after_turn_budget_exhausted(self, hermes_home):
         mgr = GoalManager(session_id="eval-sid-3", default_max_turns=2)
         mgr.set("hard goal")
 
-        with patch.object(goals, "judge_goal", return_value=("continue", "not yet", False, None)):
+        with patch.object(goals, "judge_goal", return_value=("continue", "not yet", False, None, False)):
             d1 = mgr.evaluate_after_turn("step 1")
             assert d1["should_continue"] is True
             assert mgr.state.turns_used == 1
@@ -438,9 +438,12 @@ def test_api_error_does_not_count_as_parse_failure(self):
             "agent.auxiliary_client.call_llm",
             side_effect=RuntimeError("connection reset"),
         ):
-            verdict, _, parse_failed, _wd = goals.judge_goal("goal", "response")
+            verdict, _, parse_failed, _wd, transport_failed = goals.judge_goal(
+                "goal", "response"
+            )
         assert verdict == "continue"
         assert parse_failed is False
+        assert transport_failed is True
 
     def test_empty_judge_reply_flagged_as_parse_failure(self):
         """End-to-end: judge returns empty content → parse_failed=True."""
@@ -450,7 +453,7 @@ def test_empty_judge_reply_flagged_as_parse_failure(self):
             "agent.auxiliary_client.call_llm",
             return_value=MagicMock(choices=[MagicMock(message=MagicMock(content=""))]),
         ):
-            verdict, _, parse_failed, _wd = goals.judge_goal("goal", "response")
+            verdict, _, parse_failed, _wd, _tf = goals.judge_goal("goal", "response")
         assert verdict == "continue"
         assert parse_failed is True
 
@@ -464,7 +467,7 @@ def test_auto_pause_after_three_consecutive_parse_failures(self, hermes_home):
         mgr.set("do a thing")
 
         with patch.object(
-            goals, "judge_goal", return_value=("continue", "judge returned empty response", True, None)
+            goals, "judge_goal", return_value=("continue", "judge returned empty response", True, None, False)
         ):
             d1 = mgr.evaluate_after_turn("step 1")
             assert d1["should_continue"] is True
@@ -493,7 +496,7 @@ def test_parse_failure_counter_resets_on_good_reply(self, hermes_home):
 
         # Two parse failures…
         with patch.object(
-            goals, "judge_goal", return_value=("continue", "not json", True, None)
+            goals, "judge_goal", return_value=("continue", "not json", True, None, False)
         ):
             mgr.evaluate_after_turn("step 1")
             mgr.evaluate_after_turn("step 2")
@@ -501,29 +504,50 @@ def test_parse_failure_counter_resets_on_good_reply(self, hermes_home):
 
         # …then one clean reply resets the counter.
         with patch.object(
-            goals, "judge_goal", return_value=("continue", "making progress", False, None)
+            goals, "judge_goal", return_value=("continue", "making progress", False, None, False)
         ):
             d = mgr.evaluate_after_turn("step 3")
             assert d["should_continue"] is True
             assert mgr.state.consecutive_parse_failures == 0
 
-    def test_parse_failure_counter_not_incremented_by_api_errors(self, hermes_home):
-        """API/transport errors must NOT count toward the auto-pause threshold."""
+    def test_transport_failures_do_not_increment_parse_counter(self, hermes_home):
+        """Transport failures use their own counter and a good reply resets both."""
         from hermes_cli import goals
         from hermes_cli.goals import GoalManager
 
         mgr = GoalManager(session_id="parse-fail-sid-3", default_max_turns=20)
         mgr.set("goal")
+        assert mgr.state is not None
 
         with patch.object(
-            goals, "judge_goal", return_value=("continue", "judge error: RuntimeError", False, None)
+            goals,
+            "judge_goal",
+            return_value=(
+                "continue",
+                "judge error: RuntimeError",
+                False,
+                None,
+                True,
+            ),
         ):
-            for _ in range(5):
+            for _ in range(2):
                 d = mgr.evaluate_after_turn("still going")
                 assert d["should_continue"] is True
             assert mgr.state.consecutive_parse_failures == 0
+            assert mgr.state.consecutive_transport_failures == 2
             assert mgr.state.status == "active"
 
+        with patch.object(
+            goals,
+            "judge_goal",
+            return_value=("continue", "making progress", False, None, False),
+        ):
+            d = mgr.evaluate_after_turn("recovered")
+
+        assert d["should_continue"] is True
+        assert mgr.state.consecutive_parse_failures == 0
+        assert mgr.state.consecutive_transport_failures == 0
+
     def test_consecutive_parse_failures_persists_across_goalmanager_reloads(
         self, hermes_home
     ):
@@ -535,7 +559,7 @@ def test_consecutive_parse_failures_persists_across_goalmanager_reloads(
         mgr.set("persistent goal")
 
         with patch.object(
-            goals, "judge_goal", return_value=("continue", "empty", True, None)
+            goals, "judge_goal", return_value=("continue", "empty", True, None, False)
         ):
             mgr.evaluate_after_turn("r")
             mgr.evaluate_after_turn("r")
@@ -732,7 +756,7 @@ def _fake_call_llm(**kwargs):
             return _FakeResp()
 
         with patch("agent.auxiliary_client.call_llm", side_effect=_fake_call_llm):
-            verdict, reason, parse_failed, _wd = goals.judge_goal(
+            verdict, reason, parse_failed, _wd, _tf = goals.judge_goal(
                 "ship the feature",
                 "ok shipped",
                 subgoals=["write tests", "update docs"],
@@ -837,7 +861,7 @@ def test_parked_on_live_pid_does_not_continue_or_judge(self, hermes_home):
             assert mgr.is_waiting() is True
 
             # The judge must NOT be called while parked, and no turn is burned.
-            judge = MagicMock(return_value=("continue", "x", False, None))
+            judge = MagicMock(return_value=("continue", "x", False, None, False))
             with patch.object(goals, "judge_goal", judge):
                 decision = mgr.evaluate_after_turn("still waiting on CI")
 
@@ -869,7 +893,7 @@ def test_barrier_auto_clears_when_process_exits_and_loop_resumes(self, hermes_ho
         assert mgr.is_waiting() is False  # lazy auto-clear
         assert mgr.state.waiting_on_pid is None
 
-        with patch.object(goals, "judge_goal", return_value=("continue", "more", False, None)):
+        with patch.object(goals, "judge_goal", return_value=("continue", "more", False, None, False)):
             decision = mgr.evaluate_after_turn("process finished, here are results")
 
         assert decision["verdict"] == "continue"
@@ -886,7 +910,7 @@ def test_dead_pid_never_parks(self, hermes_home):
         # is_waiting clears the stale barrier immediately.
         assert mgr.is_waiting() is False
 
-        with patch.object(goals, "judge_goal", return_value=("continue", "go", False, None)):
+        with patch.object(goals, "judge_goal", return_value=("continue", "go", False, None, False)):
             decision = mgr.evaluate_after_turn("response")
         assert decision["should_continue"] is True
 
@@ -986,7 +1010,7 @@ def test_judge_wait_pid_parks_loop(self, hermes_home):
             # Judge sees the running process and says wait-on-pid.
             with patch.object(
                 goals, "judge_goal",
-                return_value=("wait", "CI watcher still running", False, {"pid": proc.pid}),
+                return_value=("wait", "CI watcher still running", False, {"pid": proc.pid}, False),
             ):
                 decision = mgr.evaluate_after_turn(
                     "Pushed the PR, watching CI.",
@@ -1020,7 +1044,7 @@ def test_judge_wait_seconds_parks_loop(self, hermes_home):
         mgr.set("retry after backoff")
         with patch.object(
             goals, "judge_goal",
-            return_value=("wait", "rate limited", False, {"seconds": 120}),
+            return_value=("wait", "rate limited", False, {"seconds": 120}, False),
         ):
             decision = mgr.evaluate_after_turn("Hit a 429, backing off.")
         assert decision["verdict"] == "wait"
@@ -1050,7 +1074,7 @@ def test_continue_verdict_still_continues_with_background(self, hermes_home):
         mgr.set("do work")
         with patch.object(
             goals, "judge_goal",
-            return_value=("continue", "more to do", False, None),
+            return_value=("continue", "more to do", False, None, False),
         ):
             decision = mgr.evaluate_after_turn(
                 "made progress",
@@ -1118,7 +1142,7 @@ def test_goal_parks_on_session_and_releases_on_trigger(self, hermes_home):
         mgr.set("wait for the build to succeed")
         with patch.object(
             goals, "judge_goal",
-            return_value=("wait", "blocked on build", False, {"session_id": "proc_t4"}),
+            return_value=("wait", "blocked on build", False, {"session_id": "proc_t4"}, False),
         ):
             decision = mgr.evaluate_after_turn(
                 "Started the build watcher.",
@@ -1146,7 +1170,7 @@ def test_goal_parks_on_session_and_releases_on_trigger(self, hermes_home):
 
         # Loop resumes with a real judge verdict.
         with patch.object(goals, "judge_goal",
-                          return_value=("continue", "build done", False, None)):
+                          return_value=("continue", "build done", False, None, False)):
             d3 = mgr.evaluate_after_turn("build succeeded")
         assert d3["should_continue"] is True
 
@@ -1462,7 +1486,7 @@ def test_judge_prompt_carries_contract_and_background(self, hermes_home):
         }]
         with patch("agent.auxiliary_client.call_llm",
                    side_effect=self._capture_call_llm(captured)):
-            verdict, reason, parse_failed, wait_directive = goals.judge_goal(
+            verdict, reason, parse_failed, wait_directive, _tf = goals.judge_goal(
                 "ship the PR",
                 "I pushed and started the CI watcher; waiting on it now.",
                 contract=GoalContract(verification="PR CI goes green"),
@@ -1492,7 +1516,7 @@ def test_contract_goal_can_still_complete_on_evidence(self, hermes_home):
                        captured,
                        content='{"verdict": "done", "reason": "CI is green, evidence shown"}',
                    )):
-            verdict, reason, parse_failed, wait_directive = goals.judge_goal(
+            verdict, reason, parse_failed, wait_directive, _tf = goals.judge_goal(
                 "ship the PR",
                 "CI finished: 30 passed, 0 failed. Done.",
                 contract=GoalContract(verification="PR CI goes green"),

tests/hermes_cli/test_kanban_goal_mode.py

@@ -181,8 +181,8 @@ def _patch_judge(monkeypatch, verdicts):
 
     def _fake_judge(goal, response, subgoals=None, background_processes=None, **_kw):
         v = seq.pop(0) if seq else "done"
-        # 4-tuple contract: (verdict, reason, parse_failed, wait_directive)
-        return v, f"scripted:{v}", False, None
+        # 5-tuple contract: verdict, reason, parse failure, wait, transport failure.
+        return v, f"scripted:{v}", False, None, False
 
     monkeypatch.setattr(goals, "judge_goal", _fake_judge)
 
@@ -347,10 +347,10 @@ def fake_complete_task(conn, tid, **kw):
             lambda name: _aux_client,
         )
         # Match the real judge_goal contract:
-        # (verdict, reason, parse_failed, wait_directive)
+        # (verdict, reason, parse_failed, wait_directive, transport_failed)
         monkeypatch.setattr(
             "hermes_cli.goals.judge_goal",
-            lambda **kw: (verdict, reason, False, None),
+            lambda **kw: (verdict, reason, False, None, False),
         )
 
         args = argparse.Namespace(task_ids=["t1"], summary=summary, result=None, metadata=None)

tests/tools/test_kanban_tools.py

@@ -649,8 +649,8 @@ def test_complete_goal_mode_rejected_by_judge(monkeypatch, tmp_path):
     # judge is reachable, so force the availability probe True as well.
     def mock_judge_goal(goal, last_response, *, timeout=30.0, subgoals=None):
         # Match the real judge_goal contract:
-        # (verdict, reason, parse_failed, wait_directive)
-        return "continue", "missing verification evidence", False, None
+        # (verdict, reason, parse_failed, wait_directive, transport_failed)
+        return "continue", "missing verification evidence", False, None, False
 
     monkeypatch.setattr("tools.kanban_tools.judge_goal", mock_judge_goal)
     monkeypatch.setattr("tools.kanban_tools._goal_judge_available", lambda: True)

tools/kanban_tools.py

@@ -603,11 +603,11 @@ def _handle_complete(args: dict, **kw) -> str:
                 reason = ""
                 try:
                     # judge_goal returns (verdict, reason, parse_failed,
-                    # wait_directive) — see hermes_cli/goals.py. Unpacking
-                    # fewer raises ValueError, which the defensive handler
-                    # below swallows, leaving verdict="done" and silently
-                    # disabling the gate.
-                    verdict, reason, _, _ = judge_goal(
+                    # wait_directive, transport_failed) — see
+                    # hermes_cli/goals.py. Unpacking fewer raises ValueError,
+                    # which the defensive handler below swallows, leaving
+                    # verdict="done" and silently disabling the gate.
+                    verdict, reason, _, _, _ = judge_goal(
                         goal=f"{task.title}\n\n{task.body or ''}".strip(),
                         last_response=(summary or result or "").strip(),
                     )

2 PRs closed without merging

From the close comments: PR volume cleanup 1 · other 1

#27752 fix(goals): pause loop when judge errors and response is terminal (#27585) · PR volume cleanup
#27760 fix(goals): add consecutive API error auto-pause to prevent goal spam (#27585) · other → #67997, #54387

Closed issues · 1

Closed#27585[Bug]: /goal can spam repeated completion messages when goal_judge errors fail-open to continue

How this cluster was grouped

Link evidence (structural links throughout): every member is connected by structural evidence: shared changed files, or a closing/superseded reference.
Pairwise link density 0.50: the share of item pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36. 1.00 means every pair is linked.

Pick up a Lab task

Check whether a fix covers the open issues here, with your own agent. Bring back evidence for maintainers.

Open the Lab

At a glance

Subsystem: agent
Merged PRs: #67997
Closed issues: 1
Attempts: 2 PRs closed without merging
Fix link checked: 20 Jul 2026

For your agent

The same facts, typed.

cluster/27585.json
cluster/27585.md