Timeout misclassification and fallback loop fixes
Cluster 22548. Data as of
Nothing open on GitHub. Fix merged in #22857 on 10 May 2026. Check it covers your case.
The short answer
- Known?
- Yes. 4 related items grouped here, 1 issue already closed.
- Fix event
- A fix was merged to main in PR #22857. Check it covers your case. No release data is tracked.
- Tracker
- The maintainers' tracker marks this
resolved, deciding PR #22780. That is separate from the fix event. - Still open
- Nothing open on GitHub.
These items address a bug where custom provider timeouts were misreported as empty responses and incorrectly triggered fallback loops, including fixes to error classification and fallback chain logic to prevent self-selection.
Open issues0
Open PRs0
Fixes on record
-
#22780 fix(fallback): skip chain entries matching current provider/model/base_url
files:
run_agent.py,tests/run_agent/test_provider_fallback.pyShow the diff
-
run_agent.py@@ -8042,6 +8042,32 @@ def _try_activate_fallback(self, reason: "FailoverReason | None" = None) -> bool if not fb_provider or not fb_model: return self._try_activate_fallback() # skip invalid, try next + # Skip entries that resolve to the current (provider, model) — falling + # back to the same backend that just failed loops the failure. Compare + # base_url too so two distinct custom_providers entries pointing at the + # same shim/proxy URL also dedup. See issue #22548. + current_provider = (getattr(self, "provider", "") or "").strip().lower() + current_model = (getattr(self, "model", "") or "").strip() + current_base_url = str(getattr(self, "base_url", "") or "").rstrip("/").lower() + fb_base_url_for_dedup = (fb.get("base_url") or "").strip().rstrip("/").lower() + if fb_provider == current_provider and fb_model == current_model: + logging.warning( + "Fallback skip: chain entry %s/%s matches current provider/model", + fb_provider, fb_model, + ) + return self._try_activate_fallback() + if ( + fb_base_url_for_dedup + and current_base_url + and fb_base_url_for_dedup == current_base_url + and fb_model == current_model + ): + logging.warning( + "Fallback skip: chain entry base_url %s matches current backend", + fb_base_url_for_dedup, + ) + return self._try_activate_fallback() + # Use centralized router for client construction. # raw_codex=True because the main agent needs direct responses.stream() # access for Codex providers. -
tests/run_agent/test_provider_fallback.py@@ -220,3 +220,88 @@ def test_multiple_credentials_all_in_cooldown_returns_false(self): def test_many_credentials_available_returns_true(self): assert _pool_may_recover_from_rate_limit(_pool(10)) is True + + +# ── Skip-self dedup (#22548) ─────────────────────────────────────────────── + + +class TestFallbackChainDedup: + """A fallback chain entry that resolves to the current provider/model + (or the same custom-provider base_url) must be skipped, not retried. + Otherwise a misconfigured chain or two custom_providers entries pointing + at the same shim loop the same failure. See issue #22548.""" + + def test_skips_entry_matching_current_provider_and_model(self): + """Chain has [same-as-current, real-fallback]; activate must skip + the first and use the second.""" + fbs = [ + # First entry == current state. Should be skipped. + {"provider": "openrouter", "model": "z-ai/glm-4.7"}, + # Second entry: real fallback. + {"provider": "zai", "model": "glm-4.7"}, + ] + agent = _make_agent(fallback_model=fbs) + agent.provider = "openrouter" + agent.model = "z-ai/glm-4.7" + agent.base_url = "https://openrouter.ai/api/v1" + + # Stub out resolve_provider_client so we can assert which entry was + # actually used — return a MagicMock client tagged with the provider. + called = [] + def _resolve(provider, model=None, raw_codex=False, **kwargs): + called.append((provider, model)) + return _mock_client(), model + with patch("agent.auxiliary_client.resolve_provider_client", side_effect=_resolve): + with patch("hermes_cli.model_normalize.normalize_model_for_provider", side_effect=lambda m, p: m): + ok = agent._try_activate_fallback() + + assert ok is True + # The first entry was skipped — only the second reached resolve. + assert called == [("zai", "glm-4.7")], ( + f"expected fallback to skip same-state entry, got call order: {called}" + ) + + def test_skips_entry_matching_current_base_url_and_model(self): + """Two custom_providers entries pointing at the same shim URL + with the same model should dedup even if their provider names differ.""" + fbs = [ + # Different provider name but same shim URL + model — same backend. + {"provider": "claude-cli-alt", "model": "claude-opus-4.7", + "base_url": "http://127.0.0.1:7891/v1"}, + # Real different fallback. + {"provider": "openrouter", "model": "anthropic/claude-opus-4.7"}, + ] + agent = _make_agent(fallback_model=fbs) + agent.provider = "claude-cli" + agent.model = "claude-opus-4.7" + agent.base_url = "http://127.0.0.1:7891/v1" + + called = [] + def _resolve(provider, model=None, raw_codex=False, **kwargs): + called.append((provider, model)) + return _mock_client(), model + with patch("agent.auxiliary_client.resolve_provider_client", side_effect=_resolve): + with patch("hermes_cli.model_normalize.normalize_model_for_provider", side_effect=lambda m, p: m): + ok = agent._try_activate_fallback() + + assert ok is True + # Same shim/base_url+model entry skipped, second one used. + assert called == [("openrouter", "anthropic/claude-opus-4.7")], ( + f"expected base_url-aware dedup, got call order: {called}" + ) + + def test_returns_false_when_only_self_matching_entries(self): + """A chain with only self-matching entries exhausts to False.""" + fbs = [ + {"provider": "openrouter", "model": "z-ai/glm-4.7"}, + ] + agent = _make_agent(fallback_model=fbs) + agent.provider = "openrouter" + agent.model = "z-ai/glm-4.7" + agent.base_url = "https://openrouter.ai/api/v1" + + with patch("agent.auxiliary_client.resolve_provider_client") as mock_resolve: + ok = agent._try_activate_fallback() + + assert ok is False + mock_resolve.assert_not_called()
-
-
#22857 fix(error_classifier): classify generic-typed timeout messages as transient (carve-out of #22664)
files:
agent/error_classifier.py,tests/agent/test_error_classifier.py
1 PR closed without merging
From the close comments: merged via another change 1
-
#22664 fix(error_classifier,fallback): classify custom-provider timeouts correctly, prevent self-selection fallback loop · merged via another change →
#22857,#22780,#22643,#22798
Closed issues · 1
- Closed#22548[Bug]: Local claude-cli custom provider timeout is reported as Empty response and fallback loops
How this cluster was grouped
-
Link evidence (
structural links throughout): every member is connected by structural evidence: shared changed files, or a closing/superseded reference. -
Pairwise link density
0.50: the share of item pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36.1.00means every pair is linked.
Pick up a Lab task
Check whether a fix covers the open issues here, with your own agent. Bring back evidence for maintainers.
Open the Lab