Auxiliary and vision LLM timeout configuration issues

NousResearch/hermes-agent · cluster 2107 · agent

Data as of 2026-10-22 12:15 UTC

3 open items. Fix merged in #2480. Check it covers your case.

open issues

open PR

items grouped

Known?: Yes. 17 related items grouped here, 3 issues already closed.
Fix event: A fix was merged to main in PR #2480. Check it covers your case. No release data is tracked.
Tracker: The maintainers' tracker marks this unresolved. That is separate from the fix event.
Still open: 2 issues and 1 PR.

Next step: read #2480, then check whether your report is one of the 2 still open.

Read #2480 Pick up a Lab task

These issues and PRs concern default HTTP timeouts (often 30s) being too short for auxiliary LLM calls, vision analysis, and context compression when using slow or local model endpoints. The work spans bug reports, fixes, and feature requests for making these timeouts configurable and auto-adjusted for local inference.

Open issues2

Issue#8120[Bug]: vision_analyze frequently times out when analyzing local images in WSL2
Issue#21566Auxiliary LLM calls (session_search, skills_hub, etc.) use short 30s default timeout with no local-endpoint detection — causes retry storms on slow local inference

Open PRs1

PR#21569fix(agent): auto-bump auxiliary LLM timeout for local endpoints

Fixes on record

Merged to main. A stamp counts the issues a fix is linked to.

#2480 fix: make vision_analyze timeout configurable via config.yaml (salvage #2306)

merged to main

files: hermes_cli/config.py, tools/vision_tools.py, website/docs/user-guide/configuration.md

Show the diff

hermes_cli/config.py

@@ -182,6 +182,7 @@ def ensure_hermes_home():
             "model": "",           # e.g. "google/gemini-2.5-flash", "gpt-4o"
             "base_url": "",        # direct OpenAI-compatible endpoint (takes precedence over provider)
             "api_key": "",         # API key for base_url (falls back to OPENAI_API_KEY)
+            "timeout": 30,         # seconds — increase for slow local vision models
         },
         "web_extract": {
             "provider": "auto",

tools/vision_tools.py

@@ -298,12 +298,23 @@ async def vision_analyze_tool(
         
         logger.info("Processing image with vision model...")
         
-        # Call the vision API via centralized router
+        # Call the vision API via centralized router.
+        # Read timeout from config.yaml (auxiliary.vision.timeout), default 30s.
+        vision_timeout = 30.0
+        try:
+            from hermes_cli.config import load_config
+            _cfg = load_config()
+            _vt = _cfg.get("auxiliary", {}).get("vision", {}).get("timeout")
+            if _vt is not None:
+                vision_timeout = float(_vt)
+        except Exception:
+            pass
         call_kwargs = {
             "task": "vision",
             "messages": messages,
             "temperature": 0.1,
             "max_tokens": 2000,
+            "timeout": vision_timeout,
         }
         if model:
             call_kwargs["model"] = model

website/docs/user-guide/configuration.md

@@ -907,6 +907,7 @@ auxiliary:
     model: ""                  # e.g. "openai/gpt-4o", "google/gemini-2.5-flash"
     base_url: ""               # Custom OpenAI-compatible endpoint (overrides provider)
     api_key: ""                # API key for base_url (falls back to OPENAI_API_KEY)
+    timeout: 30                # seconds — increase for slow local vision models
 
   # Web page summarization + browser page text extraction
   web_extract:

#3597 feat(agent): configurable timeouts for auxiliary LLM calls via config.yaml (salvage #3406)

merged to main

files: agent/auxiliary_client.py, agent/context_compressor.py, agent/title_generator.py, hermes_cli/config.py

Show the diff

agent/auxiliary_client.py

@@ -1458,6 +1458,29 @@ def _resolve_task_provider_model(
     return "auto", resolved_model, None, None
 
 
+_DEFAULT_AUX_TIMEOUT = 30.0
+
+
+def _get_task_timeout(task: str, default: float = _DEFAULT_AUX_TIMEOUT) -> float:
+    """Read timeout from auxiliary.{task}.timeout in config, falling back to *default*."""
+    if not task:
+        return default
+    try:
+        from hermes_cli.config import load_config
+        config = load_config()
+    except ImportError:
+        return default
+    aux = config.get("auxiliary", {}) if isinstance(config, dict) else {}
+    task_config = aux.get(task, {}) if isinstance(aux, dict) else {}
+    raw = task_config.get("timeout")
+    if raw is not None:
+        try:
+            return float(raw)
+        except (ValueError, TypeError):
+            pass
+    return default
+
+
 def _build_call_kwargs(
     provider: str,
     model: str,
@@ -1515,7 +1538,7 @@ def call_llm(
     temperature: float = None,
     max_tokens: int = None,
     tools: list = None,
-    timeout: float = 30.0,
+    timeout: float = None,
     extra_body: dict = None,
 ) -> Any:
     """Centralized synchronous LLM call.
@@ -1533,7 +1556,7 @@ def call_llm(
         temperature: Sampling temperature (None = provider default).
         max_tokens: Max output tokens (handles max_tokens vs max_completion_tokens).
         tools: Tool definitions (for function calling).
-        timeout: Request timeout in seconds.
+        timeout: Request timeout in seconds (None = read from auxiliary.{task}.timeout config).
         extra_body: Additional request body fields.
 
     Returns:
@@ -1598,10 +1621,12 @@ def call_llm(
                 f"No LLM provider configured for task={task} provider={resolved_provider}. "
                 f"Run: hermes setup")
 
+    effective_timeout = timeout if timeout is not None else _get_task_timeout(task)
+
     kwargs = _build_call_kwargs(
         resolved_provider, final_model, messages,
         temperature=temperature, max_tokens=[redacted:secret-assignment]
-        tools=tools, timeout=timeout, extra_body=extra_body,
+        tools=tools, timeout=effective_timeout, extra_body=extra_body,
         base_url=resolved_base_url)
 
     # Handle max_tokens vs max_completion_tokens retry
@@ -1683,7 +1708,7 @@ async def async_call_llm(
     temperature: float = None,
     max_tokens: int = None,
     tools: list = None,
-    timeout: float = 30.0,
+    timeout: float = None,
     extra_body: dict = None,
 ) -> Any:
     """Centralized asynchronous LLM call.
@@ -1744,10 +1769,12 @@ async def async_call_llm(
                 f"No LLM provider configured for task={task} provider={resolved_provider}. "
                 f"Run: hermes setup")
 
+    effective_timeout = timeout if timeout is not None else _get_task_timeout(task)
+
     kwargs = _build_call_kwargs(
         resolved_provider, final_model, messages,
         temperature=temperature, max_tokens=[redacted:secret-assignment]
-        tools=tools, timeout=timeout, extra_body=extra_body,
+        tools=tools, timeout=effective_timeout, extra_body=extra_body,
         base_url=resolved_base_url)
 
     try:

agent/context_compressor.py

@@ -347,7 +347,7 @@ def _generate_summary(self, turns_to_summarize: List[Dict[str, Any]]) -> Optiona
                 "messages": [{"role": "user", "content": prompt}],
                 "temperature": 0.3,
                 "max_tokens": summary_budget * 2,
-                "timeout": 45.0,
+                # timeout resolved from auxiliary.compression.timeout config by call_llm
             }
             if self.summary_model:
                 call_kwargs["model"] = self.summary_model

agent/title_generator.py

@@ -19,7 +19,7 @@
 )
 
 
-def generate_title(user_message: str, assistant_response: str, timeout: float = 15.0) -> Optional[str]:
+def generate_title(user_message: str, assistant_response: str, timeout: float = 30.0) -> Optional[str]:
     """Generate a session title from the first exchange.
 
     Uses the auxiliary LLM client (cheapest/fastest available model).

hermes_cli/config.py

@@ -227,42 +227,49 @@ def ensure_hermes_home():
             "model": "",
             "base_url": "",
             "api_key": "",
+            "timeout": 30,         # seconds — increase for slow local models
         },
         "compression": {
             "provider": "auto",
             "model": "",
             "base_url": "",
             "api_key": "",
+            "timeout": 120,        # seconds — compression summarises large contexts; increase for local models
         },
         "session_search": {
             "provider": "auto",
             "model": "",
             "base_url": "",
             "api_key": "",
+            "timeout": 30,
         },
         "skills_hub": {
             "provider": "auto",
             "model": "",
             "base_url": "",
             "api_key": "",
+            "timeout": 30,
         },
         "approval": {
             "provider": "auto",
             "model": "",           # fast/cheap model recommended (e.g. gemini-flash, haiku)
             "base_url": "",
             "api_key": "",
+            "timeout": 30,
         },
         "mcp": {
             "provider": "auto",
             "model": "",
             "base_url": "",
             "api_key": "",
+            "timeout": 30,
         },
         "flush_memories": {
             "provider": "auto",
             "model": "",
             "base_url": "",
             "api_key": "",
+            "timeout": 30,
         },
     },

9 PRs closed without merging

From the close comments: done elsewhere 3 · merged via another change 2 · stale or abandoned 2 · duplicate 1 · other 1

#2126 fix(vision): make timeout configurable via HERMES_VISION_TIMEOUT_SECONDS · duplicate → #2306
#2257 fix(vision): increase LLM timeout for vision_analyze to support local models · other
#2306 fix: make vision_analyze timeout configurable · merged via another change → #2480
#3406 feat(agent): configurable timeouts for auxiliary LLM calls and context compression · merged via another change → #3597
#8168 fix: prevent vision_analyze timeouts on larger local images · done elsewhere

Show 4 more

#21575 fix(auxiliary): bump aux-call timeout for local endpoints (#21566) · stale or abandoned
#26483 fix(agent): forward fallback_providers[].extra_body during fallback activation (#26460) · stale or abandoned
#36258 fix(agent): add 300s timeout to all auxiliary OpenAI clients (#35517) · done elsewhere → 370a489
#129734 fix(vision): bound the aux vision payload before the request, not after a provider rejection · done elsewhere → fe52e62

Closed issues · 3

Closed#2107[Bug]: `vision_analyze` on a slow enough local model will timeout due to default 30s timout
Closed#3404[Feature]: Configurable timeouts for auxiliary call_llm and context compression
Closed#35517[Bug]: Auxiliary client HTTP timeout defaults to 30s, causing compression to fail with slow local LLMs

How this cluster was grouped

Link evidence (partly structural links): some members are connected by structural evidence (shared changed files, or a closing/superseded reference); the rest are joined by an automated relevance judgement over their text.
Members outside that structural core may belong to a different underlying problem.
Pairwise link density 0.10: the share of item pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36. 1.00 means every pair is linked.

Pick up a Lab task

Check whether a fix covers the open issues here, with your own agent. Bring back evidence for maintainers.

Open the Lab

At a glance

Subsystem: agent
Merged PRs: #2480, #3597
Closed issues: 3
Attempts: 9 PRs closed without merging
Fix link checked: 28 Mar 2026. This confirmation is old and may be out of date (a revert or reopen would not have invalidated it here); re-verify.

For your agent

The same facts, typed.

cluster/2107.json
cluster/2107.md