# NousResearch/hermes-agent · Auxiliary and vision LLM timeout configuration issues

## Open now

- 2 issues, 1 PR open on GitHub.
- #8120: \[Bug\]: vision_analyze frequently times out when analyzing local images in WSL2
- #21566: Auxiliary LLM calls (session_search, skills_hub, etc.) use short 30s default timeout with no local-endpoint detection — causes retry storms on slow local inference
- PR #21569: fix(agent): auto-bump auxiliary LLM timeout for local endpoints

These issues and PRs concern default HTTP timeouts (often 30s) being too short for auxiliary LLM calls, vision analysis, and context compression when using slow or local model endpoints. The work spans bug reports, fixes, and feature requests for making these timeouts configurable and auto-adjusted for local inference.

## TL;DR

- 17 member(s): 2 open issue(s), 3 closed issue(s), 1 open PR(s), 2 merged PR(s), 9 PR(s) closed without merging.
- A fix was merged to main in PR #2480. Verify it addresses your case.
- **Recommended:** read PR #2480 (and any competing PRs below) before filing. Confirm the merged change covers your symptom.

## Maintainer resolution verdict

- Internal-tracker verdict over this cluster's issue members: `unresolved`. This is distinct from the fix status above. Weigh it yourself.

## How this cluster was grouped

- Link evidence (`STRUCTURAL_PARTIAL`): some members are connected by structural evidence (shared changed files, or a closing/superseded reference); the rest are joined by an automated relevance judgement over their text.
- Members outside that structural core may belong to a different underlying problem.
- Pairwise link density `0.10`: the share of member pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36. `1.00` means every pair is linked.

## Open related issues

- #8120: \[Bug\]: vision_analyze frequently times out when analyzing local images in WSL2
- #21566: Auxiliary LLM calls (session_search, skills_hub, etc.) use short 30s default timeout with no local-endpoint detection — causes retry storms on slow local inference

## Closed / duplicate issues

- #2107: \[Bug\]: \`vision_analyze\` on a slow enough local model will timeout due to default 30s timout
- #3404: \[Feature\]: Configurable timeouts for auxiliary call_llm and context compression
- #35517: \[Bug\]: Auxiliary client HTTP timeout defaults to 30s, causing compression to fail with slow local LLMs

## Prior & competing PRs

- **#2480: fix: make vision_analyze timeout configurable via config.yaml (salvage #2306)**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`; implication `superseded`
  - files: `hermes_cli/config.py`, `tools/vision_tools.py`, `website/docs/user-guide/configuration.md`
  - fix hunks:
  - `hermes_cli/config.py`

    ```diff
    @@ -182,6 +182,7 @@ def ensure_hermes_home():
                 "model": "",           # e.g. "google/gemini-2.5-flash", "gpt-4o"
                 "base_url": "",        # direct OpenAI-compatible endpoint (takes precedence over provider)
                 "api_key": "",         # API key for base_url (falls back to OPENAI_API_KEY)
    +            "timeout": 30,         # seconds — increase for slow local vision models
             },
             "web_extract": {
                 "provider": "auto",
    ```
  - `tools/vision_tools.py`

    ```diff
    @@ -298,12 +298,23 @@ async def vision_analyze_tool(
             
             logger.info("Processing image with vision model...")
             
    -        # Call the vision API via centralized router
    +        # Call the vision API via centralized router.
    +        # Read timeout from config.yaml (auxiliary.vision.timeout), default 30s.
    +        vision_timeout = 30.0
    +        try:
    +            from hermes_cli.config import load_config
    +            _cfg = load_config()
    +            _vt = _cfg.get("auxiliary", {}).get("vision", {}).get("timeout")
    +            if _vt is not None:
    +                vision_timeout = float(_vt)
    +        except Exception:
    +            pass
             call_kwargs = {
                 "task": "vision",
                 "messages": messages,
                 "temperature": 0.1,
                 "max_tokens": 2000,
    +            "timeout": vision_timeout,
             }
             if model:
                 call_kwargs["model"] = model
    ```
  - `website/docs/user-guide/configuration.md`

    ```diff
    @@ -907,6 +907,7 @@ auxiliary:
         model: ""                  # e.g. "openai/gpt-4o", "google/gemini-2.5-flash"
         base_url: ""               # Custom OpenAI-compatible endpoint (overrides provider)
         api_key: ""                # API key for base_url (falls back to OPENAI_API_KEY)
    +    timeout: 30                # seconds — increase for slow local vision models
     
       # Web page summarization + browser page text extraction
       web_extract:
    ```
- **#3597: feat(agent): configurable timeouts for auxiliary LLM calls via config.yaml (salvage #3406)**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`; implication `superseded`
  - files: `agent/auxiliary_client.py`, `agent/context_compressor.py`, `agent/title_generator.py`, `hermes_cli/config.py`
  - fix hunks:
  - `agent/auxiliary_client.py`

    ```diff
    @@ -1458,6 +1458,29 @@ def _resolve_task_provider_model(
         return "auto", resolved_model, None, None
     
     
    +_DEFAULT_AUX_TIMEOUT = 30.0
    +
    +
    +def _get_task_timeout(task: str, default: float = _DEFAULT_AUX_TIMEOUT) -> float:
    +    """Read timeout from auxiliary.{task}.timeout in config, falling back to *default*."""
    +    if not task:
    +        return default
    +    try:
    +        from hermes_cli.config import load_config
    +        config = load_config()
    +    except ImportError:
    +        return default
    +    aux = config.get("auxiliary", {}) if isinstance(config, dict) else {}
    +    task_config = aux.get(task, {}) if isinstance(aux, dict) else {}
    +    raw = task_config.get("timeout")
    +    if raw is not None:
    +        try:
    +            return float(raw)
    +        except (ValueError, TypeError):
    +            pass
    +    return default
    +
    +
     def _build_call_kwargs(
         provider: str,
         model: str,
    ```

    ```diff
    @@ -1515,7 +1538,7 @@ def call_llm(
         temperature: float = None,
         max_tokens: int = None,
         tools: list = None,
    -    timeout: float = 30.0,
    +    timeout: float = None,
         extra_body: dict = None,
     ) -> Any:
         """Centralized synchronous LLM call.
    ```

    ```diff
    @@ -1533,7 +1556,7 @@ def call_llm(
             temperature: Sampling temperature (None = provider default).
             max_tokens: Max output tokens (handles max_tokens vs max_completion_tokens).
             tools: Tool definitions (for function calling).
    -        timeout: Request timeout in seconds.
    +        timeout: Request timeout in seconds (None = read from auxiliary.{task}.timeout config).
             extra_body: Additional request body fields.
     
         Returns:
    ```

    ```diff
    @@ -1598,10 +1621,12 @@ def call_llm(
                     f"No LLM provider configured for task={task} provider={resolved_provider}. "
                     f"Run: hermes setup")
     
    +    effective_timeout = timeout if timeout is not None else _get_task_timeout(task)
    +
         kwargs = _build_call_kwargs(
             resolved_provider, final_model, messages,
             temperature=temperature, max_tokens=[redacted:secret-assignment]
    -        tools=tools, timeout=timeout, extra_body=extra_body,
    +        tools=tools, timeout=effective_timeout, extra_body=extra_body,
             base_url=resolved_base_url)
     
         # Handle max_tokens vs max_completion_tokens retry
    ```

    ```diff
    @@ -1683,7 +1708,7 @@ async def async_call_llm(
         temperature: float = None,
         max_tokens: int = None,
         tools: list = None,
    -    timeout: float = 30.0,
    +    timeout: float = None,
         extra_body: dict = None,
     ) -> Any:
         """Centralized asynchronous LLM call.
    ```

    ```diff
    @@ -1744,10 +1769,12 @@ async def async_call_llm(
                     f"No LLM provider configured for task={task} provider={resolved_provider}. "
                     f"Run: hermes setup")
     
    +    effective_timeout = timeout if timeout is not None else _get_task_timeout(task)
    +
         kwargs = _build_call_kwargs(
             resolved_provider, final_model, messages,
             temperature=temperature, max_tokens=[redacted:secret-assignment]
    -        tools=tools, timeout=timeout, extra_body=extra_body,
    +        tools=tools, timeout=effective_timeout, extra_body=extra_body,
             base_url=resolved_base_url)
     
         try:
    ```
  - `agent/context_compressor.py`

    ```diff
    @@ -347,7 +347,7 @@ def _generate_summary(self, turns_to_summarize: List[Dict[str, Any]]) -> Optiona
                     "messages": [{"role": "user", "content": prompt}],
                     "temperature": 0.3,
                     "max_tokens": summary_budget * 2,
    -                "timeout": 45.0,
    +                # timeout resolved from auxiliary.compression.timeout config by call_llm
                 }
                 if self.summary_model:
                     call_kwargs["model"] = self.summary_model
    ```
  - `agent/title_generator.py`

    ```diff
    @@ -19,7 +19,7 @@
     )
     
     
    -def generate_title(user_message: str, assistant_response: str, timeout: float = 15.0) -> Optional[str]:
    +def generate_title(user_message: str, assistant_response: str, timeout: float = 30.0) -> Optional[str]:
         """Generate a session title from the first exchange.
     
         Uses the auxiliary LLM client (cheapest/fastest available model).
    ```
  - `hermes_cli/config.py`

    ```diff
    @@ -227,42 +227,49 @@ def ensure_hermes_home():
                 "model": "",
                 "base_url": "",
                 "api_key": "",
    +            "timeout": 30,         # seconds — increase for slow local models
             },
             "compression": {
                 "provider": "auto",
                 "model": "",
                 "base_url": "",
                 "api_key": "",
    +            "timeout": 120,        # seconds — compression summarises large contexts; increase for local models
             },
             "session_search": {
                 "provider": "auto",
                 "model": "",
                 "base_url": "",
                 "api_key": "",
    +            "timeout": 30,
             },
             "skills_hub": {
                 "provider": "auto",
                 "model": "",
                 "base_url": "",
                 "api_key": "",
    +            "timeout": 30,
             },
             "approval": {
                 "provider": "auto",
                 "model": "",           # fast/cheap model recommended (e.g. gemini-flash, haiku)
                 "base_url": "",
                 "api_key": "",
    +            "timeout": 30,
             },
             "mcp": {
                 "provider": "auto",
                 "model": "",
                 "base_url": "",
                 "api_key": "",
    +            "timeout": 30,
             },
             "flush_memories": {
                 "provider": "auto",
                 "model": "",
                 "base_url": "",
                 "api_key": "",
    +            "timeout": 30,
             },
         },
         
    ```
- **#2126: fix(vision): make timeout configurable via HERMES_VISION_TIMEOUT_SECONDS**. closed without merge
  - maintainer direction: category `duplicate_of`; implication `duplicate`; refs `#2306`
- **#2257: fix(vision): increase LLM timeout for vision_analyze to support local models**. closed without merge
  - maintainer direction: category `uncategorized`; implication `needs_llm`
- **#2306: fix: make vision_analyze timeout configurable**. closed without merge
  - maintainer direction: category `merged_via`; implication `superseded`; refs `#2480`
- **#3406: feat(agent): configurable timeouts for auxiliary LLM calls and context compression**. closed without merge
  - maintainer direction: category `merged_via`; implication `superseded`; refs `#3597`
- **#8168: fix: prevent vision_analyze timeouts on larger local images**. closed without merge
  - maintainer direction: category `implemented_elsewhere`; implication `superseded`
- **#21569: fix(agent): auto-bump auxiliary LLM timeout for local endpoints**. open
  - maintainer direction: category `uncategorized`; implication `needs_llm`; refs `#21575`, `#21566`
  - files: `agent/auxiliary_client.py`, `gateway/platforms/whatsapp.py`, `tests/agent/test_auxiliary_client.py`, `tests/tools/test_file_tools.py`, `tools/file_tools.py`
- **#21575: fix(auxiliary): bump aux-call timeout for local endpoints (#21566)**. closed without merge
  - maintainer direction: category `stale_abandoned`; implication `active`
- **#26483: fix(agent): forward fallback_providers\[\].extra_body during fallback activation (#26460)**. closed without merge
  - maintainer direction: category `stale_abandoned`; implication `active`
- **#36258: fix(agent): add 300s timeout to all auxiliary OpenAI clients (#35517)**. closed without merge
  - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `370a489fb4adb7268f6510439e25e8213bddb624`
- **#129734: fix(vision): bound the aux vision payload before the request, not after a provider rejection**. closed without merge
  - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `fe52e6272cac`

## Fix status

- Merged to main in PR #2480 (cited above). No release data is tracked. Verify downstream.
- Fix last verified (merged) `2026-03-28`. This confirmation is old and may be out of date (a revert/reopen would not have invalidated it here); re-verify.
- Subsystem: `agent`