# NousResearch/hermes-agent · Context length config override and detection fixes

## Open now

- 3 issues, 2 PRs open on GitHub.
- #22930: \[Setup\]: For some small 2b、4b、8B、14B parameter models, how can the Hermes agent be run offline when the computing power resources are insufficient?
- #32048: \[Feature\]: token context window is too high for small offline users as well as small LLM model APIs
- #52595: Model picker can save Grok Imagine models that fail Hermes 64K context minimum
- PR #25464: fix: handle stale low context metadata for curated models
- PR #52627: fix(cli): reject xai media models as main model

These issues and fixes address problems where the configured or API-reported context length was ignored in favor of hardcoded minimums or stale metadata, particularly affecting small models, Kimi variants, and Ollama-hosted models. The work spans bug reports, config override enforcement, and improved fallback resolution for context window values.

## TL;DR

- 25 member(s): 3 open issue(s), 8 closed issue(s), 2 open PR(s), 4 merged PR(s), 8 PR(s) closed without merging.
- A fix was merged to main in PR #23980. Verify it addresses your case.
- **Recommended:** read PR #23980 (and any competing PRs below) before filing. Confirm the merged change covers your symptom.

## Maintainer resolution verdict

- Internal-tracker verdict over this cluster's issue members: `partially_resolved` · deciding PR(s): `pr-23980`. This is distinct from the fix status above. Weigh it yourself.

## How this cluster was grouped

- Link evidence (`STRUCTURAL_PARTIAL`): some members are connected by structural evidence (shared changed files, or a closing/superseded reference); the rest are joined by an automated relevance judgement over their text.
- Members outside that structural core may belong to a different underlying problem.
- Pairwise link density `0.07`: the share of member pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36. `1.00` means every pair is linked.

## Open related issues

- #22930: \[Setup\]: For some small 2b、4b、8B、14B parameter models, how can the Hermes agent be run offline when the computing power resources are insufficient?
- #32048: \[Feature\]: token context window is too high for small offline users as well as small LLM model APIs
- #52595: Model picker can save Grok Imagine models that fail Hermes 64K context minimum

## Closed / duplicate issues

- #8430: \[Bug\]: ## ## 问题描述 当使用上下文窗口大小小于 64,000 tokens 的模型时，即使在 config.yaml 中设置了 \`model.context_length\`，仍然会收到错误：
- #10449: \[Bug\]: Local model context setting
- #23949: kimi-k2.6 on Ollama Cloud detected as 32K context despite API reporting 256K
- #24000: provider: nous falls back to 32,768-token context, blocking boot with model.context_length workaround required
- #24119: get_model_context_length() unconditionally queries OpenRouter metadata even for non-OpenRouter users, causing false "below minimum 64K" rejections
- #24140: All models rejected with "context window below minimum 64,000 tokens" — Telegram completely down
- #24268: Wrong context length for kimi-k2.6 family: OpenRouter returns 32K, overrides correct hardcoded 256K default
- #24427: \[Bug\]: discord-kimi-context-bug-issue

## Prior & competing PRs

- **#23980: fix: correct context-length resolution for kimi-k2.6 on Ollama Cloud and Kimi Coding**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`; implication `superseded`
  - files: `agent/model_metadata.py`, `agent/models_dev.py`, `scripts/release.py`
  - fix hunks:
  - `agent/model_metadata.py`

    ```diff
    @@ -1006,6 +1006,79 @@ def query_ollama_num_ctx(model: str, base_url: str, api_key: str = "") -> Option
         return None
     
     
    +def _query_ollama_api_show(model: str, base_url: str, api_key: str = "") -> Optional[int]:
    +    """Query an Ollama server's native ``/api/show`` for context length.
    +
    +    Provider-agnostic: works against ANY Ollama-compatible server regardless
    +    of hostname — local Ollama, Ollama Cloud (``ollama.com``), custom Ollama
    +    hosting behind a reverse proxy, etc.  For non-Ollama servers the POST
    +    returns 404/405 quickly; the function handles errors gracefully.
    +
    +    For hosted servers the GGUF ``model_info.*.context_length`` is the
    +    authoritative source: the user can't set their own ``num_ctx``, and the
    +    OpenAI-compat ``/v1/models`` endpoint correctly omits ``context_length``
    +    per the OpenAI schema.
    +
    +    Resolution order for hosted Ollama:
    +      1. ``model_info.*.context_length`` — GGUF training max (authoritative)
    +      2. ``parameters`` → ``num_ctx`` — server-side Modelfile override
    +    The order is flipped vs ``query_ollama_num_ctx()`` because local users
    +    control ``num_ctx`` themselves; hosted users can't.
    +    """
    +    import httpx
    +
    +    server_url = base_url.rstrip("/")
    +    if server_url.endswith("/v1"):
    +        server_url = server_url[:-3]
    +
    +    headers = _auth_headers(api_key)
    +
    +    try:
    +        with httpx.Client(timeout=5.0, headers=headers) as client:
    +            resp = client.post(f"{server_url}/api/show", json={"name": model})
    +            if resp.status_code != 200:
    +                return None
    +            data = resp.json()
    +
    +            # Hosted Ollama: GGUF model_info is the real max — prefer it over
    +            # num_ctx which the Cloud operator may have capped arbitrarily.
    +            model_info = data.get("model_info", {})
    +            for key, value in model_info.items():
    +                if "context_length" in key and isinstance(value, (int, float)):
    +                    ctx = int(value)
    +                    if ctx >= 1024:
    +                        return ctx
    +
    +            # Fall back to num_ctx from Modelfile parameters (rare on Cloud)
    +            params = data.get("parameters", "")
    +            if "num_ctx" in params:
    +                for line in params.split("\n"):
    +                    if "num_ctx" in line:
    +                        parts = line.strip().split()
    +                        if len(parts) >= 2:
    +                            try:
    +                                ctx = int(parts[-1])
    +                                if ctx >= 1024:
    +                                    return ctx
    +                            except ValueError:
    +                                pass
    +    except Exception:
    +        pass
    +    return None
    +
    +
    +def _model_name_suggests_kimi(model: str) -> bool:
    +    """Return True if the model name looks like a Kimi-family model.
    +
    +    Catches ``kimi-k2.6``, ``kimi-k2.5``, ``kimi-k2-thinking``,
    +    ``moonshotai/Kimi-K2.6``, and similar variants.  Used as a guard
    +    against stale OpenRouter metadata that underreports these models
    +    as 32K context when they actually support 262K+.
    +    """
    +    lower = model.lower()
    +    return lower.startswith("kimi") or "moonshot" in lower
    +
    +
     def _query_local_context_length(model: str, base_url: str, api_key: str = "") -> Optional[int]:
         """Query a local server for the model's context length."""
         import httpx
    ```

    ```diff
    @@ -1307,12 +1380,17 @@ def get_model_context_length(
         2. Active endpoint metadata (/models for explicit custom endpoints)
         3. Local server query (for local endpoints)
         4. Anthropic /v1/models API (API-key users only, not OAuth)
    -    5. OpenRouter live API metadata
    -    6. Nous suffix-match via OpenRouter cache
    -    7. models.dev registry lookup (provider-aware)
    -    8. Thin hardcoded defaults (broad family patterns)
    -    9. Default fallback (256K)
    -    """
    +    5. Provider-aware lookups (before generic OpenRouter cache):
    +       a. Copilot live /models API
    +       b. Nous suffix-match via OpenRouter cache
    +       c. Codex OAuth /models probe
    +       d. GMI /models endpoint
    +       e. Ollama native /api/show probe (any base_url, provider-agnostic)
    +       f. models.dev registry lookup (with :cloud/-cloud suffix fallback)
    +    6. OpenRouter live API metadata (Kimi-family 32k guard)
    +    7. Hardcoded defaults (broad family patterns, longest-key-first)
    +    8. Local server query (last resort)
    +    9. Default fallback (256K)"""
         # 0. Explicit config override — user knows best
         if config_context_length is not None and isinstance(config_context_length, int) and config_context_length > 0:
             return config_context_length
    ```

    ```diff
    @@ -1392,6 +1470,13 @@ def get_model_context_length(
             if context_length is not None:
                 return context_length
             if not _is_known_provider_base_url(base_url):
    +            # 2b. Ollama native /api/show — any URL might be an Ollama server
    +            # (local, cloud, or custom hosting).  Non-Ollama servers return
    +            # 404/405 quickly.  Fall through on failure.
    +            ctx = _query_ollama_api_show(model, base_url, api_key=[redacted:secret-assignment]
    +            if ctx is not None:
    +                save_context_length(model, base_url, ctx)
    +                return ctx
                 # 3. Try querying local server directly
                 if is_local_endpoint(base_url):
                     local_ctx = _query_local_context_length(model, base_url, api_key=[redacted:secret-assignment]
    ```

    ```diff
    @@ -1461,16 +1546,53 @@ def get_model_context_length(
             ctx = _resolve_endpoint_context_length(model, base_url, api_key=[redacted:secret-assignment]
             if ctx is not None:
                 return ctx
    +    # 5e. Ollama native /api/show probe — runs for ANY provider with a
    +    # base_url, not just ollama-cloud.  Ollama-compatible servers expose
    +    # this endpoint regardless of hostname (local Ollama, Ollama Cloud,
    +    # custom Ollama hosting).  The OpenAI-compat /v1/models endpoint
    +    # correctly omits context_length per the OpenAI schema, but /api/show
    +    # returns the authoritative GGUF model_info.context_length.
    +    # For non-Ollama servers (OpenAI, Anthropic, etc.), the POST returns
    +    # 404/405 quickly.  Results are cached, so the hit is per-model+URL,
    +    # once per hour.
    +    if base_url:
    +        ctx = _query_ollama_api_show(model, base_url, api_key=[redacted:secret-assignment]
    +        if ctx is not None:
    +            save_context_length(model, base_url, ctx)
    +            return ctx
         if effective_provider:
             from agent.models_dev import lookup_models_dev_context
             ctx = lookup_models_dev_context(effective_provider, model)
             if ctx:
                 return ctx
     
    -    # 6. OpenRouter live API metadata (provider-unaware fallback)
    -    metadata = fetch_model_metadata()
    -    if model in metadata:
    -        return metadata[model].get("context_length", DEFAULT_FALLBACK_CONTEXT)
    +    # 6. OpenRouter live API metadata — provider-unaware fallback.
    +    # Only consulted when the provider is unknown (no effective_provider),
    +    # because OpenRouter data is community-maintained and can be incorrect
    +    # for models that belong to known providers with curated defaults.
    +    if not effective_provider:
    +        metadata = fetch_model_metadata()
    +        if model in metadata:
    +            or_ctx = metadata[model].get("context_length", DEFAULT_FALLBACK_CONTEXT)
    +            # Guard against stale OpenRouter metadata for Kimi-family models.
    +            # OpenRouter reports 32768 for moonshotai/kimi-k2.6, but the model
    +            # actually supports 262144 (models.dev + official Kimi docs agree).
    +            # Providers that host their own Kimi endpoints (Ollama Cloud, Kimi
    +            # Coding, Moonshot) would otherwise trip the 64k minimum-context
    +            # guard and reject a perfectly capable model.
    +            # The filter is narrow: only reject exactly 32768 for Kimi-named
    +            # models.  If OpenRouter ever updates its data, the stale path
    +            # becomes dead code with no impact.
    +            if or_ctx == 32768 and _model_name_suggests_kimi(model):
    +                logger.info(
    +                    "Rejecting OpenRouter metadata context=%s for %r "
    +                    "(Kimi-family underreport); falling through to hardcoded defaults",
    +                    or_ctx, model,
    +                )
    +            else:
    +                return or_ctx
    +
    +    # 7. (reserved)
     
         # 8. Hardcoded defaults (fuzzy match — longest key first for specificity)
         # Only check `default_model in model` (is the key a substring of the input).
    ```
  - `agent/models_dev.py`

    ```diff
    @@ -145,7 +145,9 @@ class ProviderInfo:
         "openai": "openai",
         "openai-codex": "openai",
         "zai": "zai",
    +    "kimi": "kimi-for-coding",
         "kimi-coding": "kimi-for-coding",
    +    "moonshot": "kimi-for-coding",
         "stepfun": "stepfun",
         "kimi-coding-cn": "kimi-for-coding",
         "minimax": "minimax",
    ```

    ```diff
    @@ -347,6 +349,28 @@ def lookup_models_dev_context(provider: str, model: str) -> Optional[int]:
                 if ctx:
                     return ctx
     
    +    # Suffix-aware fallback: some providers (e.g. ollama-cloud) store
    +    # model IDs with :cloud / -cloud suffixes in models.dev while the
    +    # live API returns bare names.  Without this, kimi-k2.6 misses the
    +    # kimi-k2.6:cloud entry and falls through to stale OpenRouter metadata
    +    # reporting 32768 — tripping the 64k minimum-context guard.
    +    # The suffix-stripping in fetch_ollama_cloud_models() handles the
    +    # model-picker UX; this handles the context-length lookup path.
    +    for suffix in (":cloud", "-cloud"):
    +        suffixed_key = model + suffix
    +        entry = models.get(suffixed_key)
    +        if entry:
    +            ctx = _extract_context(entry)
    +            if ctx:
    +                return ctx
    +        # Also try case-insensitive
    +        suffixed_lower = model_lower + suffix
    +        for mid, mdata in models.items():
    +            if mid.lower() == suffixed_lower:
    +                ctx = _extract_context(mdata)
    +                if ctx:
    +                    return ctx
    +
         return None
     
     
    ```
  - `scripts/release.py`

    ```diff
    @@ -88,6 +88,7 @@
         "[redacted:email]": "Qwinty",
         "[redacted:email]": "CalmProton",
         "[redacted:email]": "mbac",
    +    "[redacted:email]": "nicoechaniz",
         "[redacted:email]": "Ninso112",
         "[redacted:email]": "wesleysimplicio",
         "[redacted:email]": "SiliconID",
    ```
- **#24156: fix: always consult OpenRouter metadata as last-resort fallback for context length**. closed without merge · fix tier `suggested`
  - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `#24268`
  - _suggested-tier fix detail withheld; re-request with `?tier=all` to include it._
- **#24502: Use nous portal as model metadata authority**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`; implication `superseded`
  - files: `agent/model_metadata.py`, `tests/agent/test_model_metadata.py`
- **#42986: fix(models): read OpenRouter live context_length before hardcoded catch-all**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`; implication `superseded`
  - files: `agent/model_metadata.py`, `tests/agent/test_model_metadata.py`
- **#54636: fix(agent): respect model.context_length config override for minimum check (#8430)**. closed without merge · fix tier `suggested`
  - maintainer direction: category `uncategorized`; implication `needs_llm`; refs `#8430`, `#8430`
- **#55231: fix: honor explicit context_length config override for small-context models**. closed without merge · fix tier `suggested`
  - maintainer direction: category `uncategorized`; implication `needs_llm`; refs `#8430`
- **#104159: OpenRouter provider pins can now be set per model (provider_routing.models, #24495 / #100711)**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`; implication `superseded`
  - files: `agent/chat_completion_helpers.py`, `agent/turn_recovery.py`, `cli-config.yaml.example`, `hermes_constants.py`, `plugins/model-providers/openrouter/__init__.py`, `tests/agent/test_per_model_provider_routing.py`, `website/docs/integrations/providers.md`, `website/docs/user-guide/features/provider-routing.md`
- **#8590: fix(model): respect explicit context_length override**. closed without merge
  - maintainer direction: category `duplicate_of`; implication `duplicate`; refs `#9142`, `#9142`
- **#9142: fix(config): respect model.context_length override for sub-64K models**. closed without merge
  - maintainer direction: category `wontfix_bydesign`; implication `wontfix`; refs `#53569`
- **#24066: fix(model-metadata): extend Kimi 32k guard to Nous OpenRouter suffix-match**. closed without merge
  - maintainer direction: category `superseded_by_pr`; implication `superseded`; refs `528bba673`, `528bba673`, `#24502`
- **#24441: fix(model-metadata): add explicit context length for kimi-k2.6 family**. closed without merge
  - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `967c325da8a04e9722794e0d2b01dd79659d7127`, `#42986`
- **#25464: fix: handle stale low context metadata for curated models**. open
  - maintainer direction: category `uncategorized`; implication `needs_llm`
  - files: `agent/model_metadata.py`, `tests/agent/test_model_metadata.py`
- **#37548: fix(agent): respect model.context_length config**. closed without merge
  - maintainer direction: category `wontfix_bydesign`; implication `wontfix`; refs `#53569`
- **#52627: fix(cli): reject xai media models as main model**. open
  - maintainer direction: category `uncategorized`; implication `needs_llm`
  - files: `agent/model_metadata.py`, `hermes_cli/model_switch.py`, `hermes_cli/models.py`, `tests/hermes_cli/test_xai_main_model_context_floor.py`

## Fix status

- Merged to main in PR #23980 (cited above). No release data is tracked. Verify downstream.
- Fix last verified (merged) `2026-09-06`.
- Subsystem: `agent`