Ollama reasoning/thinking control and propagation fixes
NousResearch/hermes-agent · cluster 3191 · agent
Data as of
3 open items. Fix merged in #10782. Check it covers your case.
The short answer
- Known?
- Yes. 11 related items grouped here, 2 issues already closed.
- Fix event
- A fix was merged to main in PR #10782. Check it covers your case. No release data is tracked.
- Tracker
- The maintainers' tracker marks this
partially resolved, deciding PR #10782. That is separate from the fix event. - Still open
- 2 issues and 1 PR.
Next step: read #10782, then check whether your report is one of the 2 still open.
Read #10782Pick up a Lab taskThese issues and pull requests address passing reasoning_effort or think=false to Ollama and related custom providers to suppress or control thinking/reasoning mode. A recurring concern is that reasoning settings were silently ignored, causing models to run in unintended reasoning modes.
Open issues2
Open PRs1
- PR#46489fix(transports): promote Ollama reasoning field to content when content is empty
Fixes on record
-
#10782 feat: Ollama improvements — Cloud provider, GLM continuation, think=false, surrogate sanitization, /v1 hint
files:
.env.example,agent/auxiliary_client.py,agent/model_metadata.py,agent/models_dev.py,cli-config.yaml.example,hermes_cli/auth.py,hermes_cli/config.py,hermes_cli/main.py,hermes_cli/model_normalize.py,hermes_cli/models.py,hermes_cli/providers.py,run_agent.py,tests/hermes_cli/test_ollama_cloud_provider.py,tests/run_agent/test_run_agent.pyShow the diff
-
.env.example@@ -24,6 +24,15 @@ # Optional base URL override (default: Google's OpenAI-compatible endpoint) # GEMINI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai +# ============================================================================= +# LLM PROVIDER (Ollama Cloud) +# ============================================================================= +# Cloud-hosted open models via Ollama's OpenAI-compatible endpoint. +# Get your key at: https://ollama.com/settings +# OLLAMA_API_KEY=[redacted:secret-assignment] +# Optional base URL override (default: https://ollama.com/v1) +# OLLAMA_BASE_URL=https://ollama.com/v1 + # ============================================================================= # LLM PROVIDER (z.ai / GLM) # ============================================================================= -
agent/auxiliary_client.py@@ -104,6 +104,7 @@ def _normalize_aux_provider(provider: Optional[str]) -> str: "opencode-zen": "gemini-3-flash", "opencode-go": "glm-5", "kilocode": "google/gemini-3-flash-preview", + "ollama-cloud": "nemotron-3-nano:30b", } # Vision-specific model overrides for direct providers. -
agent/model_metadata.py@@ -23,7 +23,7 @@ # are preserved so the full model name reaches cache lookups and server queries. _PROVIDER_PREFIXES: frozenset[str] = frozenset({ "openrouter", "nous", "openai-codex", "copilot", "copilot-acp", - "gemini", "zai", "kimi-coding", "kimi-coding-cn", "minimax", "minimax-cn", "anthropic", "deepseek", + "gemini", "ollama-cloud", "zai", "kimi-coding", "kimi-coding-cn", "minimax", "minimax-cn", "anthropic", "deepseek", "opencode-zen", "opencode-go", "ai-gateway", "kilocode", "alibaba", "qwen-oauth", "xiaomi", @@ -33,6 +33,7 @@ "google", "google-gemini", "google-ai-studio", "glm", "z-ai", "z.ai", "zhipu", "github", "github-copilot", "github-models", "kimi", "moonshot", "kimi-cn", "moonshot-cn", "claude", "deep-seek", + "ollama", "opencode", "zen", "go", "vercel", "kilo", "dashscope", "aliyun", "qwen", "mimo", "xiaomi-mimo", "arcee-ai", "arceeai", @@ -239,6 +240,7 @@ def _is_custom_endpoint(base_url: str) -> bool: "api.x.ai": "xai", "api.xiaomimimo.com": "xiaomi", "xiaomimimo.com": "xiaomi", + "ollama.com": "ollama-cloud", } -
agent/models_dev.py@@ -169,6 +169,7 @@ class ProviderInfo: "togetherai": "togetherai", "perplexity": "perplexity", "cohere": "cohere", + "ollama-cloud": "ollama-cloud", } # Reverse mapping: models.dev → Hermes (built lazily) -
cli-config.yaml.example@@ -26,6 +26,7 @@ model: # "huggingface" - Hugging Face Inference (requires: HF_TOKEN) # "xiaomi" - Xiaomi MiMo (requires: XIAOMI_API_KEY) # "arcee" - Arcee AI Trinity models (requires: ARCEEAI_API_KEY) + # "ollama-cloud" - Ollama Cloud (requires: OLLAMA_API_KEY — https://ollama.com/settings) # "kilocode" - KiloCode gateway (requires: KILOCODE_API_KEY) # "ai-gateway" - Vercel AI Gateway (requires: AI_GATEWAY_API_KEY) # @@ -37,12 +38,6 @@ model: # base_url: "http://localhost:1234/v1" # No API key needed — local servers typically ignore auth. # - # For Ollama Cloud (https://ollama.com/pricing): - # provider: "custom" - # base_url: "https://ollama.com/v1" - # Set OLLAMA_API_KEY in .env — automatically picked up when base_url - # points to ollama.com. - # # Can also be overridden with --provider flag or HERMES_INFERENCE_PROVIDER env var. provider: "auto" @@ -337,6 +332,7 @@ compression: # "openrouter" - Force OpenRouter (requires OPENROUTER_API_KEY) # "nous" - Force Nous Portal (requires: hermes login) # "gemini" - Force Google AI Studio direct (requires: GOOGLE_API_KEY or GEMINI_API_KEY) +# "ollama-cloud" - Ollama Cloud (requires: OLLAMA_API_KEY) # "codex" - Force Codex OAuth (requires: hermes model → Codex). # Uses gpt-5.3-codex which supports vision. # "main" - Use your custom endpoint (OPENAI_BASE_URL + OPENAI_API_KEY). -
hermes_cli/auth.py@@ -70,6 +70,7 @@ DEFAULT_QWEN_BASE_URL = "https://portal.qwen.ai/v1" DEFAULT_GITHUB_MODELS_BASE_URL = "https://api.githubcopilot.com" DEFAULT_COPILOT_ACP_BASE_URL = "acp://copilot" +DEFAULT_OLLAMA_CLOUD_BASE_URL = "https://ollama.com/v1" CODEX_OAUTH_CLIENT_ID = "app_EMoamEEZ73f0CkXaXp7hrann" CODEX_OAUTH_TOKEN_URL = "[redacted:secret-assignment]" CODEX_ACCESS_TOKEN_REFRESH_SKEW_SECONDS = 120 @@ -274,6 +275,14 @@ class ProviderConfig: api_key_env_vars=("XIAOMI_API_KEY",), base_url_env_var="XIAOMI_BASE_URL", ), + "ollama-cloud": ProviderConfig( + id="ollama-cloud", + name="Ollama Cloud", + auth_type="api_key", + inference_base_url=DEFAULT_OLLAMA_CLOUD_BASE_URL, + api_key_env_vars=("OLLAMA_API_KEY",), + base_url_env_var="OLLAMA_BASE_URL", + ), "bedrock": ProviderConfig( id="bedrock", name="AWS Bedrock", @@ -937,7 +946,8 @@ def resolve_provider( "kilo": "kilocode", "kilo-code": "kilocode", "kilo-gateway": "kilocode", # Local server aliases — route through the generic custom provider "lmstudio": "custom", "lm-studio": "custom", "lm_studio": "custom", - "ollama": "custom", "vllm": "custom", "llamacpp": "custom", + "ollama": "custom", "ollama_cloud": "ollama-cloud", + "vllm": "custom", "llamacpp": "custom", "llama.cpp": "custom", "llama-cpp": "custom", } normalized = _PROVIDER_ALIASES.get(normalized, normalized) -
hermes_cli/config.py@@ -1024,6 +1024,22 @@ def _ensure_hermes_home_managed(home: Path): "category": "provider", "advanced": True, }, + "OLLAMA_API_KEY": { + "description": "Ollama Cloud API key (ollama.com — cloud-hosted open models)", + "prompt": "Ollama Cloud API key", + "url": "https://ollama.com/settings", + "password": True, + "category": "provider", + "advanced": True, + }, + "OLLAMA_BASE_URL": { + "description": "Ollama Cloud base URL override (default: https://ollama.com/v1)", + "prompt": "Ollama base URL (leave empty for default)", + "url": None, + "password": False, + "category": "provider", + "advanced": True, + }, "XIAOMI_API_KEY": { "description": "Xiaomi MiMo API key for MiMo models (mimo-v2-pro, mimo-v2-omni, mimo-v2-flash)", "prompt": "Xiaomi MiMo API Key", -
hermes_cli/main.py@@ -1141,7 +1141,7 @@ def _named_custom_provider_map(cfg) -> dict[str, dict[str, str]]: _model_flow_kimi(config, current_model) elif selected_provider == "bedrock": _model_flow_bedrock(config, current_model) - elif selected_provider in ("gemini", "deepseek", "xai", "zai", "kimi-coding-cn", "minimax", "minimax-cn", "kilocode", "opencode-zen", "opencode-go", "ai-gateway", "alibaba", "huggingface", "xiaomi", "arcee"): + elif selected_provider in ("gemini", "deepseek", "xai", "zai", "kimi-coding-cn", "minimax", "minimax-cn", "kilocode", "opencode-zen", "opencode-go", "ai-gateway", "alibaba", "huggingface", "xiaomi", "arcee", "ollama-cloud"): _model_flow_api_key_provider(config, selected_provider, current_model) # ── Post-switch cleanup: clear stale OPENAI_BASE_URL ────────────── @@ -1568,6 +1568,27 @@ def _model_flow_custom(config): effective_key = api_key or current_key + # Hint: most local model servers (Ollama, vLLM, llama.cpp) require /v1 + # in the base URL for OpenAI-compatible chat completions. Prompt the + # user if the URL looks like a local server without /v1. + _url_lower = effective_url.rstrip("/").lower() + _looks_local = any(h in _url_lower for h in ("localhost", "127.0.0.1", "0.0.0.0", ":11434", ":8080", ":5000")) + if _looks_local and not _url_lower.endswith("/v1"): + print() + print(f" Hint: Did you mean to add /v1 at the end?") + print(f" Most local model servers (Ollama, vLLM, llama.cpp) require it.") + print(f" e.g. {effective_url.rstrip('/')}/v1") + try: + _add_v1 = input(" Add /v1? [Y/n]: ").strip().lower() + except (KeyboardInterrupt, EOFError): + _add_v1 = "n" + if _add_v1 in ("", "y", "yes"): + effective_url = effective_url.rstrip("/") + "/v1" + if base_url: + base_url = effective_url + print(f" Updated URL: {effective_url}") + print() + from hermes_cli.models import probe_api_models probe = probe_api_models(effective_key, effective_url) @@ -2734,34 +2755,43 @@ def _model_flow_api_key_provider(config, provider_id, current_model=""): # 1. models.dev registry (cached, filtered for agentic/tool-capable models) # 2. Curated static fallback list (offline insurance) # 3. Live /models endpoint probe (small providers without models.dev data) - curated = _PROVIDER_MODELS.get(provider_id, []) + # + # Ollama Cloud: dedicated merged discovery (live API + models.dev + disk cache) + if provider_id == "ollama-cloud": + from hermes_cli.models import fetch_ollama_cloud_models + api_key_for_probe = [redacted:secret-assignment] or (get_env_value(key_env) if key_env else "") + model_list = fetch_ollama_cloud_models(api_key=[redacted:secret-assignment] base_url=effective_base) + if model_list: + print(f" Found {len(model_list)} model(s) from Ollama Cloud") + else: + curated = _PROVIDER_MODELS.get(provider_id, []) - # Try models.dev first — returns tool-capable models, filtered for noise - mdev_models: list = [] - try: - from agent.models_dev import list_agentic_models - mdev_models = list_agentic_models(provider_id) - except Exception: - pass + # Try models.dev first — returns tool-capable models, filtered for noise + mdev_models: list = [] + try: + from agent.models_dev import list_agentic_models + mdev_models = list_agentic_models(provider_id) + except Exception: + pass - if mdev_models: - model_list = mdev_models - print(f" Found {len(model_list)} model(s) from models.dev registry") - elif curated and len(curated) >= 8: - # Curated list is substantial — use it directly, skip live probe - model_list = curated - print(f" Showing {len(model_list)} curated models — use \"Enter custom model name\" for others.") - else: - api_key_for_probe = [redacted:secret-assignment] or (get_env_value(key_env) if key_env else "") - live_models = fetch_api_models(api_key_for_probe, effective_base) - if live_models and len(live_models) >= len(curated): - model_list = live_models - print(f" Found {len(model_list)} model(s) from {pconfig.name} API") - else: + if mdev_models: + model_list = mdev_models + print(f" Found {len(model_list)} model(s) from models.dev registry") + elif curated and len(curated) >= 8: + # Curated list is substantial — use it directly, skip live probe model_list = curated - if model_list: - print(f" Showing {len(model_list)} curated models — use \"Enter custom model name\" for others.") - # else: no defaults either, will fall through to raw input + print(f" Showing {len(model_list)} curated models — use \"Enter custom model name\" for others.") + else: + api_key_for_probe = [redacted:secret-assignment] or (get_env_value(key_env) if key_env else "") + live_models = fetch_api_models(api_key_for_probe, effective_base) + if live_models and len(live_models) >= len(curated): + model_list = live_models + print(f" Found {len(model_list)} model(s) from {pconfig.name} API") + else: + model_list = curated + if model_list: + print(f" Showing {len(model_list)} curated models — use \"Enter custom model name\" for others.") + # else: no defaults either, will fall through to raw input if provider_id in {"opencode-zen", "opencode-go"}: model_list = [normalize_opencode_model_id(provider_id, mid) for mid in model_list] @@ -4860,7 +4890,7 @@ def main(): ) chat_parser.add_argument( "--provider", - choices=["auto", "openrouter", "nous", "openai-codex", "copilot-acp", "copilot", "anthropic", "gemini", "huggingface", "zai", "kimi-coding", "kimi-coding-cn", "minimax", "minimax-cn", "kilocode", "xiaomi", "arcee"], + choices=["auto", "openrouter", "nous", "openai-codex", "copilot-acp", "copilot", "anthropic", "gemini", "ollama-cloud", "huggingface", "zai", "kimi-coding", "kimi-coding-cn", "minimax", "minimax-cn", "kilocode", "xiaomi", "arcee"], default=None, help="Inference provider (default: auto)" ) -
hermes_cli/model_normalize.py@@ -96,6 +96,7 @@ "qwen-oauth", "xiaomi", "arcee", + "ollama-cloud", "custom", }) -
hermes_cli/models.py@@ -11,7 +11,9 @@ import os import urllib.request import urllib.error +import time from difflib import get_close_matches +from pathlib import Path from typing import Any, NamedTuple, Optional COPILOT_BASE_URL = "https://api.githubcopilot.com" @@ -547,6 +549,7 @@ class ProviderEntry(NamedTuple): ProviderEntry("minimax", "MiniMax", "MiniMax (global direct API)"), ProviderEntry("minimax-cn", "MiniMax (China)", "MiniMax China (domestic direct API)"), ProviderEntry("alibaba", "Alibaba Cloud (DashScope)","Alibaba Cloud / DashScope Coding (Qwen + multi-provider)"), + ProviderEntry("ollama-cloud", "Ollama Cloud", "Ollama Cloud (cloud-hosted open models — ollama.com)"), ProviderEntry("arcee", "Arcee AI", "Arcee AI (Trinity models — direct API)"), ProviderEntry("kilocode", "Kilo Code", "Kilo Code (Kilo Gateway API)"), ProviderEntry("opencode-zen", "OpenCode Zen", "OpenCode Zen (35+ curated models, pay-as-you-go)"), @@ -559,6 +562,7 @@ class ProviderEntry(NamedTuple): _PROVIDER_LABELS = {p.slug: p.label for p in CANONICAL_PROVIDERS} _PROVIDER_LABELS["custom"] = "Custom endpoint" # special case: not a named provider + _PROVIDER_ALIASES = { "glm": "zai", "z-ai": "zai", @@ -611,6 +615,8 @@ class ProviderEntry(NamedTuple): "grok": "xai", "x-ai": "xai", "x.ai": "xai", + "ollama": "custom", # bare "ollama" = local; use "ollama-cloud" for cloud + "ollama_cloud": "ollama-cloud", } @@ -1786,6 +1792,125 @@ def fetch_api_models( return probe_api_models(api_key, base_url, timeout=timeout).get("models") +# --------------------------------------------------------------------------- +# Ollama Cloud — merged model discovery with disk cache +# --------------------------------------------------------------------------- + + + +_OLLAMA_CLOUD_CACHE_TTL = 3600 # 1 hour + + +def _ollama_cloud_cache_path() -> Path: + """Return the path for the Ollama Cloud model cache.""" + from hermes_constants import get_hermes_home + return get_hermes_home() / "ollama_cloud_models_cache.json" + + +def _load_ollama_cloud_cache(*, ignore_ttl: bool = False) -> Optional[dict]: + """Load cached Ollama Cloud models from disk. + + Args: + ignore_ttl: If True, return data even if the TTL has expired (stale fallback). + """ + try: + cache_path = _ollama_cloud_cache_path() + if not cache_path.exists(): + return None + with open(cache_path, encoding="utf-8") as f: + data = json.load(f) + if not isinstance(data, dict): + return None + models = data.get("models") + if not (isinstance(models, list) and models): + return None + if not ignore_ttl: + cached_at = data.get("cached_at", 0) + if (time.time() - cached_at) > _OLLAMA_CLOUD_CACHE_TTL: + return None # stale + return data + except Exception: + pass + return None + + +def _save_ollama_cloud_cache(models: list[str]) -> None: + """Persist the merged Ollama Cloud model list to disk.""" + try: + from utils import atomic_json_write + cache_path = _ollama_cloud_cache_path() + cache_path.parent.mkdir(parents=True, exist_ok=True) + atomic_json_write(cache_path, {"models": models, "cached_at": time.time()}, indent=None) + except Exception: + pass + + +def fetch_ollama_cloud_models( + api_key: [redacted:secret-assignment] = None, + base_url: Optional[str] = None, + *, + force_refresh: bool = False, +) -> list[str]: + """Fetch Ollama Cloud models by merging live API + models.dev, with disk cache. + + Resolution order: + 1. Disk cache (if fresh, < 1 hour, and not force_refresh) + 2. Live ``/v1/models`` endpoint (primary — freshest source) + 3. models.dev registry (secondary — fills gaps for unlisted models) + 4. Merge: live models first, then models.dev additions (deduped) + + Returns a list of model IDs (never None — empty list on total failure). + """ + # 1. Check disk cache + if not force_refresh: + cached = _load_ollama_cloud_cache() + if cached is not None: + return cached["models"] + + # 2. Live API probe + if not api_key: + api_key = [redacted:secret-assignment]"OLLAMA_API_KEY", "") + if not base_url: + base_url = os.getenv("OLLAMA_BASE_URL", "") or "https://ollama.com/v1" + + live_models: list[str] = [] + if api_key: + result = fetch_api_models(api_key, base_url, timeout=8.0) + if result: + live_models = result + + # 3. models.dev registry + mdev_models: list[str] = [] + try: + from agent.models_dev import list_agentic_models + mdev_models = list_agentic_models("ollama-cloud") + except Exception: + pass + + # 4. Merge: live first, then models.dev additions (deduped, order-preserving) + if live_models or mdev_models: + seen: set[str] = set() + merged: list[str] = [] + for m in live_models: + if m and m not in seen: + seen.add(m) + merged.append(m) + for m in mdev_models: + if m and m not in seen: + seen.add(m) + merged.append(m) + if merged: + _save_ollama_cloud_cache(merged) + return merged + + # Total failure — return stale cache if available (ignore TTL) + stale = _load_ollama_cloud_cache(ignore_ttl=True) + if stale is not None: + return stale["models"] + + return [] + + def validate_requested_model( model_name: str, provider: Optional[str], -
hermes_cli/providers.py@@ -141,6 +141,10 @@ class HermesOverlay: base_url_override="https://api.arcee.ai/api/v1", base_url_env_var="ARCEE_BASE_URL", ), + "ollama-cloud": HermesOverlay( + transport="openai_chat", + base_url_env_var="OLLAMA_BASE_URL", + ), } @@ -250,7 +254,7 @@ class ProviderDef: "lmstudio": "lmstudio", "lm-studio": "lmstudio", "lm_studio": "lmstudio", - "ollama": "ollama-cloud", + "ollama": "custom", # bare "ollama" = local; use "ollama-cloud" for cloud "vllm": "local", "llamacpp": "local", "llama.cpp": "local", @@ -269,6 +273,7 @@ class ProviderDef: "xiaomi": "Xiaomi MiMo", "local": "Local endpoint", "bedrock": "AWS Bedrock", + "ollama-cloud": "Ollama Cloud", } -
run_agent.py@@ -2103,6 +2103,59 @@ def _strip_think_blocks(self, content: str) -> str: content = re.sub(r'</?(?:think|thinking|reasoning|thought|REASONING_SCRATCHPAD)>\s*', '', content, flags=re.IGNORECASE) return content + @staticmethod + def _has_natural_response_ending(content: str) -> bool: + """Heuristic: does visible assistant text look intentionally finished?""" + if not content: + return False + stripped = content.rstrip() + if not stripped: + return False + if stripped.endswith("```"): + return True + return stripped[-1] in '.!?:)"\']}。!?:)】」』》' + + def _is_ollama_glm_backend(self) -> bool: + """Detect the narrow backend family affected by Ollama/GLM stop misreports.""" + model_lower = (self.model or "").lower() + provider_lower = (self.provider or "").lower() + if "glm" not in model_lower and provider_lower != "zai": + return False + if "ollama" in self._base_url_lower or ":11434" in self._base_url_lower: + return True + return bool(self.base_url and is_local_endpoint(self.base_url)) + + def _should_treat_stop_as_truncated( + self, + finish_reason: str, + assistant_message, + messages: Optional[list] = None, + ) -> bool: + """Detect conservative stop->length misreports for Ollama-hosted GLM models.""" + if finish_reason != "stop" or self.api_mode != "chat_completions": + return False + if not self._is_ollama_glm_backend(): + return False + if not any( + isinstance(msg, dict) and msg.get("role") == "tool" + for msg in (messages or []) + ): + return False + if assistant_message is None or getattr(assistant_message, "tool_calls", None): + return False + + content = getattr(assistant_message, "content", None) + if not isinstance(content, str): + return False + + visible_text = self._strip_think_blocks(content).strip() + if not visible_text: + return False + if len(visible_text) < 20 or not re.search(r"\s", visible_text): + return False + + return not self._has_natural_response_ending(visible_text) + def _looks_like_codex_intermediate_ack( self, user_message: str, @@ -6635,6 +6688,18 @@ def _build_api_kwargs(self, api_messages: list) -> dict: options["num_ctx"] = self._ollama_num_ctx extra_body["options"] = options + # Ollama / custom provider: pass think=false when reasoning is disabled. + # Ollama does not recognise the OpenRouter-style `reasoning` extra_body + # field, so we use its native `think` parameter instead. + # This prevents thinking-capable models (Qwen3, etc.) from generating + # <think> blocks and producing empty-response errors when the user has + # set reasoning_effort: none. + if self.provider == "custom" and self.reasoning_config and isinstance(self.reasoning_config, dict): + _effort = (self.reasoning_config.get("effort") or "").strip().lower() + _enabled = self.reasoning_config.get("enabled", True) + if _effort == "none" or _enabled is False: + extra_body["think"] = False + if self._is_qwen_portal(): extra_body["vl_high_resolution_images"] = True @@ -6757,9 +6822,16 @@ def _build_assistant_message(self, assistant_message, finish_reason: str) -> dic except Exception: pass + # Sanitize surrogates from API response — some models (e.g. Kimi/GLM via Ollama) + # can return invalid surrogate code points that crash json.dumps() on persist. + _raw_content = assistant_message.content or "" + _san_content = _sanitize_surrogates(_raw_content) + if reasoning_text: + reasoning_text = _sanitize_surrogates(reasoning_text) + msg = { "role": "assistant", - "content": assistant_message.content or "", + "content": _san_content, "reasoning": reasoning_text, "finish_reason": finish_reason, } @@ -8640,6 +8712,12 @@ def run_conversation( new_tcs.append(tc) am["tool_calls"] = new_tcs + # Proactively strip any surrogate characters before the API call. + # Models served via Ollama (Kimi K2.5, GLM-5, Qwen) can return + # lone surrogates (U+D800-U+DFFF) that crash json.dumps() inside + # the OpenAI SDK. Sanitizing here prevents the 3-retry cycle. + _sanitize_messages_surrogates(api_messages) + # Calculate approximate request size for logging total_chars = sum(len(str(msg)) for msg in api_messages) approx_tokens = [redacted:secret-assignment] @@ -9038,6 +9116,17 @@ def _stop_spinner(): finish_reason = stop_reason_map.get(response.stop_reason, "stop") else: finish_reason = response.choices[0].finish_reason + assistant_message = response.choices[0].message + if self._should_treat_stop_as_truncated( + finish_reason, + assistant_message, + messages, + ): + self._vprint( + f"{self.log_prefix}⚠️ Treating suspicious Ollama/GLM stop response as truncated", + force=True, + ) + finish_reason = "length" if finish_reason == "length": self._vprint(f"{self.log_prefix}⚠️ Response truncated (finish_reason='length') - model hit max output tokens", force=True) -
tests/hermes_cli/test_ollama_cloud_provider.py@@ -0,0 +1,351 @@ +"""Tests for Ollama Cloud provider integration.""" + +import os +import pytest +from unittest.mock import patch, MagicMock + +from hermes_cli.auth import PROVIDER_REGISTRY, resolve_provider, resolve_api_key_provider_credentials +from hermes_cli.models import _PROVIDER_MODELS, _PROVIDER_LABELS, _PROVIDER_ALIASES, normalize_provider +from hermes_cli.model_normalize import normalize_model_for_provider +from agent.model_metadata import _URL_TO_PROVIDER, _PROVIDER_PREFIXES +from agent.models_dev import PROVIDER_TO_MODELS_DEV, list_agentic_models + + +# ── Provider Registry ── + +class TestOllamaCloudProviderRegistry: + def test_ollama_cloud_in_registry(self): + assert "ollama-cloud" in PROVIDER_REGISTRY + + def test_ollama_cloud_config(self): + pconfig = PROVIDER_REGISTRY["ollama-cloud"] + assert pconfig.id == "ollama-cloud" + assert pconfig.name == "Ollama Cloud" + assert pconfig.auth_type == "api_key" + assert pconfig.inference_base_url == "https://ollama.com/v1" + + def test_ollama_cloud_env_vars(self): + pconfig = PROVIDER_REGISTRY["ollama-cloud"] + assert pconfig.api_key_env_vars == ("OLLAMA_API_KEY",) + assert pconfig.base_url_env_var == "OLLAMA_BASE_URL" + + def test_ollama_cloud_base_url(self): + assert "ollama.com" in PROVIDER_REGISTRY["ollama-cloud"].inference_base_url + + +# ── Provider Aliases ── + +PROVIDER_ENV_VARS = ( + "OPENROUTER_API_KEY", "OPENAI_API_KEY", "ANTHROPIC_API_KEY", + "GOOGLE_API_KEY", "GEMINI_API_KEY", "OLLAMA_API_KEY", + "GLM_API_KEY", "ZAI_API_KEY", "KIMI_API_KEY", + "MINIMAX_API_KEY", "DEEPSEEK_API_KEY", +) + [redacted:email](autouse=True) +def _clean_provider_env(monkeypatch): + for var in PROVIDER_ENV_VARS: + monkeypatch.delenv(var, raising=False) + + +class TestOllamaCloudAliases: + def test_explicit_ollama_cloud(self): + assert resolve_provider("ollama-cloud") == "ollama-cloud" + + def test_alias_ollama_underscore(self): + """ollama_cloud (underscore) is the unambiguous cloud alias.""" + assert resolve_provider("ollama_cloud") == "ollama-cloud" + + def test_bare_ollama_stays_local(self): + """Bare 'ollama' alias routes to 'custom' (local) — not cloud.""" + assert resolve_provider("ollama") == "custom" + + def test_models_py_aliases(self): + assert _PROVIDER_ALIASES.get("ollama_cloud") == "ollama-cloud" + # bare "ollama" stays local + assert _PROVIDER_ALIASES.get("ollama") == "custom" + + def test_normalize_provider(self): + assert normalize_provider("ollama-cloud") == "ollama-cloud" + + +# ── Auto-detection ── + +class TestOllamaCloudAutoDetection: + def test_auto_detects_ollama_api_key(self, monkeypatch): + monkeypatch.setenv("OLLAMA_API_KEY", "test-ollama-key") + assert resolve_provider("auto") == "ollama-cloud" + + +# ── Credential Resolution ── + +class TestOllamaCloudCredentials: + def test_resolve_with_ollama_api_key(self, monkeypatch): + monkeypatch.setenv("OLLAMA_API_KEY", "ollama-secret") + creds = resolve_api_key_provider_credentials("ollama-cloud") + assert creds["provider"] == "ollama-cloud" + assert creds["api_key"] == "ollama-secret" + assert creds["base_url"] == "https://ollama.com/v1" + + def test_resolve_with_custom_base_url(self, monkeypatch): + monkeypatch.setenv("OLLAMA_API_KEY", "key") + monkeypatch.setenv("OLLAMA_BASE_URL", "https://custom.ollama/v1") + creds = resolve_api_key_provider_credentials("ollama-cloud") + assert creds["base_url"] == "https://custom.ollama/v1" + + def test_runtime_ollama_cloud(self, monkeypatch): + monkeypatch.setenv("OLLAMA_API_KEY", "ollama-key") + from hermes_cli.runtime_provider import resolve_runtime_provider + result = resolve_runtime_provider(requested="ollama-cloud") + assert result["provider"] == "ollama-cloud" + assert result["api_mode"] == "chat_completions" + assert result["api_key"] == "ollama-key" + assert result["base_url"] == "https://ollama.com/v1" + + +# ── Model Catalog (dynamic — no static list) ── + +class TestOllamaCloudModelCatalog: + def test_no_static_model_list(self): + """Ollama Cloud models are fetched dynamically — no static list to maintain.""" + assert "ollama-cloud" not in _PROVIDER_MODELS + + def test_provider_label(self): + assert "ollama-cloud" in _PROVIDER_LABELS + assert _PROVIDER_LABELS["ollama-cloud"] == "Ollama Cloud" + + +# ── Merged Model Discovery ── + +class TestOllamaCloudMergedDiscovery: + def test_merges_live_and_models_dev(self, tmp_path, monkeypatch): + """Live API models appear first, models.dev additions fill gaps.""" + from hermes_cli.models import fetch_ollama_cloud_models + + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + monkeypatch.setenv("OLLAMA_API_KEY", "test-key") + + mock_mdev = { + "ollama-cloud": { + "models": { + "glm-5": {"tool_call": True}, + "kimi-k2.5": {"tool_call": True}, + "nemotron-3-super": {"tool_call": True}, + } + } + } + with patch("hermes_cli.models.fetch_api_models", return_value=["qwen3.5:397b", "glm-5"]), \ + patch("agent.models_dev.fetch_models_dev", return_value=mock_mdev): + result = fetch_ollama_cloud_models(force_refresh=True) + + # Live models first, then models.dev additions (deduped) + assert result[0] == "qwen3.5:397b" # from live API + assert result[1] == "glm-5" # from live API (also in models.dev) + assert "kimi-k2.5" in result # from models.dev only + assert "nemotron-3-super" in result # from models.dev only + assert result.count("glm-5") == 1 # no duplicates + + def test_falls_back_to_models_dev_without_api_key(self, tmp_path, monkeypatch): + """Without API key, only models.dev results are returned.""" + from hermes_cli.models import fetch_ollama_cloud_models + + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + monkeypatch.delenv("OLLAMA_API_KEY", raising=False) + + mock_mdev = { + "ollama-cloud": { + "models": { + "glm-5": {"tool_call": True}, + } + } + } + with patch("agent.models_dev.fetch_models_dev", return_value=mock_mdev): + result = fetch_ollama_cloud_models(force_refresh=True) + + assert result == ["glm-5"] + + def test_uses_disk_cache(self, tmp_path, monkeypatch): + """Second call returns cached results without hitting APIs.""" + from hermes_cli.models import fetch_ollama_cloud_models + + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + monkeypatch.setenv("OLLAMA_API_KEY", "test-key") + + with patch("hermes_cli.models.fetch_api_models", return_value=["model-a"]) as mock_api, \ + patch("agent.models_dev.fetch_models_dev", return_value={}): + first = fetch_ollama_cloud_models(force_refresh=True) + assert first == ["model-a"] + assert mock_api.call_count == 1 + + # Second call — should use disk cache, not call API + second = fetch_ollama_cloud_models() + assert second == ["model-a"] + assert mock_api.call_count == 1 # no extra API call + + def test_force_refresh_bypasses_cache(self, tmp_path, monkeypatch): + """force_refresh=True always hits the API even with fresh cache.""" + from hermes_cli.models import fetch_ollama_cloud_models + + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + monkeypatch.setenv("OLLAMA_API_KEY", "test-key") + + with patch("hermes_cli.models.fetch_api_models", return_value=["model-a"]) as mock_api, \ + patch("agent.models_dev.fetch_models_dev", return_value={}): + fetch_ollama_cloud_models(force_refresh=True) + fetch_ollama_cloud_models(force_refresh=True) + assert mock_api.call_count == 2 + + def test_stale_cache_used_on_total_failure(self, tmp_path, monkeypatch): + """If both API and models.dev fail, stale cache is returned.""" + from hermes_cli.models import fetch_ollama_cloud_models, _save_ollama_cloud_cache + + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + monkeypatch.setenv("OLLAMA_API_KEY", "test-key") + + # Pre-populate a stale cache + _save_ollama_cloud_cache(["stale-model"]) + + # Make the cache appear stale by backdating it + import json + cache_path = tmp_path / "ollama_cloud_models_cache.json" + with open(cache_path) as f: + data = json.load(f) + data["cached_at"] = 0 # epoch = very stale + with open(cache_path, "w") as f: + json.dump(data, f) + + with patch("hermes_cli.models.fetch_api_models", return_value=None), \ + patch("agent.models_dev.fetch_models_dev", return_value={}): + result = fetch_ollama_cloud_models(force_refresh=True) + + assert result == ["stale-model"] + + def test_empty_on_total_failure_no_cache(self, tmp_path, monkeypatch): + """Returns empty list when everything fails and no cache exists.""" + from hermes_cli.models import fetch_ollama_cloud_models + + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + monkeypatch.delenv("OLLAMA_API_KEY", raising=False) + + with patch("agent.models_dev.fetch_models_dev", return_value={}): + result = fetch_ollama_cloud_models(force_refresh=True) + + assert result == [] + + +# ── Model Normalization ── + +class TestOllamaCloudModelNormalization: + def test_passthrough_bare_name(self): + """Ollama Cloud is a passthrough provider — model names used as-is.""" + assert normalize_model_for_provider("qwen3.5:397b", "ollama-cloud") == "qwen3.5:397b" + + def test_passthrough_with_tag(self): + assert normalize_model_for_provider("cogito-2.1:671b", "ollama-cloud") == "cogito-2.1:671b" + + def test_passthrough_no_tag(self): + assert normalize_model_for_provider("glm-5", "ollama-cloud") == "glm-5" + + +# ── URL-to-Provider Mapping ── + +class TestOllamaCloudUrlMapping: + def test_url_to_provider(self): + assert _URL_TO_PROVIDER.get("ollama.com") == "ollama-cloud" + + def test_provider_prefix_canonical(self): + assert "ollama-cloud" in _PROVIDER_PREFIXES + + def test_provider_prefix_alias(self): + assert "ollama" in _PROVIDER_PREFIXES + + +# ── models.dev Integration ── + +class TestOllamaCloudModelsDev: + def test_ollama_cloud_mapped(self): + assert PROVIDER_TO_MODELS_DEV.get("ollama-cloud") == "ollama-cloud" + + def test_list_agentic_models_with_mock_data(self): + """list_agentic_models filters correctly from mock models.dev data.""" + mock_data = { + "ollama-cloud": { + "models": { + "qwen3.5:397b": {"tool_call": True}, + "glm-5": {"tool_call": True}, + "nemotron-3-nano:30b": {"tool_call": True}, + "some-embedding:latest": {"tool_call": False}, + } + } + } + with patch("agent.models_dev.fetch_models_dev", return_value=mock_data): + result = list_agentic_models("ollama-cloud") + assert "qwen3.5:397b" in result + assert "glm-5" in result + assert "nemotron-3-nano:30b" in result + assert "some-embedding:latest" not in result # no tool_call + + +# ── Agent Init (no SyntaxError) ── + +class TestOllamaCloudAgentInit: + def test_agent_imports_without_error(self): + """Verify run_agent.py has no SyntaxError.""" + import importlib + import run_agent + importlib.reload(run_agent) + + def test_ollama_cloud_agent_uses_chat_completions(self, monkeypatch): + """Ollama Cloud falls through to chat_completions — no special elif needed.""" + monkeypatch.setenv("OLLAMA_API_KEY", "test-key") + with patch("run_agent.OpenAI") as mock_openai: + mock_openai.return_value = MagicMock() + from run_agent import AIAgent + agent = AIAgent( + model="qwen3.5:397b", + provider="ollama-cloud", + api_key="[redacted:secret-assignment]", + base_url="https://ollama.com/v1", + ) + assert agent.api_mode == "chat_completions" + assert agent.provider == "ollama-cloud" + + +# ── providers.py New System ── + +class TestOllamaCloudProvidersNew: + def test_overlay_exists(self): + from hermes_cli.providers import HERMES_OVERLAYS + assert "ollama-cloud" in HERMES_OVERLAYS + overlay = HERMES_OVERLAYS["ollama-cloud"] + assert overlay.transport == "openai_chat" + assert overlay.base_url_env_var == "OLLAMA_BASE_URL" + + def test_alias_resolves(self): + from hermes_cli.providers import normalize_provider as np + assert np("ollama") == "custom" # bare "ollama" = local + assert np("ollama-cloud") == "ollama-cloud" + + def test_label_override(self): + from hermes_cli.providers import _LABEL_OVERRIDES + assert _LABEL_OVERRIDES.get("ollama-cloud") == "Ollama Cloud" + + def test_get_label(self): + from hermes_cli.providers import get_label + assert get_label("ollama-cloud") == "Ollama Cloud" + + def test_get_provider(self): + from hermes_cli.providers import get_provider + pdef = get_provider("ollama-cloud") + assert pdef is not None + assert pdef.id == "ollama-cloud" + assert pdef.transport == "openai_chat" + + +# ── Auxiliary Model ── + +class TestOllamaCloudAuxiliary: + def test_aux_model_defined(self): + from agent.auxiliary_client import _API_KEY_PROVIDER_AUX_MODELS + assert "ollama-cloud" in _API_KEY_PROVIDER_AUX_MODELS + assert _API_KEY_PROVIDER_AUX_MODELS["ollama-cloud"] == "nemotron-3-nano:30b" -
tests/run_agent/test_run_agent.py@@ -928,6 +928,7 @@ def test_max_tokens_injected(self, agent): kwargs = agent._build_api_kwargs(messages) assert kwargs["max_tokens"] == 4096 + def test_qwen_portal_formats_messages_and_metadata(self, agent): agent.base_url = "https://portal.qwen.ai/v1" agent._base_url_lower = agent.base_url.lower() @@ -984,6 +985,46 @@ def test_qwen_portal_default_max_tokens(self, agent): kwargs = agent._build_api_kwargs(messages) assert kwargs["max_tokens"] == 65536 + def test_ollama_think_false_on_effort_none(self, agent): + """Custom (Ollama) provider with effort=none should inject think=false.""" + agent.provider = "custom" + agent.base_url = "http://localhost:11434/v1" + agent._base_url_lower = agent.base_url.lower() + agent.reasoning_config = {"effort": "none"} + messages = [{"role": "user", "content": "hi"}] + kwargs = agent._build_api_kwargs(messages) + assert kwargs.get("extra_body", {}).get("think") is False + + def test_ollama_think_false_on_enabled_false(self, agent): + """Custom (Ollama) provider with enabled=false should inject think=false.""" + agent.provider = "custom" + agent.base_url = "http://localhost:11434/v1" + agent._base_url_lower = agent.base_url.lower() + agent.reasoning_config = {"enabled": False} + messages = [{"role": "user", "content": "hi"}] + kwargs = agent._build_api_kwargs(messages) + assert kwargs.get("extra_body", {}).get("think") is False + + def test_ollama_no_think_param_when_reasoning_enabled(self, agent): + """Custom provider with reasoning enabled should NOT inject think=false.""" + agent.provider = "custom" + agent.base_url = "http://localhost:11434/v1" + agent._base_url_lower = agent.base_url.lower() + agent.reasoning_config = {"enabled": True, "effort": "medium"} + messages = [{"role": "user", "content": "hi"}] + kwargs = agent._build_api_kwargs(messages) + assert kwargs.get("extra_body", {}).get("think") is None + + def test_non_custom_provider_unaffected(self, agent): + """OpenRouter provider with effort=none should NOT inject think=false.""" + agent.provider = "openrouter" + agent.model = "qwen/qwen3.5-plus-02-15" + agent.reasoning_config = {"effort": "none"} + messages = [{"role": "user", "content": "hi"}] + kwargs = agent._build_api_kwargs(messages) + assert kwargs.get("extra_body", {}).get("think") is None + + class TestBuildAssistantMessage: def test_basic_message(self, agent): @@ -2202,6 +2243,114 @@ def test_length_finish_reason_requests_continuation(self, agent): assert second_call_messages[-1]["role"] == "user" assert "truncated by the output length limit" in second_call_messages[-1]["content"] + def test_ollama_glm_stop_after_tools_without_terminal_boundary_requests_continuation(self, agent): + """Ollama-hosted GLM responses can misreport truncated output as stop.""" + self._setup_agent(agent) + agent.base_url = "http://localhost:11434/v1" + agent._base_url_lower = agent.base_url.lower() + agent.model = "glm-5.1:cloud" + + tool_turn = _mock_response( + content="", + finish_reason="tool_calls", + tool_calls=[_mock_tool_call(name="web_search", arguments="{}", call_id="c1")], + ) + misreported_stop = _mock_response( + content="Based on the search results, the best next", + finish_reason="stop", + ) + continued = _mock_response( + content=" step is to update the config.", + finish_reason="stop", + ) + agent.client.chat.completions.create.side_effect = [ + tool_turn, + misreported_stop, + continued, + ] + + with ( + patch("run_agent.handle_function_call", return_value="search result"), + patch.object(agent, "_persist_session"), + patch.object(agent, "_save_trajectory"), + patch.object(agent, "_cleanup_task_resources"), + ): + result = agent.run_conversation("hello") + + assert result["completed"] is True + assert result["api_calls"] == 3 + assert ( + result["final_response"] + == "Based on the search results, the best next step is to update the config." + ) + + third_call_messages = agent.client.chat.completions.create.call_args_list[2].kwargs["messages"] + assert third_call_messages[-1]["role"] == "user" + assert "truncated by the output length limit" in third_call_messages[-1]["content"] + + def test_ollama_glm_stop_with_terminal_boundary_does_not_continue(self, agent): + """Complete Ollama/GLM responses should not be reclassified as truncated.""" + self._setup_agent(agent) + agent.base_url = "http://localhost:11434/v1" + agent._base_url_lower = agent.base_url.lower() + agent.model = "glm-5.1:cloud" + + tool_turn = _mock_response( + content="", + finish_reason="tool_calls", + tool_calls=[_mock_tool_call(name="web_search", arguments="{}", call_id="c1")], + ) + complete_stop = _mock_response( + content="Based on the search results, the best next step is to update the config.", + finish_reason="stop", + ) + agent.client.chat.completions.create.side_effect = [tool_turn, complete_stop] + + with ( + patch("run_agent.handle_function_call", return_value="search result"), + patch.object(agent, "_persist_session"), + patch.object(agent, "_save_trajectory"), + patch.object(agent, "_cleanup_task_resources"), + ): + result = agent.run_conversation("hello") + + assert result["completed"] is True + assert result["api_calls"] == 2 + assert ( + result["final_response"] + == "Based on the search results, the best next step is to update the config." + ) + + def test_non_ollama_stop_without_terminal_boundary_does_not_continue(self, agent): + """The stop->length workaround should stay scoped to Ollama/GLM backends.""" + self._setup_agent(agent) + agent.base_url = "https://api.openai.com/v1" + agent._base_url_lower = agent.base_url.lower() + agent.model = "gpt-4o-mini" + + tool_turn = _mock_response( + content="", + finish_reason="tool_calls", + tool_calls=[_mock_tool_call(name="web_search", arguments="{}", call_id="c1")], + ) + normal_stop = _mock_response( + content="Based on the search results, the best next", + finish_reason="stop", + ) + agent.client.chat.completions.create.side_effect = [tool_turn, normal_stop] + + with ( + patch("run_agent.handle_function_call", return_value="search result"), + patch.object(agent, "_persist_session"), + patch.object(agent, "_save_trajectory"), + patch.object(agent, "_cleanup_task_resources"), + ): + result = agent.run_conversation("hello") + + assert result["completed"] is True + assert result["api_calls"] == 2 + assert result["final_response"] == "Based on the search results, the best next" + def test_length_thinking_exhausted_skips_continuation(self, agent): """When finish_reason='length' but content is only thinking, skip retries.""" self._setup_agent(agent)
-
5 PRs closed without merging
From the close comments: done elsewhere 2 · duplicate 1 · merged via another change 1 · other 1
-
#3197 feat(ollama): pass think=false to custom providers when reasoning_effort is none · merged via another change →
#10782,#3191 - #25866 fix(agent): propagate reasoning_effort to Ollama + curator fork · done elsewhere
-
#29820 fix(ollama): emit top-level reasoning_effort=none on /v1/chat/complet… · duplicate →
#25866,#25758,#6152 -
#42201 Disable qwen3 thinking on local Ollama so tool-calling works (fixes #6152) · done elsewhere →
67df958 -
#105996 fix(transport): enforce reasoning output floor so tight max_tokens can't starve the answer · other →
fcd7df3,#9452
Closed issues · 2
How this cluster was grouped
-
Link evidence (
partly structural links): some members are connected by structural evidence (shared changed files, or a closing/superseded reference); the rest are joined by an automated relevance judgement over their text. - Members outside that structural core may belong to a different underlying problem.
-
Pairwise link density
0.17: the share of item pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36.1.00means every pair is linked.
Pick up a Lab task
Check whether a fix covers the open issues here, with your own agent. Bring back evidence for maintainers.
Open the LabAt a glance
- Subsystem
- agent
- Merged PRs
- #10782
- Closed issues
- 2
- Attempts
- 5 PRs closed without merging
- Fix link checked
- 16 Apr 2026. This confirmation is old and may be out of date (a revert or reopen would not have invalidated it here); re-verify.