Model name normalization and provider prefix handling
NousResearch/hermes-agent · cluster 6211 · agent
Data as of
7 open items. Fix merged in #7187. Check it covers your case.
The short answer
- Known?
- Yes. 27 related items grouped here, 5 issues already closed.
- Fix event
- A fix was merged to main in PR #7187. Check it covers your case. No release data is tracked.
- Tracker
- The maintainers' tracker marks this
partially resolved, deciding PRs #7935, #54093, #115756. That is separate from the fix event. - Still open
- 3 issues and 4 PRs.
Next step: read #7187, then check whether your report is one of the 3 still open.
Read #7187Pick up a Lab taskThese issues and fixes concern how model names and provider prefixes are parsed, stripped, and forwarded across CLI commands, config repair, and API requests. Recurring themes include vendor/provider prefixes not being stripped, model fields not propagating correctly to API calls, and inconsistencies between model.name, model.default, and model.model config fields.
Open issues3
Open PRs4
Fixes on record
-
#7187 fix(model): salvage model/provider resolution PRs #6860 #6844 #6863 #6943
files:
agent/auxiliary_client.py,agent/error_classifier.py,cli.py,hermes_cli/main.py,hermes_cli/model_normalize.py,hermes_cli/model_switch.py,run_agent.py,tests/agent/test_auxiliary_named_custom_providers.py,tests/agent/test_error_classifier.py,tests/hermes_cli/test_codex_models.py,tests/hermes_cli/test_custom_provider_model_switch.py,tests/hermes_cli/test_model_normalize.py,tests/hermes_cli/test_opencode_go_in_model_list.py,tests/run_agent/test_fallback_model.py,tests/run_agent/test_run_agent.py,tests/run_agent/test_switch_model_context.py -
#7935 fix(custom-providers): propagate model field from config so API receives the correct model name
files:
cli.py,hermes_cli/config.py,hermes_cli/runtime_provider.py,tests/hermes_cli/test_runtime_provider_resolution.pyShow the diff
-
cli.py@@ -2710,6 +2710,15 @@ def _ensure_runtime_credentials(self) -> bool: self.api_key = [redacted:secret-assignment] self.base_url = base_url + # When a custom_provider entry carries an explicit `model` field, + # use it as the effective model name. Without this, running + # `hermes chat --model <provider-name>` sends the provider name + # (e.g. "my-provider") as the model string to the API instead of + # the configured model (e.g. "qwen3.6-plus"), causing 400 errors. + runtime_model = runtime.get("model") + if runtime_model and isinstance(runtime_model, str): + self.model = runtime_model + # Normalize model for the resolved provider (e.g. swap non-Codex # models when provider is openai-codex). Fixes #651. model_changed = self._normalize_model_for_provider(resolved_provider) -
hermes_cli/config.py@@ -1497,7 +1497,7 @@ def check_config_version() -> Tuple[int, int]: # Valid fields inside a custom_providers list entry _VALID_CUSTOM_PROVIDER_FIELDS = { - "name", "base_url", "api_key", "api_mode", "models", + "name", "base_url", "api_key", "api_mode", "model", "models", "context_length", "rate_limit_delay", } -
hermes_cli/runtime_provider.py@@ -304,6 +304,9 @@ def _get_named_custom_provider(requested_provider: str) -> Optional[Dict[str, An api_mode = _parse_api_mode(entry.get("api_mode")) if api_mode: result["api_mode"] = api_mode + model_name = str(entry.get("model", "") or "").strip() + if model_name: + result["model"] = model_name return result return None @@ -329,6 +332,11 @@ def _resolve_named_custom_runtime( # Check if a credential pool exists for this custom endpoint pool_result = _try_resolve_from_custom_pool(base_url, "custom", custom_provider.get("api_mode")) if pool_result: + # Propagate the model name even when using pooled credentials — + # the pool doesn't know about the custom_providers model field. + model_name = custom_provider.get("model") + if model_name: + pool_result["model"] = model_name return pool_result api_key_candidates = [ @@ -339,7 +347,7 @@ def _resolve_named_custom_runtime( ] api_key = [redacted:secret-assignment] for candidate in api_key_candidates if has_usable_secret(candidate)), "") - return { + result = { "provider": "custom", "api_mode": custom_provider.get("api_mode") or _detect_api_mode_for_url(base_url) @@ -348,6 +356,11 @@ def _resolve_named_custom_runtime( "api_key": api_key or "no-key-required", "source": f"custom_provider:{custom_provider.get('name', requested_provider)}", } + # Propagate the model name so callers can override self.model when the + # provider name differs from the actual model string the API expects. + if custom_provider.get("model"): + result["model"] = custom_provider["model"] + return result def _resolve_openrouter_runtime( -
tests/hermes_cli/test_runtime_provider_resolution.py@@ -1214,3 +1214,115 @@ def test_openrouter_provider_not_affected_by_custom_fix(monkeypatch): resolved = rp.resolve_runtime_provider(requested="openrouter") assert resolved["provider"] == "openrouter" + + +# ------------------------------------------------------------------ +# fix #7828 — custom_providers model field must propagate to runtime +# ------------------------------------------------------------------ + + +def test_get_named_custom_provider_includes_model(monkeypatch): + """_get_named_custom_provider should include the model field from config.""" + monkeypatch.setattr(rp, "load_config", lambda: { + "custom_providers": [{ + "name": "my-dashscope", + "base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1", + "api_key": "test-key", + "api_mode": "chat_completions", + "model": "qwen3.6-plus", + }], + }) + + result = rp._get_named_custom_provider("my-dashscope") + assert result is not None + assert result["model"] == "qwen3.6-plus" + + +def test_get_named_custom_provider_excludes_empty_model(monkeypatch): + """Empty or whitespace-only model field should not appear in result.""" + for model_val in ["", " ", None]: + entry = { + "name": "test-ep", + "base_url": "https://example.com/v1", + "api_key": "key", + } + if model_val is not None: + entry["model"] = model_val + + monkeypatch.setattr(rp, "load_config", lambda e=entry: { + "custom_providers": [e], + }) + + result = rp._get_named_custom_provider("test-ep") + assert result is not None + assert "model" not in result, ( + f"model field {model_val!r} should not be included in result" + ) + + +def test_named_custom_runtime_propagates_model_direct_path(monkeypatch): + """Model should propagate through the direct (non-pool) resolution path.""" + monkeypatch.setattr(rp, "resolve_provider", lambda *a, **k: "my-server") + monkeypatch.setattr( + rp, "_get_named_custom_provider", + lambda p: { + "name": "my-server", + "base_url": "http://localhost:8000/v1", + "api_key": "test-key", + "model": "qwen3.6-plus", + }, + ) + # Ensure pool doesn't intercept + monkeypatch.setattr(rp, "_try_resolve_from_custom_pool", lambda *a, **k: None) + + resolved = rp.resolve_runtime_provider(requested="my-server") + assert resolved["model"] == "qwen3.6-plus" + assert resolved["provider"] == "custom" + + +def test_named_custom_runtime_propagates_model_pool_path(monkeypatch): + """Model should propagate even when credential pool handles credentials.""" + monkeypatch.setattr(rp, "resolve_provider", lambda *a, **k: "my-server") + monkeypatch.setattr( + rp, "_get_named_custom_provider", + lambda p: { + "name": "my-server", + "base_url": "http://localhost:8000/v1", + "api_key": "test-key", + "model": "qwen3.6-plus", + }, + ) + # Pool returns a result (intercepting the normal path) + monkeypatch.setattr( + rp, "_try_resolve_from_custom_pool", + lambda *a, **k: { + "provider": "custom", + "api_mode": "chat_completions", + "base_url": "http://localhost:8000/v1", + "api_key": "pool-key", + "source": "pool:custom:my-server", + }, + ) + + resolved = rp.resolve_runtime_provider(requested="my-server") + assert resolved["model"] == "qwen3.6-plus", ( + "model must be injected into pool result" + ) + assert resolved["api_key"] == "pool-key", "pool credentials should be used" + + +def test_named_custom_runtime_no_model_when_absent(monkeypatch): + """When custom_providers entry has no model field, runtime should not either.""" + monkeypatch.setattr(rp, "resolve_provider", lambda *a, **k: "my-server") + monkeypatch.setattr( + rp, "_get_named_custom_provider", + lambda p: { + "name": "my-server", + "base_url": "http://localhost:8000/v1", + "api_key": "test-key", + }, + ) + monkeypatch.setattr(rp, "_try_resolve_from_custom_pool", lambda *a, **k: None) + + resolved = rp.resolve_runtime_provider(requested="my-server") + assert "model" not in resolved
-
-
#54093 fix(config): canonicalize model.name/model.model to model.default (#34500)
files:
gateway/run.py,hermes_cli/config.py,infographic/model-name-canon/infographic.png,tests/cli/test_cli_init.py -
#71144 fix(model_switch): don't send a picker prefix as the custom provider model id
files:
contributors/emails/[redacted:email],hermes_cli/model_switch.py,tests/hermes_cli/test_model_switch_custom_providers.py -
#115756 fix(cli): `-m openai-codex:<model>` and other provider:model switches no longer send the prefix to the API (#64787, salvage #64788)
files:
contributors/emails/[redacted:email],hermes_cli/model_normalize.py,tests/hermes_cli/test_model_normalize.py
10 PRs closed without merging
From the close comments: merged via another change 5 · done elsewhere 3 · closed without comment 1 · duplicate 1
- #6224 fix: strip provider prefix from model name for direct API providers · closed without comment
-
#6943 fix(model): normalize native provider-prefixed model ids · merged via another change →
#7187 -
#7916 fix(custom-providers): propagate model field from config so API receives the correct model name · merged via another change →
#7935 -
#34502 fix(cli): treat model.name as alias for model.default · done elsewhere →
#54093 -
#34527 fix: honor model.name for configured providers · done elsewhere →
#54093
Show 5 more
-
#55162 fix(cli): strip custom: prefix in provider_model_ids for catalog lookups · done elsewhere →
c97d9a4 -
#64788 fix(codex): strip `openai-codex:` colon prefix in model normalization · merged via another change →
7540079,#115756,#64787,#64788 -
#64916 fix(openai-codex): strip colon provider prefix in normalize_model_for_provider (#64787) · duplicate →
#64788 -
#68436 fix(models): resolve custom provider model ids · merged via another change →
#71144 -
#71558 fix(cli): centralize matching provider: and provider/ prefix stripping · merged via another change →
7540079,#115756,#64787,#64788
Closed issues · 5
- Closed#6211Z.AI provider prefix not stripped from model name, causing HTTP 400
- Closed#7828[Bug]: custom_providers model field not preserved, causing "Model Not Exist" errors
- Closed#34500custom_providers: model name not forwarded to API request
- Closed#64787openai-codex:<model> colon prefix not stripped in normalize_model_for_provider -> HTTP 400
- Closed#68347custom_providers ignores model id field when selecting from model picker; sends prefixed provider/model string instead
How this cluster was grouped
-
Link evidence (
partly structural links): some members are connected by structural evidence (shared changed files, or a closing/superseded reference); the rest are joined by an automated relevance judgement over their text. - Members outside that structural core may belong to a different underlying problem.
-
Pairwise link density
0.06: the share of item pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36.1.00means every pair is linked.
Pick up a Lab task
Check whether a fix covers the open issues here, with your own agent. Bring back evidence for maintainers.
Open the Lab