# NousResearch/hermes-agent · Title generator leaking think/reasoning blocks from LLM output ## Open now - Nothing open on GitHub. Multiple fixes address the title generator reading raw LLM output that includes or blocks instead of using the existing extract_content_or_reasoning() helper. One member also adds multi-message context, noise filtering, retry logic, and provider override to title generation. ## TL;DR - 9 member(s): 0 open issue(s), 1 closed issue(s), 0 open PR(s), 1 merged PR(s), 7 PR(s) closed without merging. - A fix was merged to main in PR #56298. Verify it addresses your case. - **Recommended:** read PR #56298 (and any competing PRs below) before filing. Confirm the merged change covers your symptom. ## Maintainer resolution verdict - Internal-tracker verdict over this cluster's issue members: `unresolved`. This is distinct from the fix status above. Weigh it yourself. ## How this cluster was grouped - Link evidence (`STRUCTURAL_SPANNING`): every member is connected by structural evidence: shared changed files, or a closing/superseded reference. - Pairwise link density `0.22`: the share of member pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36. `1.00` means every pair is linked. ## Closed / duplicate issues - #18529: \[Bug\]: title_generator.py read response.choices\[0\].message.content directly instead of use extract_content_or_reasoning() to extract tag ## Prior & competing PRs - **#18643: fix: use extract_content_or_reasoning() for title generation with reasoning models**. closed without merge · fix tier `suggested` - maintainer direction: category `uncategorized`; implication `needs_llm`; refs `#17797`, `#20266` - _suggested-tier fix detail withheld; re-request with `?tier=all` to include it._ - **#20266: fix: support reasoning models in title generation**. closed without merge · fix tier `suggested` - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `3b739b990`, `#18529` - _suggested-tier fix detail withheld; re-request with `?tier=all` to include it._ - **#56298: fix(title_generator): strip think blocks from LLM output before extracting title**. merged to main · fix tier `verified` - maintainer direction: category `merged`; implication `superseded` - files: `agent/title_generator.py`, `scripts/release.py`, `tests/agent/test_title_generator.py` - fix hunks: - `agent/title_generator.py` ```diff @@ -87,7 +87,15 @@ def generate_title( timeout=timeout, main_runtime=main_runtime, ) - title = (response.choices[0].message.content or "").strip() + content = response.choices[0].message.content or "" + # Strip thinking/reasoning blocks that think-enabled models + # (MiniMax M2.7, DeepSeek, etc.) emit even for simple prompts like + # title generation. Without this the raw ... XML + # leaks into session titles. Reuses the canonical scrubber so all + # tag variants (unterminated blocks, orphan closes, mixed case) + # are handled, not just a single literal pair. + from agent.agent_runtime_helpers import strip_think_blocks + title = strip_think_blocks(None, content).strip() # Clean up: remove quotes, trailing punctuation, prefixes like "Title: " title = title.strip('"\'') if title.lower().startswith("title:"): ``` - `scripts/release.py` ```diff @@ -48,6 +48,7 @@ "[redacted:email]": "bitcryptic-gw", # PR #53997 salvage (Matrix E2EE: resolve device_id via query_keys({mxid: []}) when whoami returns none; guard verification call sites so query_keys is never sent [null]; reset _device_id_unverified at connect() start; disconnect before reconnect) "[redacted:email]": "abchiaravalle", # PR #46997 salvage (recover resume_pending sessions: dual freshness signal + empty-turn safety net so restart auto-resume never sends a blank user turn) "[redacted:email]": "swissly", # PR #47167 salvage (wrap cron delivery thread-pool fallback in its own try/except so a per-target failure can't escape the except-RuntimeError block and crash the multi-target delivery loop; #47163) + "[redacted:email]": "shawchanshek", # PR #44126 salvage (strip ... reasoning blocks from title-generator LLM output via the canonical strip_think_blocks scrubber so reasoning-model output can't leak into session titles) "[redacted:email]": "YLChen-007", # PR #27289 salvage (case-insensitive streaming reasoning-tag filter in cli.py _stream_delta + gateway stream_consumer so mixed-case variants like / are suppressed, not just the hardcoded case literals) "[redacted:email]": "kangsoo-bit", # PR #47508 salvage (keep Telegram gateway alive on transient bootstrap network errors: best-effort deleteWebhook + resilient start_polling degrade to background recovery instead of failing startup) "[redacted:email]": "testingbuddies24", # PR #43192 salvage (strip orphan think-tag close tags in progressive gateway stream so a bare whose open was dropped upstream can't leak to the user) ``` - `tests/agent/test_title_generator.py` ```diff @@ -68,6 +68,37 @@ def test_strips_quotes(self): title = generate_title("how do I set up docker", "First install...") assert title == "Setting Up Docker Environment" + def test_strips_think_blocks(self): + """Reasoning-model output wrapped in ... must not + leak into the session title.""" + mock_response = MagicMock() + mock_response.choices = [MagicMock()] + mock_response.choices[0].message.content = ( + "The user wants a title. I'll summarize the topic " + "concisely.Debugging Python Import Errors" + ) + + with patch("agent.title_generator.call_llm", return_value=mock_response): + title = generate_title("help me fix this import", "Sure...") + assert title == "Debugging Python Import Errors" + assert "" not in title + assert "summarize" not in title + + def test_strips_unterminated_think_block(self): + """An unterminated block (no close tag) must still be + stripped so the leaked reasoning doesn't become the title.""" + mock_response = MagicMock() + mock_response.choices = [MagicMock()] + mock_response.choices[0].message.content = ( + "Let me reason about a good title for this session" + ) + + with patch("agent.title_generator.call_llm", return_value=mock_response): + title = generate_title("hello", "hi there") + # Everything from the unterminated open tag onward is stripped, + # leaving nothing → None. + assert title is None + def test_strips_title_prefix(self): mock_response = MagicMock() mock_response.choices = [MagicMock()] ``` - **#18537: Fix/title generation thinking leak**. closed without merge - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `3b739b990b3f624a6cbd823c6b91bf27d83a694a`, `#18529` - **#19008: fix(title): strip thinking blocks from auto-generated session titles**. closed without merge - maintainer direction: category `merged_via`; implication `superseded`; refs `3b739b990b3f624a6cbd823c6b91bf27d83a694a`, `#44126`, `#56298` - **#22568: fix(title-generator): strip / blocks before saving session title**. closed without merge - maintainer direction: category `duplicate_of`; implication `duplicate`; refs `#19008`, `#18529` - **#26676: feat(title-generator): multi-message context, noise filtering, retry, explicit provider override**. closed without merge - maintainer direction: category `silent_close`; implication `active` - **#27763: fix(title-gen): use extract_content_or_reasoning for reasoning models**. closed without merge - maintainer direction: category `superseded_by_pr`; implication `superseded`; refs `#18643`, `#18643`, `#20266`, `#20754`, `#18529`, `#18643` ## Fix status - Merged to main in PR #56298 (cited above). No release data is tracked. Verify downstream. - Fix last verified (merged) `2026-07-01`. - Subsystem: `agent`