Cairn

Reduce token overhead via lazy/deferred tool loading

NousResearch/hermes-agent · cluster 4379 · agent

Data as of

9 open items. Fix merged in #97979. Check it covers your case.

The short answer

9open issues (8 distinct)
1open PR
19items grouped
Known?
Yes. 19 related items grouped here, 4 issues already closed.
Fix event
A fix was merged to main in PR #97979. Check it covers your case. No release data is tracked.
Tracker
The maintainers' tracker marks this unresolved. That is separate from the fix event.
Still open
9 issues and 1 PR.

These items focus on the high token cost of including all tool schemas in every API request, proposing approaches such as deferred or lazy loading, stub modes, tiered schemas, and opt-in deferral for built-in toolsets. Several also explore RAG-style or semantic tool selection to inject only relevant schemas on demand.

Open issues9

  • Issue#13332Feature: Hybrid Tool Pre-Selection (Semantic + Keyword) — RAG-style schema injection to reduce token overhead without extra LLM round trips
  • Issue#34038RFC: On-demand tool/skill/MCP discovery — decouple schema registration from process lifecycle
  • Issue#54963Shrink the Core: Migrate Non-Essential Tools to Plugins/MCP Servers
  • Issue#55139Shrink the Core: Migrate Non-Essential Tools to Plugins/MCP Servers
  • Issue#57044Core tool schemas are ~15k tokens on every request — add a slim/tiered schema mode
  • Issue#60181[Feature]: Let built-in toolsets opt into tool-search deferral (defer_toolsets)
  • Issue#60184feat(tool_search): let built-in toolsets opt into deferral via defer_toolsets
  • Issue#63033[Feature Request] /compress should report per-segment token distribution and consider tool schema compression
  • Issue#67273Tool schemas consume 83% of every API request — per-session filtering and lazy loading needed

Open PRs1

  • PR#35457feat(tool_search): optional embedding reranker for progressive tool disclosure

Fixes on record

Merged to main. A stamp counts the issues a fix is linked to.

  • #97979 feat(tool-search): core-tool deferral — curated event-triggered set behind the bridge (−6.5K tok/call desktop, −49% schemas)

    merged to main

    files: agent/agent_runtime_helpers.py, agent/coding_context.py, agent/context_compressor.py, agent/conversation_loop.py, agent/display.py, agent/tool_executor.py, agent/tool_guardrails.py, agent/turn_summary.py, evals/core_tool_deferral/README.md, evals/core_tool_deferral/orchestrator.py, evals/core_tool_deferral/report.py, evals/core_tool_deferral/results/.gitignore, evals/core_tool_deferral/results/SUMMARY.md, evals/core_tool_deferral/tasks.py, evals/core_tool_deferral/worker.py, hermes_cli/tools_config.py, model_tools.py, tests/agent/test_curator.py, tests/agent/test_display_todo_progress.py, tests/agent/test_phantom_tool_references.py (+32 more)

4 PRs closed without merging

From the close comments: done elsewhere 2 · stale or abandoned 1 · superseded by another PR 1

  • #43521 feat(tool_search): config-gated builtin tool deferral (#6839) · done elsewhere → 7e310f5, #97979
  • #48622 feat(tools): add stub_mode to reduce main-thread tool schema tokens · superseded by another PR
  • #60182 feat(tool_search): defer_toolsets — opt built-in toolsets into deferral · done elsewhere → 7e310f5, #97979
  • #61744 feat(tool_search): opt-in deferral for infrequent built-in toolsets · stale or abandoned
Closed issues · 4
  • Closed#4379Token overhead analysis: 73% of each API call is fixed overhead (~13.9K tokens) — data + suggestions
  • Closed#6839Feature: Lazy Tool Schema Loading — Two-Pass Tool Injection to Reduce Token Overhead
  • Closed#7961[Feature] Per-profile tool selection — opt-in lightweight toolsets
  • Closed#66473Feature: MCP Smart Loading — Lazy Connection, Tool Budget, Per-Session Scoping & On-Demand Servers
How this cluster was grouped
  • Link evidence (partly structural links): some members are connected by structural evidence (shared changed files, or a closing/superseded reference); the rest are joined by an automated relevance judgement over their text.
  • Members outside that structural core may belong to a different underlying problem.
  • Pairwise link density 0.08: the share of item pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36. 1.00 means every pair is linked.

Pick up a Lab task

Check whether a fix covers the open issues here, with your own agent. Bring back evidence for maintainers.

Open the Lab

At a glance

Subsystem
agent
Merged PRs
#97979
Closed issues
4
Attempts
4 PRs closed without merging
Fix link checked
2 Sep 2026

For your agent

The same facts, typed.

cluster/4379.json
cluster/4379.md