# NousResearch/hermes-agent · MCP tool result content handling and provider fixes ## Open now - 3 issues, 3 PRs open on GitHub. - #55092: Bedrock Converse: empty tool result produces an empty text block and 400s the request - #55134: Bedrock Converse: image tool results are flattened to a base64 text dump, hiding them from the model - #55434: Native Gemini silently drops remote (http) image URLs: _extract_multimodal_parts only handles data: URLs - PR #55135: Convert image tool results to Converse image blocks for Bedrock - PR #55460: fix(gemini): fetch remote http(s) image URLs instead of silently dropping - PR #117729: fix(bedrock): tool-result images reach the model as Converse image blocks (#55134, salvage #55135) These issues and PRs address multiple related problems: MCP tool results silently dropping EmbeddedResource and image content blocks, empty or unhelpful MCP error messages, and provider-specific bugs in Bedrock Converse and Gemini around handling image and empty tool results. A BlueBubbles iMessage platform adapter is also included. ## TL;DR - 28 member(s): 3 open issue(s), 7 closed issue(s), 3 open PR(s), 6 merged PR(s), 9 PR(s) closed without merging. - A fix was merged to main in PR #21292. Verify it addresses your case. - **Recommended:** read PR #21292 (and any competing PRs below) before filing. Confirm the merged change covers your symptom. ## Maintainer resolution verdict - Internal-tracker verdict over this cluster's issue members: `partially_resolved` · deciding PR(s): `pr-6437,pr-21289`. This is distinct from the fix status above. Weigh it yourself. ## How this cluster was grouped - Link evidence (`STRUCTURAL_PARTIAL`): some members are connected by structural evidence (shared changed files, or a closing/superseded reference); the rest are joined by an automated relevance judgement over their text. - Members outside that structural core may belong to a different underlying problem. - Pairwise link density `0.06`: the share of member pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36. `1.00` means every pair is linked. ## Open related issues - #55092: Bedrock Converse: empty tool result produces an empty text block and 400s the request - #55134: Bedrock Converse: image tool results are flattened to a base64 text dump, hiding them from the model - #55434: Native Gemini silently drops remote (http) image URLs: _extract_multimodal_parts only handles data: URLs ## Closed / duplicate issues - #3892: mcp_bluebubbles_send_imessage failing with empty error message - #10759: \[Bug\]: MCP tool responses with ImageContent blocks are silently discarded - #19417: \[Bug\]: MCP tool calls fail with \`ClosedResourceError\` and empty error message - #21559: MCP tool responses: image content silently dropped + large text results truncated - #26972: \[Feature\]: Fix for: Hermes Agent Silently Drops \`EmbeddedResource\` Content Blocks from MCP Tool Results - #30601: MCP tool results with EmbeddedResource (resource) blocks are silently dropped - #55426: Native Gemini drops images returned by tools: _translate_tool_result_to_gemini coerces tool-result content to text only ## Prior & competing PRs - **#13736: fix(mcp): extract text from EmbeddedResource content blocks in tool results**. closed without merge · fix tier `suggested` - maintainer direction: category `superseded_by_pr`; implication `superseded`; refs `#18118` - _suggested-tier fix detail withheld; re-request with `?tier=all` to include it._ - **#21292: fix(mcp): include exception type in error messages when str(exc) is empty (salvage #19425)**. merged to main · fix tier `verified` - maintainer direction: category `merged`; implication `superseded` - files: `tests/tools/test_mcp_empty_error_message.py`, `tools/mcp_tool.py` - **#21328: fix(mcp): surface image tool results as MEDIA tags instead of dropping them**. merged to main · fix tier `verified` - maintainer direction: category `merged`; implication `superseded` - files: `tests/tools/test_mcp_image_content.py`, `tools/mcp_tool.py` - fix hunks: - `tests/tools/test_mcp_image_content.py` ```diff @@ -0,0 +1,138 @@ +"""Regression tests for MCP ImageContent block handling. + +Background +========== +MCP tool results may include ``ImageContent`` blocks (screenshots from +Playwright / Blockbench / Puppeteer / any server that returns renders). +The tool result handler in ``tools/mcp_tool.py`` used to iterate content +blocks looking only for ``block.text`` — image blocks were silently dropped +and the agent saw an empty result. Distilled from @c3115644151's PR #17915 +and @gnanirahulnutakki's PR #10848 (both too stale to cherry-pick); this +test file locks in #10848's approach of plumbing the bytes through +Hermes' existing ``cache_image_from_bytes`` so a ``MEDIA:`` tag +goes back to the agent and through to messaging adapters that render +images natively. +""" + +from __future__ import annotations + +import base64 +from types import SimpleNamespace +from unittest.mock import patch + +import pytest + + +def _png_bytes(): + """Return a minimal valid PNG byte sequence. + + Hermes' ``cache_image_from_bytes`` has a format-sniff guard that rejects + non-image payloads — use a real PNG signature so the test exercises the + full pipeline instead of the reject path. + """ + # 1x1 transparent PNG + return base64.b64decode( + "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=" + ) + + +class TestMimeExtension: + def test_maps_jpeg_variants_to_jpg(self): + from tools.mcp_tool import _mcp_image_extension_for_mime_type + assert _mcp_image_extension_for_mime_type("image/jpeg") == ".jpg" + assert _mcp_image_extension_for_mime_type("image/jpg") == ".jpg" + assert _mcp_image_extension_for_mime_type("IMAGE/JPEG") == ".jpg" + assert _mcp_image_extension_for_mime_type("image/jpeg; charset=utf-8") == ".jpg" + + def test_png_falls_through_to_mimetypes(self): + from tools.mcp_tool import _mcp_image_extension_for_mime_type + assert _mcp_image_extension_for_mime_type("image/png") == ".png" + + def test_unknown_defaults_to_png(self): + from tools.mcp_tool import _mcp_image_extension_for_mime_type + assert _mcp_image_extension_for_mime_type("") == ".png" + assert _mcp_image_extension_for_mime_type("image/unheard-of-format") == ".png" + + +class TestCacheMcpImageBlock: + def test_returns_media_tag_for_valid_image_block(self, tmp_path, monkeypatch): + """A well-formed ImageContent block with valid PNG bytes caches + to the image dir and the helper returns a ``MEDIA:`` tag.""" + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + from tools.mcp_tool import _cache_mcp_image_block + + block = SimpleNamespace( + data=base64.b64encode(_png_bytes()).decode("ascii"), + mimeType="image/png", + ) + tag = _cache_mcp_image_block(block) + assert tag.startswith("MEDIA:"), f"expected MEDIA: tag, got {tag!r}" + # The cached file should be in Hermes' image cache dir + from gateway.platforms.base import get_image_cache_dir + cache_dir = str(get_image_cache_dir().resolve()) + assert tag.startswith(f"MEDIA:{cache_dir}"), ( + f"cached file not under HERMES_HOME image cache dir. " + f"tag={tag!r}, cache_dir={cache_dir!r}" + ) + # And it should exist + have the PNG bytes + path = tag[len("MEDIA:"):] + with open(path, "rb") as fh: + assert fh.read() == _png_bytes() + + def test_returns_empty_when_block_is_not_an_image(self, tmp_path, monkeypatch): + """Non-image MIME types shouldn't trigger caching.""" + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + from tools.mcp_tool import _cache_mcp_image_block + + block = SimpleNamespace( + data=base64.b64encode(b"some bytes").decode("ascii"), + mimeType="application/pdf", + ) + assert _cache_mcp_image_block(block) == "" + + def test_returns_empty_when_block_has_no_data(self, tmp_path, monkeypatch): + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + from tools.mcp_tool import _cache_mcp_image_block + + block = SimpleNamespace(data=None, mimeType="image/png") + assert _cache_mcp_image_block(block) == "" + + def test_returns_empty_on_malformed_base64(self, tmp_path, monkeypatch): + """A server that sends garbage base64 shouldn't crash the handler — + we log and drop the block, letting any text blocks still come through.""" + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + from tools.mcp_tool import _cache_mcp_image_block + + block = SimpleNamespace( + data="!!!not-base64!!!", + mimeType="image/png", + ) + assert _cache_mcp_image_block(block) == "" + + def test_returns_empty_when_bytes_dont_look_like_an_image(self, tmp_path, monkeypatch): + """``cache_image_from_bytes`` has a format sniff; if the claimed + ``image/png`` is actually an HTML error page, the cache raises and + we log + drop rather than propagate.""" + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + from tools.mcp_tool import _cache_mcp_image_block + + block = SimpleNamespace( + data=base64.b64encode(b"error").decode("ascii"), + mimeType="image/png", + ) + assert _cache_mcp_image_block(block) == "" + + def test_handles_jpeg(self, tmp_path, monkeypatch): + """JPEG signature should also be accepted.""" + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + from tools.mcp_tool import _cache_mcp_image_block + + # minimal JPEG SOI marker + filler + jpeg = b"\xff\xd8\xff\xe0" + b"\x00" * 100 + b"\xff\xd9" + block = SimpleNamespace( + data=base64.b64encode(jpeg).decode("ascii"), + mimeType="image/jpeg", + ) + tag = _cache_mcp_image_block(block) + assert tag.startswith("MEDIA:") + assert tag.endswith(".jpg"), f"expected .jpg extension, got {tag!r}" ``` - `tools/mcp_tool.py` ```diff @@ -426,6 +426,64 @@ def _resolve_stdio_command(command: str, env: dict) -> tuple[str, dict]: return resolved_command, resolved_env +# --------------------------------------------------------------------------- +# MCP ImageContent block → Hermes MEDIA tag +# --------------------------------------------------------------------------- + + +def _mcp_image_extension_for_mime_type(mime_type: str) -> str: + """Return a reasonable file extension for an MCP image MIME type.""" + import mimetypes + normalized = (mime_type or "").split(";", 1)[0].strip().lower() + if normalized in {"image/jpeg", "image/jpg"}: + return ".jpg" + return mimetypes.guess_extension(normalized) or ".png" + + +def _cache_mcp_image_block(block) -> str: + """Cache an MCP ``ImageContent`` block to the shared image cache and + return a ``MEDIA:`` tag that Hermes gateways know how to render. + + Returns an empty string when *block* is not an image, when the base64 + payload is malformed, or when the cache helper rejects the bytes (e.g. + non-image MIME masquerading as an image). Errors are logged, not raised: + a single bad block shouldn't kill the tool result, and the caller will + fall through to any text blocks that did parse. + """ + import base64 + + data = getattr(block, "data", None) + mime_type = getattr(block, "mimeType", None) + normalized_mime = str(mime_type or "").split(";", 1)[0].strip().lower() + if data is None or not normalized_mime.startswith("image/"): + return "" + + try: + raw_bytes = base64.b64decode(data) + except (TypeError, ValueError) as exc: + logger.warning("MCP image block decode failed (%s): %s", normalized_mime, exc) + return "" + + try: + from gateway.platforms.base import cache_image_from_bytes + + image_path = cache_image_from_bytes( + raw_bytes, + ext=_mcp_image_extension_for_mime_type(normalized_mime), + ) + except ImportError: + # gateway.platforms.base not importable in this process (e.g. cron + # without gateway deps). Fall back to silently dropping — callers + # get any text blocks that did parse. + logger.debug("MCP image caching skipped — gateway.platforms.base unavailable") + return "" + except Exception as exc: + logger.warning("MCP image block cache failed: %s", exc) + return "" + + return f"MEDIA:{image_path}" + + def _format_connect_error(exc: BaseException) -> str: """Render nested MCP connection errors into an actionable short message.""" ``` ```diff @@ -2146,11 +2204,25 @@ async def _call(): ) }, ensure_ascii=False) - # Collect text from content blocks + # Collect text from content blocks. MCP tool results can also + # include ImageContent blocks (screenshot / Blockbench / Playwright + # etc.); cache those via the gateway's image-cache helper so they + # flow through Hermes' MEDIA: tag convention and out to messaging + # adapters that render images natively. Without this, image blocks + # were silently dropped and the agent got an empty response. + # + # Distilled from #17915 (c3115644151) and #10848 (gnanirahulnutakki), + # both too stale to cherry-pick. #10848's approach (integrate with + # Hermes' MEDIA tag + cache_image_from_bytes) was the cleaner of + # the two — plugs into existing infrastructure. parts: List[str] = [] for block in (result.content or []): - if hasattr(block, "text"): + if hasattr(block, "text") and block.text: parts.append(block.text) + continue + image_tag = _cache_mcp_image_block(block) + if image_tag: + parts.append(image_tag) text_result = "\n".join(parts) if parts else "" # Combine content + structuredContent when both are present. ``` - **#64556: fix(mcp): materialize ResourceLink/EmbeddedResource/Audio blocks instead of dropping them (salvage #64061)**. merged to main · fix tier `verified` - maintainer direction: category `merged`; implication `superseded` - files: `tests/tools/test_mcp_resource_content.py`, `tools/mcp_tool.py` - fix hunks: - `tests/tools/test_mcp_resource_content.py` ```diff @@ -0,0 +1,281 @@ +"""Tests for MCP ResourceLink / EmbeddedResource / AudioContent handling. + +Regression coverage for a customer report (2026-07): non-image binary +resources returned through MCP resource blocks were silently dropped from +tool results, so a PDF-returning MCP tool appeared to return metadata only. +""" + +import base64 +import json +from types import SimpleNamespace + +import pytest + + +PDF_BYTES = b"%PDF-1.4 fake pdf payload for tests" + + +def _blob_resource(data: bytes, uri="slack://files/F123/report.pdf", mime="application/pdf"): + return SimpleNamespace( + uri=uri, + mimeType=mime, + blob=base64.b64encode(data).decode("ascii"), + text=None, + ) + + +def _embedded(resource): + return SimpleNamespace(type="resource", resource=resource) + + [redacted:email]() +def doc_cache(tmp_path, monkeypatch): + """Point the document cache at a temp dir.""" + import gateway.platforms.base as base + + monkeypatch.setattr(base, "DOCUMENT_CACHE_DIR", tmp_path) + monkeypatch.setenv("HERMES_DOCUMENT_CACHE_DIR", str(tmp_path)) + # _resolve_cache_dir consults the module constant; patching the constant + # is sufficient because the import-default comparison detects the change. + return tmp_path + + +class TestRenderResourceBlock: + def test_embedded_pdf_blob_is_materialized(self, doc_cache): + from tools.mcp_tool import _render_mcp_resource_block + + out = _render_mcp_resource_block(_embedded(_blob_resource(PDF_BYTES)), "slack") + assert "saved to" in out + assert "application/pdf" in out + # Extract path and verify bytes round-trip + path = out.split("saved to ", 1)[1].split(" (", 1)[0] + with open(path, "rb") as fh: + assert fh.read() == PDF_BYTES + assert "report.pdf" in path + + def test_embedded_text_resource_is_inlined(self): + from tools.mcp_tool import _render_mcp_resource_block + + res = SimpleNamespace(uri="mem://notes", mimeType="text/plain", text="hello world", blob=None) + assert _render_mcp_resource_block(_embedded(res), "srv") == "hello world" + + def test_resource_link_preserves_uri_and_points_at_reader(self): + from tools.mcp_tool import _render_mcp_resource_block + + link = SimpleNamespace( + type="resource_link", + uri="slack://files/F123", + name="report.pdf", + mimeType="application/pdf", + ) + out = _render_mcp_resource_block(link, "slack") + assert "slack://files/F123" in out + # Must be the real wire name (mcp____read_resource), not a + # made-up "_read_resource" the agent can't actually call. + assert "mcp__slack__read_resource" in out + assert "report.pdf" in out + + def test_oversized_blob_fails_explicitly_without_writing(self, doc_cache, monkeypatch): + import tools.mcp_tool as m + + monkeypatch.setattr(m, "_MCP_RESOURCE_MAX_BYTES", 8) + out = m._render_mcp_resource_block(_embedded(_blob_resource(PDF_BYTES)), "srv") + assert "too large" in out + assert not list(doc_cache.glob("doc_*")) + + def test_malformed_base64_fails_explicitly(self): + from tools.mcp_tool import _render_mcp_resource_block + + res = SimpleNamespace(uri="x://y", mimeType="application/pdf", blob="!!!not-base64!!!", text=None) + out = _render_mcp_resource_block(_embedded(res), "srv") + assert "could not be decoded" in out + + def test_non_resource_block_returns_empty(self): + from tools.mcp_tool import _render_mcp_resource_block + + assert _render_mcp_resource_block(SimpleNamespace(type="text", text="hi"), "srv") == "" + + def test_path_traversal_uri_is_neutralized(self, doc_cache): + from tools.mcp_tool import _render_mcp_resource_block + + res = _blob_resource(PDF_BYTES, uri="evil://host/../../etc/passwd") + out = _render_mcp_resource_block(_embedded(res), "srv") + assert "saved to" in out + path = out.split("saved to ", 1)[1].split(" (", 1)[0] + assert str(doc_cache) in path + assert "/etc/passwd" not in path + + +class TestResourceFilename: + def test_uri_last_segment_used(self): + from tools.mcp_tool import _mcp_resource_filename + + assert _mcp_resource_filename("slack://f/ABC/quarterly.pdf", "application/pdf") == "quarterly.pdf" + + def test_fallback_to_mime_extension(self): + from tools.mcp_tool import _mcp_resource_filename + + name = _mcp_resource_filename("", "application/pdf") + assert name.endswith(".pdf") + + def test_dotdot_rejected(self): + from tools.mcp_tool import _mcp_resource_filename + + assert _mcp_resource_filename("x://y/..", "application/pdf") != ".." + + def test_control_chars_stripped(self): + from tools.mcp_tool import _mcp_resource_filename + + name = _mcp_resource_filename("x://h/report.pdf%0Ainjected%1b[31m", "application/pdf") + assert "\n" not in name and "\x1b" not in name + + def test_long_filename_capped_preserving_extension(self): + from tools.mcp_tool import _mcp_resource_filename + + name = _mcp_resource_filename("x://h/" + "a" * 500 + ".pdf", "application/pdf") + assert len(name) <= 150 + assert name.endswith(".pdf") + + +class TestPreDecodeSizeCap: + def test_oversized_b64_rejected_before_decode(self, monkeypatch): + import tools.mcp_tool as m + + monkeypatch.setattr(m, "_MCP_RESOURCE_MAX_B64_CHARS", 16) + res = SimpleNamespace( + uri="x://y/big.pdf", mimeType="application/pdf", + blob="A" * 100, text=None, + ) + called = [] + monkeypatch.setattr(base64, "b64decode", lambda *a, **k: called.append(1)) + out = m._render_mcp_resource_block(_embedded(res), "srv") + assert "too large" in out + assert not called + + +class TestAudioBlock: + def test_non_audio_returns_empty(self): + from tools.mcp_tool import _cache_mcp_audio_block + + block = SimpleNamespace(data=base64.b64encode(b"x").decode(), mimeType="application/pdf") + assert _cache_mcp_audio_block(block) == "" + + def test_audio_block_cached_as_media(self, tmp_path, monkeypatch): + import gateway.platforms.base as base + from tools.mcp_tool import _cache_mcp_audio_block + + monkeypatch.setattr(base, "AUDIO_CACHE_DIR", tmp_path) + block = SimpleNamespace( + data=base64.b64encode(b"RIFFfakewav").decode(), + mimeType="audio/wav", + ) + out = _cache_mcp_audio_block(block) + assert out.startswith("MEDIA:") + + +class TestToolResultLoopOrdering: + def test_mixed_blocks_preserve_order(self, doc_cache): + """Simulate the tool-result block loop with text + pdf resource.""" + from tools.mcp_tool import ( + _cache_mcp_image_block, + _cache_mcp_audio_block, + _render_mcp_resource_block, + ) + + blocks = [ + SimpleNamespace(type="text", text="File ID: F123\nMIME Type: application/pdf"), + _embedded(_blob_resource(PDF_BYTES)), + ] + parts = [] + for block in blocks: + if getattr(block, "text", None): + parts.append(block.text) + continue + tag = _cache_mcp_image_block(block) or _cache_mcp_audio_block(block) + if tag: + parts.append(tag) + continue + rendered = _render_mcp_resource_block(block, "slack") + if rendered: + parts.append(rendered) + assert len(parts) == 2 + assert parts[0].startswith("File ID") + assert "saved to" in parts[1] + + def test_existing_image_behavior_unchanged(self): + from tools.mcp_tool import _cache_mcp_image_block + + block = SimpleNamespace( + data=base64.b64encode(b"some bytes").decode("ascii"), + mimeType="application/pdf", + ) + assert _cache_mcp_image_block(block) == "" + + +class TestErrorPathResourceText: + """isError payloads must surface EmbeddedResource text, not drop it.""" + + @pytest.fixture() + def _handler(self, monkeypatch): + import asyncio + from unittest.mock import AsyncMock, MagicMock, patch as mock_patch + + from tools import mcp_tool + + fake_session = MagicMock() + fake_server = SimpleNamespace(session=fake_session, _rpc_lock=None) + + def _fake_run_on_mcp_loop(coro_or_factory, timeout=30): + coro = coro_or_factory() if callable(coro_or_factory) else coro_or_factory + loop = asyncio.new_event_loop() + try: + async def _install_lock_and_run(): + for srv in list(mcp_tool._servers.values()): + if getattr(srv, "_rpc_lock", None) is None: + srv._rpc_lock = asyncio.Lock() + return await coro + + return loop.run_until_complete(_install_lock_and_run()) + finally: + loop.close() + + with mock_patch.dict(mcp_tool._servers, {"test-server": fake_server}), \ + mock_patch("tools.mcp_tool._run_on_mcp_loop", side_effect=_fake_run_on_mcp_loop): + fake_session.call_tool = AsyncMock() + yield fake_session, mcp_tool._make_tool_handler("test-server", "my-tool", 30.0) + + def test_error_embedded_resource_text_surfaced(self, _handler): + from unittest.mock import AsyncMock + + session, handler = _handler + res = SimpleNamespace(uri="mem://err", mimeType="text/plain", + text="quota exceeded for workspace W1", blob=None) + session.call_tool = AsyncMock(return_value=SimpleNamespace( + content=[_embedded(res)], isError=True, structuredContent=None, + )) + data = json.loads(handler({})) + assert "quota exceeded for workspace W1" in data["error"] + + def test_error_mixed_text_and_resource(self, _handler): + from unittest.mock import AsyncMock + + session, handler = _handler + res = SimpleNamespace(uri="mem://err", mimeType="text/plain", + text=" — details in resource", blob=None) + session.call_tool = AsyncMock(return_value=SimpleNamespace( + content=[SimpleNamespace(type="text", text="tool failed"), _embedded(res)], + isError=True, structuredContent=None, + )) + data = json.loads(handler({})) + assert "tool failed" in data["error"] + assert "details in resource" in data["error"] + + def test_error_with_no_text_blocks_falls_back(self, _handler): + from unittest.mock import AsyncMock + + session, handler = _handler + session.call_tool = AsyncMock(return_value=SimpleNamespace( + content=[], isError=True, structuredContent=None, + )) + data = json.loads(handler({})) + assert data["error"] == "MCP tool returned an error" ``` - `tools/mcp_tool.py` ```diff @@ -102,6 +102,7 @@ import sys import threading import time +from types import SimpleNamespace from typing import Callable from datetime import datetime from typing import Any, Coroutine, Dict, List, Optional ``` ```diff @@ -714,6 +715,169 @@ def _cache_mcp_image_block(block) -> str: return f"MEDIA:{image_path}" +# --------------------------------------------------------------------------- +# MCP resource blocks (ResourceLink / EmbeddedResource / AudioContent) +# --------------------------------------------------------------------------- + +# Hard cap on decoded resource bytes materialized from an MCP tool result. +# Prevents a misbehaving server from filling the cache disk via one block. +_MCP_RESOURCE_MAX_BYTES = 50 * 1024 * 1024 + +# Base64 expands raw bytes by ~4/3; reject oversized payloads before decoding +# so a multi-GB blob string is never transiently doubled in memory. +_MCP_RESOURCE_MAX_B64_CHARS = _MCP_RESOURCE_MAX_BYTES * 4 // 3 + 4 + + +def _mcp_resource_filename(uri: str, mime_type: str) -> str: + """Derive a safe display filename for an MCP resource. + + Only the last path segment of the URI is considered, and only as a + *name hint* — `cache_document_from_bytes` re-sanitizes and prefixes it, + so remote path components can't influence the cache location. + """ + import mimetypes + import re as _re + from pathlib import Path + from urllib.parse import urlparse, unquote + + name = "" + if uri: + try: + name = Path(unquote(urlparse(str(uri)).path or "")).name + except (ValueError, TypeError): + name = "" + # Strip control characters (newlines/ANSI escapes from hostile URIs would + # otherwise land in the filename and the transcript marker) and cap the + # length, preserving the extension. + name = _re.sub(r"[\x00-\x1f\x7f]", "", name).strip() + if len(name) > 150: + stem, dot, ext = name.rpartition(".") + if dot and 0 < len(ext) <= 12: + name = stem[: 150 - len(ext) - 1] + "." + ext + else: + name = name[:150] + if not name or name in {".", ".."}: + normalized = (mime_type or "").split(";", 1)[0].strip().lower() + ext = mimetypes.guess_extension(normalized) or ".bin" + name = f"resource{ext}" + return name + + +def _cache_mcp_audio_block(block) -> str: + """Cache an MCP ``AudioContent`` block and return a ``MEDIA:`` tag. + + Returns an empty string when *block* is not audio or on any failure — + same fail-open contract as ``_cache_mcp_image_block``. + """ + import base64 + + data = getattr(block, "data", None) + mime_type = str(getattr(block, "mimeType", None) or "").split(";", 1)[0].strip().lower() + if data is None or not mime_type.startswith("audio/"): + return "" + if len(data) > _MCP_RESOURCE_MAX_B64_CHARS: + return f"[MCP audio resource too large to cache: ~{len(data) * 3 // 4} bytes]" + try: + raw_bytes = base64.b64decode(data) + except (TypeError, ValueError) as exc: + logger.warning("MCP audio block decode failed (%s): %s", mime_type, exc) + return "" + if len(raw_bytes) > _MCP_RESOURCE_MAX_BYTES: + return f"[MCP audio resource too large to cache: {len(raw_bytes)} bytes]" + try: + from gateway.platforms.base import cache_audio_from_bytes + import mimetypes + + ext = ( + {"audio/wav": ".wav", "audio/x-wav": ".wav", "audio/wave": ".wav"}.get(mime_type) + or mimetypes.guess_extension(mime_type) + or ".ogg" + ) + audio_path = cache_audio_from_bytes(raw_bytes, ext=ext) + except ImportError: + logger.debug("MCP audio caching skipped — gateway.platforms.base unavailable") + return "" + except Exception as exc: + logger.warning("MCP audio block cache failed: %s", exc) + return "" + return f"MEDIA:{audio_path}" + + +def _render_mcp_resource_block(block, server_name: str = "") -> str: + """Render an MCP ``ResourceLink`` or ``EmbeddedResource`` block as text. + + - ``EmbeddedResource`` with text contents → the text itself. + - ``EmbeddedResource`` with blob contents → bytes are decoded (size-capped) + and materialized into the Hermes document cache; returns a marker with + the local path so file/terminal tools can consume it. + - ``ResourceLink`` → the URI plus a pointer at the server's read_resource + tool. No network fetch happens here; the link is only readable through + the originating MCP session. + + Returns an empty string for non-resource blocks. Failures are logged and + reported inline rather than silently dropping the block. + """ + block_type = getattr(block, "type", "") + + if block_type == "resource_link" or ( + hasattr(block, "uri") and not hasattr(block, "resource") and block_type != "text" + ): + uri = getattr(block, "uri", None) + if not uri: + return "" + name = getattr(block, "name", "") or "" + mime = getattr(block, "mimeType", "") or "" + details = f"uri={uri}" + if name: + details += f", name={name}" + if mime: + details += f", mimeType={mime}" + reader = ( + mcp_prefixed_tool_name(server_name, "read_resource") + if server_name + else "the MCP server's read_resource tool" + ) + return f"[MCP resource link: {details} — fetch it with {reader}]" + + resource = getattr(block, "resource", None) + if resource is None: + return "" + + text = getattr(resource, "text", None) + if text is not None: + return str(text) + + blob = getattr(resource, "blob", None) + if blob is None: + return "" + + import base64 + + uri = str(getattr(resource, "uri", "") or "") + mime = str(getattr(resource, "mimeType", "") or "") + if len(blob) > _MCP_RESOURCE_MAX_B64_CHARS: + return f"[MCP embedded resource too large to cache: ~{len(blob) * 3 // 4} bytes, uri={uri}]" + try: + raw_bytes = base64.b64decode(blob) + except (TypeError, ValueError) as exc: + logger.warning("MCP embedded resource decode failed (%s): %s", mime or uri, exc) + return f"[MCP embedded resource could not be decoded: {mime or uri}]" + if len(raw_bytes) > _MCP_RESOURCE_MAX_BYTES: + return f"[MCP embedded resource too large to cache: {len(raw_bytes)} bytes, uri={uri}]" + try: + from gateway.platforms.base import cache_document_from_bytes + + path = cache_document_from_bytes(raw_bytes, _mcp_resource_filename(uri, mime)) + except ImportError: + logger.debug("MCP resource caching skipped — gateway.platforms.base unavailable") + return f"[MCP embedded resource received ({len(raw_bytes)} bytes, {mime or 'unknown type'}) but document cache unavailable in this process]" + except Exception as exc: + logger.warning("MCP embedded resource cache failed: %s", exc) + return f"[MCP embedded resource could not be cached: {mime or uri}]" + detail = mime or "unknown type" + return f"[MCP resource saved to {path} ({detail}, {len(raw_bytes)} bytes) — read it with read_file or terminal tools]" + + # --------------------------------------------------------------------------- # Remote MCP URL validation # --------------------------------------------------------------------------- ``` ```diff @@ -3947,8 +4111,15 @@ async def _call(): if result.isError: error_text = "" for block in (result.content or []): - if hasattr(block, "text"): + if getattr(block, "text", None): error_text += block.text + continue + # EmbeddedResource blocks inside error payloads carry + # their text under .resource.text — previously dropped, + # leaving a bare "MCP tool returned an error". + res_text = getattr(getattr(block, "resource", None), "text", None) + if res_text: + error_text += str(res_text) return json.dumps({ "error": _sanitize_error( error_text or "MCP tool returned an error" ``` ```diff @@ -3974,6 +4145,34 @@ async def _call(): image_tag = _cache_mcp_image_block(block) if image_tag: parts.append(image_tag) + continue + audio_tag = _cache_mcp_audio_block(block) + if audio_tag: + parts.append(audio_tag) + continue + # ResourceLink / EmbeddedResource blocks (PDFs, archives, + # office docs, ...). Previously these were silently dropped, + # so document-oriented MCP tools appeared to return metadata + # only (enterprise customer report, 2026-07). + resource_text = _render_mcp_resource_block(block, server_name) + if resource_text: + parts.append(resource_text) + continue + # Benign empty renders (empty text blocks, empty text + # resources, audio in a process without the gateway cache) + # aren't data loss — log at debug. Warn only for genuinely + # unrecognized block shapes. + block_type = getattr(block, "type", None) or type(block).__name__ + if block_type in {"text", "resource", "audio", "image"}: + logger.debug( + "MCP %s: content block type %r rendered empty", + server_name, block_type, + ) + else: + logger.warning( + "MCP %s: dropping unsupported content block type %r", + server_name, block_type, + ) text_result = "\n".join(parts) if parts else "" # Combine content + structuredContent when both are present. ``` ```diff @@ -4124,10 +4323,17 @@ async def _call(): parts: List[str] = [] contents = result.contents if hasattr(result, "contents") else [] for block in contents: - if hasattr(block, "text"): + if getattr(block, "text", None) is not None: parts.append(block.text) - elif hasattr(block, "blob"): - parts.append(f"[binary data, {len(block.blob)} bytes]") + elif getattr(block, "blob", None) is not None: + # Materialize binary resource contents into the document + # cache instead of discarding them (same contract as + # EmbeddedResource blocks in tool results). + rendered = _render_mcp_resource_block( + SimpleNamespace(type="resource", resource=block), + server_name, + ) + parts.append(rendered or f"[binary data, {len(block.blob)} bytes]") return json.dumps({"result": "\n".join(parts) if parts else ""}, ensure_ascii=False) def _call_once(): ``` - **#96466: fix(gemini): embed images in Gemini 3.x functionResponse.parts for multimodal tool results**. merged to main · fix tier `verified` - maintainer direction: category `merged`; implication `superseded` - files: `agent/gemini_native_adapter.py`, `tests/agent/test_gemini_native_adapter.py` - **#6437: feat(gateway): add BlueBubbles iMessage platform adapter**. merged to main - maintainer direction: category `merged`; implication `superseded` - files: `agent/prompt_builder.py`, `cron/scheduler.py`, `gateway/channel_directory.py`, `gateway/config.py`, `gateway/platforms/bluebubbles.py`, `gateway/run.py`, `hermes_cli/config.py`, `hermes_cli/gateway.py`, `hermes_cli/status.py`, `hermes_cli/tools_config.py`, `tests/gateway/test_bluebubbles.py`, `tools/cronjob_tools.py`, `tools/send_message_tool.py`, `toolsets.py` - **#10848: fix(mcp): preserve ImageContent tool results**. closed without merge - maintainer direction: category `merged_via`; implication `superseded`; refs `c8e3e39185`, `c3115644151`, `#21328`, `#17915` - **#19425: fix(mcp): include exception type in error messages when str(exc) is empty**. closed without merge - maintainer direction: category `silent_close`; implication `active` - **#20026: fix: improve MCP error messages for exceptions with empty str()**. closed without merge - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `f9b4b8af3410e13ba002129e25c3f212568ee031`, `#19425`, `#19417` - **#21289: fix(mcp): retry stale pipe transport failures as session-expired**. merged to main - maintainer direction: category `merged`; implication `superseded` - files: `scripts/release.py`, `tests/tools/test_mcp_tool_session_expired.py`, `tools/mcp_tool.py` - **#31356: fix(mcp): preserve embedded resource text in tool results (#30601)**. closed without merge - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `#64556` - **#32352: fix(agent): embed images in Gemini 3.x functionResponse.parts for multimodal tool results**. closed without merge - maintainer direction: category `merged_via`; implication `superseded`; refs `#96466`, `#55474` - **#50999: fix(mcp): extract EmbeddedResource content blocks from tool results**. closed without merge - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `#64556`, `#64061` - **#55093: Guard empty Bedrock Converse tool results against empty text blocks**. closed without merge - maintainer direction: category `superseded_by_pr`; implication `superseded`; refs `#66167` - **#55135: Convert image tool results to Converse image blocks for Bedrock**. open - maintainer direction: category `duplicate_of`; implication `duplicate`; refs `#117729`, `#117729` - files: `agent/bedrock_adapter.py`, `tests/agent/test_bedrock_adapter.py` - **#55460: fix(gemini): fetch remote http(s) image URLs instead of silently dropping**. open - maintainer direction: category `uncategorized`; implication `needs_llm` - files: `agent/gemini_native_adapter.py` - **#55474: fix(gemini): preserve image parts from tool results in functionResponse**. closed without merge - maintainer direction: category `merged_via`; implication `superseded`; refs `#32352`, `#96466`, `#32352` - **#117729: fix(bedrock): tool-result images reach the model as Converse image blocks (#55134, salvage #55135)**. open - maintainer direction: category `uncategorized`; implication `needs_llm`; refs `#14524`, `#128064`, `#128064`, `#128064` - files: `agent/bedrock_adapter.py`, `tests/agent/test_bedrock_adapter.py` ## Fix status - Merged to main in PR #21292 (cited above). No release data is tracked. Verify downstream. - Fix last verified (merged) `2026-08-27`. - Subsystem: `agent`