{"repo":"NousResearch/hermes-agent","public_id":"1100","display_label":"STT enable/disable config and transcription backends","summary":"These issues and PRs address honoring the stt.enabled configuration flag to skip transcription, plus adding a local faster-whisper backend as an alternative to OpenAI Whisper. One unrelated fix handles non-string content in a context compressor response.","subsystem":"gateway","counts":{"members":6,"open_issues":0,"open_issues_distinct":0,"open_prs":0,"closed_issues":1,"merged_prs":2,"attempt_prs":3},"landed_fix":{"status":"merged_to_main","basis":"structural","pr":1185},"resolution":{"state":"unresolved","by":null},"freshness":{"state":"stale","last_verified_at":"2026-03-15T05:11:47+00:00","is_stale":true},"grouping":{"backbone":"STRUCTURAL_SPANNING","link_density":0.3333333333333333},"fix_edges":[{"issue_number":1100,"pr_number":1185,"tier":"verified","basis":"superseded_ref","confidence":0.9,"recoverable_from_timeline":false},{"issue_number":1100,"pr_number":1394,"tier":"verified","basis":"superseded_ref","confidence":0.9,"recoverable_from_timeline":false}],"members":[{"kind":"issue","number":1100,"title":"STT config ignored: Gateway hardcodes OpenAI Whisper despite stt.enabled: false","url":null,"state":"closed"},{"kind":"pr","number":888,"title":"fix: Handle non-string content in context compressor response","url":null,"state":"closed","close_reason":{"refs":[{"kind":"pr","value":"#1185","number":1185},{"kind":"pr","value":"#1185","number":1185}],"category":"superseded_by_pr","implication":"superseded"},"merged":false},{"kind":"pr","number":1110,"title":"fix(gateway): honor stt.enabled: false — skip transcription when STT disabled","url":null,"state":"closed","close_reason":{"refs":[{"kind":"pr","value":"#1394","number":1394}],"category":"merged_via","implication":"superseded"},"merged":false},{"kind":"pr","number":1111,"title":"fix: Honor stt_enabled config to disable voice transcription","url":null,"state":"closed","close_reason":{"refs":[{"kind":"pr","value":"#1394","number":1394},{"kind":"pr","value":"#1110","number":1110}],"category":"superseded_by_pr","implication":"superseded"},"merged":false},{"kind":"pr","number":1185,"title":"feat(stt): add free local whisper transcription via faster-whisper","url":null,"state":"merged","close_reason":{"category":"merged","implication":"superseded"},"merged":true,"is_fix_pr":true,"fix_tier":"verified","files_changed":["agent/context_compressor.py","hermes_cli/config.py","pyproject.toml","tests/agent/test_context_compressor.py","tests/tools/test_transcription.py","tools/transcription_tools.py","website/docs/user-guide/features/tts.md"],"fix_hunks":[{"file":"agent/context_compressor.py","hunks":["@@ -132,7 +132,11 @@ def _generate_summary(self, turns_to_summarize: List[Dict[str, Any]]) -> Optiona\n             if self.summary_model:\n                 call_kwargs[\"model\"] = self.summary_model\n             response = call_llm(**call_kwargs)\n-            summary = response.choices[0].message.content.strip()\n+            content = response.choices[0].message.content\n+            # Handle cases where content is not a string (e.g., dict from llama.cpp)\n+            if not isinstance(content, str):\n+                content = str(content) if content else \"\"\n+            summary = content.strip()\n             if not summary.startswith(\"[CONTEXT SUMMARY]:\"):\n                 summary = \"[CONTEXT SUMMARY]: \" + summary\n             return summary"]},{"file":"hermes_cli/config.py","hunks":["@@ -194,8 +194,13 @@ def ensure_hermes_home():\n     },\n     \n     \"stt\": {\n-        \"enabled\": True,\n-        \"model\": \"whisper-1\",\n+        \"provider\": \"local\",  # \"local\" (free, faster-whisper) | \"openai\" (Whisper API)\n+        \"local\": {\n+            \"model\": \"base\",  # tiny, base, small, medium, large-v3\n+        },\n+        \"openai\": {\n+            \"model\": \"whisper-1\",  # whisper-1, gpt-4o-mini-transcribe, gpt-4o-transcribe\n+        },\n     },\n     \n     \"human_delay\": {"]},{"file":"pyproject.toml","hunks":["@@ -30,6 +30,7 @@ dependencies = [\n   \"fal-client\",\n   # Text-to-speech (Edge TTS is free, no API key needed)\n   \"edge-tts\",\n+  \"faster-whisper>=1.0.0\",\n   # mini-swe-agent deps (terminal tool)\n   \"litellm>=1.75.5\",\n   \"typer\","]},{"file":"tests/agent/test_context_compressor.py","hunks":["@@ -153,6 +153,47 @@ def test_none_content_in_system_message_compress(self):\n         assert len(result) < len(msgs)\n \n \n+class TestNonStringContent:\n+    \"\"\"Regression: content as dict (e.g., llama.cpp tool calls) must not crash.\"\"\"\n+\n+    def test_dict_content_coerced_to_string(self):\n+        mock_response = MagicMock()\n+        mock_response.choices = [MagicMock()]\n+        mock_response.choices[0].message.content = {\"text\": \"some summary\"}\n+\n+        with patch(\"agent.context_compressor.get_model_context_length\", return_value=100000):\n+            c = ContextCompressor(model=\"test\", quiet_mode=True)\n+\n+        messages = [\n+            {\"role\": \"user\", \"content\": \"do something\"},\n+            {\"role\": \"assistant\", \"content\": \"ok\"},\n+        ]\n+\n+        with patch(\"agent.context_compressor.call_llm\", return_value=mock_response):\n+            summary = c._generate_summary(messages)\n+        assert isinstance(summary, str)\n+        assert \"CONTEXT SUMMARY\" in summary\n+\n+    def test_none_content_coerced_to_empty(self):\n+        mock_response = MagicMock()\n+        mock_response.choices = [MagicMock()]\n+        mock_response.choices[0].message.content = None\n+\n+        with patch(\"agent.context_compressor.get_model_context_length\", return_value=100000):\n+            c = ContextCompressor(model=\"test\", quiet_mode=True)\n+\n+        messages = [\n+            {\"role\": \"user\", \"content\": \"do something\"},\n+            {\"role\": \"assistant\", \"content\": \"ok\"},\n+        ]\n+\n+        with patch(\"agent.context_compressor.call_llm\", return_value=mock_response):\n+            summary = c._generate_summary(messages)\n+        # None content → empty string → \"[CONTEXT SUMMARY]: \" prefix added\n+        assert summary is not None\n+        assert \"CONTEXT SUMMARY\" in summary\n+\n+\n class TestCompressWithClient:\n     def test_summarization_path(self):\n         mock_client = MagicMock()"]},{"file":"tests/tools/test_transcription.py","hunks":["@@ -0,0 +1,223 @@\n+\"\"\"Tests for transcription_tools.py — local (faster-whisper) and OpenAI providers.\n+\n+Tests cover provider selection, config loading, validation, and transcription\n+dispatch.  All external dependencies (faster_whisper, openai) are mocked.\n+\"\"\"\n+\n+import json\n+import os\n+import tempfile\n+from pathlib import Path\n+from unittest.mock import MagicMock, patch, mock_open\n+\n+import pytest\n+\n+\n+# ---------------------------------------------------------------------------\n+# Provider selection\n+# ---------------------------------------------------------------------------\n+\n+\n+class TestGetProvider:\n+    \"\"\"_get_provider() picks the right backend based on config + availability.\"\"\"\n+\n+    def test_local_when_available(self):\n+        with patch(\"tools.transcription_tools._HAS_FASTER_WHISPER\", True):\n+            from tools.transcription_tools import _get_provider\n+            assert _get_provider({\"provider\": \"local\"}) == \"local\"\n+\n+    def test_local_fallback_to_openai(self, monkeypatch):\n+        monkeypatch.setenv(\"VOICE_TOOLS_OPENAI_KEY\", \"sk-test\")\n+        with patch(\"tools.transcription_tools._HAS_FASTER_WHISPER\", False), \\\n+             patch(\"tools.transcription_tools._HAS_OPENAI\", True):\n+            from tools.transcription_tools import _get_provider\n+            assert _get_provider({\"provider\": \"local\"}) == \"openai\"\n+\n+    def test_local_nothing_available(self, monkeypatch):\n+        monkeypatch.delenv(\"VOICE_TOOLS_OPENAI_KEY\", raising=False)\n+        with patch(\"tools.transcription_tools._HAS_FASTER_WHISPER\", False), \\\n+             patch(\"tools.transcription_tools._HAS_OPENAI\", False):\n+            from tools.transcription_tools import _get_provider\n+            assert _get_provider({\"provider\": \"local\"}) == \"none\"\n+\n+    def test_openai_when_key_set(self, monkeypatch):\n+        monkeypatch.setenv(\"VOICE_TOOLS_OPENAI_KEY\", \"sk-test\")\n+        with patch(\"tools.transcription_tools._HAS_OPENAI\", True):\n+            from tools.transcription_tools import _get_provider\n+            assert _get_provider({\"provider\": \"openai\"}) == \"openai\"\n+\n+    def test_openai_fallback_to_local(self, monkeypatch):\n+        monkeypatch.delenv(\"VOICE_TOOLS_OPENAI_KEY\", raising=False)\n+        with patch(\"tools.transcription_tools._HAS_FASTER_WHISPER\", True), \\\n+             patch(\"tools.transcription_tools._HAS_OPENAI\", True):\n+            from tools.transcription_tools import _get_provider\n+            assert _get_provider({\"provider\": \"openai\"}) == \"local\"\n+\n+    def test_default_provider_is_local(self):\n+        with patch(\"tools.transcription_tools._HAS_FASTER_WHISPER\", True):\n+            from tools.transcription_tools import _get_provider\n+            assert _get_provider({}) == \"local\"\n+\n+\n+# ---------------------------------------------------------------------------\n+# File validation\n+# ---------------------------------------------------------------------------\n+\n+\n+class TestValidateAudioFile:\n+\n+    def test_missing_file(self, tmp_path):\n+        from tools.transcription_tools import _validate_audio_file\n+        result = _validate_audio_file(str(tmp_path / \"nope.ogg\"))\n+        assert result is not None\n+        assert \"not found\" in result[\"error\"]\n+\n+    def test_unsupported_format(self, tmp_path):\n+        f = tmp_path / \"test.xyz\"\n+        f.write_bytes(b\"data\")\n+        from tools.transcription_tools import _validate_audio_file\n+        result = _validate_audio_file(str(f))\n+        assert result is not None\n+        assert \"Unsupported\" in result[\"error\"]\n+\n+    def test_valid_file_returns_none(self, tmp_path):\n+        f = tmp_path / \"test.ogg\"\n+        f.write_bytes(b\"fake audio data\")\n+        from tools.transcription_tools import _validate_audio_file\n+        assert _validate_audio_file(str(f)) is None\n+\n+    def test_too_large(self, tmp_path):\n+        import stat as stat_mod\n+        f = tmp_path / \"big.ogg\"\n+        f.write_bytes(b\"x\")\n+        from tools.transcription_tools import _validate_audio_file, MAX_FILE_SIZE\n+        real_stat = f.stat()\n+        with patch.object(type(f), \"stat\", return_value=os.stat_result((\n+            real_stat.st_mode, real_stat.st_ino, real_stat.st_dev,\n+            real_stat.st_nlink, real_stat.st_uid, real_stat.st_gid,\n+            MAX_FILE_SIZE + 1,  # st_size\n+            real_stat.st_atime, real_stat.st_mtime, real_stat.st_ctime,\n+        ))):\n+            result = _validate_audio_file(str(f))\n+        assert result is not None\n+        assert \"too large\" in result[\"error\"]\n+\n+\n+# ---------------------------------------------------------------------------\n+# Local transcription\n+# ---------------------------------------------------------------------------\n+\n+\n+class TestTranscribeLocal:\n+\n+    def test_successful_transcription(self, tmp_path):\n+        audio_file = tmp_path / \"test.ogg\"\n+        audio_file.write_bytes(b\"fake audio\")\n+\n+        mock_segment = MagicMock()\n+        mock_segment.text = \"Hello world\"\n+        mock_info = MagicMock()\n+        mock_info.language = \"en\"\n+        mock_info.duration = 2.5\n+\n+        mock_model = MagicMock()\n+        mock_model.transcribe.return_value = ([mock_segment], mock_info)\n+\n+        with patch(\"tools.transcription_tools._HAS_FASTER_WHISPER\", True), \\\n+             patch(\"tools.transcription_tools.WhisperModel\", return_value=mock_model), \\\n+             patch(\"tools.transcription_tools._local_model\", None):\n+            from tools.transcription_tools import _transcribe_local\n+            result = _transcribe_local(str(audio_file), \"base\")\n+\n+        assert result[\"success\"] is True\n+        assert result[\"transcript\"] == \"Hello world\"\n+\n+    def test_not_installed(self):\n+        with patch(\"tools.transcription_tools._HAS_FASTER_WHISPER\", False):\n+            from tools.transcription_tools import _transcribe_local\n+            result = _transcribe_local(\"/tmp/test.ogg\", \"base\")\n+        assert result[\"success\"] is False\n+        assert \"not installed\" in result[\"error\"]\n+\n+\n+# ---------------------------------------------------------------------------\n+# OpenAI transcription\n+# ---------------------------------------------------------------------------\n+\n+\n+class TestTranscribeOpenAI:\n+\n+    def test_no_key(self, monkeypatch):\n+        monkeypatch.delenv(\"VOICE_TOOLS_OPENAI_KEY\", raising=False)\n+        from tools.transcription_tools import _transcribe_openai\n+        result = _transcribe_openai(\"/tmp/test.ogg\", \"whisper-1\")\n+        assert result[\"success\"] is False\n+        assert \"VOICE_TOOLS_OPENAI_KEY\" in result[\"error\"]\n+\n+    def test_successful_transcription(self, monkeypatch, tmp_path):\n+        monkeypatch.setenv(\"VOICE_TOOLS_OPENAI_KEY\", \"sk-test\")\n+        audio_file = tmp_path / \"test.ogg\"\n+        audio_file.write_bytes(b\"fake audio\")\n+\n+        mock_client = MagicMock()\n+        mock_client.audio.transcriptions.create.return_value = \"Hello from OpenAI\"\n+\n+        with patch(\"tools.transcription_tools._HAS_OPENAI\", True), \\\n+             patch(\"tools.transcription_tools.OpenAI\", return_value=mock_client):\n+            from tools.transcription_tools import _transcribe_openai\n+            result = _transcribe_openai(str(audio_file), \"whisper-1\")\n+\n+        assert result[\"success\"] is True\n+        assert result[\"transcript\"] == \"Hello from OpenAI\"\n+\n+\n+# ---------------------------------------------------------------------------\n+# Main transcribe_audio() dispatch\n+# ---------------------------------------------------------------------------\n+\n+\n+class TestTranscribeAudio:\n+\n+    def test_dispatches_to_local(self, tmp_path):\n+        audio_file = tmp_path / \"test.ogg\"\n+        audio_file.write_bytes(b\"fake audio\")\n+\n+        with patch(\"tools.transcription_tools._load_stt_config\", return_value={\"provider\": \"local\"}), \\\n+             patch(\"tools.transcription_tools._get_provider\", return_value=\"local\"), \\\n+             patch(\"tools.transcription_tools._transcribe_local\", return_value={\"success\": True, \"transcript\": \"hi\"}) as mock_local:\n+            from tools.transcription_tools import transcribe_audio\n+            result = transcribe_audio(str(audio_file))\n+\n+        assert result[\"success\"] is True\n+        mock_local.assert_called_once()\n+\n+    def test_dispatches_to_openai(self, tmp_path):\n+        audio_file = tmp_path / \"test.ogg\"\n+        audio_file.write_bytes(b\"fake audio\")\n+\n+        with patch(\"tools.transcription_tools._load_stt_config\", return_value={\"provider\": \"openai\"}), \\\n+             patch(\"tools.transcription_tools._get_provider\", return_value=\"openai\"), \\\n+             patch(\"tools.transcription_tools._transcribe_openai\", return_value={\"success\": True, \"transcript\": \"hi\"}) as mock_openai:\n+            from tools.transcription_tools import transcribe_audio\n+            result = transcribe_audio(str(audio_file))\n+\n+        assert result[\"success\"] is True\n+        mock_openai.assert_called_once()\n+\n+    def test_no_provider_returns_error(self, tmp_path):\n+        audio_file = tmp_path / \"test.ogg\"\n+        audio_file.write_bytes(b\"fake audio\")\n+\n+        with patch(\"tools.transcription_tools._load_stt_config\", return_value={}), \\\n+             patch(\"tools.transcription_tools._get_provider\", return_value=\"none\"):\n+            from tools.transcription_tools import transcribe_audio\n+            result = transcribe_audio(str(audio_file))\n+\n+        assert result[\"success\"] is False\n+        assert \"No STT provider\" in result[\"error\"]\n+\n+    def test_invalid_file_returns_error(self):\n+        from tools.transcription_tools import transcribe_audio\n+        result = transcribe_audio(\"/nonexistent/file.ogg\")\n+        assert result[\"success\"] is False\n+        assert \"not found\" in result[\"error\"]"]},{"file":"tools/transcription_tools.py","hunks":["@@ -2,18 +2,19 @@\n \"\"\"\n Transcription Tools Module\n \n-Provides speech-to-text transcription using OpenAI's Whisper API.\n-Used by the messaging gateway to automatically transcribe voice messages\n-sent by users on Telegram, Discord, WhatsApp, and Slack.\n+Provides speech-to-text transcription with two providers:\n+\n+  - **local** (default, free) — faster-whisper running locally, no API key needed.\n+    Auto-downloads the model (~150 MB for ``base``) on first use.\n+  - **openai** — OpenAI Whisper API, requires ``VOICE_TOOLS_OPENAI_KEY``.\n \n-Supported models:\n-  - whisper-1        (cheapest, good quality)\n-  - gpt-4o-mini-transcribe  (better quality, higher cost)\n-  - gpt-4o-transcribe       (best quality, highest cost)\n+Used by the messaging gateway to automatically transcribe voice messages\n+sent by users on Telegram, Discord, WhatsApp, Slack, and Signal.\n \n Supported input formats: mp3, mp4, mpeg, mpga, m4a, wav, webm, ogg\n \n-Usage:\n+Usage::\n+\n     from tools.transcription_tools import transcribe_audio\n \n     result = transcribe_audio(\"/path/to/audio.ogg\")","@@ -28,142 +29,237 @@\n \n logger = logging.getLogger(__name__)\n \n+# ---------------------------------------------------------------------------\n+# Optional imports — graceful degradation\n+# ---------------------------------------------------------------------------\n+\n+try:\n+    from faster_whisper import WhisperModel\n+    _HAS_FASTER_WHISPER = True\n+except ImportError:\n+    _HAS_FASTER_WHISPER = False\n+    WhisperModel = None  # type: ignore[assignment,misc]\n+\n+try:\n+    from openai import OpenAI, APIError, APIConnectionError, APITimeoutError\n+    _HAS_OPENAI = True\n+except ImportError:\n+    _HAS_OPENAI = False\n+\n+# ---------------------------------------------------------------------------\n+# Constants\n+# ---------------------------------------------------------------------------\n \n-# Default STT model -- cheapest and widely available\n-DEFAULT_STT_MODEL = \"whisper-1\"\n+DEFAULT_PROVIDER = \"local\"\n+DEFAULT_LOCAL_MODEL = \"base\"\n+DEFAULT_OPENAI_MODEL = \"whisper-1\"\n \n-# Supported audio formats\n SUPPORTED_FORMATS = {\".mp3\", \".mp4\", \".mpeg\", \".mpga\", \".m4a\", \".wav\", \".webm\", \".ogg\"}\n+MAX_FILE_SIZE = 25 * 1024 * 1024  # 25 MB\n \n-# Maximum file size (25MB - OpenAI limit)\n-MAX_FILE_SIZE = 25 * 1024 * 1024\n+# Singleton for the local model — loaded once, reused across calls\n+_local_model: Optional[\"WhisperModel\"] = None\n+_local_model_name: Optional[str] = None\n \n+# ---------------------------------------------------------------------------\n+# Config helpers\n+# ---------------------------------------------------------------------------\n \n-def transcribe_audio(file_path: str, model: Optional[str] = None) -> Dict[str, Any]:\n-    \"\"\"\n-    Transcribe an audio file using OpenAI's Whisper API.\n \n-    This function calls the OpenAI Audio Transcriptions endpoint directly\n-    (not via OpenRouter, since Whisper isn't available there).\n+def _load_stt_config() -> dict:\n+    \"\"\"Load the ``stt`` section from user config, falling back to defaults.\"\"\"\n+    try:\n+        from hermes_cli.config import load_config\n+        return load_config().get(\"stt\", {})\n+    except Exception:\n+        return {}\n \n-    Args:\n-        file_path: Absolute path to the audio file to transcribe.\n-        model:     Whisper model to use. Defaults to config or \"whisper-1\".\n \n-    Returns:\n-        dict with keys:\n-          - \"success\" (bool): Whether transcription succeeded\n-          - \"transcript\" (str): The transcribed text (empty on failure)\n-          - \"error\" (str, optional): Error message if success is False\n+def _get_provider(stt_config: dict) -> str:\n+    \"\"\"Determine which STT provider to use.\n+\n+    Priority:\n+      1. Explicit config value  (``stt.provider``)\n+      2. Auto-detect: local if faster-whisper available, else openai if key set\n+      3. Disabled (returns \"none\")\n     \"\"\"\n-    api_key = [redacted:secret-assignment]\"VOICE_TOOLS_OPENAI_KEY\")\n-    if not api_key:\n-        return {\n-            \"success\": False,\n-            \"transcript\": \"\",\n-            \"error\": \"VOICE_TOOLS_OPENAI_KEY not set\",\n-        }\n+    provider = stt_config.get(\"provider\", DEFAULT_PROVIDER)\n+\n+    if provider == \"local\":\n+        if _HAS_FASTER_WHISPER:\n+            return \"local\"\n+        # Local requested but not available — fall back to openai if possible\n+        if _HAS_OPENAI and os.getenv(\"VOICE_TOOLS_OPENAI_KEY\"):\n+            logger.info(\"faster-whisper not installed, falling back to OpenAI Whisper API\")\n+            return \"openai\"\n+        return \"none\"\n+\n+    if provider == \"openai\":\n+        if _HAS_OPENAI and os.getenv(\"VOICE_TOOLS_OPENAI_KEY\"):\n+            return \"openai\"\n+        # OpenAI requested but no key — fall back to local if possible\n+        if _HAS_FASTER_WHISPER:\n+            logger.info(\"VOICE_TOOLS_OPENAI_KEY not set, falling back to local faster-whisper\")\n+            return \"local\"\n+        return \"none\"\n \n+    return provider  # Unknown — let it fail downstream\n+\n+# ---------------------------------------------------------------------------\n+# Shared validation\n+# ---------------------------------------------------------------------------\n+\n+\n+def _validate_audio_file(file_path: str) -> Optional[Dict[str, Any]]:\n+    \"\"\"Validate the audio file.  Returns an error dict or None if OK.\"\"\"\n     audio_path = Path(file_path)\n-    \n-    # Validate file exists\n+\n     if not audio_path.exists():\n-        return {\n-            \"success\": False,\n-            \"transcript\": \"\",\n-            \"error\": f\"Audio file not found: {file_path}\",\n-        }\n-    \n+        return {\"success\": False, \"transcript\": \"\", \"error\": f\"Audio file not found: {file_path}\"}\n     if not audio_path.is_file():\n-        return {\n-            \"success\": False,\n-            \"transcript\": \"\",\n-            \"error\": f\"Path is not a file: {file_path}\",\n-        }\n-    \n-    # Validate file extension\n+        return {\"success\": False, \"transcript\": \"\", \"error\": f\"Path is not a file: {file_path}\"}\n     if audio_path.suffix.lower() not in SUPPORTED_FORMATS:\n         return {\n             \"success\": False,\n             \"transcript\": \"\",\n-            \"error\": f\"Unsupported file format: {audio_path.suffix}. Supported formats: {', '.join(sorted(SUPPORTED_FORMATS))}\",\n+            \"error\": f\"Unsupported format: {audio_path.suffix}. Supported: {', '.join(sorted(SUPPORTED_FORMATS))}\",\n         }\n-    \n-    # Validate file size\n     try:\n         file_size = audio_path.stat().st_size\n         if file_size > MAX_FILE_SIZE:\n             return {\n                 \"success\": False,\n                 \"transcript\": \"\",\n-                \"error\": f\"File too large: {file_size / (1024*1024):.1f}MB (max {MAX_FILE_SIZE / (1024*1024)}MB)\",\n+                \"error\": f\"File too large: {file_size / (1024*1024):.1f}MB (max {MAX_FILE_SIZE / (1024*1024):.0f}MB)\",\n             }\n     except OSError as e:\n-        logger.error(\"Failed to get file size for %s: %s\", file_path, e, exc_info=True)\n-        return {\n-            \"success\": False,\n-            \"transcript\": \"\",\n-            \"error\": f\"Failed to access file: {e}\",\n-        }\n+        return {\"success\": False, \"transcript\": \"\", \"error\": f\"Failed to access file: {e}\"}\n+\n+    return None\n+\n+# ---------------------------------------------------------------------------\n+# Provider: local (faster-whisper)\n+# ---------------------------------------------------------------------------\n+\n \n-    # Use provided model, or fall back to default\n-    if model is None:\n-        model = DEFAULT_STT_MODEL\n+def _transcribe_local(file_path: str, model_name: str) -> Dict[str, Any]:\n+    \"\"\"Transcribe using faster-whisper (local, free).\"\"\"\n+    global _local_model, _local_model_name\n+\n+    if not _HAS_FASTER_WHISPER:\n+        return {\"success\": False, \"transcript\": \"\", \"error\": \"faster-whisper not installed\"}\n \n     try:\n-        from openai import OpenAI, APIError, APIConnectionError, APITimeoutError\n+        # Lazy-load the model (downloads on first use, ~150 MB for 'base')\n+        if _local_model is None or _local_model_name != model_name:\n+            logger.info(\"Loading faster-whisper model '%s' (first load downloads the model)...\", model_name)\n+            _local_model = WhisperModel(model_name, device=\"auto\", compute_type=\"auto\")\n+            _local_model_name = model_name\n+\n+        segments, info = _local_model.transcribe(file_path, beam_size=5)\n+        transcript = \" \".join(segment.text.strip() for segment in segments)\n+\n+        logger.info(\n+            \"Transcribed %s via local whisper (%s, lang=%s, %.1fs audio)\",\n+            Path(file_path).name, model_name, info.language, info.duration,\n+        )\n+\n+        return {\"success\": True, \"transcript\": transcript}\n+\n+    except Exception as e:\n+        logger.error(\"Local transcription failed: %s\", e, exc_info=True)\n+        return {\"success\": False, \"transcript\": \"\", \"error\": f\"Local transcription failed: {e}\"}\n \n+# ---------------------------------------------------------------------------\n+# Provider: openai (Whisper API)\n+# ---------------------------------------------------------------------------\n+\n+\n+def _transcribe_openai(file_path: str, model_name: str) -> Dict[str, Any]:\n+    \"\"\"Transcribe using OpenAI Whisper API (paid).\"\"\"\n+    api_key = [redacted:secret-assignment]\"VOICE_TOOLS_OPENAI_KEY\")\n+    if not api_key:\n+        return {\"success\": False, \"transcript\": \"\", \"error\": \"VOICE_TOOLS_OPENAI_KEY not set\"}\n+\n+    if not _HAS_OPENAI:\n+        return {\"success\": False, \"transcript\": \"\", \"error\": \"openai package not installed\"}\n+\n+    try:\n         client = OpenAI(api_key=[redacted:secret-assignment] base_url=\"https://api.openai.com/v1\")\n \n         with open(file_path, \"rb\") as audio_file:\n             transcription = client.audio.transcriptions.create(\n-                model=model,\n+                model=model_name,\n                 file=audio_file,\n                 response_format=\"text\",\n             )\n \n-        # The response is a plain string when response_format=\"text\"\n         transcript_text = str(transcription).strip()\n+        logger.info(\"Transcribed %s via OpenAI API (%s, %d chars)\",\n+                     Path(file_path).name, model_name, len(transcript_text))\n \n-        logger.info(\"Transcribed %s (%d chars)\", audio_path.name, len(transcript_text))\n-\n-        return {\n-            \"success\": True,\n-            \"transcript\": transcript_text,\n-        }\n+        return {\"success\": True, \"transcript\": transcript_text}\n \n     except PermissionError:\n-        logger.error(\"Permission denied accessing file: %s\", file_path, exc_info=True)\n-        return {\n-            \"success\": False,\n-            \"transcript\": \"\",\n-            \"error\": f\"Permission denied: {file_path}\",\n-        }\n+        return {\"success\": False, \"transcript\": \"\", \"error\": f\"Permission denied: {file_path}\"}\n     except APIConnectionError as e:\n-        logger.error(\"API connection error during transcription: %s\", e, exc_info=True)\n-        return {\n-            \"success\": False,\n-            \"transcript\": \"\",\n-            \"error\": f\"Connection error: {e}\",\n-        }\n+        return {\"success\": False, \"transcript\": \"\", \"error\": f\"Connection error: {e}\"}\n     except APITimeoutError as e:\n-        logger.error(\"API timeout during transcription: %s\", e, exc_info=True)\n-        return {\n-            \"success\": False,\n-            \"transcript\": \"\",\n-            \"error\": f\"Request timeout: {e}\",\n-        }\n+        return {\"success\": False, \"transcript\": \"\", \"error\": f\"Request timeout: {e}\"}\n     except APIError as e:\n-        logger.error(\"OpenAI API error during transcription: %s\", e, exc_info=True)\n-        return {\n-            \"success\": False,\n-            \"transcript\": \"\",\n-            \"error\": f\"API error: {e}\",\n-        }\n+        return {\"success\": False, \"transcript\": \"\", \"error\": f\"API error: {e}\"}\n     except Exception as e:\n-        logger.error(\"Unexpected error during transcription: %s\", e, exc_info=True)\n-        return {\n-            \"success\": False,\n-            \"transcript\": \"\",\n-            \"error\": f\"Transcription failed: {e}\",\n-        }\n+        logger.error(\"OpenAI transcription failed: %s\", e, exc_info=True)\n+        return {\"success\": False, \"transcript\": \"\", \"error\": f\"Transcription failed: {e}\"}\n+\n+# ---------------------------------------------------------------------------\n+# Public API\n+# ---------------------------------------------------------------------------\n+\n+\n+def transcribe_audio(file_path: str, model: Optional[str] = None) -> Dict[str, Any]:\n+    \"\"\"\n+    Transcribe an audio file using the configured STT provider.\n+\n+    Provider priority:\n+      1. User config (``stt.provider`` in config.yaml)\n+      2. Auto-detect: local faster-whisper if available, else OpenAI API\n+\n+    Args:\n+        file_path: Absolute path to the audio file to transcribe.\n+        model:     Override the model. If None, uses config or provider default.\n+\n+    Returns:\n+        dict with keys:\n+          - \"success\" (bool): Whether transcription succeeded\n+          - \"transcript\" (str): The transcribed text (empty on failure)\n+          - \"error\" (str, optional): Error message if success is False\n+    \"\"\"\n+    # Validate input\n+    error = _validate_audio_file(file_path)\n+    if error:\n+        return error\n+\n+    # Load config and determine provider\n+    stt_config = _load_stt_config()\n+    provider = _get_provider(stt_config)\n+\n+    if provider == \"local\":\n+        local_cfg = stt_config.get(\"local\", {})\n+        model_name = model or local_cfg.get(\"model\", DEFAULT_LOCAL_MODEL)\n+        return _transcribe_local(file_path, model_name)\n+\n+    if provider == \"openai\":\n+        openai_cfg = stt_config.get(\"openai\", {})\n+        model_name = model or openai_cfg.get(\"model\", DEFAULT_OPENAI_MODEL)\n+        return _transcribe_openai(file_path, model_name)\n+\n+    # No provider available\n+    return {\n+        \"success\": False,\n+        \"transcript\": \"\",\n+        \"error\": (\n+            \"No STT provider available. Install faster-whisper for free local \"\n+            \"transcription, or set VOICE_TOOLS_OPENAI_KEY for the OpenAI Whisper API.\"\n+        ),\n+    }"]},{"file":"website/docs/user-guide/features/tts.md","hunks":["@@ -67,23 +67,48 @@ Without ffmpeg, Edge TTS audio is sent as a regular audio file (playable, but sh\n If you want voice bubbles without installing ffmpeg, switch to the OpenAI or ElevenLabs provider.\n :::\n \n-## Voice Message Transcription\n+## Voice Message Transcription (STT)\n \n-Voice messages sent on Telegram, Discord, WhatsApp, or Slack are automatically transcribed and injected as text into the conversation. The agent sees the transcript as normal text.\n+Voice messages sent on Telegram, Discord, WhatsApp, Slack, or Signal are automatically transcribed and injected as text into the conversation. The agent sees the transcript as normal text.\n \n-| Provider | Model | Quality | Cost |\n-|----------|-------|---------|------|\n-| **OpenAI Whisper** | `whisper-1` (default) | Good | Low |\n-| **OpenAI GPT-4o** | `gpt-4o-mini-transcribe` | Better | Medium |\n-| **OpenAI GPT-4o** | `gpt-4o-transcribe` | Best | Higher |\n+| Provider | Quality | Cost | API Key |\n+|----------|---------|------|---------| \n+| **Local Whisper** (default) | Good | Free | None needed |\n+| **OpenAI Whisper API** | Good–Best | Paid | `VOICE_TOOLS_OPENAI_KEY` |\n \n-Requires `VOICE_TOOLS_OPENAI_KEY` in `~/.hermes/.env`.\n+:::info Zero Config\n+Local transcription works out of the box — no API key needed. The `faster-whisper` model (~150 MB for `base`) is auto-downloaded on first voice message.\n+:::\n \n ### Configuration\n \n ```yaml\n # In ~/.hermes/config.yaml\n stt:\n-  enabled: true\n-  model: \"whisper-1\"\n+  provider: \"local\"           # \"local\" (free, faster-whisper) | \"openai\" (API)\n+  local:\n+    model: \"base\"             # tiny, base, small, medium, large-v3\n+  openai:\n+    model: \"whisper-1\"        # whisper-1, gpt-4o-mini-transcribe, gpt-4o-transcribe\n ```\n+\n+### Provider Details\n+\n+**Local (faster-whisper)** — Runs Whisper locally via [faster-whisper](https://github.com/SYSTRAN/faster-whisper). Uses CPU by default, GPU if available. Model sizes:\n+\n+| Model | Size | Speed | Quality |\n+|-------|------|-------|---------|\n+| `tiny` | ~75 MB | Fastest | Basic |\n+| `base` | ~150 MB | Fast | Good (default) |\n+| `small` | ~500 MB | Medium | Better |\n+| `medium` | ~1.5 GB | Slower | Great |\n+| `large-v3` | ~3 GB | Slowest | Best |\n+\n+**OpenAI API** — Requires `VOICE_TOOLS_OPENAI_KEY`. Supports `whisper-1`, `gpt-4o-mini-transcribe`, and `gpt-4o-transcribe`.\n+\n+### Fallback Behavior\n+\n+If your configured provider isn't available, Hermes automatically falls back:\n+- **Local not installed** → Falls back to OpenAI API (if key is set)\n+- **OpenAI key not set** → Falls back to local Whisper (if installed)\n+- **Neither available** → Voice messages pass through with a note to the user\n"]}]},{"kind":"pr","number":1394,"title":"fix: honor stt.enabled false across gateway transcription","url":null,"state":"merged","close_reason":{"category":"merged","implication":"superseded"},"merged":true,"is_fix_pr":true,"fix_tier":"verified","files_changed":["gateway/config.py","gateway/run.py","hermes_cli/config.py","tests/gateway/test_stt_config.py","tests/tools/test_transcription.py","tools/transcription_tools.py","tools/voice_mode.py"]}],"trust":{"default_tier":"verified","includes_suggested":false}}