CJK/wide character rendering and alignment issues
NousResearch/hermes-agent · cluster 4212 · environments
Data as of
8 open items. Fix merged in #5665. Check it covers your case.
The short answer
- Known?
- Yes. 25 related items grouped here, 8 issues already closed.
- Fix event
- A fix was merged to main in PR #5665. Check it covers your case. No release data is tracked.
- Tracker
- The maintainers' tracker marks this
unresolved. That is separate from the fix event. - Still open
- 3 issues and 5 PRs.
Next step: read #5665, then check whether your report is one of the 3 still open.
Read #5665Pick up a Lab taskThis cluster centers on bugs and fixes related to CJK and other wide characters causing misalignment, ghosting, cursor offset, and incorrect rendering in TUI tables, input boxes, and session lists. Most members share this theme, with a small number of adjacent or unrelated fixes bundled into broader PRs.
Open issues3
- Issue#20621CJK/Emoji characters overflow and misalign panel borders due to len()-based width calculation
- Issue#35272Two TUI rendering bugs: CJK cursor offset + infinite scroll loop on long voice conversations (v0.15.2)
- Issue#44199fix(cli): CJK characters misaligned in /resume and sessions list tables
Open PRs5
- PR#17722fix: align image picker columns for CJK text
- PR#20714fix: four easy bug fixes (docs, TUI, Termux)
- PR#20750fix(cli): CJK/emoji panel border alignment
- PR#44203fix(cli): use rich.Table for resume session list to fix CJK alignment
- PR#44211fix(cli): align session tables with CJK content via display-width padding
Fixes on record
-
#5665 fix: batch gateway/platform fixes — matrix E2EE, CJK input, Windows browser, Feishu reconnect + ACL
files:
cli.py,gateway/config.py,gateway/platforms/feishu.py,gateway/platforms/matrix.py,hermes_cli/config.py,tests/gateway/test_feishu.py,tests/gateway/test_matrix.py,tests/test_cli_browser_connect.py,tests/test_cli_status_bar.pyShow the diff
-
cli.py@@ -120,6 +120,63 @@ def _parse_reasoning_config(effort: str) -> dict | None: return result +def _get_chrome_debug_candidates(system: str) -> list[str]: + """Return likely browser executables for local CDP auto-launch.""" + candidates: list[str] = [] + seen: set[str] = set() + + def _add_candidate(path: str | None) -> None: + if not path: + return + normalized = os.path.normcase(os.path.normpath(path)) + if normalized in seen: + return + if os.path.isfile(path): + candidates.append(path) + seen.add(normalized) + + def _add_from_path(*names: str) -> None: + for name in names: + _add_candidate(shutil.which(name)) + + if system == "Darwin": + for app in ( + "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome", + "/Applications/Chromium.app/Contents/MacOS/Chromium", + "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser", + "/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge", + ): + _add_candidate(app) + elif system == "Windows": + _add_from_path( + "chrome.exe", "msedge.exe", "brave.exe", "chromium.exe", + "chrome", "msedge", "brave", "chromium", + ) + + for base in ( + os.environ.get("ProgramFiles"), + os.environ.get("ProgramFiles(x86)"), + os.environ.get("LOCALAPPDATA"), + ): + if not base: + continue + for parts in ( + ("Google", "Chrome", "Application", "chrome.exe"), + ("Chromium", "Application", "chrome.exe"), + ("Chromium", "Application", "chromium.exe"), + ("BraveSoftware", "Brave-Browser", "Application", "brave.exe"), + ("Microsoft", "Edge", "Application", "msedge.exe"), + ): + _add_candidate(os.path.join(base, *parts)) + else: + _add_from_path( + "google-chrome", "google-chrome-stable", "chromium-browser", + "chromium", "brave-browser", "microsoft-edge", + ) + + return candidates + + def load_cli_config() -> Dict[str, Any]: """ Load CLI configuration from config files. @@ -4838,27 +4895,9 @@ def _try_launch_chrome_debug(port: int, system: str) -> bool: Returns True if a launch command was executed (doesn't guarantee success). """ - import shutil import subprocess as _sp - candidates = [] - if system == "Darwin": - # macOS: try common app bundle locations - for app in ( - "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome", - "/Applications/Chromium.app/Contents/MacOS/Chromium", - "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser", - "/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge", - ): - if os.path.isfile(app): - candidates.append(app) - else: - # Linux: try common binary names - for name in ("google-chrome", "google-chrome-stable", "chromium-browser", - "chromium", "brave-browser", "microsoft-edge"): - path = shutil.which(name) - if path: - candidates.append(path) + candidates = _get_chrome_debug_candidates(system) if not candidates: return False @@ -7469,18 +7508,26 @@ def get_prompt(): # wrapping of long lines so the input area always fits its content. def _input_height(): try: + from prompt_toolkit.application import get_app + from prompt_toolkit.utils import get_cwidth + doc = input_area.buffer.document - prompt_width = max(2, len(self._get_tui_prompt_text())) - available_width = shutil.get_terminal_size().columns - prompt_width + prompt_width = max(2, get_cwidth(self._get_tui_prompt_text())) + try: + available_width = get_app().output.get_size().columns - prompt_width + except Exception: + available_width = shutil.get_terminal_size((80, 24)).columns - prompt_width if available_width < 10: available_width = 40 visual_lines = 0 for line in doc.lines: - # Each logical line takes at least 1 visual row; long lines wrap - if len(line) == 0: + # Each logical line takes at least 1 visual row; long lines wrap. + # Use prompt_toolkit's cell width so CJK wide characters count as 2. + line_width = get_cwidth(line) + if line_width <= 0: visual_lines += 1 else: - visual_lines += max(1, -(-len(line) // available_width)) # ceil division + visual_lines += max(1, -(-line_width // available_width)) # ceil division return min(max(visual_lines, 1), 8) except Exception: return 1 -
gateway/config.py@@ -779,6 +779,9 @@ def _apply_env_overrides(config: GatewayConfig) -> None: config.platforms[Platform.MATRIX].extra["password"] = matrix_password matrix_e2ee = os.getenv("MATRIX_ENCRYPTION", "").lower() in ("true", "1", "yes") config.platforms[Platform.MATRIX].extra["encryption"] = matrix_e2ee + matrix_device_id = os.getenv("MATRIX_DEVICE_ID", "") + if matrix_device_id: + config.platforms[Platform.MATRIX].extra["device_id"] = matrix_device_id matrix_home = os.getenv("MATRIX_HOME_ROOM") if matrix_home and Platform.MATRIX in config.platforms: config.platforms[Platform.MATRIX].home_channel = HomeChannel( -
gateway/platforms/feishu.py@@ -270,6 +270,22 @@ class FeishuAdapterSettings: webhook_host: str webhook_port: int webhook_path: str + ws_reconnect_nonce: int = 30 + ws_reconnect_interval: int = 120 + ws_ping_interval: Optional[int] = None + ws_ping_timeout: Optional[int] = None + admins: frozenset[str] = frozenset() + default_group_policy: str = "" + group_rules: Dict[str, FeishuGroupRule] = field(default_factory=dict) + + +@dataclass +class FeishuGroupRule: + """Per-group policy rule for controlling which users may interact with the bot.""" + + policy: str # "open" | "allowlist" | "blacklist" | "admin_only" | "disabled" + allowlist: set[str] = field(default_factory=set) + blacklist: set[str] = field(default_factory=set) @dataclass @@ -358,6 +374,24 @@ def _strip_markdown_to_plain_text(text: str) -> str: return plain.strip() +def _coerce_int(value: Any, default: Optional[int] = None, min_value: int = 0) -> Optional[int]: + """Coerce value to int with optional default and minimum constraint.""" + try: + parsed = int(value) + except (TypeError, ValueError): + return default + return parsed if parsed >= min_value else default + + +def _coerce_required_int(value: Any, default: int, min_value: int = 0) -> int: + parsed = _coerce_int(value, default=default, min_value=min_value) + return default if parsed is None else parsed + + +def _is_loop_ready(loop: Optional[asyncio.AbstractEventLoop]) -> bool: + return loop is not None and not bool(getattr(loop, "is_closed", lambda: False)()) + + # --------------------------------------------------------------------------- # Post payload builders and parsers # --------------------------------------------------------------------------- @@ -913,14 +947,66 @@ def _unique_lines(lines: List[str]) -> List[str]: return unique -def _run_official_feishu_ws_client(ws_client: Any) -> None: +def _run_official_feishu_ws_client(ws_client: Any, adapter: Any) -> None: """Run the official Lark WS client in its own thread-local event loop.""" import lark_oapi.ws.client as ws_client_module loop = asyncio.new_event_loop() asyncio.set_event_loop(loop) ws_client_module.loop = loop - ws_client.start() + adapter._ws_thread_loop = loop + + original_connect = ws_client_module.websockets.connect + original_configure = getattr(ws_client, "_configure", None) + + def _apply_runtime_ws_overrides() -> None: + try: + setattr(ws_client, "_reconnect_nonce", adapter._ws_reconnect_nonce) + setattr(ws_client, "_reconnect_interval", adapter._ws_reconnect_interval) + if adapter._ws_ping_interval is not None: + setattr(ws_client, "_ping_interval", adapter._ws_ping_interval) + except Exception: + logger.debug("[Feishu] Failed to apply websocket runtime overrides", exc_info=True) + + async def _connect_with_overrides(*args: Any, **kwargs: Any) -> Any: + if adapter._ws_ping_interval is not None and "ping_interval" not in kwargs: + kwargs["ping_interval"] = adapter._ws_ping_interval + if adapter._ws_ping_timeout is not None and "ping_timeout" not in kwargs: + kwargs["ping_timeout"] = adapter._ws_ping_timeout + return await original_connect(*args, **kwargs) + + def _configure_with_overrides(conf: Any) -> Any: + assert original_configure is not None + result = original_configure(conf) + _apply_runtime_ws_overrides() + return result + + ws_client_module.websockets.connect = _connect_with_overrides + if original_configure is not None: + setattr(ws_client, "_configure", _configure_with_overrides) + _apply_runtime_ws_overrides() + try: + ws_client.start() + except Exception: + pass + finally: + ws_client_module.websockets.connect = original_connect + if original_configure is not None: + setattr(ws_client, "_configure", original_configure) + pending = [t for t in asyncio.all_tasks(loop) if not t.done()] + for task in pending: + task.cancel() + if pending: + loop.run_until_complete(asyncio.gather(*pending, return_exceptions=True)) + try: + loop.stop() + except Exception: + pass + try: + loop.close() + except Exception: + pass + adapter._ws_thread_loop = None def check_feishu_requirements() -> bool: @@ -945,10 +1031,11 @@ def __init__(self, config: PlatformConfig): self._client: Optional[Any] = None self._ws_client: Optional[Any] = None self._ws_future: Optional[asyncio.Future] = None + self._ws_thread_loop: Optional[asyncio.AbstractEventLoop] = None self._loop: Optional[asyncio.AbstractEventLoop] = None self._webhook_runner: Optional[Any] = None self._webhook_site: Optional[Any] = None - self._event_handler = self._build_event_handler() + self._event_handler: Optional[Any] = None self._seen_message_ids: Dict[str, float] = {} # message_id → seen_at (time.time()) self._seen_message_order: List[str] = [] self._dedup_state_path = get_hermes_home() / "feishu_seen_message_ids.json" @@ -974,6 +1061,26 @@ def __init__(self, config: PlatformConfig): @staticmethod def _load_settings(extra: Dict[str, Any]) -> FeishuAdapterSettings: + # Parse per-group rules from config + raw_group_rules = extra.get("group_rules", {}) + group_rules: Dict[str, FeishuGroupRule] = {} + if isinstance(raw_group_rules, dict): + for chat_id, rule_cfg in raw_group_rules.items(): + if not isinstance(rule_cfg, dict): + continue + group_rules[str(chat_id)] = FeishuGroupRule( + policy=str(rule_cfg.get("policy", "open")).strip().lower(), + allowlist=set(str(u).strip() for u in rule_cfg.get("allowlist", []) if str(u).strip()), + blacklist=set(str(u).strip() for u in rule_cfg.get("blacklist", []) if str(u).strip()), + ) + + # Bot-level admins + raw_admins = extra.get("admins", []) + admins = frozenset(str(u).strip() for u in raw_admins if str(u).strip()) + + # Default group policy (for groups not in group_rules) + default_group_policy = str(extra.get("default_group_policy", "")).strip().lower() + return FeishuAdapterSettings( app_id=str(extra.get("app_id") or os.getenv("FEISHU_APP_ID", "")).strip(), app_secret=[redacted:secret-assignment]"app_secret") or os.getenv("FEISHU_APP_SECRET", "")).strip(), @@ -1020,6 +1127,13 @@ def _load_settings(extra: Dict[str, Any]) -> FeishuAdapterSettings: str(extra.get("webhook_path") or os.getenv("FEISHU_WEBHOOK_PATH", _DEFAULT_WEBHOOK_PATH)).strip() or _DEFAULT_WEBHOOK_PATH ), + ws_reconnect_nonce=_coerce_required_int(extra.get("ws_reconnect_nonce"), default=30, min_value=0), + ws_reconnect_interval=_coerce_required_int(extra.get("ws_reconnect_interval"), default=120, min_value=1), + ws_ping_interval=_coerce_int(extra.get("ws_ping_interval"), default=None, min_value=1), + ws_ping_timeout=_coerce_int(extra.get("ws_ping_timeout"), default=None, min_value=1), + admins=admins, + default_group_policy=default_group_policy, + group_rules=group_rules, ) def _apply_settings(self, settings: FeishuAdapterSettings) -> None: @@ -1031,6 +1145,9 @@ def _apply_settings(self, settings: FeishuAdapterSettings) -> None: self._verification_token = [redacted:secret-assignment] self._group_policy = settings.group_policy self._allowed_group_users = set(settings.allowed_group_users) + self._admins = set(settings.admins) + self._default_group_policy = settings.default_group_policy or settings.group_policy + self._group_rules = settings.group_rules self._bot_open_id = settings.bot_open_id self._bot_user_id = settings.bot_user_id self._bot_name = settings.bot_name @@ -1042,6 +1159,10 @@ def _apply_settings(self, settings: FeishuAdapterSettings) -> None: self._webhook_host = settings.webhook_host self._webhook_port = settings.webhook_port self._webhook_path = settings.webhook_path + self._ws_reconnect_nonce = settings.ws_reconnect_nonce + self._ws_reconnect_interval = settings.ws_reconnect_interval + self._ws_ping_interval = settings.ws_ping_interval + self._ws_ping_timeout = settings.ws_ping_timeout def _build_event_handler(self) -> Any: if EventDispatcherHandler is None: @@ -1116,8 +1237,37 @@ async def disconnect(self) -> None: self._reset_batch_buffers() self._disable_websocket_auto_reconnect() await self._stop_webhook_server() + + ws_thread_loop = self._ws_thread_loop + if ws_thread_loop is not None and not ws_thread_loop.is_closed(): + logger.debug("[Feishu] Cancelling websocket thread tasks and stopping loop") + + def cancel_all_tasks() -> None: + tasks = [t for t in asyncio.all_tasks(ws_thread_loop) if not t.done()] + logger.debug("[Feishu] Found %d pending tasks in websocket thread", len(tasks)) + for task in tasks: + task.cancel() + ws_thread_loop.call_later(0.1, ws_thread_loop.stop) + + ws_thread_loop.call_soon_threadsafe(cancel_all_tasks) + + ws_future = self._ws_future + if ws_future is not None: + try: + logger.debug("[Feishu] Waiting for websocket thread to exit (timeout=10s)") + await asyncio.wait_for(asyncio.shield(ws_future), timeout=10.0) + logger.debug("[Feishu] Websocket thread exited cleanly") + except asyncio.TimeoutError: + logger.warning("[Feishu] Websocket thread did not exit within 10s - may be stuck") + except asyncio.CancelledError: + logger.debug("[Feishu] Websocket thread cancelled during disconnect") + except Exception as exc: + logger.debug("[Feishu] Websocket thread exited with error: %s", exc, exc_info=True) + self._ws_future = None + self._ws_thread_loop = None self._loop = None + self._event_handler = None self._persist_seen_message_ids() await self._release_app_lock() @@ -1476,12 +1626,13 @@ def format_message(self, content: str) -> str: def _on_message_event(self, data: Any) -> None: """Normalize Feishu inbound events into MessageEvent.""" - if self._loop is None: + loop = self._loop + if loop is None or bool(getattr(loop, "is_closed", lambda: False)()): logger.warning("[Feishu] Dropping inbound message before adapter loop is ready") return future = asyncio.run_coroutine_threadsafe( self._handle_message_event_data(data), - self._loop, + loop, ) future.add_done_callback(self._log_background_failure) @@ -1504,7 +1655,8 @@ async def _handle_message_event_data(self, data: Any) -> None: return chat_type = getattr(message, "chat_type", "p2p") - if chat_type != "p2p" and not self._should_accept_group_message(message, sender_id): + chat_id = getattr(message, "chat_id", "") or "" + if chat_type != "p2p" and not self._should_accept_group_message(message, sender_id, chat_id): logger.debug("[Feishu] Dropping group message that failed mention/policy gate: %s", message_id) return await self._process_inbound_message( @@ -1553,27 +1705,30 @@ def _on_reaction_event(self, event_type: str, data: Any) -> None: ) # Only process reactions from real users. Ignore app/bot-generated reactions # and Hermes' own ACK emoji to avoid feedback loops. + loop = self._loop if ( operator_type in {"bot", "app"} or emoji_type == _FEISHU_ACK_EMOJI or not message_id - or self._loop is None + or loop is None + or bool(getattr(loop, "is_closed", lambda: False)()) ): return future = asyncio.run_coroutine_threadsafe( self._handle_reaction_event(event_type, data), - self._loop, + loop, ) future.add_done_callback(self._log_background_failure) def _on_card_action_trigger(self, data: Any) -> Any: """Schedule Feishu card actions on the adapter loop and acknowledge immediately.""" - if self._loop is None: + loop = self._loop + if loop is None or bool(getattr(loop, "is_closed", lambda: False)()): logger.warning("[Feishu] Dropping card action before adapter loop is ready") else: future = asyncio.run_coroutine_threadsafe( self._handle_card_action_event(data), - self._loop, + loop, ) future.add_done_callback(self._log_background_failure) if P2CardActionTriggerResponse is None: @@ -2083,7 +2238,7 @@ async def _handle_webhook_request(self, request: Any) -> Any: event_type = str((payload.get("header") or {}).get("event_type") or "") data = self._namespace_from_mapping(payload) if event_type == "im.message.receive_v1": - await self._handle_message_event_data(data) + self._on_message_event(data) elif event_type == "im.message.message_read_v1": self._on_message_read_event(data) elif event_type == "im.chat.member.bot.added_v1": @@ -2093,7 +2248,7 @@ async def _handle_webhook_request(self, request: Any) -> Any: elif event_type in ("im.message.reaction.created_v1", "im.message.reaction.deleted_v1"): self._on_reaction_event(event_type, data) elif event_type == "card.action.trigger": - asyncio.ensure_future(self._handle_card_action_event(data)) + self._on_card_action_trigger(data) else: logger.debug("[Feishu] Ignoring webhook event type: %s", event_type or "unknown") return web.json_response({"code": 0, "msg": "ok"}) @@ -2657,18 +2812,41 @@ def _log_background_failure(future: Any) -> None: # Group policy and mention gating # ========================================================================= - def _allow_group_message(self, sender_id: Any) -> bool: - """Current group policy gate for non-DM traffic.""" - if self._group_policy == "disabled": + def _allow_group_message(self, sender_id: Any, chat_id: str = "") -> bool: + """Per-group policy gate for non-DM traffic.""" + sender_open_id = getattr(sender_id, "open_id", None) + sender_user_id = getattr(sender_id, "user_id", None) + sender_ids = {sender_open_id, sender_user_id} - {None} + + if sender_ids and self._admins and (sender_ids & self._admins): + return True + + rule = self._group_rules.get(chat_id) if chat_id else None + if rule: + policy = rule.policy + allowlist = rule.allowlist + blacklist = rule.blacklist + else: + policy = self._default_group_policy or self._group_policy + allowlist = self._allowed_group_users + blacklist = set() + + if policy == "disabled": return False - sender_open_id = getattr(sender_id, "open_id", None) or getattr(sender_id, "user_id", None) - if self._group_policy == "open": + if policy == "open": return True - return bool(sender_open_id and sender_open_id in self._allowed_group_users) + if policy == "admin_only": + return False + if policy == "allowlist": + return bool(sender_ids and (sender_ids & allowlist)) + if policy == "blacklist": + return bool(sender_ids and not (sender_ids & blacklist)) - def _should_accept_group_message(self, message: Any, sender_id: Any) -> bool: + return bool(sender_ids and (sender_ids & self._allowed_group_users)) + + def _should_accept_group_message(self, message: Any, sender_id: Any, chat_id: str = "") -> bool: """Require an explicit @mention before group messages enter the agent.""" - if not self._allow_group_message(sender_id): + if not self._allow_group_message(sender_id, chat_id): return False # @_all is Feishu's @everyone placeholder — always route to the bot. raw_content = getattr(message, "content", "") or "" @@ -2965,6 +3143,12 @@ async def _connect_websocket(self) -> None: raise RuntimeError("websockets not installed; websocket mode unavailable") domain = FEISHU_DOMAIN if self._domain_name != "lark" else LARK_DOMAIN self._client = self._build_lark_client(domain) + self._event_handler = self._build_event_handler() + if self._event_handler is None: + raise RuntimeError("failed to build Feishu event handler") + loop = self._loop + if loop is None or loop.is_closed(): + raise RuntimeError("adapter loop is not ready") await self._hydrate_bot_identity() self._ws_client = FeishuWSClient( app_id=self._app_id, @@ -2973,10 +3157,11 @@ async def _connect_websocket(self) -> None: event_handler=self._event_handler, domain=domain, ) - self._ws_future = self._loop.run_in_executor( + self._ws_future = loop.run_in_executor( None, _run_official_feishu_ws_client, self._ws_client, + self, ) async def _connect_webhook(self) -> None: @@ -2984,6 +3169,9 @@ async def _connect_webhook(self) -> None: raise RuntimeError("aiohttp not installed; webhook mode unavailable") domain = FEISHU_DOMAIN if self._domain_name != "lark" else LARK_DOMAIN self._client = self._build_lark_client(domain) + self._event_handler = self._build_event_handler() + if self._event_handler is None: + raise RuntimeError("failed to build Feishu event handler") await self._hydrate_bot_identity() app = web.Application() app.router.add_post(self._webhook_path, self._handle_webhook_request) -
gateway/platforms/matrix.py@@ -10,6 +10,7 @@ MATRIX_USER_ID Full user ID (@bot:server) — required for password login MATRIX_PASSWORD Password (alternative to access token) MATRIX_ENCRYPTION Set "true" to enable E2EE + MATRIX_DEVICE_ID Stable device ID for E2EE persistence across restarts MATRIX_ALLOWED_USERS Comma-separated Matrix user IDs (@user:server) MATRIX_HOME_ROOM Room ID for cron/notification delivery MATRIX_REACTIONS Set "false" to disable processing lifecycle reactions @@ -65,6 +66,21 @@ _PENDING_EVENT_TTL = 300 # seconds — stop retrying after 5 min +_E2EE_INSTALL_HINT = ( + "Install with: pip install 'matrix-nio[e2e]' " + "(requires libolm C library)" +) + + +def _check_e2ee_deps() -> bool: + """Return True if matrix-nio E2EE dependencies (python-olm) are available.""" + try: + from nio.crypto import ENCRYPTION_ENABLED + return bool(ENCRYPTION_ENABLED) + except (ImportError, AttributeError): + return False + + def check_matrix_requirements() -> bool: """Return True if the Matrix adapter can be used.""" token = [redacted:secret-assignment]"MATRIX_ACCESS_TOKEN", "") @@ -79,7 +95,6 @@ def check_matrix_requirements() -> bool: return False try: import nio # noqa: F401 - return True except ImportError: logger.warning( "Matrix: matrix-nio not installed. " @@ -87,6 +102,20 @@ def check_matrix_requirements() -> bool: ) return False + # If encryption is requested, verify E2EE deps are available at startup + # rather than silently degrading to plaintext-only at connect time. + encryption_requested = os.getenv("MATRIX_ENCRYPTION", "").lower() in ("true", "1", "yes") + if encryption_requested and not _check_e2ee_deps(): + logger.error( + "Matrix: MATRIX_ENCRYPTION=true but E2EE dependencies are missing. %s. " + "Without this, encrypted rooms will not work. " + "Set MATRIX_ENCRYPTION=false to disable E2EE.", + _E2EE_INSTALL_HINT, + ) + return False + + return True + class MatrixAdapter(BasePlatformAdapter): """Gateway adapter for Matrix (any homeserver).""" @@ -111,6 +140,10 @@ def __init__(self, config: PlatformConfig): "encryption", os.getenv("MATRIX_ENCRYPTION", "").lower() in ("true", "1", "yes"), ) + self._device_id: str = ( + config.extra.get("device_id", "") + or os.getenv("MATRIX_DEVICE_ID", "") + ) self._client: Any = None # nio.AsyncClient self._sync_task: Optional[asyncio.Task] = None @@ -169,24 +202,42 @@ async def connect(self) -> bool: _STORE_DIR.mkdir(parents=True, exist_ok=True) # Create the client. + # When a stable device_id is configured, pass it to the constructor + # so matrix-nio binds to it from the start (important for E2EE + # crypto-store persistence across restarts). + ctor_device_id = self._device_id or None if self._encryption: + if not _check_e2ee_deps(): + logger.error( + "Matrix: MATRIX_ENCRYPTION=true but E2EE dependencies are missing. %s. " + "Refusing to connect — encrypted rooms would silently fail.", + _E2EE_INSTALL_HINT, + ) + return False try: client = nio.AsyncClient( self._homeserver, self._user_id or "", + device_id=ctor_device_id, store_path=store_path, ) - logger.info("Matrix: E2EE enabled (store: %s)", store_path) + logger.info( + "Matrix: E2EE enabled (store: %s%s)", + store_path, + f", device_id={self._device_id}" if self._device_id else "", + ) except Exception as exc: - logger.warning( - "Matrix: failed to create E2EE client (%s), " - "falling back to plain client. Install: " - "pip install 'matrix-nio[e2e]'", - exc, + logger.error( + "Matrix: failed to create E2EE client: %s. %s", + exc, _E2EE_INSTALL_HINT, ) - client = nio.AsyncClient(self._homeserver, self._user_id or "") + return False else: - client = nio.AsyncClient(self._homeserver, self._user_id or "") + client = nio.AsyncClient( + self._homeserver, + self._user_id or "", + device_id=ctor_device_id, + ) self._client = client @@ -205,30 +256,36 @@ async def connect(self) -> bool: if resolved_user_id: self._user_id = resolved_user_id + # Prefer the user-configured device_id (MATRIX_DEVICE_ID) so + # the bot reuses a stable identity across restarts. Fall back + # to whatever whoami returned. + effective_device_id = self._device_id or resolved_device_id + # restore_login() is the matrix-nio path that binds the access # token to a specific device and loads the crypto store. - if resolved_device_id and hasattr(client, "restore_login"): + if effective_device_id and hasattr(client, "restore_login"): client.restore_login( self._user_id or resolved_user_id, - resolved_device_id, + effective_device_id, self._access_token, ) else: if self._user_id: client.user_id = self._user_id - if resolved_device_id: - client.device_id = resolved_device_id + if effective_device_id: + client.device_id = effective_device_id client.access_token = [redacted:secret-assignment] if self._encryption: logger.warning( "Matrix: access-token login did not restore E2EE state; " - "encrypted rooms may fail until a device_id is available" + "encrypted rooms may fail until a device_id is available. " + "Set MATRIX_DEVICE_ID to a stable value." ) logger.info( "Matrix: using access token for %s%s", self._user_id or "(unknown user)", - f" (device {resolved_device_id})" if resolved_device_id else "", + f" (device {effective_device_id})" if effective_device_id else "", ) else: logger.error( @@ -271,10 +328,15 @@ async def connect(self) -> bool: except Exception as exc: logger.debug("Matrix: could not import keys: %s", exc) elif self._encryption: - logger.warning( - "Matrix: E2EE requested but crypto store is not loaded; " - "encrypted rooms may fail" + # E2EE was requested but the crypto store failed to load — + # this means encrypted rooms will silently not work. Hard-fail. + logger.error( + "Matrix: E2EE requested but crypto store is not loaded — " + "cannot decrypt or encrypt messages. %s", + _E2EE_INSTALL_HINT, ) + await client.close() + return False # Register event callbacks. client.add_event_callback(self._on_room_message, nio.RoomMessageText) -
hermes_cli/config.py@@ -42,7 +42,7 @@ "TERMINAL_ENV", "TERMINAL_SSH_KEY", "TERMINAL_SSH_PORT", "WHATSAPP_MODE", "WHATSAPP_ENABLED", "MATTERMOST_HOME_CHANNEL", "MATTERMOST_REPLY_MODE", - "MATRIX_PASSWORD", "MATRIX_ENCRYPTION", "MATRIX_HOME_ROOM", + "MATRIX_PASSWORD", "MATRIX_ENCRYPTION", "MATRIX_DEVICE_ID", "MATRIX_HOME_ROOM", "MATRIX_REQUIRE_MENTION", "MATRIX_FREE_RESPONSE_ROOMS", "MATRIX_AUTO_THREAD", }) import yaml @@ -1079,6 +1079,14 @@ def ensure_hermes_home(): "category": "messaging", "advanced": True, }, + "MATRIX_DEVICE_ID": { + "description": "Stable Matrix device ID for E2EE persistence across restarts (e.g. HERMES_BOT)", + "prompt": "Matrix device ID (stable across restarts)", + "url": None, + "password": False, + "category": "messaging", + "advanced": True, + }, "GATEWAY_ALLOW_ALL_USERS": { "description": "Allow all users to interact with messaging bots (true/false). Default: false.", "prompt": "Allow all users (true/false)", -
tests/gateway/test_feishu.py@@ -8,7 +8,7 @@ import unittest from pathlib import Path from types import SimpleNamespace -from unittest.mock import AsyncMock, patch +from unittest.mock import AsyncMock, Mock, patch try: import lark_oapi @@ -17,6 +17,18 @@ _HAS_LARK_OAPI = False +def _mock_event_dispatcher_builder(mock_handler_class): + mock_builder = Mock() + mock_builder.register_p2_im_message_message_read_v1 = Mock(return_value=mock_builder) + mock_builder.register_p2_im_message_receive_v1 = Mock(return_value=mock_builder) + mock_builder.register_p2_im_message_reaction_created_v1 = Mock(return_value=mock_builder) + mock_builder.register_p2_im_message_reaction_deleted_v1 = Mock(return_value=mock_builder) + mock_builder.register_p2_card_action_trigger = Mock(return_value=mock_builder) + mock_builder.build = Mock(return_value=object()) + mock_handler_class.builder = Mock(return_value=mock_builder) + return mock_builder + + class TestPlatformEnum(unittest.TestCase): def test_feishu_in_platform_enum(self): from gateway.config import Platform @@ -262,12 +274,14 @@ def test_connect_webhook_mode_starts_local_server(self): with ( patch("gateway.platforms.feishu.FEISHU_AVAILABLE", True), patch("gateway.platforms.feishu.FEISHU_WEBHOOK_AVAILABLE", True), + patch("gateway.platforms.feishu.EventDispatcherHandler") as mock_handler_class, patch("gateway.platforms.feishu.acquire_scoped_lock", return_value=(True, None)), patch("gateway.platforms.feishu.release_scoped_lock"), patch.object(adapter, "_hydrate_bot_identity", new=AsyncMock()), patch.object(adapter, "_build_lark_client", return_value=SimpleNamespace()), patch("gateway.platforms.feishu.web", web_module), ): + _mock_event_dispatcher_builder(mock_handler_class) connected = asyncio.run(adapter.connect()) self.assertTrue(connected) @@ -283,13 +297,13 @@ def test_connect_acquires_scoped_lock_and_disconnect_releases_it(self): from gateway.platforms.feishu import FeishuAdapter adapter = FeishuAdapter(PlatformConfig()) - ws_client = object() + ws_client = SimpleNamespace() with ( patch("gateway.platforms.feishu.FEISHU_AVAILABLE", True), patch("gateway.platforms.feishu.FEISHU_WEBSOCKET_AVAILABLE", True), patch("gateway.platforms.feishu.lark", SimpleNamespace(LogLevel=SimpleNamespace(INFO="INFO", WARNING="WARNING"))), - patch("gateway.platforms.feishu.EventDispatcherHandler", object()), + patch("gateway.platforms.feishu.EventDispatcherHandler") as mock_handler_class, patch("gateway.platforms.feishu.FeishuWSClient", return_value=ws_client), patch("gateway.platforms.feishu._run_official_feishu_ws_client"), patch("gateway.platforms.feishu.acquire_scoped_lock", return_value=(True, None)) as acquire_lock, @@ -297,6 +311,8 @@ def test_connect_acquires_scoped_lock_and_disconnect_releases_it(self): patch.object(adapter, "_hydrate_bot_identity", new=AsyncMock()), patch.object(adapter, "_build_lark_client", return_value=SimpleNamespace()), ): + _mock_event_dispatcher_builder(mock_handler_class) + loop = asyncio.new_event_loop() future = loop.create_future() future.set_result(None) @@ -305,6 +321,9 @@ class _Loop: def run_in_executor(self, *_args, **_kwargs): return future + def is_closed(self): + return False + try: with patch("gateway.platforms.feishu.asyncio.get_running_loop", return_value=_Loop()): connected = asyncio.run(adapter.connect()) @@ -313,6 +332,7 @@ def run_in_executor(self, *_args, **_kwargs): loop.close() self.assertTrue(connected) + self.assertIsNone(adapter._event_handler) acquire_lock.assert_called_once_with( "feishu-app-id", "cli_app", @@ -354,14 +374,14 @@ def test_connect_retries_transient_startup_failure(self): from gateway.platforms.feishu import FeishuAdapter adapter = FeishuAdapter(PlatformConfig()) - ws_client = object() + ws_client = SimpleNamespace() sleeps = [] with ( patch("gateway.platforms.feishu.FEISHU_AVAILABLE", True), patch("gateway.platforms.feishu.FEISHU_WEBSOCKET_AVAILABLE", True), patch("gateway.platforms.feishu.lark", SimpleNamespace(LogLevel=SimpleNamespace(INFO="INFO", WARNING="WARNING"))), - patch("gateway.platforms.feishu.EventDispatcherHandler", object()), + patch("gateway.platforms.feishu.EventDispatcherHandler") as mock_handler_class, patch("gateway.platforms.feishu.FeishuWSClient", return_value=ws_client), patch("gateway.platforms.feishu.acquire_scoped_lock", return_value=(True, None)), patch("gateway.platforms.feishu.release_scoped_lock"), @@ -369,6 +389,8 @@ def test_connect_retries_transient_startup_failure(self): patch("gateway.platforms.feishu.asyncio.sleep", side_effect=lambda delay: sleeps.append(delay)), patch.object(adapter, "_build_lark_client", return_value=SimpleNamespace()), ): + _mock_event_dispatcher_builder(mock_handler_class) + loop = asyncio.new_event_loop() future = loop.create_future() future.set_result(None) @@ -383,6 +405,9 @@ def run_in_executor(self, *_args, **_kwargs): raise OSError("temporary websocket failure") return future + def is_closed(self): + return False + fake_loop = _Loop() try: with patch("gateway.platforms.feishu.asyncio.get_running_loop", return_value=fake_loop): @@ -536,6 +561,113 @@ def test_adapter_registers_reaction_and_card_handlers_for_websocket(self): self.assertIn("register_p2_im_message_reaction_deleted_v1", source) self.assertIn("register_p2_card_action_trigger", source) + def test_load_settings_uses_sdk_defaults_for_invalid_ws_reconnect_values(self): + from gateway.platforms.feishu import FeishuAdapter + + settings = FeishuAdapter._load_settings( + { + "ws_reconnect_nonce": -1, + "ws_reconnect_interval": "bad", + } + ) + + self.assertEqual(settings.ws_reconnect_nonce, 30) + self.assertEqual(settings.ws_reconnect_interval, 120) + + def test_load_settings_accepts_custom_ws_reconnect_values(self): + from gateway.platforms.feishu import FeishuAdapter + + settings = FeishuAdapter._load_settings( + { + "ws_reconnect_nonce": 0, + "ws_reconnect_interval": 3, + } + ) + + self.assertEqual(settings.ws_reconnect_nonce, 0) + self.assertEqual(settings.ws_reconnect_interval, 3) + + def test_load_settings_accepts_custom_ws_ping_values(self): + from gateway.platforms.feishu import FeishuAdapter + + settings = FeishuAdapter._load_settings( + { + "ws_ping_interval": 10, + "ws_ping_timeout": 8, + } + ) + + self.assertEqual(settings.ws_ping_interval, 10) + self.assertEqual(settings.ws_ping_timeout, 8) + + def test_load_settings_ignores_invalid_ws_ping_values(self): + from gateway.platforms.feishu import FeishuAdapter + + settings = FeishuAdapter._load_settings( + { + "ws_ping_interval": 0, + "ws_ping_timeout": -1, + } + ) + + self.assertIsNone(settings.ws_ping_interval) + self.assertIsNone(settings.ws_ping_timeout) + + def test_runtime_ws_overrides_reapply_after_sdk_configure(self): + import sys + from types import ModuleType + + class _FakeWSClient: + def __init__(self): + self._reconnect_nonce = 30 + self._reconnect_interval = 120 + self._ping_interval = 120 + self.configure_calls = [] + + def _configure(self, conf): + self.configure_calls.append(conf) + self._reconnect_nonce = conf.ReconnectNonce + self._reconnect_interval = conf.ReconnectInterval + self._ping_interval = conf.PingInterval + + def start(self): + conf = SimpleNamespace(ReconnectNonce=99, ReconnectInterval=88, PingInterval=77) + self._configure(conf) + raise RuntimeError("stop test client") + + fake_client = _FakeWSClient() + fake_adapter = SimpleNamespace( + _ws_thread_loop=None, + _ws_reconnect_nonce=2, + _ws_reconnect_interval=3, + _ws_ping_interval=4, + _ws_ping_timeout=5, + ) + fake_client_module = ModuleType("lark_oapi.ws.client") + fake_client_module.loop = None + fake_client_module.websockets = SimpleNamespace(connect=AsyncMock()) + fake_ws_module = ModuleType("lark_oapi.ws") + fake_ws_module.client = fake_client_module + fake_root_module = ModuleType("lark_oapi") + fake_root_module.ws = fake_ws_module + + original_modules = sys.modules.copy() + sys.modules["lark_oapi"] = fake_root_module + sys.modules["lark_oapi.ws"] = fake_ws_module + sys.modules["lark_oapi.ws.client"] = fake_client_module + try: + from gateway.platforms.feishu import _run_official_feishu_ws_client + + _run_official_feishu_ws_client(fake_client, fake_adapter) + finally: + sys.modules.clear() + sys.modules.update(original_modules) + + self.assertEqual(len(fake_client.configure_calls), 1) + self.assertEqual(fake_client._reconnect_nonce, 2) + self.assertEqual(fake_client._reconnect_interval, 3) + self.assertEqual(fake_client._ping_interval, 4) + class TestAdapterBehavior(unittest.TestCase): @patch.dict(os.environ, {}, clear=True) @@ -690,10 +822,10 @@ def test_group_message_requires_mentions_even_when_policy_open(self): adapter = FeishuAdapter(PlatformConfig()) message = SimpleNamespace(mentions=[]) sender_id = SimpleNamespace(open_id="ou_any", user_id=None) - self.assertFalse(adapter._should_accept_group_message(message, sender_id)) + self.assertFalse(adapter._should_accept_group_message(message, sender_id, "")) message_with_mention = SimpleNamespace(mentions=[SimpleNamespace(key="@_user_1")]) - self.assertFalse(adapter._should_accept_group_message(message_with_mention, sender_id)) + self.assertFalse(adapter._should_accept_group_message(message_with_mention, sender_id, "")) @patch.dict(os.environ, {"FEISHU_GROUP_POLICY": "open"}, clear=True) def test_group_message_with_other_user_mention_is_rejected_when_bot_identity_unknown(self): @@ -707,7 +839,7 @@ def test_group_message_with_other_user_mention_is_rejected_when_bot_identity_unk id=SimpleNamespace(open_id="ou_other", user_id="u_other"), ) - self.assertFalse(adapter._should_accept_group_message(SimpleNamespace(mentions=[other_mention]), sender_id)) + self.assertFalse(adapter._should_accept_group_message(SimpleNamespace(mentions=[other_mention]), sender_id, "")) @patch.dict( os.environ, @@ -736,28 +868,222 @@ def test_group_message_allowlist_and_mention_both_required(self): adapter._should_accept_group_message( mentioned, SimpleNamespace(open_id="ou_allowed", user_id=None), + "", ) ) self.assertFalse( adapter._should_accept_group_message( mentioned, SimpleNamespace(open_id="ou_blocked", user_id=None), + "", ) ) - @patch.dict( - os.environ, - { - "FEISHU_GROUP_POLICY": "open", - "FEISHU_BOT_OPEN_ID": "ou_bot", - }, - clear=True, - ) + def test_per_group_allowlist_policy_gates_by_sender(self): + from gateway.config import PlatformConfig + from gateway.platforms.feishu import FeishuAdapter + + config = PlatformConfig( + extra={ + "group_rules": { + "oc_chat_a": { + "policy": "allowlist", + "allowlist": ["ou_alice", "ou_bob"], + } + } + } + ) + adapter = FeishuAdapter(config) + adapter._bot_open_id = "ou_bot" + + message = SimpleNamespace( + mentions=[SimpleNamespace(name="Bot", id=SimpleNamespace(open_id="ou_bot", user_id=None))] + ) + + self.assertTrue( + adapter._should_accept_group_message( + message, + SimpleNamespace(open_id="ou_alice", user_id=None), + "oc_chat_a", + ) + ) + self.assertFalse( + adapter._should_accept_group_message( + message, + SimpleNamespace(open_id="ou_charlie", user_id=None), + "oc_chat_a", + ) + ) + + def test_per_group_blacklist_policy_blocks_specific_users(self): + from gateway.config import PlatformConfig + from gateway.platforms.feishu import FeishuAdapter + + config = PlatformConfig( + extra={ + "group_rules": { + "oc_chat_b": { + "policy": "blacklist", + "blacklist": ["ou_blocked"], + } + } + } + ) + adapter = FeishuAdapter(config) + adapter._bot_open_id = "ou_bot" + + message = SimpleNamespace( + mentions=[SimpleNamespace(name="Bot", id=SimpleNamespace(open_id="ou_bot", user_id=None))] + ) + + self.assertTrue( + adapter._should_accept_group_message( + message, + SimpleNamespace(open_id="ou_alice", user_id=None), + "oc_chat_b", + ) + ) + self.assertFalse( + adapter._should_accept_group_message( + message, + SimpleNamespace(open_id="ou_blocked", user_id=None), + "oc_chat_b", + ) + ) + + def test_per_group_admin_only_policy_requires_admin(self): + from gateway.config import PlatformConfig + from gateway.platforms.feishu import FeishuAdapter + + config = PlatformConfig( + extra={ + "admins": ["ou_admin"], + "group_rules": { + "oc_chat_c": { + "policy": "admin_only", + } + }, + } + ) + adapter = FeishuAdapter(config) + adapter._bot_open_id = "ou_bot" + + message = SimpleNamespace( + mentions=[SimpleNamespace(name="Bot", id=SimpleNamespace(open_id="ou_bot", user_id=None))] + ) + + self.assertTrue( + adapter._should_accept_group_message( + message, + SimpleNamespace(open_id="ou_admin", user_id=None), + "oc_chat_c", + ) + ) + self.assertFalse( + adapter._should_accept_group_message( + message, + SimpleNamespace(open_id="ou_regular", user_id=None), + "oc_chat_c", + ) + ) + + def test_per_group_disabled_policy_blocks_all(self): + from gateway.config import PlatformConfig + from gateway.platforms.feishu import FeishuAdapter + + config = PlatformConfig( + extra={ + "admins": ["ou_admin"], + "group_rules": { + "oc_chat_d": { + "policy": "disabled", + } + }, + } + ) + adapter = FeishuAdapter(config) + adapter._bot_open_id = "ou_bot" + + message = SimpleNamespace( + mentions=[SimpleNamespace(name="Bot", id=SimpleNamespace(open_id="ou_bot", user_id=None))] + ) + + self.assertTrue( + adapter._should_accept_group_message( + message, + SimpleNamespace(open_id="ou_admin", user_id=None), + "oc_chat_d", + ) + ) + self.assertFalse( + adapter._should_accept_group_message( + message, + SimpleNamespace(open_id="ou_regular", user_id=None), + "oc_chat_d", + ) + ) + + def test_global_admins_bypass_all_group_rules(self): + from gateway.config import PlatformConfig + from gateway.platforms.feishu import FeishuAdapter + + config = PlatformConfig( + extra={ + "admins": ["ou_admin"], + "group_rules": { + "oc_chat_e": { + "policy": "allowlist", + "allowlist": ["ou_alice"], + } + }, + } + ) + adapter = FeishuAdapter(config) + adapter._bot_open_id = "ou_bot" + + message = SimpleNamespace( + mentions=[SimpleNamespace(name="Bot", id=SimpleNamespace(open_id="ou_bot", user_id=None))] + ) + + self.assertTrue( + adapter._should_accept_group_message( + message, + SimpleNamespace(open_id="ou_admin", user_id=None), + "oc_chat_e", + ) + ) + + def test_default_group_policy_fallback_for_chats_without_explicit_rule(self): + from gateway.config import PlatformConfig + from gateway.platforms.feishu import FeishuAdapter + + config = PlatformConfig( + extra={ + "default_group_policy": "open", + } + ) + adapter = FeishuAdapter(config) + adapter._bot_open_id = "ou_bot" + + message = SimpleNamespace( + mentions=[SimpleNamespace(name="Bot", id=SimpleNamespace(open_id="ou_bot", user_id=None))] + ) + + self.assertTrue( + adapter._should_accept_group_message( + message, + SimpleNamespace(open_id="ou_anyone", user_id=None), + "oc_chat_unknown", + ) + ) + + @patch.dict(os.environ, {"FEISHU_GROUP_POLICY": "open"}, clear=True) def test_group_message_matches_bot_open_id_when_configured(self): from gateway.config import PlatformConfig from gateway.platforms.feishu import FeishuAdapter adapter = FeishuAdapter(PlatformConfig()) + adapter._bot_open_id = "ou_bot" sender_id = SimpleNamespace(open_id="ou_any", user_id=None) bot_mention = SimpleNamespace( @@ -769,22 +1095,16 @@ def test_group_message_matches_bot_open_id_when_configured(self): id=SimpleNamespace(open_id="ou_other", user_id="u_other"), ) - self.assertTrue(adapter._should_accept_group_message(SimpleNamespace(mentions=[bot_mention]), sender_id)) - self.assertFalse(adapter._should_accept_group_message(SimpleNamespace(mentions=[other_mention]), sender_id)) + self.assertTrue(adapter._should_accept_group_message(SimpleNamespace(mentions=[bot_mention]), sender_id, "")) + self.assertFalse(adapter._should_accept_group_message(SimpleNamespace(mentions=[other_mention]), sender_id, "")) - @patch.dict( - os.environ, - { - "FEISHU_GROUP_POLICY": "open", - "FEISHU_BOT_NAME": "Hermes Bot", - }, - clear=True, - ) + @patch.dict(os.environ, {"FEISHU_GROUP_POLICY": "open"}, clear=True) def test_group_message_matches_bot_name_when_only_name_available(self): from gateway.config import PlatformConfig from gateway.platforms.feishu import FeishuAdapter adapter = FeishuAdapter(PlatformConfig()) + adapter._bot_name = "Hermes Bot" sender_id = SimpleNamespace(open_id="ou_any", user_id=None) named_mention = SimpleNamespace( @@ -796,22 +1116,16 @@ def test_group_message_matches_bot_name_when_only_name_available(self): id=SimpleNamespace(open_id="ou_other", user_id="u_other"), ) - self.assertTrue(adapter._should_accept_group_message(SimpleNamespace(mentions=[named_mention]), sender_id)) - self.assertFalse(adapter._should_accept_group_message(SimpleNamespace(mentions=[different_mention]), sender_id)) + self.assertTrue(adapter._should_accept_group_message(SimpleNamespace(mentions=[named_mention]), sender_id, "")) + self.assertFalse(adapter._should_accept_group_message(SimpleNamespace(mentions=[different_mention]), sender_id, "")) - @patch.dict( - os.environ, - { - "FEISHU_GROUP_POLICY": "open", - "FEISHU_BOT_OPEN_ID": "ou_bot", - }, - clear=True, - ) + @patch.dict(os.environ, {"FEISHU_GROUP_POLICY": "open"}, clear=True) def test_group_post_message_uses_parsed_mentions_when_sdk_mentions_missing(self): from gateway.config import PlatformConfig from gateway.platforms.feishu import FeishuAdapter adapter = FeishuAdapter(PlatformConfig()) + adapter._bot_open_id = "ou_bot" sender_id = SimpleNamespace(open_id="ou_any", user_id=None) message = SimpleNamespace( message_type="post", @@ -819,7 +1133,7 @@ def test_group_post_message_uses_parsed_mentions_when_sdk_mentions_missing(self) content='{"en_us":{"content":[[{"tag":"at","user_name":"Hermes","open_id":"ou_bot"}]]}}', ) - self.assertTrue(adapter._should_accept_group_message(message, sender_id)) + self.assertTrue(adapter._should_accept_group_message(message, sender_id, "")) @patch.dict(os.environ, {}, clear=True) def test_extract_post_message_as_text(self): @@ -1196,7 +1510,12 @@ def test_message_event_submits_to_adapter_loop(self): from gateway.platforms.feishu import FeishuAdapter adapter = FeishuAdapter(PlatformConfig()) - adapter._loop = object() + + class _Loop: + def is_closed(self): + return False + + adapter._loop = _Loop() message = SimpleNamespace( message_id="om_text", @@ -1210,6 +1529,7 @@ def test_message_event_submits_to_adapter_loop(self): data = SimpleNamespace(event=SimpleNamespace(message=message, sender=sender)) future = SimpleNamespace(add_done_callback=lambda *_args, **_kwargs: None) + def _submit(coro, _loop): coro.close() return future @@ -1219,6 +1539,30 @@ def _submit(coro, _loop): self.assertTrue(submit.called) + @patch.dict(os.environ, {}, clear=True) + def test_webhook_request_uses_same_message_dispatch_path(self): + from gateway.config import PlatformConfig + from gateway.platforms.feishu import FeishuAdapter + + adapter = FeishuAdapter(PlatformConfig()) + adapter._on_message_event = Mock() + + body = json.dumps({ + "header": {"event_type": "im.message.receive_v1"}, + "event": {"message": {"message_id": "om_test"}}, + }).encode("utf-8") + request = SimpleNamespace( + remote="127.0.0.1", + content_length=None, + headers={}, + read=AsyncMock(return_value=body), + ) + + response = asyncio.run(adapter._handle_webhook_request(request)) + + self.assertEqual(response.status, 200) + adapter._on_message_event.assert_called_once() + @patch.dict(os.environ, {}, clear=True) def test_process_inbound_message_uses_event_sender_identity_only(self): from gateway.config import PlatformConfig @@ -2456,7 +2800,7 @@ def test_at_all_in_content_accepts_without_explicit_bot_mention(self): mentions=[], ) sender_id = SimpleNamespace(open_id="ou_any", user_id=None) - self.assertTrue(adapter._should_accept_group_message(message, sender_id)) + self.assertTrue(adapter._should_accept_group_message(message, sender_id, "")) @patch.dict(os.environ, {"FEISHU_GROUP_POLICY": "allowlist", "FEISHU_ALLOWED_USERS": "ou_allowed"}, clear=True) def test_at_all_still_requires_policy_gate(self): @@ -2468,10 +2812,10 @@ def test_at_all_still_requires_policy_gate(self): message = SimpleNamespace(content='{"text":"@_all attention"}', mentions=[]) # Non-allowlisted user — should be blocked even with @_all. blocked_sender = SimpleNamespace(open_id="ou_blocked", user_id=None) - self.assertFalse(adapter._should_accept_group_message(message, blocked_sender)) + self.assertFalse(adapter._should_accept_group_message(message, blocked_sender, "")) # Allowlisted user — should pass. allowed_sender = SimpleNamespace(open_id="ou_allowed", user_id=None) - self.assertTrue(adapter._should_accept_group_message(message, allowed_sender)) + self.assertTrue(adapter._should_accept_group_message(message, allowed_sender, "")) @unittest.skipUnless(_HAS_LARK_OAPI, "lark-oapi not installed") -
tests/gateway/test_matrix.py@@ -428,6 +428,7 @@ class TestMatrixRequirements: def test_check_requirements_with_token(self, monkeypatch): monkeypatch.setenv("MATRIX_ACCESS_TOKEN", "syt_test") monkeypatch.setenv("MATRIX_HOMESERVER", "https://matrix.example.org") + monkeypatch.delenv("MATRIX_ENCRYPTION", raising=False) from gateway.platforms.matrix import check_matrix_requirements try: import nio # noqa: F401 @@ -448,6 +449,45 @@ def test_check_requirements_without_homeserver(self, monkeypatch): from gateway.platforms.matrix import check_matrix_requirements assert check_matrix_requirements() is False + def test_check_requirements_encryption_true_no_e2ee_deps(self, monkeypatch): + """MATRIX_ENCRYPTION=true should fail if python-olm is not installed.""" + monkeypatch.setenv("MATRIX_ACCESS_TOKEN", "syt_test") + monkeypatch.setenv("MATRIX_HOMESERVER", "https://matrix.example.org") + monkeypatch.setenv("MATRIX_ENCRYPTION", "true") + + from gateway.platforms import matrix as matrix_mod + with patch.object(matrix_mod, "_check_e2ee_deps", return_value=False): + assert matrix_mod.check_matrix_requirements() is False + + def test_check_requirements_encryption_false_no_e2ee_deps_ok(self, monkeypatch): + """Without encryption, missing E2EE deps should not block startup.""" + monkeypatch.setenv("MATRIX_ACCESS_TOKEN", "syt_test") + monkeypatch.setenv("MATRIX_HOMESERVER", "https://matrix.example.org") + monkeypatch.delenv("MATRIX_ENCRYPTION", raising=False) + + from gateway.platforms import matrix as matrix_mod + with patch.object(matrix_mod, "_check_e2ee_deps", return_value=False): + # Still needs nio itself to be importable + try: + import nio # noqa: F401 + assert matrix_mod.check_matrix_requirements() is True + except ImportError: + assert matrix_mod.check_matrix_requirements() is False + + def test_check_requirements_encryption_true_with_e2ee_deps(self, monkeypatch): + """MATRIX_ENCRYPTION=true should pass if E2EE deps are available.""" + monkeypatch.setenv("MATRIX_ACCESS_TOKEN", "syt_test") + monkeypatch.setenv("MATRIX_HOMESERVER", "https://matrix.example.org") + monkeypatch.setenv("MATRIX_ENCRYPTION", "true") + + from gateway.platforms import matrix as matrix_mod + with patch.object(matrix_mod, "_check_e2ee_deps", return_value=True): + try: + import nio # noqa: F401 + assert matrix_mod.check_matrix_requirements() is True + except ImportError: + assert matrix_mod.check_matrix_requirements() is False + # --------------------------------------------------------------------------- # Access-token auth / E2EE bootstrap @@ -516,10 +556,12 @@ def _restore_login(user_id, device_id, access_token): fake_nio.InviteMemberEvent = type("InviteMemberEvent", (), {}) fake_nio.MegolmEvent = type("MegolmEvent", (), {}) - with patch.dict("sys.modules", {"nio": fake_nio}): - with patch.object(adapter, "_refresh_dm_cache", AsyncMock()): - with patch.object(adapter, "_sync_loop", AsyncMock(return_value=None)): - assert await adapter.connect() is True + from gateway.platforms import matrix as matrix_mod + with patch.object(matrix_mod, "_check_e2ee_deps", return_value=True): + with patch.dict("sys.modules", {"nio": fake_nio}): + with patch.object(adapter, "_refresh_dm_cache", AsyncMock()): + with patch.object(adapter, "_sync_loop", AsyncMock(return_value=None)): + assert await adapter.connect() is True fake_client.restore_login.assert_called_once_with( "@bot:example.org", "DEV123", "syt_test_access_token" @@ -532,6 +574,326 @@ def _restore_login(user_id, device_id, access_token): await adapter.disconnect() +class TestMatrixE2EEHardFail: + """connect() must refuse to start when E2EE is requested but deps are missing.""" + + @pytest.mark.asyncio + async def test_connect_fails_when_encryption_true_but_no_e2ee_deps(self): + from gateway.platforms.matrix import MatrixAdapter + + config = PlatformConfig( + enabled=True, + token="[redacted:secret-assignment]", + extra={ + "homeserver": "https://matrix.example.org", + "user_id": "@bot:example.org", + "encryption": True, + }, + ) + adapter = MatrixAdapter(config) + + fake_nio = MagicMock() + fake_nio.AsyncClient = MagicMock() + + from gateway.platforms import matrix as matrix_mod + with patch.object(matrix_mod, "_check_e2ee_deps", return_value=False): + with patch.dict("sys.modules", {"nio": fake_nio}): + result = await adapter.connect() + + assert result is False + + @pytest.mark.asyncio + async def test_connect_fails_when_olm_not_loaded_after_login(self): + """Even if _check_e2ee_deps passes, if olm is None after auth, hard-fail.""" + from gateway.platforms.matrix import MatrixAdapter + + config = PlatformConfig( + enabled=True, + token="[redacted:secret-assignment]", + extra={ + "homeserver": "https://matrix.example.org", + "user_id": "@bot:example.org", + "encryption": True, + }, + ) + adapter = MatrixAdapter(config) + + class FakeWhoamiResponse: + def __init__(self, user_id, device_id): + self.user_id = user_id + self.device_id = device_id + + fake_client = MagicMock() + fake_client.whoami = AsyncMock(return_value=FakeWhoamiResponse("@bot:example.org", "DEV123")) + fake_client.close = AsyncMock() + # olm is None — crypto store not loaded + fake_client.olm = None + fake_client.should_upload_keys = False + + def _restore_login(user_id, device_id, access_token): + fake_client.user_id = user_id + fake_client.device_id = device_id + fake_client.access_token = [redacted:secret-assignment] + + fake_client.restore_login = MagicMock(side_effect=_restore_login) + + fake_nio = MagicMock() + fake_nio.AsyncClient = MagicMock(return_value=fake_client) + fake_nio.WhoamiResponse = FakeWhoamiResponse + + from gateway.platforms import matrix as matrix_mod + with patch.object(matrix_mod, "_check_e2ee_deps", return_value=True): + with patch.dict("sys.modules", {"nio": fake_nio}): + result = await adapter.connect() + + assert result is False + fake_client.close.assert_awaited_once() + + +class TestMatrixDeviceId: + """MATRIX_DEVICE_ID should be used for stable device identity.""" + + def test_device_id_from_config_extra(self): + from gateway.platforms.matrix import MatrixAdapter + + config = PlatformConfig( + enabled=True, + token="[redacted:secret-assignment]", + extra={ + "homeserver": "https://matrix.example.org", + "device_id": "HERMES_BOT_STABLE", + }, + ) + adapter = MatrixAdapter(config) + assert adapter._device_id == "HERMES_BOT_STABLE" + + def test_device_id_from_env(self, monkeypatch): + monkeypatch.setenv("MATRIX_DEVICE_ID", "FROM_ENV") + + from gateway.platforms.matrix import MatrixAdapter + + config = PlatformConfig( + enabled=True, + token="[redacted:secret-assignment]", + extra={ + "homeserver": "https://matrix.example.org", + }, + ) + adapter = MatrixAdapter(config) + assert adapter._device_id == "FROM_ENV" + + def test_device_id_config_takes_precedence_over_env(self, monkeypatch): + monkeypatch.setenv("MATRIX_DEVICE_ID", "FROM_ENV") + + from gateway.platforms.matrix import MatrixAdapter + + config = PlatformConfig( + enabled=True, + token="[redacted:secret-assignment]", + extra={ + "homeserver": "https://matrix.example.org", + "device_id": "FROM_CONFIG", + }, + ) + adapter = MatrixAdapter(config) + assert adapter._device_id == "FROM_CONFIG" + + @pytest.mark.asyncio + async def test_connect_uses_configured_device_id_over_whoami(self): + """When MATRIX_DEVICE_ID is set, it should be used instead of whoami device_id.""" + from gateway.platforms.matrix import MatrixAdapter + + config = PlatformConfig( + enabled=True, + token="[redacted:secret-assignment]", + extra={ + "homeserver": "https://matrix.example.org", + "user_id": "@bot:example.org", + "encryption": True, + "device_id": "MY_STABLE_DEVICE", + }, + ) + adapter = MatrixAdapter(config) + + class FakeWhoamiResponse: + def __init__(self, user_id, device_id): + self.user_id = user_id + self.device_id = device_id + + class FakeSyncResponse: + def __init__(self): + self.rooms = MagicMock(join={}) + + fake_client = MagicMock() + fake_client.whoami = AsyncMock(return_value=FakeWhoamiResponse("@bot:example.org", "WHOAMI_DEV")) + fake_client.sync = AsyncMock(return_value=FakeSyncResponse()) + fake_client.keys_upload = AsyncMock() + fake_client.keys_query = AsyncMock() + fake_client.keys_claim = AsyncMock() + fake_client.send_to_device_messages = AsyncMock(return_value=[]) + fake_client.get_users_for_key_claiming = MagicMock(return_value={}) + fake_client.close = AsyncMock() + fake_client.add_event_callback = MagicMock() + fake_client.rooms = {} + fake_client.account_data = {} + fake_client.olm = object() + fake_client.should_upload_keys = False + fake_client.should_query_keys = False + fake_client.should_claim_keys = False + + def _restore_login(user_id, device_id, access_token): + fake_client.user_id = user_id + fake_client.device_id = device_id + fake_client.access_token = [redacted:secret-assignment] + + fake_client.restore_login = MagicMock(side_effect=_restore_login) + + fake_nio = MagicMock() + fake_nio.AsyncClient = MagicMock(return_value=fake_client) + fake_nio.WhoamiResponse = FakeWhoamiResponse + fake_nio.SyncResponse = FakeSyncResponse + fake_nio.LoginResponse = type("LoginResponse", (), {}) + fake_nio.RoomMessageText = type("RoomMessageText", (), {}) + fake_nio.RoomMessageImage = type("RoomMessageImage", (), {}) + fake_nio.RoomMessageAudio = type("RoomMessageAudio", (), {}) + fake_nio.RoomMessageVideo = type("RoomMessageVideo", (), {}) + fake_nio.RoomMessageFile = type("RoomMessageFile", (), {}) + fake_nio.InviteMemberEvent = type("InviteMemberEvent", (), {}) + fake_nio.MegolmEvent = type("MegolmEvent", (), {}) + + from gateway.platforms import matrix as matrix_mod + with patch.object(matrix_mod, "_check_e2ee_deps", return_value=True): + with patch.dict("sys.modules", {"nio": fake_nio}): + with patch.object(adapter, "_refresh_dm_cache", AsyncMock()): + with patch.object(adapter, "_sync_loop", AsyncMock(return_value=None)): + assert await adapter.connect() is True + + # The configured device_id should override the whoami device_id + fake_client.restore_login.assert_called_once_with( + "@bot:example.org", "MY_STABLE_DEVICE", "syt_test_access_token" + ) + assert fake_client.device_id == "MY_STABLE_DEVICE" + + # Verify device_id was passed to nio.AsyncClient constructor + ctor_call = fake_nio.AsyncClient.call_args + assert ctor_call.kwargs.get("device_id") == "MY_STABLE_DEVICE" + + await adapter.disconnect() + + +class TestMatrixE2EEClientConstructorFailure: + """connect() should hard-fail if nio.AsyncClient() raises when encryption is on.""" + + @pytest.mark.asyncio + async def test_connect_fails_when_e2ee_client_constructor_raises(self): + from gateway.platforms.matrix import MatrixAdapter + + config = PlatformConfig( + enabled=True, + token="[redacted:secret-assignment]", + extra={ + "homeserver": "https://matrix.example.org", + "user_id": "@bot:example.org", + "encryption": True, + }, + ) + adapter = MatrixAdapter(config) + + fake_nio = MagicMock() + fake_nio.AsyncClient = MagicMock(side_effect=Exception("olm init failed")) + + from gateway.platforms import matrix as matrix_mod + with patch.object(matrix_mod, "_check_e2ee_deps", return_value=True): + with patch.dict("sys.modules", {"nio": fake_nio}): + result = await adapter.connect() + + assert result is False + + +class TestMatrixPasswordLoginDeviceId: + """MATRIX_DEVICE_ID should be passed to nio.AsyncClient even with password login.""" + + @pytest.mark.asyncio + async def test_password_login_passes_device_id_to_constructor(self): + from gateway.platforms.matrix import MatrixAdapter + + config = PlatformConfig( + enabled=True, + extra={ + "homeserver": "https://matrix.example.org", + "user_id": "@bot:example.org", + "password": "secret", + "device_id": "STABLE_PW_DEVICE", + }, + ) + adapter = MatrixAdapter(config) + + class FakeLoginResponse: + pass + + class FakeSyncResponse: + def __init__(self): + self.rooms = MagicMock(join={}) + + fake_client = MagicMock() + fake_client.login = AsyncMock(return_value=FakeLoginResponse()) + fake_client.sync = AsyncMock(return_value=FakeSyncResponse()) + fake_client.close = AsyncMock() + fake_client.add_event_callback = MagicMock() + fake_client.rooms = {} + fake_client.account_data = {} + + fake_nio = MagicMock() + fake_nio.AsyncClient = MagicMock(return_value=fake_client) + fake_nio.LoginResponse = FakeLoginResponse + fake_nio.SyncResponse = FakeSyncResponse + fake_nio.RoomMessageText = type("RoomMessageText", (), {}) + fake_nio.RoomMessageImage = type("RoomMessageImage", (), {}) + fake_nio.RoomMessageAudio = type("RoomMessageAudio", (), {}) + fake_nio.RoomMessageVideo = type("RoomMessageVideo", (), {}) + fake_nio.RoomMessageFile = type("RoomMessageFile", (), {}) + fake_nio.InviteMemberEvent = type("InviteMemberEvent", (), {}) + + with patch.dict("sys.modules", {"nio": fake_nio}): + with patch.object(adapter, "_refresh_dm_cache", AsyncMock()): + with patch.object(adapter, "_sync_loop", AsyncMock(return_value=None)): + assert await adapter.connect() is True + + # Verify device_id was passed to the nio.AsyncClient constructor + ctor_call = fake_nio.AsyncClient.call_args + assert ctor_call.kwargs.get("device_id") == "STABLE_PW_DEVICE" + + await adapter.disconnect() + + +class TestMatrixDeviceIdConfig: + """MATRIX_DEVICE_ID should be plumbed through gateway config.""" + + def test_device_id_in_config_extra(self, monkeypatch): + monkeypatch.setenv("MATRIX_ACCESS_TOKEN", "syt_abc123") + monkeypatch.setenv("MATRIX_HOMESERVER", "https://matrix.example.org") + monkeypatch.setenv("MATRIX_DEVICE_ID", "HERMES_BOT") + + from gateway.config import GatewayConfig, _apply_env_overrides + config = GatewayConfig() + _apply_env_overrides(config) + + mc = config.platforms[Platform.MATRIX] + assert mc.extra.get("device_id") == "HERMES_BOT" + + def test_device_id_not_set_when_env_empty(self, monkeypatch): + monkeypatch.setenv("MATRIX_ACCESS_TOKEN", "syt_abc123") + monkeypatch.setenv("MATRIX_HOMESERVER", "https://matrix.example.org") + monkeypatch.delenv("MATRIX_DEVICE_ID", raising=False) + + from gateway.config import GatewayConfig, _apply_env_overrides + config = GatewayConfig() + _apply_env_overrides(config) + + mc = config.platforms[Platform.MATRIX] + assert "device_id" not in mc.extra + + class TestMatrixE2EEMaintenance: @pytest.mark.asyncio async def test_sync_loop_runs_e2ee_maintenance_requests(self): @@ -1071,10 +1433,12 @@ class FakeMegolmEvent: ... fake_nio.InviteMemberEvent = FakeInviteMemberEvent fake_nio.MegolmEvent = FakeMegolmEvent - with patch.dict("sys.modules", {"nio": fake_nio}): - with patch.object(adapter, "_refresh_dm_cache", AsyncMock()): - with patch.object(adapter, "_sync_loop", AsyncMock(return_value=None)): - assert await adapter.connect() is True + from gateway.platforms import matrix as matrix_mod + with patch.object(matrix_mod, "_check_e2ee_deps", return_value=True): + with patch.dict("sys.modules", {"nio": fake_nio}): + with patch.object(adapter, "_refresh_dm_cache", AsyncMock()): + with patch.object(adapter, "_sync_loop", AsyncMock(return_value=None)): + assert await adapter.connect() is True callback_classes = [call.args[1] for call in fake_client.add_event_callback.call_args_list] assert FakeRoomEncryptedImage in callback_classes -
tests/test_cli_browser_connect.py@@ -0,0 +1,46 @@ +"""Tests for CLI browser CDP auto-launch helpers.""" + +import os +from unittest.mock import patch + +from cli import HermesCLI + + +class TestChromeDebugLaunch: + def test_windows_launch_uses_browser_found_on_path(self): + captured = {} + + def fake_popen(cmd, **kwargs): + captured["cmd"] = cmd + captured["kwargs"] = kwargs + return object() + + with patch("cli.shutil.which", side_effect=lambda name: r"C:\Chrome\chrome.exe" if name == "chrome.exe" else None), \ + patch("cli.os.path.isfile", side_effect=lambda path: path == r"C:\Chrome\chrome.exe"), \ + patch("subprocess.Popen", side_effect=fake_popen): + assert HermesCLI._try_launch_chrome_debug(9333, "Windows") is True + + assert captured["cmd"] == [r"C:\Chrome\chrome.exe", "--remote-debugging-port=9333"] + assert captured["kwargs"]["start_new_session"] is True + + def test_windows_launch_falls_back_to_common_install_dirs(self, monkeypatch): + captured = {} + program_files = r"C:\Program Files" + # Use os.path.join so path separators match cross-platform + installed = os.path.join(program_files, "Google", "Chrome", "Application", "chrome.exe") + + def fake_popen(cmd, **kwargs): + captured["cmd"] = cmd + captured["kwargs"] = kwargs + return object() + + monkeypatch.setenv("ProgramFiles", program_files) + monkeypatch.delenv("ProgramFiles(x86)", raising=False) + monkeypatch.delenv("LOCALAPPDATA", raising=False) + + with patch("cli.shutil.which", return_value=None), \ + patch("cli.os.path.isfile", side_effect=lambda path: path == installed), \ + patch("subprocess.Popen", side_effect=fake_popen): + assert HermesCLI._try_launch_chrome_debug(9222, "Windows") is True + + assert captured["cmd"] == [installed, "--remote-debugging-port=9222"] -
tests/test_cli_status_bar.py@@ -1,5 +1,6 @@ from datetime import datetime, timedelta from types import SimpleNamespace +from unittest.mock import MagicMock, patch from cli import HermesCLI @@ -78,6 +79,92 @@ def test_build_status_bar_text_for_wide_terminal(self): assert "$0.06" not in text # cost hidden by default assert "15m" in text + def test_input_height_counts_wide_characters_using_cell_width(self): + cli_obj = _make_cli() + + class _Doc: + lines = ["你" * 10] + + class _Buffer: + document = _Doc() + + input_area = SimpleNamespace(buffer=_Buffer()) + + def _input_height(): + try: + from prompt_toolkit.application import get_app + from prompt_toolkit.utils import get_cwidth + + doc = input_area.buffer.document + prompt_width = max(2, get_cwidth(cli_obj._get_tui_prompt_text())) + try: + available_width = get_app().output.get_size().columns - prompt_width + except Exception: + import shutil + available_width = shutil.get_terminal_size((80, 24)).columns - prompt_width + if available_width < 10: + available_width = 40 + visual_lines = 0 + for line in doc.lines: + line_width = get_cwidth(line) + if line_width <= 0: + visual_lines += 1 + else: + visual_lines += max(1, -(-line_width // available_width)) + return min(max(visual_lines, 1), 8) + except Exception: + return 1 + + mock_app = MagicMock() + mock_app.output.get_size.return_value = MagicMock(columns=14) + with patch.object(HermesCLI, "_get_tui_prompt_text", return_value="❯ "), \ + patch("prompt_toolkit.application.get_app", return_value=mock_app): + assert _input_height() == 2 + + def test_input_height_uses_prompt_toolkit_width_over_shutil(self): + cli_obj = _make_cli() + + class _Doc: + lines = ["你" * 10] + + class _Buffer: + document = _Doc() + + input_area = SimpleNamespace(buffer=_Buffer()) + + def _input_height(): + try: + from prompt_toolkit.application import get_app + from prompt_toolkit.utils import get_cwidth + + doc = input_area.buffer.document + prompt_width = max(2, get_cwidth(cli_obj._get_tui_prompt_text())) + try: + available_width = get_app().output.get_size().columns - prompt_width + except Exception: + import shutil + available_width = shutil.get_terminal_size((80, 24)).columns - prompt_width + if available_width < 10: + available_width = 40 + visual_lines = 0 + for line in doc.lines: + line_width = get_cwidth(line) + if line_width <= 0: + visual_lines += 1 + else: + visual_lines += max(1, -(-line_width // available_width)) + return min(max(visual_lines, 1), 8) + except Exception: + return 1 + + mock_app = MagicMock() + mock_app.output.get_size.return_value = MagicMock(columns=14) + with patch.object(HermesCLI, "_get_tui_prompt_text", return_value="❯ "), \ + patch("prompt_toolkit.application.get_app", return_value=mock_app), \ + patch("shutil.get_terminal_size") as mock_shutil: + assert _input_height() == 2 + mock_shutil.assert_not_called() + def test_build_status_bar_text_no_cost_in_status_bar(self): cli_obj = _attach_agent( _make_cli(),
-
-
#26195 fix(tui): width-aware markdown table rendering with vertical fallback
files:
ui-tui/src/components/markdown.tsx,ui-tui/src/components/messageLine.tsx,ui-tui/src/components/streamingMarkdown.tsxShow the diff
-
ui-tui/src/components/markdown.tsx@@ -200,44 +200,288 @@ export const stripInlineMarkup = (v: string) => .replace(/(?<!\$)\$([^\s$](?:[^$\n]*?[^\s$])?)\$(?!\$)/g, '$1') .replace(/\\\(([^\n]+?)\\\)/g, '$1') -const renderTable = (k: number, rows: string[][], t: Theme) => { - // Column widths in *display cells*, not UTF-16 code units. CJK - // glyphs and most emoji render as two cells but `String#length` - // counts them as one, which collapses Chinese / Japanese / Korean - // tables into drift across rows. `stringWidth` (Bun.stringWidth - // fast path + an East-Asian-width-aware fallback, memoised in - // @hermes/ink) returns the actual cell count. - const cellWidth = (raw: string) => stringWidth(stripInlineMarkup(raw)) - - const widths = rows[0]!.map((_, ci) => Math.max(...rows.map(r => cellWidth(r[ci] ?? '')))) - - // Thin divider under the header. Without it tables look like prose - // with extra spacing because the header is just accent-coloured text - // (#15534). We avoid full borders on purpose — column widths come - // from `stringWidth(...)`, so the dividers and the row content stay - // in sync on CJK / emoji tables; tab-style column gaps still read - // cleanly without the boxed look. - const sep = widths.map(w => '─'.repeat(Math.max(1, w))).join(' ') +const SAFETY_MARGIN = 4 +const MIN_COL_WIDTH = 3 +const COL_GAP = 2 // the ' ' between columns +const TABLE_PADDING_LEFT = 2 // paddingLeft={2} on the outer <Box> + +const renderTable = (k: number, rows: string[][], t: Theme, cols?: number) => { + // Guard: empty table + if (rows.length === 0 || rows[0]!.length === 0) return null + + const cellDisplayWidth = (raw: string) => stringWidth(stripInlineMarkup(raw)) + + // Minimum width: longest word in a cell (to avoid breaking words) + const minCellWidth = (raw: string) => { + const text = stripInlineMarkup(raw) + const words = text.split(/\s+/).filter(w => w.length > 0) + if (words.length === 0) return MIN_COL_WIDTH + return Math.max(...words.map(w => stringWidth(w)), MIN_COL_WIDTH) + } - return ( - <Box flexDirection="column" key={k} paddingLeft={2}> - {rows.map((row, ri) => ( - <Fragment key={ri}> - <Box> - {widths.map((w, ci) => ( - <Text bold={ri === 0} color={ri === 0 ? t.color.accent : undefined} key={ci}> - <MdInline t={t} text={row[ci] ?? ''} /> - {' '.repeat(Math.max(0, w - cellWidth(row[ci] ?? '')))} - {ci < widths.length - 1 ? ' ' : ''} - </Text> - ))} - </Box> - {ri === 0 && rows.length > 1 ? ( - <Text color={t.color.muted} dimColor> - {sep} + const numCols = rows[0]!.length + + // Normalize ragged rows: ensure every row has exactly numCols cells + const normalizedRows = rows.map(row => { + if (row.length >= numCols) return row.slice(0, numCols) + return [...row, ...Array<string>(numCols - row.length).fill('')] + }) + + // Ideal widths: max cell content per column + const idealWidths = normalizedRows[0]!.map((_, ci) => + Math.max(...normalizedRows.map(r => cellDisplayWidth(r[ci] ?? '')), MIN_COL_WIDTH) + ) + + // Min widths: longest word per column + const minWidths = normalizedRows[0]!.map((_, ci) => + Math.max(...normalizedRows.map(r => minCellWidth(r[ci] ?? '')), MIN_COL_WIDTH) + ) + + // Available width: cols minus table padding minus column gaps minus safety. + // transcriptBodyWidth (source of cols) subtracts message gutter + scrollbar, + // but NOT this table's paddingLeft — we subtract it here. + const gapOverhead = (numCols - 1) * COL_GAP + const availableWidth = cols + ? Math.max(cols - TABLE_PADDING_LEFT - gapOverhead - SAFETY_MARGIN, numCols * MIN_COL_WIDTH) + : Infinity + + const totalIdeal = idealWidths.reduce((a, b) => a + b, 0) + const totalMin = minWidths.reduce((a, b) => a + b, 0) + + let columnWidths: number[] + let needsWrap = false + + if (totalIdeal <= availableWidth) { + // Tier 1: everything fits at ideal widths + columnWidths = idealWidths + } else if (totalMin <= availableWidth) { + // Tier 2: proportional shrink — distribute extra space beyond minimums + needsWrap = true + const extraSpace = availableWidth - totalMin + const overflows = idealWidths.map((ideal, i) => ideal - minWidths[i]!) + const totalOverflow = overflows.reduce((a, b) => a + b, 0) + if (totalOverflow === 0) { + columnWidths = [...minWidths] + } else { + const rawAlloc = minWidths.map((min, i) => + min + (overflows[i]! / totalOverflow) * extraSpace + ) + columnWidths = rawAlloc.map(v => Math.floor(v)) + // Distribute rounding remainders to columns with largest fractional part + let remainder = availableWidth - columnWidths.reduce((a, b) => a + b, 0) + const fracs = rawAlloc.map((v, i) => ({ i, frac: v - Math.floor(v) })) + .sort((a, b) => b.frac - a.frac) + for (const { i } of fracs) { + if (remainder <= 0) break + columnWidths[i]!++ + remainder-- + } + } + } else { + // Tier 3: even min-widths don't fit — scale proportionally, allow hard breaks. + // NOTE: Math.max(..., MIN_COL_WIDTH) can push total above availableWidth when + // many columns are scaled below 3. This is caught by safetyOverflow → vertical fallback. + needsWrap = true + const scaleFactor = availableWidth / totalMin + const rawAlloc = minWidths.map(w => w * scaleFactor) + columnWidths = rawAlloc.map(v => Math.max(Math.floor(v), MIN_COL_WIDTH)) + let remainder = availableWidth - columnWidths.reduce((a, b) => a + b, 0) + const fracs = rawAlloc.map((v, i) => ({ i, frac: v - Math.floor(v) })) + .sort((a, b) => b.frac - a.frac) + for (const { i } of fracs) { + if (remainder <= 0) break + columnWidths[i]!++ + remainder-- + } + } + + // Grapheme-safe hard-break: prefer Intl.Segmenter, fall back to code-point split + const segmenter = typeof Intl !== 'undefined' && 'Segmenter' in Intl + ? new (Intl as any).Segmenter(undefined, { granularity: 'grapheme' }) + : null + + const graphemes = (s: string): string[] => + segmenter + ? [...segmenter.segment(s)].map((seg: { segment: string }) => seg.segment) + : [...s] + + // Word-wrap plain text to fit within `width` display columns. + // Operates on stripped text for correct width measurement. + const wrapCell = (raw: string, width: number, hard: boolean): string[] => { + const text = stripInlineMarkup(raw) + if (width <= 0) return [text] + if (stringWidth(text) <= width) return [text] + + const words = text.split(/\s+/).filter(w => w.length > 0) + const lines: string[] = [] + let current = '' + let currentWidth = 0 + + for (const word of words) { + const w = stringWidth(word) + if (currentWidth === 0) { + if (hard && w > width) { + for (const ch of graphemes(word)) { + const cw = stringWidth(ch) + if (currentWidth + cw > width && current) { + lines.push(current) + current = '' + currentWidth = 0 + } + current += ch + currentWidth += cw + } + } else { + current = word + currentWidth = w + } + } else if (currentWidth + 1 + w <= width) { + current += ' ' + word + currentWidth += 1 + w + } else { + lines.push(current) + current = word + currentWidth = w + } + } + if (current) lines.push(current) + return lines.length > 0 ? lines : [''] + } + + const isHard = totalMin > availableWidth // tier 3 needs hard word breaks + const sep = columnWidths.map(w => '─'.repeat(Math.max(1, w))).join(' ') + + // When wrapping isn't needed, build single-line strings per row. + // All cells render as plain text via stripInlineMarkup. + // TODO: follow-up — format to ANSI then wrap with wrapAnsi for inline markdown preservation. + // See free-code/src/components/MarkdownTable.tsx L44-L62 for approach. + if (!needsWrap) { + const buildRowString = (row: string[]): string => + row.map((cell, ci) => { + const text = stripInlineMarkup(cell) + const pad = ' '.repeat(Math.max(0, columnWidths[ci]! - stringWidth(text))) + const gap = ci < numCols - 1 ? ' ' : '' + return text + pad + gap + }).join('') + + return ( + <Box flexDirection="column" key={k} paddingLeft={TABLE_PADDING_LEFT}> + {normalizedRows.map((row, ri) => ( + <Fragment key={ri}> + <Text + bold={ri === 0} + color={ri === 0 ? t.color.accent : undefined} + wrap="truncate-end" + > + {buildRowString(row)} </Text> - ) : null} - </Fragment> + {ri === 0 && normalizedRows.length > 1 ? ( + <Text color={t.color.muted} dimColor wrap="truncate-end">{sep}</Text> + ) : null} + </Fragment> + ))} + </Box> + ) + } + + // Wrapping path: build multi-line rows as complete strings. + type LineEntry = { text: string; kind: 'header' | 'separator' | 'body' } + + const buildRowLines = (row: string[]): string[] => { + const cellLines = row.map((cell, ci) => + wrapCell(cell, columnWidths[ci]!, isHard) + ) + const maxLines = Math.max(...cellLines.map(l => l.length), 1) + + const result: string[] = [] + for (let li = 0; li < maxLines; li++) { + let line = '' + for (let ci = 0; ci < numCols; ci++) { + const cl = cellLines[ci] ?? [''] + const cellText = li < cl.length ? cl[li]! : '' + const pad = ' '.repeat(Math.max(0, columnWidths[ci]! - stringWidth(cellText))) + line += cellText + pad + if (ci < numCols - 1) line += ' ' + } + result.push(line) + } + return result + } + + // Build all lines with metadata for styling, tracking tallest body row + const allEntries: LineEntry[] = [] + let tallestBodyRow = 0 + normalizedRows.forEach((row, ri) => { + const kind = ri === 0 ? 'header' as const : 'body' as const + const rowLines = buildRowLines(row) + rowLines.forEach(text => allEntries.push({ text, kind })) + if (ri > 0) tallestBodyRow = Math.max(tallestBodyRow, rowLines.length) + if (ri === 0 && normalizedRows.length > 1) { + allEntries.push({ text: sep, kind: 'separator' }) + } + }) + + // Post-render safety condition: compute max line width. + const maxLineWidth = Math.max(...allEntries.map(e => stringWidth(e.text))) + const safetyOverflow = cols != null && maxLineWidth > cols - TABLE_PADDING_LEFT - SAFETY_MARGIN + + // Scaled vertical threshold — 2-3 col tables stay tabular even with tall cells + const maxRowLinesThreshold = numCols <= 3 ? 8 : numCols <= 6 ? 5 : 4 + + const useVertical = tallestBodyRow > maxRowLinesThreshold || safetyOverflow + + if (useVertical) { + // Edge case: header-only table + if (normalizedRows.length <= 1) { + return ( + <Box flexDirection="column" key={k} paddingLeft={TABLE_PADDING_LEFT}> + <Text bold color={t.color.accent} wrap="wrap-trim"> + {normalizedRows[0]!.map(h => stripInlineMarkup(h)).join(' · ')} + </Text> + </Box> + ) + } + + const headers = normalizedRows[0]! + const dataRows = normalizedRows.slice(1) + const sepWidth = Math.max(1, cols ? Math.min(cols - TABLE_PADDING_LEFT - 1, 40) : 40) + + return ( + <Box flexDirection="column" key={k} paddingLeft={TABLE_PADDING_LEFT}> + {dataRows.map((row, ri) => ( + <Fragment key={ri}> + {ri > 0 ? ( + <Text color={t.color.muted} dimColor>{'─'.repeat(sepWidth)}</Text> + ) : null} + {headers.map((header, ci) => { + const cell = row[ci] ?? '' + const label = stripInlineMarkup(header) || `Col ${ci + 1}` + return ( + <Text key={ci} wrap="wrap-trim"> + <Text bold color={t.color.accent}>{label}:</Text> + {' '}{stripInlineMarkup(cell)} + </Text> + ) + })} + </Fragment> + ))} + </Box> + ) + } + + // Render wrapped horizontal rows — one <Text> per visual line. + return ( + <Box flexDirection="column" key={k} paddingLeft={TABLE_PADDING_LEFT}> + {allEntries.map((entry, i) => ( + <Text + bold={entry.kind === 'header'} + color={entry.kind === 'header' ? t.color.accent : entry.kind === 'separator' ? t.color.muted : undefined} + dimColor={entry.kind === 'separator'} + key={i} + wrap="truncate-end" + > + {entry.text} + </Text> ))} </Box> ) @@ -395,10 +639,10 @@ const cacheSet = (b: Map<string, ReactNode[]>, key: string, v: ReactNode[]) => { } } -function MdImpl({ compact, t, text }: MdProps) { +function MdImpl({ cols, compact, t, text }: MdProps) { const nodes = useMemo(() => { const bucket = cacheBucket(t) - const cacheKey = `${compact ? '1' : '0'}|${text}` + const cacheKey = `${compact ? '1' : '0'}|${cols ?? ''}|${text}` const cached = cacheGet(bucket, cacheKey) if (cached) { @@ -490,7 +734,7 @@ function MdImpl({ compact, t, text }: MdProps) { if (['md', 'markdown'].includes(lang)) { start('paragraph') - nodes.push(<Md compact={compact} key={key} t={t} text={block.join('\n')} />) + nodes.push(<Md cols={cols} compact={compact} key={key} t={t} text={block.join('\n')} />) continue } @@ -785,7 +1029,7 @@ function MdImpl({ compact, t, text }: MdProps) { rows.push(splitRow(lines[i]!)) } - nodes.push(renderTable(key, rows, t)) + nodes.push(renderTable(key, rows, t, cols)) continue } @@ -838,7 +1082,7 @@ function MdImpl({ compact, t, text }: MdProps) { } if (rows.length) { - nodes.push(renderTable(key, rows, t)) + nodes.push(renderTable(key, rows, t, cols)) } continue @@ -852,7 +1096,7 @@ function MdImpl({ compact, t, text }: MdProps) { cacheSet(bucket, cacheKey, nodes) return nodes - }, [compact, t, text]) + }, [cols, compact, t, text]) return <Box flexDirection="column">{nodes}</Box> } @@ -862,6 +1106,7 @@ export const Md = memo(MdImpl) type Kind = 'blank' | 'code' | 'heading' | 'list' | 'paragraph' | 'quote' | 'rule' | 'table' | null interface MdProps { + cols?: number compact?: boolean t: Theme text: string -
ui-tui/src/components/messageLine.tsx@@ -139,13 +139,15 @@ export const MessageLine = memo(function MessageLine({ } if (msg.role === 'assistant') { + const bodyWidth = transcriptBodyWidth(cols, msg.role, t.brand.prompt) + return isStreaming ? ( // Incremental markdown: split at the last stable block boundary so // only the in-flight tail re-tokenizes per delta. See // streamingMarkdown.tsx for the cost model. - <StreamingMd compact={compact} t={t} text={boundedLiveRenderText(msg.text)} /> + <StreamingMd cols={bodyWidth} compact={compact} t={t} text={boundedLiveRenderText(msg.text)} /> ) : ( - <Md compact={compact} t={t} text={limitHistoryRender ? boundedHistoryRenderText(msg.text) : msg.text} /> + <Md cols={bodyWidth} compact={compact} t={t} text={limitHistoryRender ? boundedHistoryRenderText(msg.text) : msg.text} /> ) } -
ui-tui/src/components/streamingMarkdown.tsx@@ -128,7 +128,7 @@ export const findStableBoundary = (text: string) => { return -1 } -export const StreamingMd = memo(function StreamingMd({ compact, t, text }: StreamingMdProps) { +export const StreamingMd = memo(function StreamingMd({ cols, compact, t, text }: StreamingMdProps) { const stablePrefixRef = useRef('') // Reset if the text no longer starts with our recorded prefix (defensive; @@ -151,22 +151,23 @@ export const StreamingMd = memo(function StreamingMd({ compact, t, text }: Strea const unstableSuffix = text.slice(stablePrefix.length) if (!stablePrefix) { - return <Md compact={compact} t={t} text={unstableSuffix} /> + return <Md cols={cols} compact={compact} t={t} text={unstableSuffix} /> } if (!unstableSuffix) { - return <Md compact={compact} t={t} text={stablePrefix} /> + return <Md cols={cols} compact={compact} t={t} text={stablePrefix} /> } return ( <Box flexDirection="column"> - <Md compact={compact} t={t} text={stablePrefix} /> - <Md compact={compact} t={t} text={unstableSuffix} /> + <Md cols={cols} compact={compact} t={t} text={stablePrefix} /> + <Md cols={cols} compact={compact} t={t} text={unstableSuffix} /> </Box> ) }) interface StreamingMdProps { + cols?: number compact?: boolean t: Theme text: string
-
7 PRs closed without merging
From the close comments: done elsewhere 3 · closed without comment 1 · merged via another change 1 · stale or abandoned 1 · other 1
- #4213 fix(cli): use display width for chat input; fix(cron): init MCP servers · done elsewhere
-
#5539 fix(cli): handle CJK wide chars in TUI input height · merged via another change →
#5665 - #17721 fix: use display-aware column width for CJK model names · done elsewhere
-
#20631 fix: use stringWidth() for Markdown table column widths to support CJ… · done elsewhere →
55c9f32,#26195,#20621 - #20641 fix(cli): use display-width for CJK/emoji panel border alignment · stale or abandoned
Closed issues · 8
- Closed#4212[Bug]: chat input box, caused by the 236 characters
- Closed#5221[Bug]: TUI input box renders incorrectly for Chinese / East Asian wide characters
- Closed#7443[Bug]: CLI TUI renders and deletes Chinese characters incorrectly
- Closed#17602TUI rendering glitch: Chinese text causes character scattering and ghosting
- Closed#17603[TUI Bug] Chinese text rendering: character scattering and ghosting
- Closed#17698Terminal ASCII tables misalign with mixed Chinese/ASCII content
- Closed#17700[Bug]: Terminal ASCII tables misalign with mixed Chinese/ASCII content
- Closed#38234[Bug]: After resizing terminal, (Chinese) characters are always duplicated automatically in input field
How this cluster was grouped
-
Link evidence (
partly structural links): some members are connected by structural evidence (shared changed files, or a closing/superseded reference); the rest are joined by an automated relevance judgement over their text. - Members outside that structural core may belong to a different underlying problem.
-
Pairwise link density
0.07: the share of item pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36.1.00means every pair is linked.
Pick up a Lab task
Check whether a fix covers the open issues here, with your own agent. Bring back evidence for maintainers.
Open the Lab