# NousResearch/hermes-agent · Windows subprocess Unicode encoding errors and fixes

## Open now

- Nothing open on GitHub.

These issues and pull requests concern UnicodeDecodeError crashes on Windows when subprocess output uses non-UTF-8 locale encodings (e.g., cp1251, GBK). The members propose various fixes including explicit UTF-8 encoding, errors='replace' fallbacks, and environment variable overrides across multiple components.

## TL;DR

- 47 member(s): 0 open issue(s), 18 closed issue(s), 0 open PR(s), 5 merged PR(s), 24 PR(s) closed without merging.
- A fix was merged to main in PR #17435. Verify it addresses your case.
- **Recommended:** read PR #17435 (and any competing PRs below) before filing. Confirm the merged change covers your symptom.

## Maintainer resolution verdict

- Internal-tracker verdict over this cluster's issue members: `partially_resolved` · deciding PR(s): `pr-17435,pr-38334,pr-70850,pr-70875`. This is distinct from the fix status above. Weigh it yourself.

## How this cluster was grouped

- Link evidence (`STRUCTURAL_PARTIAL`): some members are connected by structural evidence (shared changed files, or a closing/superseded reference); the rest are joined by an automated relevance judgement over their text.
- Members outside that structural core may belong to a different underlying problem.
- Pairwise link density `0.05`: the share of member pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36. `1.00` means every pair is linked.

## Closed / duplicate issues

- #17049: \[Bug\]: UnicodeDecodeError and AttributeError in Windows gateway process scanning
- #31420: \[Bug\] Windows: subprocess env missing PYTHONUTF8=1 causes UnicodeEncodeError on non-ASCII output in CP936/legacy locales
- #38172: Windows: hermes gateway status emits UnicodeDecodeError on localized schtasks output
- #39029: Windows: UnicodeDecodeError in subprocess._readerthread — cp1251 can't decode byte 0x98
- #45140: \[Bug\]: UnicodeDecodeError in copilot_acp_client.py _stdout_reader on Windows
- #47939: \[Bug\]: BUG_WINDOWS_UNICODE_SUBPROCESS
- #49491: UnicodeDecodeError-on-Windows-doctor
- #49492: Bug-Windows-UnicodeDecodeError-doctor
- #49496: hermes doctor crashes with UnicodeDecodeError on Chinese Windows GBK locale
- #49499: hermes doctor crashes with UnicodeDecodeError on Chinese Windows GBK locale
- #52649: Windows: Gateway crashes with UnicodeDecodeError (cp950) in _readerthread — causes zombie cycle every ~580s
- #53137: fix: TUI gateway crash on Windows - UnicodeDecodeError in subprocess _readerthread
- #53428: subprocess.run(text=True) without encoding param triggers GBK crash on Chinese Windows (21 locations)
- #55020: Windows Chinese locale (GBK) causes subprocess._readerthread UnicodeDecodeError crash
- #57238: Bug: GBK encoding crash on Chinese Windows\[Bug\] GBK encoding crash in terminal tool on Chinese Windows (cp936)
- #60760: \[Windows\] Detached gateway crashes with \`UnicodeDecodeError: 'gbk'\` when a spawned child process emits UTF-8 output
- #61595: \[Windows\] subprocess._readerthread crashes with UnicodeDecodeError on GBK output
- #63686: Windows CJK locale: UnicodeDecodeError in _SlashWorker crashes gateway (text=True defaults to GBK)

## Prior & competing PRs

- **#17074: Fix UnicodeDecodeError in Windows wmic process scanning  #17049**. closed without merge · fix tier `suggested`
  - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `cf83982da`, `#17049`
  - _suggested-tier fix detail withheld; re-request with `?tier=all` to include it._
- **#17435: fix(windows): decode wmic output with explicit encoding to survive non-UTF-8 locales (#17049)**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`; implication `superseded`
  - files: `hermes_cli/gateway.py`, `hermes_cli/main.py`, `scripts/release.py`, `tests/hermes_cli/test_update_stale_dashboard.py`
  - fix hunks:
  - `hermes_cli/gateway.py`

    ```diff
    @@ -279,9 +279,11 @@ def _matches_current_profile(command: str) -> bool:
                     ["wmic", "process", "get", "ProcessId,CommandLine", "/FORMAT:LIST"],
                     capture_output=True,
                     text=True,
    +                encoding="utf-8",
    +                errors="ignore",
                     timeout=10,
                 )
    -            if result.returncode != 0:
    +            if result.returncode != 0 or result.stdout is None:
                     return []
                 current_cmd = ""
                 for line in result.stdout.split("\n"):
    ```
  - `hermes_cli/main.py`

    ```diff
    @@ -5274,12 +5274,20 @@ def _warn_stale_dashboard_processes() -> None:
     
         try:
             if sys.platform == "win32":
    +            # wmic may emit text in the system code page (for example cp936
    +            # on zh-CN systems), not UTF-8. In text mode, subprocess output
    +            # decoding depends on Python's configuration (locale-dependent
    +            # by default, or UTF-8 in UTF-8 mode). The important protection
    +            # here is errors="ignore": it prevents a reader-thread
    +            # UnicodeDecodeError from leaving result.stdout=None and turning
    +            # the later .split() into an AttributeError (#17049).
                 result = subprocess.run(
                     ["wmic", "process", "get", "ProcessId,CommandLine",
                      "/FORMAT:LIST"],
                     capture_output=True, text=True, timeout=10,
    +                encoding="utf-8", errors="ignore",
                 )
    -            if result.returncode != 0:
    +            if result.returncode != 0 or result.stdout is None:
                     return
                 current_cmd = ""
                 for line in result.stdout.split("\n"):
    ```
  - `scripts/release.py`

    ```diff
    @@ -263,6 +263,7 @@
         "[redacted:email]": "Bartok9",
         "[redacted:email]": "CruxExperts",
         "[redacted:email]": "LeonSGP43",
    +    "[redacted:email]": "Kailigithub",
         "[redacted:email]": "vlwkaos",
         "[redacted:email]": "Tranquil-Flow",
         "[redacted:email]": "mcndjxlefnd",
    ```
  - `tests/hermes_cli/test_update_stale_dashboard.py`

    ```diff
    @@ -175,3 +175,55 @@ def test_grep_lines_ignored(self, capsys):
             output = capsys.readouterr().out
             assert "PID 99999" not in output
             assert "PID 12345" in output
    +
    +
    +class TestWindowsWmicEncoding:
    +    """Regression tests for #17049 — the Windows wmic branch must not crash
    +    `hermes update` on non-UTF-8 system locales (e.g. cp936 on zh-CN).
    +    """
    +
    +    def test_wmic_invoked_with_utf8_ignore_errors(self, monkeypatch):
    +        """The wmic subprocess.run call must pass encoding='utf-8' and
    +        errors='ignore' so the subprocess reader thread cannot raise
    +        UnicodeDecodeError on non-UTF-8 wmic output."""
    +        monkeypatch.setattr(sys, "platform", "win32")
    +        with patch("subprocess.run") as mock_run:
    +            # Provide a minimal valid wmic /FORMAT:LIST response.
    +            mock_run.return_value = MagicMock(
    +                returncode=0,
    +                stdout=(
    +                    "CommandLine=python -m hermes_cli.main dashboard\n"
    +                    "ProcessId=12345\n"
    +                ),
    +                stderr="",
    +            )
    +            _warn_stale_dashboard_processes()
    +
    +        assert mock_run.called, "subprocess.run was not invoked"
    +        kwargs = mock_run.call_args.kwargs
    +        assert kwargs.get("encoding") == "utf-8", (
    +            "encoding kwarg must be 'utf-8' so wmic output is decoded "
    +            "deterministically rather than via the implicit reader-thread "
    +            "default that crashes on non-UTF-8 locales (#17049)."
    +        )
    +        assert kwargs.get("errors") == "ignore", (
    +            "errors kwarg must be 'ignore' so undecodable bytes don't take "
    +            "down the reader thread (#17049)."
    +        )
    +
    +    def test_wmic_returns_none_stdout_does_not_crash(self, monkeypatch, capsys):
    +        """If subprocess.run returns successfully but stdout is None — which
    +        is what Python 3.11 leaves behind when the reader thread silently
    +        crashed on UnicodeDecodeError before this fix landed — the warning
    +        must short-circuit instead of raising AttributeError on
    +        ``None.split('\\n')`` and aborting `hermes update` (#17049)."""
    +        monkeypatch.setattr(sys, "platform", "win32")
    +        with patch("subprocess.run") as mock_run:
    +            mock_run.return_value = MagicMock(
    +                returncode=0, stdout=None, stderr=""
    +            )
    +            # Must not raise.
    +            _warn_stale_dashboard_processes()
    +
    +        output = capsys.readouterr().out
    +        assert "dashboard process" not in output
    ```
- **#38187: fix(cli/windows): decode schtasks output with locale encoding to silence UnicodeDecodeError (#38172)**. closed without merge · fix tier `suggested`
  - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `973decc05048f1d6fe990d5d6457e80389d5f1c5`, `#38172`
- **#38334: fix(gateway): decode schtasks output with locale encoding on Windows (#38172)**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`; implication `superseded`
  - files: `hermes_cli/gateway_windows.py`, `tests/hermes_cli/test_gateway_windows.py`
  - fix hunks:
  - `hermes_cli/gateway_windows.py`

    ```diff
    @@ -29,6 +29,7 @@
     from __future__ import annotations
     
     import ctypes
    +import locale
     import os
     import re
     import shlex
    ```

    ```diff
    @@ -52,6 +53,20 @@
     _TASK_DESCRIPTION = "Hermes Agent Gateway - Messaging Platform Integration"
     
     
    +def _schtasks_encoding() -> str:
    +    """Best-effort console encoding for decoding ``schtasks.exe`` output.
    +
    +    On localized Windows (e.g. Chinese), ``schtasks`` emits text in the OEM/ANSI
    +    code page rather than UTF-8. Decoding with the wrong codec raised
    +    ``UnicodeDecodeError`` inside ``subprocess``' reader threads. Prefer the
    +    locale's preferred encoding and fall back to UTF-8.
    +    """
    +    try:
    +        return locale.getpreferredencoding(False) or "utf-8"
    +    except Exception:
    +        return "utf-8"
    +
    +
     # ---------------------------------------------------------------------------
     # Platform guard
     # ---------------------------------------------------------------------------
    ```

    ```diff
    @@ -112,6 +127,12 @@ def _exec_schtasks(args: list[str]) -> tuple[int, str, str]:
                 [schtasks, *args],
                 capture_output=True,
                 text=True,
    +            # Localized Windows emits schtasks output in the console code page,
    +            # not UTF-8. Decode with the locale encoding and replace undecodable
    +            # bytes so a non-UTF-8 status line never surfaces a UnicodeDecodeError
    +            # traceback from subprocess' reader threads (issue #38172).
    +            encoding=_schtasks_encoding(),
    +            errors="replace",
                 timeout=_SCHTASKS_TIMEOUT_S,
                 # CREATE_NO_WINDOW avoids a flashing console window when the CLI
                 # is itself hosted in a TUI. See tools/browser_tool.py for the
    ```
  - `tests/hermes_cli/test_gateway_windows.py`

    ```diff
    @@ -29,6 +29,49 @@ def test_schtasks_fallback_does_not_hide_unknown_errors():
         assert gateway_windows._should_fall_back(1, "ERROR: The system cannot find the file specified.") is False
     
     
    +def test_schtasks_encoding_falls_back_to_utf8(monkeypatch):
    +    """A broken/empty locale must not leave us without a decoder (issue #38172)."""
    +
    +    monkeypatch.setattr(gateway_windows.locale, "getpreferredencoding", lambda *a, **k: "")
    +    assert gateway_windows._schtasks_encoding() == "utf-8"
    +
    +    def _boom(*args, **kwargs):
    +        raise RuntimeError("locale exploded")
    +
    +    monkeypatch.setattr(gateway_windows.locale, "getpreferredencoding", _boom)
    +    assert gateway_windows._schtasks_encoding() == "utf-8"
    +
    +
    +def test_exec_schtasks_decodes_with_replace_errors(monkeypatch):
    +    """schtasks output must be decoded with errors='replace' so localized
    +    (non-UTF-8) bytes never surface a UnicodeDecodeError traceback (#38172)."""
    +
    +    captured: dict[str, object] = {}
    +
    +    class _FakeCompleted:
    +        returncode = 0
    +        stdout = "ok"
    +        stderr = ""
    +
    +    def fake_run(cmd, **kwargs):
    +        captured["cmd"] = cmd
    +        captured.update(kwargs)
    +        return _FakeCompleted()
    +
    +    monkeypatch.setattr(gateway_windows, "_assert_windows", lambda: None)
    +    monkeypatch.setattr(gateway_windows.shutil, "which", lambda name: r"C:\\Windows\\System32\\schtasks.exe")
    +    monkeypatch.setattr(gateway_windows.subprocess, "run", fake_run)
    +
    +    code, out, err = gateway_windows._exec_schtasks(["/Query", "/TN", "Hermes_Gateway"])
    +
    +    assert (code, out, err) == (0, "ok", "")
    +    assert captured["errors"] == "replace", "schtasks output must decode with errors='replace'"
    +    assert isinstance(captured["encoding"], str) and captured["encoding"], (
    +        "an explicit non-empty encoding must be passed to subprocess.run"
    +    )
    +    assert captured["text"] is True
    +
    +
     def test_build_gateway_argv_uses_base_pythonw_for_uv_venv_launcher(monkeypatch, tmp_path):
         """Avoid uv's venv pythonw launcher because it respawns console python.exe."""
     
    ```
- **#49512: fix(doctor): pin utf-8/replace decoding for captured subprocesses**. closed without merge · fix tier `suggested`
  - maintainer direction: category `superseded_by_pr`; implication `superseded`; refs `#49510`, `#49499`, `#49510`
  - _suggested-tier fix detail withheld; re-request with `?tier=all` to include it._
- **#49564: fix(cli): add encoding to subprocess calls in doctor (#49499)**. closed without merge · fix tier `suggested`
  - maintainer direction: category `uncategorized`; implication `needs_llm`
  - _suggested-tier fix detail withheld; re-request with `?tier=all` to include it._
- **#70850: fix(windows): utf-8 subprocess decode across the desktop backend — salvage #61978 + widen to sibling sites**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`; implication `superseded`
  - files: `hermes_cli/banner.py`, `hermes_cli/web_server.py`, `tests/tui_gateway/test_subprocess_encoding.py`, `tui_gateway/compute_host.py`, `tui_gateway/host_supervisor.py`, `tui_gateway/server.py`
- **#70875: fix(windows): close out the text-mode subprocess decode bug class — salvage #55339/#60741/#60751 + full sweep + linter**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`; implication `superseded`
  - files: `agent/anthropic_adapter.py`, `agent/context_references.py`, `agent/copilot_acp_client.py`, `agent/lsp/install.py`, `agent/proxy_sources/iron_proxy.py`, `agent/secret_sources/base.py`, `agent/secret_sources/bitwarden.py`, `agent/shell_hooks.py`, `agent/skill_preprocessing.py`, `agent/trace_upload.py`, `agent/transports/codex_app_server.py`, `batch_runner.py`, `cli.py`, `contributors/emails/[redacted:email]`, `contributors/emails/[redacted:email]`, `gateway/platforms/base.py`, `gateway/platforms/webhook.py`, `gateway/platforms/webhook_filters.py`, `gateway/run.py`, `gateway/shutdown_forensics.py` (+72 more)
- **#71014: fix(windows): residual encoding gaps — MCP stdio, gateway update file-I/O, STT/TTS, desktop spawn, bootstrap ver-stub**. merged to main · fix tier `verified`
  - maintainer direction: category `merged`; implication `superseded`
  - files: `apps/desktop/electron/backend-env.test.ts`, `apps/desktop/electron/backend-env.ts`, `gateway/dead_targets.py`, `gateway/delivery.py`, `gateway/platforms/qqbot/adapter.py`, `gateway/run.py`, `gateway/slash_commands.py`, `gateway/status.py`, `hermes_bootstrap.py`, `plugins/platforms/discord/adapter.py`, `plugins/platforms/feishu/adapter.py`, `plugins/platforms/google_chat/adapter.py`, `plugins/platforms/google_chat/oauth.py`, `plugins/platforms/telegram/adapter.py`, `plugins/platforms/whatsapp/adapter.py`, `tests/gateway/test_gateway_utf8_encoding.py`, `tests/test_hermes_bootstrap.py`, `tests/tools/test_mcp_stdio_encoding_handler.py`, `tools/mcp_tool.py`, `tools/transcription_tools.py` (+1 more)
- **#17073: fix(gateway): handle wmic UnicodeDecodeError gracefully**. closed without merge
  - maintainer direction: category `duplicate_of`; implication `duplicate`; refs `#17074`, `#17049`
- **#17123: fix(update): protect dashboard wmic scan against UnicodeDecodeError on Windows non-UTF-8 locales (#17049)**. closed without merge
  - maintainer direction: category `merged_via`; implication `superseded`; refs `#17435`
- **#17252: fix(gateway): handle wmic encoding errors on Windows non-English locales**. closed without merge
  - maintainer direction: category `merged_via`; implication `superseded`; refs `#17435`
- **#17271: fix(gateway): add errors='replace' to subprocess.run for Windows wmic compatibility**. closed without merge
  - maintainer direction: category `superseded_by_pr`; implication `superseded`; refs `#17435`, `#17049`
- **#38186: fix(gateway): decode schtasks output with locale encoding on Windows (#38172)**. closed without merge
  - maintainer direction: category `merged_via`; implication `superseded`; refs `973decc05`, `df848bd2d`, `#38334`, `#38172`
- **#39539: fix(windows): tolerate localized taskkill decode errors**. closed without merge
  - maintainer direction: category `uncategorized`; implication `needs_llm`; refs `0f732cb3d6`, `#70875`, `#55339`, `#60741`, `#60751`
- **#45099: fix(tools): add errors='replace' to subprocess.**. closed without merge
  - maintainer direction: category `merged_via`; implication `superseded`; refs `#71014`, `#70875`
- **#45144: fix(windows): set UTF-8 encoding for Copilot ACP subprocess**. closed without merge
  - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `#45099`, `#45099`, `#45099`
- **#47951: fix(environments): pass encoding='utf-8' errors='replace' to all text-mode Popen calls**. closed without merge
  - maintainer direction: category `superseded_by_pr`; implication `superseded`; refs `#45099`, `#45099`, `#45099`
- **#47993: fix(subprocess): add UTF-8 encoding to _popen_bash for Windows (#47939)**. closed without merge
  - maintainer direction: category `uncategorized`; implication `needs_llm`
- **#48838: fix(cli): tolerate undecodable alias lookup output**. closed without merge
  - maintainer direction: category `superseded_by_pr`; implication `superseded`; refs `c89481db5ec2400e936de6ac402c89e7a39cfe19`, `#53428`
- **#48839: fix(cli): default Windows subprocess text decoding**. closed without merge
  - maintainer direction: category `duplicate_of`; implication `duplicate`; refs `#48838`
- **#49510: fix(doctor): decode subprocess output as UTF-8 on non-UTF-8 (GBK) locales**. closed without merge
  - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `38b67a345`
- **#53460: fix: add encoding=utf-8, errors=replace to all subprocess text=True calls (#53428)**. closed without merge
  - maintainer direction: category `superseded_by_pr`; implication `superseded`; refs `#54220`, `#54236`, `#53892`, `#54417`
- **#53549: fix: add encoding='utf-8' to subprocess calls with text=True**. closed without merge
  - maintainer direction: category `superseded_by_pr`; implication `superseded`; refs `#54220`, `#54236`, `#53892`, `#54417`
- **#55339: fix: add explicit UTF-8 encoding to all subprocess text=True calls (#53428)**. closed without merge
  - maintainer direction: category `merged_via`; implication `superseded`; refs `#70875`, `#60751`
- **#60832: fix(windows): force utf8 for detached gateway**. closed without merge
  - maintainer direction: category `implemented_elsewhere`; implication `superseded`; refs `#70850`, `#70875`, `#71014`
- **#61978: fix: harden tui_gateway subprocess reads against Windows locale UnicodeDecodeError (#53137)**. closed without merge
  - maintainer direction: category `merged_via`; implication `superseded`; refs `#70850`
- **#63716: fix(tui): decode slash worker streams as UTF-8**. closed without merge
  - maintainer direction: category `merged_via`; implication `superseded`; refs `ead23aadb9`, `#70850`, `#52700`, `#61978`
- **#68546: fix(doctor): decode subprocess output as utf-8**. closed without merge
  - maintainer direction: category `uncategorized`; implication `needs_llm`; refs `0f732cb3d6`, `#70875`, `#55339`, `#60741`, `#60751`

## Fix status

- Merged to main in PR #17435 (cited above). No release data is tracked. Verify downstream.
- Fix last verified (merged) `2026-07-24`.
- Subsystem: `acp`