Binary file detection logic fixes and corrections
Cluster 80308. Data as of
Nothing open on GitHub. Fix merged in #81961. Check it covers your case.
The short answer
- Known?
- Yes. 5 related items grouped here, 1 issue already closed.
- Fix event
- A fix was merged to main in PR #81961. Check it covers your case. No release data is tracked.
- Tracker
- The maintainers' tracker marks this
unresolved. That is separate from the fix event. - Still open
- Nothing open on GitHub.
These issues and pull requests address incorrect binary file detection in file-reading and search tools, specifically around flawed heuristics involving U+FFFD replacement characters and NUL bytes that caused valid UTF-8 files (including CJK content) to be misidentified as binary.
Open issues0
Open PRs0
Fixes on record
-
#81961 fix(file-ops): stop flagging boundary-cut UTF-8 as binary across read_file/patch/search_files
files:
contributors/emails/[redacted:email],tests/tools/test_file_operations.py,tests/tools/test_read_file_utf8_binary_regression.py,tools/file_operations.py
3 PRs closed without merging
From the close comments: duplicate 2 · superseded by another PR 1
-
#76934 fix(tools): keep invalid UTF-8 boundaries binary · superseded by another PR →
#81961,#76924,#80440 -
#80349 fix: verify NUL bytes before flagging U+FFFD samples as binary (#80308) · duplicate →
#76925 -
#80749 fix(tools): use NUL-byte check for binary detection instead of U+FFFD (#80308) · duplicate →
#76925
Closed issues · 1
- Closed#80308[Bug]: read_file incorrectly flags valid UTF-8 CJK files as binary, and search_files is completely broken (returns 0 for all queries)
How this cluster was grouped
-
Link evidence (
structural links throughout): every member is connected by structural evidence: shared changed files, or a closing/superseded reference. -
Pairwise link density
0.40: the share of item pairs that carry a link, counting a structural link in full and a relevance-only link at 0.36.1.00means every pair is linked.
Pick up a Lab task
Check whether a fix covers the open issues here, with your own agent. Bring back evidence for maintainers.
Open the LabAt a glance
- Merged PRs
- #81961
- Closed issues
- 1
- Attempts
- 3 PRs closed without merging
- Fix link checked
- 8 Aug 2026