Repository navigation
refactor(parallel-read): ship source under packages/, split the 1159-line entry - #478
Open
yanfei0725 wants to merge 6 commits into
Open
yanfei0725 wants to merge 6 commits into
yanfei0725 wants to merge 6 commits into
Conversation
…ache 同一文件在一次会话里被读两遍是常态(读、改、再读确认),此前每次都重付 open + fstat + read + 解码 + SHA-256 + 切行 + 行窗口结算。open 与 fstat 必须每次 都付(它们是「这个文件现在是什么」的唯一凭据),能省的是它们之后那段纯计算。 - 键 = dev:ino:size:mtime + 全部影响正文的开关(full/hash/normalize/stripBom/ strictUtf8、行窗口、字符额度),命中即等价于同一批字节、同一个窗口。 - inode 拿不到(部分网络盘报 0)不缓存;容量 64 项 / 8 Mi 字符,LRU 淘汰。 - 实测 worker.js(17.5 KB)第二遍 1.07 ms -> 0.67 ms;mtime 变即失效。 - 新增 readCacheStats() / clearReadCache()。 - 四道闸门(并发池 / 2 MiB / 3 s / 二进制)位置不变,预算仍按输入顺序结算。 README 与 AUDIT 同步到 4.1.8。
…kages/ instead of a tarball The diff for this PR was a 42 KB tgz plus a duplicated README, so nothing in the change could be read without unpacking it first. - packages/ops-parallel-read/src/ the zero-dependency implementation plus its .d.ts - packages/ops-parallel-read/test/ concurrency gates, truncation, normalize on/off - packages/ops-parallel-read/docs/ audit and fix log - drop the tarball and the duplicated root README and AUDIT files - .gitignore: release archives and vendored .exe never enter the tree
…le modules The single src/ops_parallel_read.js carried the concurrency pool, the path fences, the read cache, and the whole dsh-ops compat layer. No behaviour changed; each piece now sits in the file that names it: ops_parallel_read.js 617 pool, decoding gates, budget, cache compat.js 393 ops_read_files contract (createCompat factory) paths.js 180 name/path validation, reparse-point fence read-cache.js 76 cross-call LRU errors.js 21 OpsError compat.js takes readOne / normalizeOptions / cutChars as injected arguments, so the split introduces no circular require. Every file is under 1000 lines. npm test still passes 22 assertions.
OldLigant
self-requested a review
October 30, 2026 15:46
read_files spawned a 3.8 MB worker for every call: ~21 ms of floor before a byte was read, and the read itself is open/read/decode. The arm now runs in the host process. - src/file/read-types.ts result shape, defaults, pool, truncation, LRU - src/file/read-fence.ts lexical fence, real-path re-check, reparse refusal - src/file/parallel-read.ts readOne (seven gates) and parallelRead - src/file/read-files-compat.ts native item receipt, whole-file default - src/file/worker.ts dispatch in-process after the policy/credential/anchoring fence - native/.../file_many.rs removed, not kept as a second implementation Parity with the shipped worker: same revision, lineCount, bytes, omitted, nextOffset, CRLF-stripped text, retained BOM and error codes. Two intentional differences: the default limit reads the whole file (200 lines silently cut a 270-line file), and receipt paths are POSIX like symbol_map and ptc. 6 x 2KB 22.18 -> 0.31 ms, 12 x 1KB 24.05 -> 0.35 ms, 24 x 512B 27.87 -> 0.85 ms.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
这次只改一件事:把 1159 行的单文件拆成五个可 review 的模块
src/ops_parallel_read.js一个文件里塞了并发池、路径围栏、读取缓存、以及整套 dsh-ops 兼容层。行为一行没动,只是让每个东西待在叫它名字的文件里。目录
没有
.tgz、没有.tar.gz、没有.exe、没有dist/、没有node_modules/。.gitignore只拦这三类产物,src/下任何.js都不受影响 —— 这个包是手写 JS,不存在编译这一步。拆分没有引入循环 require
compat.js需要主文件的三个函数,所以反过来用工厂注入:compat.js本身零require,主文件是唯一的依赖方向。单文件 1000 行
最大 617 行。全部提交在 1000 行以内。
验证
22 条断言全过:默认清洗、
normalize:false原样、stripBom与normalize正交、纯 LF 零改动、UTF-16LE 不误转、compat 外层与逐行 path 都是 POSIX 而正文逐字节。拆分前后各跑一遍,输出逐字节相同。