Skip to content

fix(relay): 边缘写队列加上限、慢读客户端被断开,空闲判定计入未收齐的帧,节点间写侧溢出立即断开 - #196

Merged
Owlbay merged 4 commits into
mainfrom
fix/relay-edge-write-queue
Nov 2, 2026
Merged

Owlbay merged 4 commits into
mainfrom
fix/relay-edge-write-queue

Conversation

@Owlbay

@Owlbay Owlbay commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

变更摘要 / Summary

中文:修 #191 的三处(审计 BA-53、BA-54、BA-55)。

  1. 边缘写队列无上限(P2):已认证客户端不读 socket、每个 ping 轮发满 128 条最大载荷的 PING,PONG 在 econn.out 里无限增长(fix(relay): 控制帧队列加硬上限,待发 PONG 只留最新一条,边缘 PING 限频 #173 只限了速率)。现在 netpoll 两种实现(Linux 事件循环、每连接协程)都对写队列里不带信用的帧(ack 为 0:PONG、关闭帧)记账,超过 netpoll.MaxCtlBytes(64 KiB)就以 netpoll.ErrQueueOverflow 断开连接并作废队列;带信用的下行数据仍由流窗口(每流 256 KiB)约束,不受这条限制。策略是「断开慢读的对端」,不对其做背压(它本来就不读)。边缘在 OnClose 里记 warn(reason=write_queue_overflow)并计 relay_edge_dropped_total{reason="write_queue_overflow"};这个指标族出现过才输出,Workers 版的黄金输出不变。
  2. 空闲判定只看整帧(P3):OnRead 在有没收齐的非 PING 帧时也刷新 lastSeen,弱网上传近 1 MiB 的帧超过 WSIdle 不再被当空闲断开;半个 PING 不算,光发 PING 仍不能赖着。
  3. 节点间链路写侧溢出不立即断开(P3):sendLocked 置 overflow 时 go c.closeTransport()(持着锁,另起协程);closeTransport 对空的 ws / done 容错,裸构造的测试连接不 panic。

未做:issue 里「PONG 与隧道侧 queuePong 一样合并」。netpoll 的帧队列不可原地替换(部分写出的头帧不能改),而 64 KiB 的上限已把队列界定住(最多约 500 条 PONG),所以没有增加替换槽的复杂度;需要的话单开 issue。

English: Fixes the three items of #191 (audit BA-53/54/55).

  1. Unbounded edge write queue (P2): an authenticated client that never reads its socket and sends 128 maximum-payload PINGs per ping round made the PONG replies pile up in econn.out without limit (fix(relay): 控制帧队列加硬上限,待发 PONG 只留最新一条,边缘 PING 限频 #173 only limited the rate). Both netpoll implementations (Linux event loop and per-connection goroutines) now account frames without credit in the write queue (ack 0: PONGs, close frames); beyond netpoll.MaxCtlBytes (64 KiB) the connection is closed with netpoll.ErrQueueOverflow and the queue is discarded. Credit-bearing downstream data stays bounded by the stream window (256 KiB per stream) and is not affected. The policy is to drop the slow consumer rather than apply backpressure (it is not reading anyway). The edge logs a warning (reason=write_queue_overflow) in OnClose and counts relay_edge_dropped_total{reason="write_queue_overflow"}; the metric family is only rendered once it has occurred, so the Workers golden output is unchanged.
  2. Idle check only on whole frames (P3): OnRead also refreshes lastSeen while a non-PING frame is incomplete, so a slow near-1 MiB upload exceeding WSIdle is no longer closed as idle; half a PING does not count, so PING-only clients still cannot linger.
  3. Inter-node write-side overflow does not close the link (P3): sendLocked now does go c.closeTransport() when it sets overflow (it holds the lock, hence the goroutine); closeTransport tolerates a nil ws / done so bare test connections do not panic.

Not done: the issue's "coalesce PONG like the tunnel's queuePong". The netpoll frame queue cannot replace frames in place (the head frame may be partially written), and the 64 KiB cap already bounds the queue (about 500 PONGs); open a follow-up issue if wanted.

关联 issue / Related issues

Closes #191(Refs #172、#173)

改动范围 / Scope

  • Go 实现(中继与控制面)/ Go implementation (relay and control plane)(go/)
  • docs(设计 §3.3、CHANGELOG)

契约与协议变化 / Contract and protocol changes

  • 契约节号 / Sections: 无 / None
  • 协议版本 / Protocol version: 无 / None
  • 生成物 / Generated files: 无 / None

数据库迁移 / Database migrations

无 / None

测试 / Testing

先写了会失败的测试,再修;把修法逐项临时去掉后,对应用例全部失败(无上限、不刷新活性、不关链路),恢复后通过。 / Tests were written to fail first; with each fix removed temporarily the matching cases fail (no cap, no liveness refresh, link left open), and pass once restored.

  • netpoll:TestCtlQueueOverflow(对端不读、控制帧排 512 MiB 前必须以 ErrQueueOverflow 断开)、TestCtlQueueBudgetReleased(带信用的帧不计入上限,写完额度归还),两种实现各跑。 / both implementations.

  • edge:TestSlowReaderPingFloodIsDropped(真实 Poller + 慢读客户端,断开、计指标、登记摘干净)、TestFastReaderPingFloodNotDropped、TestPingFlood 新增「没收齐的帧刷新活性、半个 PING 不算」。

  • saas:TestInternodeCtlFlood 新增「写侧溢出后链路立即关闭」(真实 WebSocket)。

  • obs:TestMetricsEdgeDropped。

  • Linux 事件循环:交叉编译(arm64)的 netpoll 与 edge 全包测试在容器里跑过,通过。 / Cross-compiled arm64 netpoll and edge full suites pass in a Linux container.

  • 单元测试 / Unit tests: pnpm go:check(gofmt、vet、staticcheck、go test -race、生成物新鲜度)通过;另带 Redis / PostgreSQL 跑了 netpoll、edge、obs、saas、personal、dataplane、tunnel 全包,通过。

  • 集成测试 / Integration tests: 未跑 pnpm test:integration

  • 跨仓端到端 / Cross-repo e2e: 未跑

  • 镜像 / Images: 未跑

  • pnpm check 与 / and pnpm repo:test

检查清单 / Checklist

  • 不涉及 app / 协议包 / 迁移 / 依赖;日志 msg 为简体中文短句、变量进字段,不含正文与令牌 / No apps, protocol package, migrations or dependencies touched; log messages are short Simplified Chinese sentences with variables in fields and carry no bodies or tokens
  • 用户可见的变化已记进 CHANGELOG.md 的 Unreleased / Recorded under Unreleased in CHANGELOG.md

需要维护者提供的外部条件 / External prerequisites

无 / None

已知限制 / Known limitations

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant