Skip to content

fix(protocols): bound protocol accumulation buffer size - #3820

Merged
jmthomas merged 2 commits into
mainfrom
fix/protocol-max-buffer-size
Sep 9, 2026
Merged

fix(protocols): bound protocol accumulation buffer size#3820
jmthomas merged 2 commits into
mainfrom
fix/protocol-max-buffer-size

Conversation

@jmthomas

@jmthomas jmthomas commented Sep 3, 2026

Copy link
Copy Markdown
Member

What changed

Add a maximum buffer size to BurstProtocol, defaulting to 100 MB and overridable via OPENC3_PROTOCOL_MAX_BUFFER_SIZE. Exceeding it resets the protocol and raises, which makes Interface#read disconnect the peer. LENGTH and PREIDENTIFIED additionally reject a declared length over the limit immediately rather than buffering that many bytes first. The default sits far above any realistic packet so existing configurations are unaffected, and MAX_LENGTH remains the tighter per-interface bound.

NOTE: I'm not exposing OPENC3_PROTOCOL_MAX_BUFFER_SIZE in .env or compose.yaml because it generally will never need to be set. It is documented in the protocols.md markdown file.

Why it changed

Protocols which delineate packets buffer incoming data until a complete packet arrives, and the peer controls how much that is. LENGTH and PREIDENTIFIED buffer until a length field read off the wire is satisfied; TERMINATED buffers until it sees the termination characters. MAX_LENGTH is the only existing bound and is not set by default, so a peer could declare an enormous packet, or never terminate one, and drive the interface microservice to memory exhaustion.

Testing strategy

Unit tests

Protocols which delineate packets buffer incoming data until a complete
packet arrives, and the peer controls how much that is. LENGTH and
PREIDENTIFIED buffer until a length field read off the wire is satisfied;
TERMINATED buffers until it sees the termination characters. MAX_LENGTH is
the only existing bound and is not set by default, so a peer could declare
an enormous packet, or never terminate one, and drive the interface
microservice to memory exhaustion.

Add a maximum buffer size to BurstProtocol, defaulting to 100 MB and
overridable via OPENC3_PROTOCOL_MAX_BUFFER_SIZE. Exceeding it resets the
protocol and raises, which makes Interface#read disconnect the peer. LENGTH
and PREIDENTIFIED additionally reject a declared length over the limit
immediately rather than buffering that many bytes first. The default sits
far above any realistic packet so existing configurations are unaffected,
and MAX_LENGTH remains the tighter per-interface bound.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@jmthomas
jmthomas requested review from mcosgriff and ryanmelt and a lite review from Copilot and removed request for Copilot September 3, 2026 18:56
@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.28%. Comparing base (5dbf25b) to head (54a7bff).
⚠️ Report is 52 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3820      +/-   ##
==========================================
+ Coverage   79.22%   79.28%   +0.05%     
==========================================
  Files         894      896       +2     
  Lines       67139    67406     +267     
  Branches     2553     2608      +55     
==========================================
+ Hits        53194    53445     +251     
- Misses      13281    13291      +10     
- Partials      664      670       +6     
Flag Coverage Δ
frontend 66.50% <ø> (+0.48%) ⬆️
python 79.38% <ø> (+0.01%) ⬆️
ruby-api 81.99% <ø> (-0.43%) ⬇️
ruby-backend 84.59% <100.00%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread openc3/lib/openc3/interfaces/protocols/burst_protocol.rb Outdated
Comment thread openc3/lib/openc3/interfaces/protocols/length_protocol.rb Outdated
Comment thread openc3/lib/openc3/interfaces/protocols/preidentified_protocol.rb Outdated
Comment thread openc3/python/openc3/interfaces/protocols/burst_protocol.py Outdated
Comment thread openc3/python/openc3/interfaces/protocols/length_protocol.py Outdated
Comment thread openc3/python/openc3/interfaces/protocols/preidentified_protocol.py Outdated
MAX_LENGTH does not prevent the buffer size limit from being hit, so the
error messages now point only at OPENC3_PROTOCOL_MAX_BUFFER_SIZE. Also
avoid mutating test class state in the new length protocol test.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 4, 2026 23:44
@sonarqubecloud

sonarqubecloud Bot commented Sep 4, 2026

Copy link
Copy Markdown

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new env-configured max buffer size should be validated (e.g., disallow 0/negative) and the new raised message should avoid emitting a leading “: ” when no interface name is present.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR introduces a global maximum accumulation buffer size for packet-delineating protocols to prevent unbounded buffering (and potential memory exhaustion) when peers send unterminated streams or declare extremely large frame lengths.

Changes:

  • Add OPENC3_PROTOCOL_MAX_BUFFER_SIZE (default 100,000,000 bytes) to bound protocol buffering in BurstProtocol (Ruby + Python, including the Ruby C extension path).
  • Add “reject declared length > max buffer” behavior for LENGTH and PREIDENTIFIED (Ruby + Python) and “raise when terminator never arrives” behavior for TERMINATED.
  • Add unit tests (Ruby + Python) and document the new limit in protocols.md.
File summaries
File Description
openc3/spec/interfaces/protocols/terminated_protocol_spec.rb Adds spec coverage for TERMINATED max-buffer enforcement and defaulting behavior.
openc3/spec/interfaces/protocols/preidentified_protocol_spec.rb Adds spec for rejecting oversized declared packet lengths and ensuring reducer state resets.
openc3/spec/interfaces/protocols/length_protocol_spec.rb Adds specs for oversized declared lengths and env var override behavior.
openc3/python/test/interfaces/protocols/test_terminated_protocol.py Adds Python unit tests for TERMINATED buffer enforcement and default.
openc3/python/test/interfaces/protocols/test_length_protocol.py Adds Python unit tests for LENGTH oversized declared lengths and env var override.
openc3/python/openc3/interfaces/protocols/preidentified_protocol.py Rejects oversized declared lengths up-front and resets state on violation.
openc3/python/openc3/interfaces/protocols/length_protocol.py Rejects oversized calculated packet lengths up-front and resets state on violation.
openc3/python/openc3/interfaces/protocols/burst_protocol.py Introduces global max buffer size, enforces it on reads, and reports it in details.
openc3/lib/openc3/interfaces/protocols/preidentified_protocol.rb Rejects oversized declared lengths up-front and resets state on violation.
openc3/lib/openc3/interfaces/protocols/length_protocol.rb Rejects oversized calculated packet lengths up-front and resets state on violation.
openc3/lib/openc3/interfaces/protocols/burst_protocol.rb Introduces global max buffer size and enforces it (also used by the C extension).
openc3/ext/openc3/ext/burst_protocol/burst_protocol.c Ensures the Ruby C-extension read path also triggers the buffer-size check.
docs.openc3.com/docs/configuration/protocols.md Documents Maximum Buffer Size behavior and recommends per-protocol Max Length usage.
Review details

Suppressed comments (2)

openc3/lib/openc3/interfaces/protocols/burst_protocol.rb:217

  • The raised error message always includes a ": " prefix even when the protocol has no interface/name, which can produce messages like ": Protocol buffer ...". Build the prefix conditionally so the message is clean in non-interface/unit-test contexts.
      reset()
      raise "#{@interface ? @interface.name : ""}: Protocol buffer of #{length} bytes exceeds maximum of #{@max_buffer_size} bytes. " \
            "Increase OPENC3_PROTOCOL_MAX_BUFFER_SIZE."

openc3/python/openc3/interfaces/protocols/burst_protocol.py:214

  • The raised error message always includes a ": " prefix even when the protocol has no interface/name, which can produce messages like ": Protocol buffer ...". Build the prefix conditionally so the message is clean in non-interface/unit-test contexts.
        self.reset()
        name = self.interface.name if self.interface else ""
        raise RuntimeError(
            f"{name}: Protocol buffer of {length} bytes exceeds maximum of {self.max_buffer_size} bytes. "
            "Increase OPENC3_PROTOCOL_MAX_BUFFER_SIZE."
        )
  • Files reviewed: 13/13 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread openc3/lib/openc3/interfaces/protocols/burst_protocol.rb
Comment thread openc3/python/openc3/interfaces/protocols/burst_protocol.py
@jmthomas
jmthomas requested a review from ryanmelt September 6, 2026 21:22
@jmthomas
jmthomas merged commit 651e15d into main Sep 9, 2026
37 checks passed
@jmthomas
jmthomas deleted the fix/protocol-max-buffer-size branch September 9, 2026 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants