Skip to content

Count a blocked write's record when another call flushed it - #12026

Open
yosuke-wolfssl wants to merge 3 commits into
wolfSSL:masterfrom
yosuke-wolfssl:fix/senddata-flushed-count
Open

yosuke-wolfssl wants to merge 3 commits into
wolfSSL:masterfrom
yosuke-wolfssl:fix/senddata-flushed-count

Conversation

@yosuke-wolfssl

@yosuke-wolfssl yosuke-wolfssl commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Problem

When wolfSSL_write() returns WANT_WRITE, SendData() stores the bytes already handled in prevSent/plainSz so the retry counts them instead of sending them again. Three paths sent the data twice:

  • Flushed by another call. The retry applied the credit only when its own flush emptied outputBuffer. If another call flushed the queued record first (wolfSSL_update_keys(), wolfSSL_negotiate(), wolfSSL_shutdown()), the retry found the buffer empty, skipped the credit, and encrypted the whole buffer again.
  • Early return after taking the credit. The retry took the credit, then could return WANT_WRITE while sending a deferred KeyUpdate response or a pending alert, so the next retry started from byte 0.
  • Grouped early data (WOLFSSL_EARLY_DATA_GROUP). The retry of a blocked 0-RTT write skipped its flush to group with the ClientHello again, and appended a second copy of the record.

The unused credit also stayed behind. wolfSSL_clear() did not reset it, so it could carry into the first write on the next connection.

Fix (src/internal.c)

  • SendData() applies the credit whenever outputBuffer is empty, whether this call or another one flushed it, and only after the deferred KeyUpdate response, RetrySendAlert() and IsWriteClosed() can no longer return.
  • SendData() groups early data with the ClientHello only when no credit is pending, so a retry flushes its queued record.
  • wolfSSL_clear() resets prevSent and plainSz.
  • DoTls13EncryptedExtensions() and SendTls13ClientHello() (on a HelloRetryRequest) clear the credit where the client marks early data rejected. A blocked write sent as 0-RTT and then dropped by the server is resent in full on retry, not counted as delivered.

Tests

Adds 6 tests to pin the behavior.

Verification

macOS, ASan + UBSan, --enable-all --disable-shared plus:

Extra flags Result
--enable-asynccrypt --enable-asynccrypt-sw ssl_rw 25 and tls13 192 passed, 0 failed
--enable-earlydata=group tls13 192 passed, 0 failed
-DWOLFSSL_RW_THREADED ssl_rw 25 passed, 0 failed
  • No sanitizer reports.
  • With master's src/internal.c, the flushed_by_update_keys, keyupdate_blocked, alert_blocked and early_data_group_write_retry tests fail, as does the existing test_tls13_early_data in the grouped build.
  • Removing the SendTls13ClientHello() reset fails test_tls13_early_data_write_retry.

@yosuke-wolfssl yosuke-wolfssl self-assigned this Nov 5, 2026
Copilot AI balanced review requested due to automatic review settings November 5, 2026 23:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues for the nonblocking write-retry changes.

No open issues 🎉

What changed in this PR

Fixes nonblocking write retries so wolfSSL does not resend application data when another API call has already flushed the queued record.

Changes:

  • Credits previously buffered bytes whenever the output buffer is empty.
  • Clears pending write credit when reusing a connection or rejecting TLS 1.3 early data.
  • Adds regression tests for read-triggered flushing, connection reuse, and early-data acceptance and rejection, including HelloRetryRequest.
File Description
tests/​api/​test_tls13.h Registers the early-data write-retry test.
tests/​api/​test_tls13.c Tests blocked-write retries across early-data acceptance and rejection.
tests/​api/​test_ssl_rw.h Registers write-retry and connection-reuse tests.
tests/​api/​test_ssl_rw.c Tests read-triggered flushing and pending-write cleanup on reuse.
src/​tls13.c Clears write credit when early data is rejected.
src/​ssl.c Resets write credit in wolfSSL_clear().
src/​internal.c Applies write credit after another call flushes the queued record.

Review effort: Balanced

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

MemBrowse Memory Report · Explore what changed →

gcc-arm-cortex-m3

  • FLASH: .text +12 B (+0.0%, 131,689 B / 262,144 B, total: 50% used)

gcc-arm-cortex-m4-openssl-compat

  • FLASH: .text +64 B (+0.0%, 800,740 B / 1,048,576 B, total: 76% used)

zephyr-frdm_rw612

  • FLASH: log_const_area -8 B, rodata -8 B, text +1,592 B (+0.6%, 284,040 B / 67,108,864 B, total: 0% used)

No memory changes detected for:


Catch memory regressions in CI with MemBrowse. Free for open source.

@yosuke-wolfssl

Copy link
Copy Markdown
Contributor Author

@claude review

Comment thread src/internal.c Resolved
Comment thread src/internal.c Resolved
Comment thread src/ssl.c Resolved

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #12026

Scan targets checked: wolfssl-src, wolfssl-bugs
Not reviewed (no scan target covers them): tests/api/test_ssl_rw.h, tests/api/test_tls13.h
Coverage: 5 of 5 in-scope changed file(s) opened by the reviewer

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

Review tier: Max

@yosuke-wolfssl
yosuke-wolfssl force-pushed the fix/senddata-flushed-count branch from 4de274e to 1c9a983 Compare November 6, 2026 01:26
- SendData() counts the bytes of a record left by an earlier blocked
  write whenever outputBuffer is empty, whether this call or another
  call flushed it.
- SendData() groups early data with the ClientHello only when no
  queued write credit is pending, so a retried write flushes its
  record and takes the credit.
- wolfSSL_clear() resets prevSent and plainSz.
- Add test_wolfSSL_write_retry_flushed_by_update_keys,
  test_wolfSSL_clear_write_pending and
  test_tls13_early_data_group_write_retry.
- DoTls13EncryptedExtensions() and SendTls13ClientHello() clear
  prevSent and plainSz where the client sets earlyData to
  no_early_data on a rejection, so a retried wolfSSL_write() sends
  all of its data.
- Add test_tls13_early_data_write_retry.
…ends

- SendData() moves the queued write credit after the deferred KeyUpdate
  send, RetrySendAlert() and the IsWriteClosed() check, so a WANT_WRITE
  from any of them returns with prevSent and plainSz still set.
- Add test_wolfSSL_write_retry_keyupdate_blocked and
  test_wolfSSL_write_retry_alert_blocked.

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #12026

Scan targets checked: wolfssl-src, wolfssl-bugs
Not reviewed (no scan target covers them): tests/api/test_ssl_rw.h, tests/api/test_tls13.h
Coverage: 1 of 3 in-scope changed file(s) opened by the reviewer; not opened: tests/api/test_ssl_rw.c, tests/api/test_tls13.c

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

Review tier: Lite

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants