[ID-1671] Handle incomplete upstream responses without returning partial success

[Ri-go]

Large historical responses could hit an upstream EOF or timeout after Stitch had buffered only part of the body. Stitch then flushed that partial body as a clean HTTP 200, and a syntactically valid prefix could enter a response or hash cache.

Discard failed buffered attempts and use the existing coverage, retry-budget and idempotency rules to retry or return a complete 502. Abort already committed HTTP streams on a failed upstream read. Preserve Comet WebSocket connections, EVM batch request IDs and structured errors, and prevent failed filter creation from leaving bindings. Count only the final buffered request outcome in traffic metrics; client cancellations release half-open circuit slots without penalizing the backend.

Validation: full go test -race -count=1 -timeout=10m ./..., go vet ./..., go build -mod=readonly ./..., and golangci-lint v2.12.2 passed. Real HTTP listener regressions cover premature EOF, broken chunks, body deadlines, HTTP/1 and HTTP/2 aborts, slow complete responses, cache poisoning, batch errors, filter creation and cancellation. Independent review found no remaining issues.

The published candidate passed RPC/API/gRPC/EVM checks, two real 30-second timeout cases with complete 502 responses and no cache poisoning, and successful cache recovery. Stable v0.1.10 has been deployed to both OVH archival replicas. Both replicas and the public endpoint now return complete 502 JSON for the slow historical request, with matching runtime deadline logs; none returns truncated HTTP 200. The archive-to-OVH path still has significant TCP loss, so large historical delivery remains unresolved. This change makes those failures explicit and does not claim to fix network throughput.

Linear: https://linear.app/injectivelabs/issue/ID-1671

[linear]

ID-1671

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Essentials
• Run ID: 2990d9c6-713e-4cdc-8d28-3f18daffc1cd

📥 Commits

Reviewing files that changed from the base of the PR and between e1523bb and bf03916.

📒 Files selected for processing (8)

• internal/forwarder/http.go
• internal/forwarder/http_test.go
• internal/forwarder/truncation_test.go
• internal/server/capture.go
• internal/server/cmt_rpc/truncation_test.go
• internal/server/cmt_rpc/websocket.go
• internal/server/eth_rpc/server.go
• internal/server/eth_rpc/truncation_test.go

Included review availability: This review used your included allowance. 2 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.

---

📝 Walkthrough

Walkthrough

The forwarder now resets buffered responses after upstream body-read failures, retries eligible candidates, and retains abort behavior for streamed responses. Capture and WebSocket buffers support resets. RPC tests cover truncation, cancellation, retries, metrics, caching, and recovery. EVM batch handling converts captured non-2xx responses into JSON-RPC errors.

Changes

Upstream response truncation handling

Layer / File(s)	Summary
Forwarder reset and failure handling internal/forwarder/http.go, internal/server/capture.go, internal/forwarder/http_test.go, internal/forwarder/truncation_test.go	Capture supports response resets. The forwarder resets buffered responses after upstream read failures, then handles cancellation or continues candidate iteration. Streamed responses retain abort behavior. Tests cover truncation, cancellation, retries, and metrics.
CometBFT buffered response handling internal/server/cmt_rpc/websocket.go, internal/server/cmt_rpc/truncation_test.go	The WebSocket response buffer supports resets. Tests cover truncated HTTP responses, retries, caching, slow complete responses, and WebSocket recovery.
EVM non-2xx batch responses internal/server/eth_rpc/server.go, internal/server/eth_rpc/truncation_test.go	EVM batch handling preserves existing JSON-RPC errors and converts other captured non-2xx responses into JSON-RPC errors. Tests cover batch IDs, filter creation, and response and hash caching after truncation.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to bf039

This change makes incomplete upstream responses retry or return a 502 instead of a partial success, and aborts truncated streams. No merge-blocking risk was identified from the supplied evidence.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 5.88% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 8 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: handling incomplete upstream responses without returning partial success.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}