Skip to content

Commit 02db089

Browse files
committed
Bound OAuth response bodies in the client
## Motivation and Context `MCP::Client::HTTP` caps a single message from the server at 4 MiB, but the OAuth flow talks to authorization endpoints over its own Faraday connection, and that one had no cap. Discovery, dynamic client registration, and token responses were buffered in full and handed to `JSON.parse`, so a server that answered an OAuth request with an endless body grew the client's memory for as long as it kept sending. `Flow` and `IDJAGTokenExchange` each build their own connection, so both needed the bound. The new `BoundedBody` counts bytes in a Faraday `on_data` callback and refuses the response once it passes `MAX_RESPONSE_BYTES`, which matches the 4 MiB of `MCP::Client::HTTP::MAX_MESSAGE_BYTES` and `MCP::Client::Stdio::MAX_LINE_BYTES`. It hands back the status paired with the bounded body rather than the Faraday response, so no later caller can reach the unbounded `response.body`. Those bytes are counted after decompression. The default Net::HTTP adapter negotiates `Accept-Encoding` itself and reads the body through `Net::HTTPResponse#inflater`, so a small compressed body that expands past the cap is refused partway through the expansion. That holds only while the connection leaves `Accept-Encoding` alone, which is now recorded in a comment on both `default_http_client` definitions. An over-cap response is refused rather than truncated: a partial discovery or token document cannot be validated. Protected resource metadata is the one place where refusing is not fatal, since any PRM discovery failure already selects the legacy authorization path by design. ## How Has This Been Tested? New unit tests in `test/mcp/client/oauth/bounded_body_test.rb` cover the streaming path, the at-limit boundary, and the fallback for adapters that ignore `on_data`. One of them serves a gzip body over a real socket rather than through WebMock, which returns a stubbed body verbatim and never runs the inflater; it fails if the cap ever starts measuring compressed bytes. New tests in `test/mcp/client/oauth/flow_test.rb` and `test/mcp/client/oauth/id_jag_token_exchange_test.rb` drive each of the four request wrappers end to end with an over-cap body; all five fail without the cap in place. ## Breaking Changes An OAuth endpoint response larger than 4 MiB is now rejected instead of parsed.
1 parent ad73154 commit 02db089

8 files changed

Lines changed: 353 additions & 11 deletions

File tree

‎README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3002,6 +3002,11 @@ the HTTP client connects to a moment later. The same-origin rule is what protect
30023002
If you replace the OAuth HTTP client through `MCP::Client::OAuth::Flow.new(http_client_factory:)`, do not add redirect-following middleware. Every check above runs against
30033003
the URL as written, so a connection that follows a `3xx` on its own would reach hosts these rules just refused.
30043004

3005+
The SDK also bounds what those endpoints may return. A discovery, dynamic client registration, token, or token exchange response is refused once it passes 4 MiB,
3006+
measured as the body arrives rather than after it has been buffered, so a compressed body that expands past the limit is refused partway through the expansion.
3007+
Unlike the transport's `max_message_bytes:`, this limit is not configurable: these documents run to kilobytes in normal operation, and a connection supplied through
3008+
`http_client_factory:` is bounded as well, so there is no way to opt out of it.
3009+
30053010
#### Customizing the Faraday Connection
30063011

30073012
You can pass a block to `MCP::Client::HTTP.new` to customize the underlying Faraday connection.

‎lib/mcp/client/oauth.rb‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# frozen_string_literal: true
22

3+
require_relative "oauth/bounded_body"
34
require_relative "oauth/discovery"
45
require_relative "oauth/flow"
56
require_relative "oauth/in_memory_storage"
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# frozen_string_literal: true
2+
3+
module MCP
4+
class Client
5+
module OAuth
6+
# Bounds an OAuth response body while it arrives, rather than after it has been buffered.
7+
# Discovery documents, registration responses, and token responses are all small by definition,
8+
# so a body that keeps growing is never something worth holding in memory. Matches the 4 MiB cap of
9+
# `MCP::Client::HTTP::MAX_MESSAGE_BYTES` and `MCP::Client::Stdio::MAX_LINE_BYTES`.
10+
class BoundedBody
11+
MAX_RESPONSE_BYTES = 4 * 1024 * 1024
12+
13+
# Raised while the body is read. Each caller translates it into its own error type,
14+
# so this never reaches an embedder.
15+
class TooLargeError < StandardError; end
16+
17+
# What the OAuth code reads from a response. The Faraday response itself is not passed on,
18+
# so a later caller cannot reach the unbounded `response.body` by accident.
19+
Response = Struct.new(:status, :body)
20+
21+
def initialize(max_bytes: MAX_RESPONSE_BYTES)
22+
@max_bytes = max_bytes
23+
@buffer = +""
24+
end
25+
26+
# Faraday `on_data` streaming callback. The chunks arrive decompressed: the default `Net::HTTP` adapter negotiates
27+
# `Accept-Encoding` itself and reads the body through `Net::HTTPResponse#inflater`, so a small compressed body
28+
# that expands past the cap is refused partway through the expansion rather than after it. That holds only while
29+
# the connection leaves `Accept-Encoding` to the adapter; see `Flow#default_http_client`.
30+
def on_data
31+
proc do |chunk, _received_bytes, _env|
32+
@buffer << chunk
33+
34+
raise TooLargeError, too_large_message if @buffer.bytesize > @max_bytes
35+
end
36+
end
37+
38+
# The status paired with the bounded body. Adapters that ignore `on_data` leave the buffer empty and deliver
39+
# the whole body in `response.body`, so that path is measured here instead; the bytes are already allocated by then,
40+
# but refusing them still keeps an over-cap document out of `JSON.parse`.
41+
def response_for(response)
42+
Response.new(response.status, bounded_body(response))
43+
end
44+
45+
private
46+
47+
def bounded_body(response)
48+
return @buffer unless @buffer.empty?
49+
50+
body = response.body
51+
body = body.is_a?(String) ? body : body.to_s
52+
raise TooLargeError, too_large_message if body.bytesize > @max_bytes
53+
54+
body
55+
end
56+
57+
def too_large_message
58+
# Not "the authorization server": protected resource metadata comes from the MCP server's own origin,
59+
# so this message covers endpoints on both sides of the flow.
60+
"Response body from the OAuth endpoint exceeds #{@max_bytes} bytes"
61+
end
62+
end
63+
64+
private_constant :BoundedBody
65+
end
66+
end
67+
end

‎lib/mcp/client/oauth/flow.rb‎

Lines changed: 44 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1021,26 +1021,54 @@ def basic_auth_credentials(client_id, client_secret)
10211021
end
10221022

10231023
def http_get(url)
1024-
http_client.get(url)
1024+
bounded_request do |on_data|
1025+
http_client.get(url) do |req|
1026+
req.options.on_data = on_data
1027+
end
1028+
end
10251029
end
10261030

10271031
def http_post_json(url, body)
1028-
http_client.post(url) do |req|
1029-
req.headers["Content-Type"] = "application/json"
1030-
req.headers["Accept"] = "application/json"
1031-
req.body = JSON.generate(body)
1032+
bounded_request do |on_data|
1033+
http_client.post(url) do |req|
1034+
req.headers["Content-Type"] = "application/json"
1035+
req.headers["Accept"] = "application/json"
1036+
req.options.on_data = on_data
1037+
req.body = JSON.generate(body)
1038+
end
10321039
end
10331040
end
10341041

10351042
def http_post_form(url, form, headers: {})
1036-
http_client.post(url) do |req|
1037-
req.headers["Content-Type"] = "application/x-www-form-urlencoded"
1038-
req.headers["Accept"] = "application/json"
1039-
headers.each { |key, value| req.headers[key] = value }
1040-
req.body = URI.encode_www_form(form)
1043+
bounded_request do |on_data|
1044+
http_client.post(url) do |req|
1045+
req.headers["Content-Type"] = "application/x-www-form-urlencoded"
1046+
req.headers["Accept"] = "application/json"
1047+
1048+
headers.each do |key, value|
1049+
req.headers[key] = value
1050+
end
1051+
1052+
req.options.on_data = on_data
1053+
req.body = URI.encode_www_form(form)
1054+
end
10411055
end
10421056
end
10431057

1058+
# Issues a request with the response body bounded as it arrives, and returns the status paired with
1059+
# that body. An over-cap response is refused rather than truncated: a partial discovery or token document
1060+
# cannot be validated, and `fetch_metadata_json` must not fall through to the next candidate URL either,
1061+
# since the same server would serve the same body.
1062+
def bounded_request
1063+
bounded = BoundedBody.new
1064+
1065+
response = yield(bounded.on_data)
1066+
1067+
bounded.response_for(response)
1068+
rescue BoundedBody::TooLargeError => e
1069+
raise AuthorizationError, "#{e.message}."
1070+
end
1071+
10441072
def http_client
10451073
@http_client ||= @http_client_factory.call
10461074
end
@@ -1050,8 +1078,14 @@ def http_client
10501078
# that transparently followed a `3xx` would let a server reach a host the checks just refused.
10511079
# A caller passing `http_client_factory:` takes on that responsibility: add redirect following here
10521080
# and the guards above only cover the first hop.
1081+
#
1082+
# `Accept-Encoding` is deliberately left unset. `Net::HTTP::GenericRequest` negotiates it and decodes
1083+
# the response only while the caller has not claimed that header; assigning it turns `decode_content` off,
1084+
# which would silently move `BoundedBody`'s cap onto compressed bytes and let a small body expand past it
1085+
# after the check.
10531086
def default_http_client
10541087
require "faraday"
1088+
10551089
Faraday.new do |faraday|
10561090
faraday.headers["Accept"] = "application/json"
10571091
end

‎lib/mcp/client/oauth/id_jag_token_exchange.rb‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,13 @@ class << self
3434
def request(token_endpoint:, id_token:, client_id:, audience:, resource:, http_client: nil)
3535
http_client ||= default_http_client
3636

37+
bounded = BoundedBody.new
38+
3739
response = begin
38-
http_client.post(token_endpoint) do |req|
40+
raw_response = http_client.post(token_endpoint) do |req|
3941
req.headers["Content-Type"] = "application/x-www-form-urlencoded"
4042
req.headers["Accept"] = "application/json"
43+
req.options.on_data = bounded.on_data
4144
req.body = URI.encode_www_form(
4245
"grant_type" => GRANT_TYPE,
4346
"subject_token" => id_token,
@@ -48,6 +51,10 @@ def request(token_endpoint:, id_token:, client_id:, audience:, resource:, http_c
4851
"client_id" => client_id,
4952
)
5053
end
54+
55+
bounded.response_for(raw_response)
56+
rescue BoundedBody::TooLargeError => e
57+
raise ExchangeError, "#{e.message}."
5158
rescue Faraday::Error => e
5259
raise ExchangeError, "Token exchange request to #{token_endpoint} failed: #{e.class}: #{e.message}."
5360
end
@@ -88,8 +95,12 @@ def parse_id_jag(response)
8895
assertion
8996
end
9097

98+
# `Accept-Encoding` is deliberately left unset, for the same reason as `Flow#default_http_client`:
99+
# claiming that header turns Net::HTTP's `decode_content` off and would move `BoundedBody`'s cap
100+
# onto compressed bytes.
91101
def default_http_client
92102
require "faraday"
103+
93104
Faraday.new do |faraday|
94105
faraday.headers["Accept"] = "application/json"
95106
end
Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# frozen_string_literal: true
2+
3+
require "test_helper"
4+
require "faraday"
5+
require "socket"
6+
require "stringio"
7+
require "zlib"
8+
require "mcp/client/oauth"
9+
10+
module MCP
11+
class Client
12+
module OAuth
13+
class BoundedBodyTest < Minitest::Test
14+
def test_on_data_rejects_chunks_that_together_exceed_the_limit
15+
bounded = bounded_body(max_bytes: 64)
16+
17+
error = assert_raises(too_large_error) do
18+
4.times do
19+
bounded.on_data.call("a" * 32, 32, nil)
20+
end
21+
end
22+
23+
assert_includes(error.message, "exceeds 64 bytes")
24+
end
25+
26+
def test_on_data_accepts_a_body_that_stops_at_the_limit
27+
bounded = bounded_body(max_bytes: 64)
28+
29+
2.times do
30+
bounded.on_data.call("a" * 32, 32, nil)
31+
end
32+
33+
assert_equal("a" * 64, bounded.response_for(stub_response(body: "")).body)
34+
end
35+
36+
def test_response_for_rejects_an_over_limit_body_when_the_adapter_did_not_stream
37+
bounded = bounded_body(max_bytes: 64)
38+
39+
error = assert_raises(too_large_error) do
40+
bounded.response_for(stub_response(body: "a" * 65))
41+
end
42+
43+
assert_includes(error.message, "exceeds 64 bytes")
44+
end
45+
46+
def test_response_for_falls_back_to_the_response_body_when_the_adapter_did_not_stream
47+
bounded = bounded_body(max_bytes: 64)
48+
49+
assert_equal("a" * 64, bounded.response_for(stub_response(body: "a" * 64)).body)
50+
end
51+
52+
def test_response_for_coerces_a_non_string_body
53+
bounded = bounded_body(max_bytes: 64)
54+
55+
assert_equal("", bounded.response_for(stub_response(body: nil)).body)
56+
end
57+
58+
def test_response_for_carries_the_status_through
59+
bounded = bounded_body(max_bytes: 64)
60+
61+
assert_equal(404, bounded.response_for(stub_response(status: 404, body: "")).status)
62+
end
63+
64+
# Served over a real socket rather than WebMock: WebMock hands back the stubbed body verbatim
65+
# whatever its `Content-Encoding`, so the adapter's inflater never runs and a stubbed version
66+
# of this test would assert the opposite of what it looks like.
67+
def test_the_cap_counts_decompressed_bytes
68+
bounded = bounded_body(max_bytes: 1024 * 1024)
69+
compressed = gzip("a" * (5 * 1024 * 1024))
70+
71+
assert_operator(compressed.bytesize, :<, 1024 * 1024, "the compressed body must fit under the cap")
72+
73+
serving_gzip(compressed) do |url|
74+
assert_raises(too_large_error) do
75+
Faraday.new.get(url) do |req|
76+
req.options.on_data = bounded.on_data
77+
req.options.timeout = 5
78+
end
79+
end
80+
end
81+
end
82+
83+
private
84+
85+
def gzip(content)
86+
io = StringIO.new
87+
88+
writer = Zlib::GzipWriter.new(io)
89+
writer.write(content)
90+
writer.close
91+
92+
io.string
93+
end
94+
95+
def serving_gzip(body)
96+
# Other test files load `webmock/minitest`, which blocks real connections process-wide.
97+
# This is the one place that needs the adapter's own request path.
98+
WebMock.disable! if defined?(WebMock)
99+
100+
server = TCPServer.new("127.0.0.1", 0)
101+
thread = Thread.new do
102+
socket = server.accept
103+
104+
loop do
105+
line = socket.gets
106+
107+
break if line.nil? || line == "\r\n"
108+
end
109+
110+
socket.write(
111+
"HTTP/1.1 200 OK\r\n" \
112+
"Content-Type: application/json\r\n" \
113+
"Content-Encoding: gzip\r\n" \
114+
"Content-Length: #{body.bytesize}\r\n" \
115+
"Connection: close\r\n\r\n",
116+
)
117+
socket.write(body)
118+
119+
socket.close
120+
rescue IOError, SystemCallError
121+
nil
122+
end
123+
124+
# Scoped to start after both are assigned, so the cleanup below never runs against `nil`
125+
# and a failure to open the socket surfaces as itself rather than as a `NoMethodError`.
126+
begin
127+
yield("http://127.0.0.1:#{server.addr[1]}/")
128+
ensure
129+
thread.kill
130+
server.close
131+
end
132+
ensure
133+
WebMock.enable! if defined?(WebMock)
134+
end
135+
136+
def bounded_body(max_bytes:)
137+
OAuth.const_get(:BoundedBody).new(max_bytes: max_bytes)
138+
end
139+
140+
def too_large_error
141+
OAuth.const_get(:BoundedBody)::TooLargeError
142+
end
143+
144+
def stub_response(body:, status: 200)
145+
Struct.new(:status, :body).new(status, body)
146+
end
147+
end
148+
end
149+
end
150+
end

0 commit comments

Comments
 (0)