From 0b79fbfbd5c4b56cb43a8dc2c27ec9628233680c Mon Sep 17 00:00:00 2001 From: Torrey Payne <11740989+torreypayne@users.noreply.github.com> Date: Wed, 16 Sep 2026 19:16:25 +0000 Subject: [PATCH 1/5] test: add post-quantum cryptography conformance tests for showcase Adds a dedicated suite that proves the generated Ruby clients negotiate X25519MLKEM768 with the Showcase server over both transports. Assertions read the TLS metadata that Showcase reflects onto every response (x-showcase-tls-group and x-showcase-tls-client-supported-groups) rather than inspecting CRuby's internal OpenSSL structures, whose layout is not stable across Ruby releases or platforms. Covered scenarios: - gRPC and REST negotiate the hybrid post-quantum group and advertise it in their ClientHello. - Both transports degrade cleanly to classical X25519 when the server offers only classical groups. These also assert the client still advertised X25519MLKEM768: without that, the test would pass just as happily against a client that had lost post-quantum support altogether. The classical-only case needs a second Showcase process, since --tls-groups applies to a whole server. That auxiliary process mints a certificate authority of its own, so the test points SSL_CERT_FILE at the new CA for the duration of the block and builds explicit gRPC credentials from it. REST cannot be held to post-quantum key exchange unconditionally. It delegates to the host's OpenSSL, and ML-KEM only exists from OpenSSL 3.5 onward, while GitHub Actions ubuntu-latest ships 3.0.13. Skipping on older hosts would leave the REST transport unexercised in CI entirely, which is a permanently green check that verifies nothing. Instead the negotiated group must be present (proving the connection was genuinely TLS), must be either X25519MLKEM768 or classical X25519, and must be a group the client actually offered. Setting SHOWCASE_REQUIRE_REST_PQC=1 promotes this into a strict post-quantum assertion. Group membership is tested against the split supported-groups list rather than the raw header value, because "X25519" is a substring of "X25519MLKEM768" and a string containment check could therefore never fail. This mirrors the merged conformance test in gax-php. Ruby and PHP are the only Cloud SDK languages whose REST transport binds to the system OpenSSL instead of a vendored TLS stack, so they are the only two that cannot hard-assert post-quantum key exchange on a stock runner. --- shared/test/showcase/pqc_test.rb | 294 +++++++++++++++++++++++++++++++ 1 file changed, 294 insertions(+) create mode 100644 shared/test/showcase/pqc_test.rb diff --git a/shared/test/showcase/pqc_test.rb b/shared/test/showcase/pqc_test.rb new file mode 100644 index 000000000..d315b5782 --- /dev/null +++ b/shared/test/showcase/pqc_test.rb @@ -0,0 +1,294 @@ +# frozen_string_literal: true + +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +require "test_helper" +require "google/showcase/v1beta1/echo" +# Explicit: gapic-common loads GRPC::Core but not grpc/version.rb, so +# GRPC::VERSION is otherwise undefined. +require "grpc" + +## +# Verifies that generated Ruby clients negotiate post-quantum hybrid key +# exchange with the Showcase server. +# +# Ruby never performs the key exchange itself: the gRPC transport delegates to +# the BoringSSL build vendored inside the grpc gem, and the REST transport +# delegates to the system OpenSSL that Net::HTTP is linked against. These tests +# therefore assert on what the server observed, using the TLS metadata that +# Showcase reflects back on every response: +# +# x-showcase-tls-group the group that was negotiated +# x-showcase-tls-client-supported-groups everything the client offered +# +# Reflecting the server's view keeps the assertions free of any dependence on +# CRuby's internal OpenSSL object layout, which is not stable across releases +# or platforms. +# +class PqcTest < ShowcaseTest + # Header carrying the key exchange group selected during the handshake. + NEGOTIATED_GROUP_HEADER = "x-showcase-tls-group" + + # Header carrying every key exchange group the client offered in ClientHello. + CLIENT_GROUPS_HEADER = "x-showcase-tls-client-supported-groups" + + # The hybrid post-quantum group both Go and BoringSSL prefer by default. + PQC_GROUP = "X25519MLKEM768" + + # The classical group expected once post-quantum groups are withdrawn. + CLASSICAL_GROUP = "X25519" + + # IANA codepoints for X25519 and secp256r1. Passed to --tls-groups to strip + # every post-quantum group from the server's preferences. + CLASSICAL_ONLY_CODEPOINTS = "0x001d,0x0017" + + # grpc 1.83 is the first release whose vendored BoringSSL offers PQC_GROUP. + MINIMUM_GRPC_VERSION = Gem::Version.new "1.83.0" + + # ML-KEM, and therefore X25519MLKEM768, first shipped in OpenSSL 3.5. Ruby's + # openssl gem is only a binding, so REST post-quantum support is a property + # of the host rather than of any gem we can pin. + MINIMUM_REST_OPENSSL_VERSION = Gem::Version.new "3.5.0" + + # Opt-in strict mode for the REST transport. CI sets this on jobs running an + # image that is guaranteed to provide OpenSSL >= 3.5, which turns the + # tolerant key exchange assertion below into a hard post-quantum + # requirement. Everywhere else the classical fallback remains acceptable. + REQUIRE_REST_PQC = ENV["SHOWCASE_REQUIRE_REST_PQC"] == "1" + + def test_grpc_negotiates_post_quantum_key_exchange + assert_grpc_pqc_capable + headers = grpc_tls_headers new_echo_client + + assert_equal PQC_GROUP, headers[NEGOTIATED_GROUP_HEADER], + "gRPC handshake did not negotiate post-quantum key exchange" + assert_includes offered_groups(headers), PQC_GROUP, + "gRPC client did not advertise #{PQC_GROUP} in its ClientHello" + end + + def test_rest_negotiates_post_quantum_key_exchange + headers = rest_tls_headers new_echo_rest_client + + assert_rest_key_exchange headers + end + + def test_grpc_falls_back_to_classical_key_exchange + assert_grpc_pqc_capable + with_showcase_tls_groups CLASSICAL_ONLY_CODEPOINTS do |port, ca_path| + headers = grpc_tls_headers grpc_echo_client_for(port, ca_path) + + assert_equal CLASSICAL_GROUP, headers[NEGOTIATED_GROUP_HEADER], + "gRPC client failed to fall back to classical key exchange" + # Negotiating X25519 alone proves nothing: a client that had lost + # post-quantum support entirely would produce the same result. What is + # being tested is that a PQC-capable client still interoperates with a + # classical-only server. + assert_includes offered_groups(headers), PQC_GROUP, + "gRPC client no longer advertises #{PQC_GROUP}, so no fallback was exercised" + end + end + + def test_rest_falls_back_to_classical_key_exchange + with_showcase_tls_groups CLASSICAL_ONLY_CODEPOINTS do |port| + headers = rest_tls_headers rest_echo_client_for(port) + + assert_equal CLASSICAL_GROUP, headers[NEGOTIATED_GROUP_HEADER], + "REST client failed to fall back to classical key exchange" + # Same reasoning as the gRPC case, but only checkable where the host + # OpenSSL implements ML-KEM at all; below 3.5 the client has no + # post-quantum group to withhold, so there is no fallback to observe. + if REQUIRE_REST_PQC + assert_includes offered_groups(headers), PQC_GROUP, + "REST client no longer advertises #{PQC_GROUP}, so no fallback was exercised" + end + end + end + + private + + ## + # Fails if the grpc gem predates the vendored BoringSSL that added PQC_GROUP. + # Nothing here pins grpc - it arrives through gapic-common - so without this + # a downgrade looks like a protocol bug rather than a dependency one. + # + # @return [void] + def assert_grpc_pqc_capable + assert_operator Gem::Version.new(GRPC::VERSION), :>=, MINIMUM_GRPC_VERSION, + "grpc #{GRPC::VERSION} predates #{MINIMUM_GRPC_VERSION}, where the " \ + "vendored BoringSSL gained #{PQC_GROUP}" + end + + ## + # Issues an Echo RPC and returns the TLS metadata the server attached to the + # response headers, downcased for case-insensitive lookup. + # + # @param client [Google::Showcase::V1beta1::Echo::Client] + # @return [Hash{String=>String}] + def grpc_tls_headers client + metadata = nil + response = client.echo(content: "pqc probe") do |_result, operation| + metadata = operation.metadata + end + + assert_equal "pqc probe", response.content + normalize_headers metadata + end + + ## + # Issues an Echo REST call and returns the TLS metadata the server attached + # to the HTTP response headers, downcased for case-insensitive lookup. + # + # @param client [Google::Showcase::V1beta1::Echo::Rest::Client] + # @return [Hash{String=>String}] + def rest_tls_headers client + headers = nil + response = client.echo(content: "pqc probe") do |_result, operation| + headers = operation.underlying_op.headers + end + + assert_equal "pqc probe", response.content + normalize_headers headers + end + + ## + # Flattens gRPC metadata and Faraday headers into a single case-insensitive + # string map, asserting that the TLS metadata is present at all. Absent + # headers mean the request never traversed TLS, which would silently turn + # every assertion below into a no-op. + # + # @param raw [Hash, nil] + # @return [Hash{String=>String}] + def normalize_headers raw + refute_nil raw, "no response metadata was captured" + headers = raw.to_h { |key, value| [key.to_s.downcase, Array(value).join(",")] } + + [NEGOTIATED_GROUP_HEADER, CLIENT_GROUPS_HEADER].each do |header| + refute_nil headers[header], + "showcase did not report #{header}; the connection was not TLS" + end + headers + end + + ## + # The key exchange groups the client advertised, as a list. + # + # Membership must be tested against the split list, never the raw header + # string: CLASSICAL_GROUP ("X25519") is a substring of PQC_GROUP + # ("X25519MLKEM768"), so String#include? would report a match for a group the + # client never offered. + # + # @param headers [Hash{String=>String}] + # @return [Array] + def offered_groups headers + headers[CLIENT_GROUPS_HEADER].split(",").map(&:strip) + end + + ## + # Boots an auxiliary Showcase server whose key exchange preferences are + # restricted to the given IANA codepoints, yields its port and the CA + # certificate it generated, and guarantees the process is reaped. + # + # Every server started with --tls mints its own certificate authority, so the + # auxiliary server cannot share a trust root with the main harness. Both + # transports have to be pointed at the CA yielded here: REST through + # SSL_CERT_FILE, which is scoped to the block below because Net::HTTP rebuilds + # its trust store per connection, and gRPC through explicit credentials built + # from the yielded path. + # + # The suite is not parallelized, so swapping a process-wide environment + # variable for the duration of the block is safe; adding parallelize_me! to + # this file would break that assumption. + # + # @param codepoints [String] Comma separated IANA key exchange group IDs. + # @yieldparam port [Integer] + # @yieldparam ca_path [String] + # @return [void] + def with_showcase_tls_groups codepoints + dir = ShowcaseTest.instance_variable_get :@showcase_dir + skip "requires a showcase server managed by this test run" if dir.nil? + + port = SHOWCASE_PORT + 1 + ca_path = File.join dir, "ca-#{port}.pem" + pid = spawn_showcase "#{dir}/gapic-showcase", + port: port, + ca_path: ca_path, + log_file: File.join(dir, "gapic-showcase-#{port}.log"), + extra_args: ["--tls-groups", codepoints] + + original_tls_env = TLS_ENV_KEYS.to_h { |key| [key, ENV[key]] } + begin + TLS_ENV_KEYS.each { |key| ENV[key] = ca_path } + yield port, ca_path + ensure + original_tls_env.each { |key, value| ENV[key] = value } + stop_showcase pid + end + end + + def grpc_echo_client_for port, ca_path + Google::Showcase::V1beta1::Echo::Client.new do |config| + config.endpoint = "localhost:#{port}" + config.credentials = ShowcaseTest.channel_credentials ca_path + end + end + + def rest_echo_client_for port + Google::Showcase::V1beta1::Echo::Rest::Client.new do |config| + config.endpoint = "https://localhost:#{port}" + config.credentials = :this_channel_is_insecure + end + end + + ## + # Asserts on the key exchange the server negotiated for a REST call. + # + # The gRPC transport carries its own BoringSSL inside the grpc gem, so it can + # be held to post-quantum key exchange unconditionally. REST cannot: it + # delegates to the host's OpenSSL, and ML-KEM only exists from OpenSSL 3.5 + # onward. Skipping on older hosts would leave the REST path entirely + # unverified in any environment below 3.5 - including the stock GitHub + # Actions runner - so instead the negotiated group is required to be one of + # the outcomes we consider correct, and is cross-checked against the groups + # the client actually offered. A non-TLS connection, or any group outside + # that set, still fails. This mirrors the conformance test in gax-php. + # + # Setting SHOWCASE_REQUIRE_REST_PQC=1 promotes this to a strict post-quantum + # assertion, and is used by the CI job that runs on an image pinned to + # OpenSSL >= 3.5. + # + # @param headers [Hash{String=>String}] + # @return [void] + def assert_rest_key_exchange headers + negotiated = headers[NEGOTIATED_GROUP_HEADER] + offered = offered_groups headers + + if REQUIRE_REST_PQC + assert_equal PQC_GROUP, negotiated, + "SHOWCASE_REQUIRE_REST_PQC is set but REST negotiated #{negotiated}. " \ + "Host provides #{OpenSSL::OPENSSL_LIBRARY_VERSION} and post-quantum " \ + "key exchange requires OpenSSL >= #{MINIMUM_REST_OPENSSL_VERSION}" + assert_includes offered, PQC_GROUP, + "REST client did not advertise #{PQC_GROUP} in its ClientHello" + else + assert_includes [PQC_GROUP, CLASSICAL_GROUP], negotiated, + "REST negotiated an unexpected key exchange group #{negotiated}. " \ + "Expected #{PQC_GROUP} on OpenSSL >= #{MINIMUM_REST_OPENSSL_VERSION} " \ + "or #{CLASSICAL_GROUP} on older hosts " \ + "(host provides #{OpenSSL::OPENSSL_LIBRARY_VERSION})" + assert_includes offered, negotiated, + "server negotiated #{negotiated} but the client never offered it" + end + end +end From f3a52cec57477ce3b52b7b39ced63108db49110b Mon Sep 17 00:00:00 2001 From: Torrey Payne <11740989+torreypayne@users.noreply.github.com> Date: Tue, 22 Sep 2026 17:24:39 +0000 Subject: [PATCH 2/5] test: gate the REST fallback assertion on host capability, not strict mode test_rest_falls_back_to_classical_key_exchange guarded its "client still advertises X25519MLKEM768" assertion with REQUIRE_REST_PQC, the strict-mode policy flag, while the comment directly above justifies that guard in terms of host OpenSSL capability. Two independent predicates: they coincide only because CI sets the environment variable on exactly the one PQC-capable image. Where they diverge - a workstation on OpenSSL >= 3.5, or CI itself once ubuntu-latest is upgraded and the pqc-rest job is retired - the assertion silently disappears and the test degrades to "negotiated X25519", which the gRPC twin's comment explains proves nothing: a client that had lost post-quantum support entirely produces an identical result. Verified by withholding ML-KEM from the client via OPENSSL_CONF on an OpenSSL 3.5.5 image with strict mode off. Before: 59 runs, 0 failures. After: 1 failure, raised by this assertion. --- shared/test/showcase/pqc_test.rb | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/shared/test/showcase/pqc_test.rb b/shared/test/showcase/pqc_test.rb index d315b5782..f3e3db113 100644 --- a/shared/test/showcase/pqc_test.rb +++ b/shared/test/showcase/pqc_test.rb @@ -68,6 +68,16 @@ class PqcTest < ShowcaseTest # requirement. Everywhere else the classical fallback remains acceptable. REQUIRE_REST_PQC = ENV["SHOWCASE_REQUIRE_REST_PQC"] == "1" + # Whether the host can perform post-quantum key exchange over REST at all. + # + # Distinct from REQUIRE_REST_PQC, which is a policy choice about how strict to + # be. This is a fact about the machine. The two coincide in CI only because + # the strict environment variable is set on exactly the job that runs a + # PQC-capable image; anywhere else - a workstation on OpenSSL >= 3.5, or CI + # after the runner is upgraded - they diverge. + REST_OPENSSL_SUPPORTS_PQC = + Gem::Version.new(OpenSSL::OPENSSL_LIBRARY_VERSION.split[1]) >= MINIMUM_REST_OPENSSL_VERSION + def test_grpc_negotiates_post_quantum_key_exchange assert_grpc_pqc_capable headers = grpc_tls_headers new_echo_client @@ -109,7 +119,9 @@ def test_rest_falls_back_to_classical_key_exchange # Same reasoning as the gRPC case, but only checkable where the host # OpenSSL implements ML-KEM at all; below 3.5 the client has no # post-quantum group to withhold, so there is no fallback to observe. - if REQUIRE_REST_PQC + # Gated on capability rather than on REQUIRE_REST_PQC so that a + # PQC-capable host runs the real assertion even when strict mode is off. + if REST_OPENSSL_SUPPORTS_PQC assert_includes offered_groups(headers), PQC_GROUP, "REST client no longer advertises #{PQC_GROUP}, so no fallback was exercised" end From f5bd19850c741432638999482afe7494c49d921c Mon Sep 17 00:00:00 2001 From: Torrey Payne <11740989+torreypayne@users.noreply.github.com> Date: Thu, 24 Sep 2026 18:32:24 +0000 Subject: [PATCH 3/5] test: assert advertised groups before the negotiated group and match any ML-KEM group --- shared/test/showcase/pqc_test.rb | 104 ++++++++++++++++++++----------- 1 file changed, 69 insertions(+), 35 deletions(-) diff --git a/shared/test/showcase/pqc_test.rb b/shared/test/showcase/pqc_test.rb index f3e3db113..ae17d4a42 100644 --- a/shared/test/showcase/pqc_test.rb +++ b/shared/test/showcase/pqc_test.rb @@ -44,8 +44,13 @@ class PqcTest < ShowcaseTest # Header carrying every key exchange group the client offered in ClientHello. CLIENT_GROUPS_HEADER = "x-showcase-tls-client-supported-groups" - # The hybrid post-quantum group both Go and BoringSSL prefer by default. - PQC_GROUP = "X25519MLKEM768" + # Substring shared by every hybrid post-quantum group name (X25519MLKEM768, + # SecP256r1MLKEM768, ...). Matching on it rather than on one exact name keeps + # the suite correct as TLS stacks change their preferred hybrid, and mirrors + # the .NET conformance tests (googleapis/gax-dotnet#909). No classical group + # name contains it, so unlike CLASSICAL_GROUP it cannot produce a false + # substring match. + PQC_GROUP_MARKER = "MLKEM" # The classical group expected once post-quantum groups are withdrawn. CLASSICAL_GROUP = "X25519" @@ -54,7 +59,7 @@ class PqcTest < ShowcaseTest # every post-quantum group from the server's preferences. CLASSICAL_ONLY_CODEPOINTS = "0x001d,0x0017" - # grpc 1.83 is the first release whose vendored BoringSSL offers PQC_GROUP. + # grpc 1.83 is the first release whose vendored BoringSSL offers X25519MLKEM768. MINIMUM_GRPC_VERSION = Gem::Version.new "1.83.0" # ML-KEM, and therefore X25519MLKEM768, first shipped in OpenSSL 3.5. Ruby's @@ -82,10 +87,8 @@ def test_grpc_negotiates_post_quantum_key_exchange assert_grpc_pqc_capable headers = grpc_tls_headers new_echo_client - assert_equal PQC_GROUP, headers[NEGOTIATED_GROUP_HEADER], - "gRPC handshake did not negotiate post-quantum key exchange" - assert_includes offered_groups(headers), PQC_GROUP, - "gRPC client did not advertise #{PQC_GROUP} in its ClientHello" + assert_advertises_pqc headers, "gRPC" + assert_negotiated_pqc headers, "gRPC" end def test_rest_negotiates_post_quantum_key_exchange @@ -99,14 +102,13 @@ def test_grpc_falls_back_to_classical_key_exchange with_showcase_tls_groups CLASSICAL_ONLY_CODEPOINTS do |port, ca_path| headers = grpc_tls_headers grpc_echo_client_for(port, ca_path) - assert_equal CLASSICAL_GROUP, headers[NEGOTIATED_GROUP_HEADER], - "gRPC client failed to fall back to classical key exchange" # Negotiating X25519 alone proves nothing: a client that had lost # post-quantum support entirely would produce the same result. What is # being tested is that a PQC-capable client still interoperates with a # classical-only server. - assert_includes offered_groups(headers), PQC_GROUP, - "gRPC client no longer advertises #{PQC_GROUP}, so no fallback was exercised" + assert_advertises_pqc headers, "gRPC" + assert_equal CLASSICAL_GROUP, headers[NEGOTIATED_GROUP_HEADER], + "gRPC client failed to fall back to classical key exchange" end end @@ -114,32 +116,66 @@ def test_rest_falls_back_to_classical_key_exchange with_showcase_tls_groups CLASSICAL_ONLY_CODEPOINTS do |port| headers = rest_tls_headers rest_echo_client_for(port) - assert_equal CLASSICAL_GROUP, headers[NEGOTIATED_GROUP_HEADER], - "REST client failed to fall back to classical key exchange" # Same reasoning as the gRPC case, but only checkable where the host # OpenSSL implements ML-KEM at all; below 3.5 the client has no # post-quantum group to withhold, so there is no fallback to observe. # Gated on capability rather than on REQUIRE_REST_PQC so that a # PQC-capable host runs the real assertion even when strict mode is off. - if REST_OPENSSL_SUPPORTS_PQC - assert_includes offered_groups(headers), PQC_GROUP, - "REST client no longer advertises #{PQC_GROUP}, so no fallback was exercised" - end + assert_advertises_pqc headers, "REST" if REST_OPENSSL_SUPPORTS_PQC + assert_equal CLASSICAL_GROUP, headers[NEGOTIATED_GROUP_HEADER], + "REST client failed to fall back to classical key exchange" end end private ## - # Fails if the grpc gem predates the vendored BoringSSL that added PQC_GROUP. - # Nothing here pins grpc - it arrives through gapic-common - so without this - # a downgrade looks like a protocol bug rather than a dependency one. + # Fails if the grpc gem predates the vendored BoringSSL that added + # post-quantum key exchange. Nothing here pins grpc - it arrives through + # gapic-common - so without this a downgrade looks like a protocol bug rather + # than a dependency one. # # @return [void] def assert_grpc_pqc_capable assert_operator Gem::Version.new(GRPC::VERSION), :>=, MINIMUM_GRPC_VERSION, "grpc #{GRPC::VERSION} predates #{MINIMUM_GRPC_VERSION}, where the " \ - "vendored BoringSSL gained #{PQC_GROUP}" + "vendored BoringSSL gained X25519MLKEM768" + end + + ## + # Whether a TLS group name denotes a hybrid post-quantum key exchange. + # + # @param group [String, nil] + # @return [Boolean] + def pqc_group? group + group.to_s.upcase.include? PQC_GROUP_MARKER + end + + ## + # Asserts the client offered at least one post-quantum group in its + # ClientHello. Checked before the negotiated group so that a failure + # distinguishes a client that never offered PQC from a server that declined it. + # + # @param headers [Hash{String=>String}] + # @param transport [String] Label used in failure messages. + # @return [void] + def assert_advertises_pqc headers, transport + offered = offered_groups headers + assert offered.any? { |group| pqc_group? group }, + "#{transport} client did not advertise a post-quantum group in its " \ + "ClientHello (offered: #{offered.join ', '})" + end + + ## + # Asserts the server negotiated a post-quantum group. + # + # @param headers [Hash{String=>String}] + # @param transport [String] Label used in failure messages. + # @return [void] + def assert_negotiated_pqc headers, transport + negotiated = headers[NEGOTIATED_GROUP_HEADER] + assert pqc_group?(negotiated), + "#{transport} negotiated #{negotiated}, not a post-quantum group" end ## @@ -197,9 +233,8 @@ def normalize_headers raw # The key exchange groups the client advertised, as a list. # # Membership must be tested against the split list, never the raw header - # string: CLASSICAL_GROUP ("X25519") is a substring of PQC_GROUP - # ("X25519MLKEM768"), so String#include? would report a match for a group the - # client never offered. + # string: CLASSICAL_GROUP ("X25519") is a substring of X25519MLKEM768, so + # String#include? would report a match for a group the client never offered. # # @param headers [Hash{String=>String}] # @return [Array] @@ -287,20 +322,19 @@ def assert_rest_key_exchange headers offered = offered_groups headers if REQUIRE_REST_PQC - assert_equal PQC_GROUP, negotiated, - "SHOWCASE_REQUIRE_REST_PQC is set but REST negotiated #{negotiated}. " \ - "Host provides #{OpenSSL::OPENSSL_LIBRARY_VERSION} and post-quantum " \ - "key exchange requires OpenSSL >= #{MINIMUM_REST_OPENSSL_VERSION}" - assert_includes offered, PQC_GROUP, - "REST client did not advertise #{PQC_GROUP} in its ClientHello" + assert_advertises_pqc headers, "REST" + assert pqc_group?(negotiated), + "SHOWCASE_REQUIRE_REST_PQC is set but REST negotiated #{negotiated}. " \ + "Host provides #{OpenSSL::OPENSSL_LIBRARY_VERSION} and post-quantum " \ + "key exchange requires OpenSSL >= #{MINIMUM_REST_OPENSSL_VERSION}" else - assert_includes [PQC_GROUP, CLASSICAL_GROUP], negotiated, - "REST negotiated an unexpected key exchange group #{negotiated}. " \ - "Expected #{PQC_GROUP} on OpenSSL >= #{MINIMUM_REST_OPENSSL_VERSION} " \ - "or #{CLASSICAL_GROUP} on older hosts " \ - "(host provides #{OpenSSL::OPENSSL_LIBRARY_VERSION})" assert_includes offered, negotiated, "server negotiated #{negotiated} but the client never offered it" + assert pqc_group?(negotiated) || negotiated == CLASSICAL_GROUP, + "REST negotiated an unexpected key exchange group #{negotiated}. " \ + "Expected a post-quantum group on OpenSSL >= #{MINIMUM_REST_OPENSSL_VERSION} " \ + "or #{CLASSICAL_GROUP} on older hosts " \ + "(host provides #{OpenSSL::OPENSSL_LIBRARY_VERSION})" end end end From 393ebed6ffb72df1c277b9c234dbdf3b05dda222 Mon Sep 17 00:00:00 2001 From: Torrey Payne <11740989+torreypayne@users.noreply.github.com> Date: Thu, 24 Sep 2026 18:33:01 +0000 Subject: [PATCH 4/5] test: move with_showcase_tls_groups into the showcase test helper and guard against parallelize_me! --- shared/test/showcase/pqc_test.rb | 56 ---------------------- shared/test/showcase/test_helper.rb | 73 +++++++++++++++++++++++++++++ 2 files changed, 73 insertions(+), 56 deletions(-) diff --git a/shared/test/showcase/pqc_test.rb b/shared/test/showcase/pqc_test.rb index ae17d4a42..8944deeee 100644 --- a/shared/test/showcase/pqc_test.rb +++ b/shared/test/showcase/pqc_test.rb @@ -242,62 +242,6 @@ def offered_groups headers headers[CLIENT_GROUPS_HEADER].split(",").map(&:strip) end - ## - # Boots an auxiliary Showcase server whose key exchange preferences are - # restricted to the given IANA codepoints, yields its port and the CA - # certificate it generated, and guarantees the process is reaped. - # - # Every server started with --tls mints its own certificate authority, so the - # auxiliary server cannot share a trust root with the main harness. Both - # transports have to be pointed at the CA yielded here: REST through - # SSL_CERT_FILE, which is scoped to the block below because Net::HTTP rebuilds - # its trust store per connection, and gRPC through explicit credentials built - # from the yielded path. - # - # The suite is not parallelized, so swapping a process-wide environment - # variable for the duration of the block is safe; adding parallelize_me! to - # this file would break that assumption. - # - # @param codepoints [String] Comma separated IANA key exchange group IDs. - # @yieldparam port [Integer] - # @yieldparam ca_path [String] - # @return [void] - def with_showcase_tls_groups codepoints - dir = ShowcaseTest.instance_variable_get :@showcase_dir - skip "requires a showcase server managed by this test run" if dir.nil? - - port = SHOWCASE_PORT + 1 - ca_path = File.join dir, "ca-#{port}.pem" - pid = spawn_showcase "#{dir}/gapic-showcase", - port: port, - ca_path: ca_path, - log_file: File.join(dir, "gapic-showcase-#{port}.log"), - extra_args: ["--tls-groups", codepoints] - - original_tls_env = TLS_ENV_KEYS.to_h { |key| [key, ENV[key]] } - begin - TLS_ENV_KEYS.each { |key| ENV[key] = ca_path } - yield port, ca_path - ensure - original_tls_env.each { |key, value| ENV[key] = value } - stop_showcase pid - end - end - - def grpc_echo_client_for port, ca_path - Google::Showcase::V1beta1::Echo::Client.new do |config| - config.endpoint = "localhost:#{port}" - config.credentials = ShowcaseTest.channel_credentials ca_path - end - end - - def rest_echo_client_for port - Google::Showcase::V1beta1::Echo::Rest::Client.new do |config| - config.endpoint = "https://localhost:#{port}" - config.credentials = :this_channel_is_insecure - end - end - ## # Asserts on the key exchange the server negotiated for a REST call. # diff --git a/shared/test/showcase/test_helper.rb b/shared/test/showcase/test_helper.rb index 6a4fd549f..43ece16ef 100644 --- a/shared/test/showcase/test_helper.rb +++ b/shared/test/showcase/test_helper.rb @@ -175,6 +175,79 @@ def new_compliance_rest_client end end + # Echo client for an auxiliary server started by with_showcase_tls_groups. + # + # @param port [Integer] + # @param ca_path [String] CA certificate minted by that server. + # @return [Google::Showcase::V1beta1::Echo::Client] + def grpc_echo_client_for port, ca_path + Google::Showcase::V1beta1::Echo::Client.new do |config| + config.endpoint = "localhost:#{port}" + config.credentials = ShowcaseTest.channel_credentials ca_path + end + end + + # Echo REST client for an auxiliary server started by + # with_showcase_tls_groups. Trust comes from SSL_CERT_FILE, which that helper + # points at the server's CA for the duration of its block. + # + # @param port [Integer] + # @return [Google::Showcase::V1beta1::Echo::Rest::Client] + def rest_echo_client_for port + Google::Showcase::V1beta1::Echo::Rest::Client.new do |config| + config.endpoint = "https://localhost:#{port}" + config.credentials = :this_channel_is_insecure + end + end + + ## + # Boots an auxiliary Showcase server whose key exchange preferences are + # restricted to the given IANA codepoints, yields its port and the CA + # certificate it generated, and guarantees the process is reaped. + # + # Every server started with --tls mints its own certificate authority, so the + # auxiliary server cannot share a trust root with the main harness. Both + # transports have to be pointed at the CA yielded here: REST through + # SSL_CERT_FILE, which is scoped to the block below because Net::HTTP rebuilds + # its trust store per connection, and gRPC through explicit credentials built + # from the yielded path. + # + # @param codepoints [String] Comma separated IANA key exchange group IDs. + # @yieldparam port [Integer] + # @yieldparam ca_path [String] + # @return [void] + def with_showcase_tls_groups codepoints + # This helper is only safe when tests run one at a time. It repoints the + # process-wide SSL_CERT_FILE, which every concurrent REST connection would + # pick up, and restores it on exit, which would clobber an overlapping + # caller's value. It also binds a fixed port and CA file path, so two + # overlapping calls would collide. Minitest runs serially by default; + # parallelize_me! switches the class's test_order to :parallel. + refute_equal :parallel, self.class.test_order, + "#{self.class} is parallelized, but with_showcase_tls_groups " \ + "mutates process-wide state and must run serially" + + dir = ShowcaseTest.instance_variable_get :@showcase_dir + skip "requires a showcase server managed by this test run" if dir.nil? + + port = SHOWCASE_PORT + 1 + ca_path = File.join dir, "ca-#{port}.pem" + pid = spawn_showcase "#{dir}/gapic-showcase", + port: port, + ca_path: ca_path, + log_file: File.join(dir, "gapic-showcase-#{port}.log"), + extra_args: ["--tls-groups", codepoints] + + original_tls_env = TLS_ENV_KEYS.to_h { |key| [key, ENV[key]] } + begin + TLS_ENV_KEYS.each { |key| ENV[key] = ca_path } + yield port, ca_path + ensure + original_tls_env.each { |key, value| ENV[key] = value } + stop_showcase pid + end + end + # Env vars pointing REST at Showcase's CA, saved so after_run can restore them # rather than leak a test-only root. REST only: Net::HTTP re-reads # SSL_CERT_FILE per connection, while the gRPC C core resolves From 74f1b955a5eb1ded8770702d737b9a4a2802921d Mon Sep 17 00:00:00 2001 From: Torrey Payne <11740989+torreypayne@users.noreply.github.com> Date: Thu, 24 Sep 2026 19:54:08 +0000 Subject: [PATCH 5/5] test: derive the expected REST key exchange from host OpenSSL capability --- shared/test/showcase/pqc_test.rb | 60 ++++++++++++-------------------- 1 file changed, 23 insertions(+), 37 deletions(-) diff --git a/shared/test/showcase/pqc_test.rb b/shared/test/showcase/pqc_test.rb index 8944deeee..0e1788ece 100644 --- a/shared/test/showcase/pqc_test.rb +++ b/shared/test/showcase/pqc_test.rb @@ -67,19 +67,16 @@ class PqcTest < ShowcaseTest # of the host rather than of any gem we can pin. MINIMUM_REST_OPENSSL_VERSION = Gem::Version.new "3.5.0" - # Opt-in strict mode for the REST transport. CI sets this on jobs running an - # image that is guaranteed to provide OpenSSL >= 3.5, which turns the - # tolerant key exchange assertion below into a hard post-quantum - # requirement. Everywhere else the classical fallback remains acceptable. + # Set by CI on jobs whose image is guaranteed to provide OpenSSL >= 3.5. It + # does not change which outcome the REST test expects - host capability + # decides that - it only asserts that the capability is actually there, so a + # base-image regression fails loudly instead of silently turning the REST test + # into a classical-only check. REQUIRE_REST_PQC = ENV["SHOWCASE_REQUIRE_REST_PQC"] == "1" - # Whether the host can perform post-quantum key exchange over REST at all. - # - # Distinct from REQUIRE_REST_PQC, which is a policy choice about how strict to - # be. This is a fact about the machine. The two coincide in CI only because - # the strict environment variable is set on exactly the job that runs a - # PQC-capable image; anywhere else - a workstation on OpenSSL >= 3.5, or CI - # after the runner is upgraded - they diverge. + # Whether the host can perform post-quantum key exchange over REST at all, and + # therefore which outcome the REST test expects: a post-quantum group when + # true, CLASSICAL_GROUP when false. REST_OPENSSL_SUPPORTS_PQC = Gem::Version.new(OpenSSL::OPENSSL_LIBRARY_VERSION.split[1]) >= MINIMUM_REST_OPENSSL_VERSION @@ -119,8 +116,6 @@ def test_rest_falls_back_to_classical_key_exchange # Same reasoning as the gRPC case, but only checkable where the host # OpenSSL implements ML-KEM at all; below 3.5 the client has no # post-quantum group to withhold, so there is no fallback to observe. - # Gated on capability rather than on REQUIRE_REST_PQC so that a - # PQC-capable host runs the real assertion even when strict mode is off. assert_advertises_pqc headers, "REST" if REST_OPENSSL_SUPPORTS_PQC assert_equal CLASSICAL_GROUP, headers[NEGOTIATED_GROUP_HEADER], "REST client failed to fall back to classical key exchange" @@ -248,37 +243,28 @@ def offered_groups headers # The gRPC transport carries its own BoringSSL inside the grpc gem, so it can # be held to post-quantum key exchange unconditionally. REST cannot: it # delegates to the host's OpenSSL, and ML-KEM only exists from OpenSSL 3.5 - # onward. Skipping on older hosts would leave the REST path entirely - # unverified in any environment below 3.5 - including the stock GitHub - # Actions runner - so instead the negotiated group is required to be one of - # the outcomes we consider correct, and is cross-checked against the groups - # the client actually offered. A non-TLS connection, or any group outside - # that set, still fails. This mirrors the conformance test in gax-php. - # - # Setting SHOWCASE_REQUIRE_REST_PQC=1 promotes this to a strict post-quantum - # assertion, and is used by the CI job that runs on an image pinned to - # OpenSSL >= 3.5. + # onward. Rather than skipping on older hosts, which would leave REST + # unverified on the stock GitHub Actions runner, the expected outcome follows + # from the host: a post-quantum group on OpenSSL >= 3.5, CLASSICAL_GROUP below + # it. Either way exactly one outcome passes. # # @param headers [Hash{String=>String}] # @return [void] def assert_rest_key_exchange headers - negotiated = headers[NEGOTIATED_GROUP_HEADER] - offered = offered_groups headers - if REQUIRE_REST_PQC + assert REST_OPENSSL_SUPPORTS_PQC, + "SHOWCASE_REQUIRE_REST_PQC is set but the host provides " \ + "#{OpenSSL::OPENSSL_LIBRARY_VERSION}; post-quantum key exchange " \ + "requires OpenSSL >= #{MINIMUM_REST_OPENSSL_VERSION}" + end + + if REST_OPENSSL_SUPPORTS_PQC assert_advertises_pqc headers, "REST" - assert pqc_group?(negotiated), - "SHOWCASE_REQUIRE_REST_PQC is set but REST negotiated #{negotiated}. " \ - "Host provides #{OpenSSL::OPENSSL_LIBRARY_VERSION} and post-quantum " \ - "key exchange requires OpenSSL >= #{MINIMUM_REST_OPENSSL_VERSION}" + assert_negotiated_pqc headers, "REST" else - assert_includes offered, negotiated, - "server negotiated #{negotiated} but the client never offered it" - assert pqc_group?(negotiated) || negotiated == CLASSICAL_GROUP, - "REST negotiated an unexpected key exchange group #{negotiated}. " \ - "Expected a post-quantum group on OpenSSL >= #{MINIMUM_REST_OPENSSL_VERSION} " \ - "or #{CLASSICAL_GROUP} on older hosts " \ - "(host provides #{OpenSSL::OPENSSL_LIBRARY_VERSION})" + assert_equal CLASSICAL_GROUP, headers[NEGOTIATED_GROUP_HEADER], + "REST on #{OpenSSL::OPENSSL_LIBRARY_VERSION} should negotiate " \ + "#{CLASSICAL_GROUP}" end end end