Skip to content

fix(otel): address exporter diagnostic and TLS vulnerabilities - #527

Merged
LouisLotter merged 5 commits into
stackstate-7.78.2from
cve/otel-scan35310940870
Sep 18, 2026
Merged

LouisLotter merged 5 commits into
stackstate-7.78.2from
cve/otel-scan35310940870

Conversation

@ai-collaboration-app

@ai-collaboration-app ai-collaboration-app Bot commented Sep 18, 2026

Copy link
Copy Markdown

Independent review accepted signed head 85b010341b2343d8d2f29d76c00db2f1f6b5f135 for OTel CVE-2026-81870/81871, with no required corrections. Review independently replayed all four security cases and failing old-version controls; final lint/unit, binary and DEB workflows passed.

Scope limits: seven Python/GO findings and PR511's hold remain; green inform-mode CI is not whole-image clearance. Independent local race linking failed; race coverage comes from accepted CI. Infrastructure E2E was not run; release-candidate telemetry QA and actual delivery remain outstanding. Extra wrong-CA/missing-client-certificate controls are optional.

Candidate and CNI publication evidence — preserved

OpenTelemetry trace exporters can disclose collector endpoints in verbose internal diagnostics (CVE-2026-81870), and the log gRPC exporter ignores environment CA/client certificates (CVE-2026-81871). Update the registered workspace's trace SDK/exporters to 1.45.0 and log exporters to 0.21.0. The HTTP/stdout log exporters and otelzap 0.20.0 are aligned because the new log API removes types their previous releases used. Metric exporters remain unchanged.

Adds regressions using real HTTP/gRPC trace exporters and a loopback collector requiring private-CA mTLS, covering generic and log-specific environment settings. Includes a release note, test instructions, and module-boundary guidance in AGENTS.md. Ignored fixtures and unregistered legacy manifests are unchanged; all 35 changed manifests are registered. Consolidates relevant updates from #523/#525/#526 and #524; #522 remains separate.

Head: 85b010341b2343d8d2f29d76c00db2f1f6b5f135 (GitHub-verified signed commits). Current CI tests merge 71e9a7d4e60ce5bf25594b7096e88556031af979, whose parents are this head and base 4d7fbc56f3a485e733c44c9f63b36df53750d2b4 (Louis's merge of #521). Thus current artifact validation includes CNI 1.9.0 as well as these OTel fixes.

Validation:

  • Targeted Invoke lint/tests and final-head race tests passed for security, log mapping, trace integration/stats/traceutil, and metrics-client packages.

  • Negative controls at SDK 1.44.0 / log exporter 0.20.0 fail both trace endpoint assertions and both environment-TLS cases as expected.

  • Local and current CI module tidiness passed, including independent-module checks and the generated consumer compile.

  • Binary/image workflow passed all four agent/cluster builds and both cluster-image smoke tests/scans. AMD64 scan and ARM64 scan report no assigned OTel CVEs and zero secrets; each retains UNKNOWN GO-2026-5932 in x/crypto/openpgp, with its existing expired exception unchanged.

  • ARM64 DEB artifact passed; package SHA-256 ac45cb822af523a2dd047e4921589250c4aca5e32c5aaa75d6fe46943a5ecb76. Its agent embeds CNI 1.9.0, trace SDK/exporters 1.45.0, log exporters 0.21.0, and otelzap 0.20.0. A focused Grype 0.117.0 scan of this binary, using the reviewed containerd VEX and DB built 2026-09-18T06:30:15Z, reports zero active findings and the three reviewed containerd suppressions. This is not whole-package/image clearance.

  • Full branded/unbranded unit suites and lint/module checks passed, including the new security regressions.

  • Full DEB/agent-image workflow passed for AMD64 and ARM64: packages, branding, image builds, smoke tests and scans. AMD64 DEB artifact SHA-256: c4e3477bdf552b6c3b10811fe73ff314771fd02f3dca9216a9f563ab3866a7e5. Embedded build information in both architecture packages confirms the expected CNI and OTel versions at test merge 71e9a7d4e60ce5bf25594b7096e88556031af979.

  • Final agent-image scans (AMD64, ARM64) report zero secrets and no assigned OTel or CNI CVEs. Both retain seven in-scope findings: Python CVE-2026-15310/15806/17084/19672/87910 and CVE-2025-15367, plus UNKNOWN GO-2026-5932. The latter two have existing expired exceptions. Scans run in inform mode; green CI does not mean clean images. All three required CI rollups passed.

Baseline and remaining scope:

  • Assigned dev scan35310940870 attempt1, source de6e601c4722b85675f58f120b0a331dc15dc1e6; downloaded aggregate ZIP verified as sha256:4364222d6d21a4000d718e2228136c23ce1d60c9c602c14d32422e8b04cf6c49. Canonical ticket: StackVista/cve-reporter#29 (supervisor-owned).
  • Merged VEX42, d0eecb38117425c4ec35d7101c4d445a745d1794, was also validated against immutable baseline image quay.io/stackstate/stackstate-k8s-agent@sha256:2acf83d5ecfaaac287aa5293c7d779e698689eb6d60db567e26cc2c5f8e9cabf. Containerd 50195/53492/53489 at 1.7.35 are ignored as reviewed; the eight then-unfixed CNI/Python/OTel control rows remain active. No applicability investigation or new exception is proposed.
  • CNI Update CNI plugins to 1.9.0 #521 merged externally at 2026-09-18T07:25:11Z, preserving reviewed head 0118835c767490c1c24bdcc0d59f39c612f01c6a. Retire the VEX-superseded agent CVE exceptions and repair the rest #511's human hold is preserved. Python CVE-2026-17084 (scanner lead 3.15.0rc2, not a stable 3.13 patch) and CVE-2026-87910 (no scanner fix) remain explicit. Curl Update embedded curl to 8.22.0 #520 is delivered.
  • Existing fakeintake OTel pipeline assertions were inspected; infrastructure E2E was not run. Retain release-candidate telemetry QA, independent review and complete delivery validation. This task performed no merge, production release/deployment, VEX approval, or whole-estate scan.

CNI #521 delivery follow-up (2026-09-18; existing independent review reused):

  • Post-merge lint/unit, DEB/agent publication, and binary/cluster publication all succeeded at 4d7fbc56f3a485e733c44c9f63b36df53750d2b4.
  • Published quay.io/stackstate/stackstate-k8s-agent:4d7fbc56 index: sha256:9862e634ce21c65a1fb5d735ca836ca9d291163a7eaa1b3a20834f5960a82ddb. Publication/signature verification matches the independently fetched live registry manifest. AMD64 image digest sha256:17166fb2362c19659754d41bb46df0b335423c46cd4ce156e7a20da86da95730; ARM64 sha256:b5999629f19af0c46054c4a2c379f74313344e793071b9ad0bea57e808b07429. Syft catalogs read directly from each immutable registry image confirm github.com/containernetworking/plugins v1.9.0 in /opt/stackstate-agent/bin/agent/agent.
  • Published cluster-agent :4d7fbc56 index: sha256:81a44d3d488072be7b66d094c0bedb32b6781e1415c99018f29bb3fb726151ed; live registry digest matches publication/signature verification.
  • Signed pre-release DEBs stackstate-agent_3.78.2.git.182.4d7fbc5-1_{amd64,arm64}.deb were published normally. Independently downloaded from https://sts-agent-prerelease.s3.amazonaws.com/pool/stackstate-7.78.2/s/st/; SHA-256 matches each live dists/stackstate-7.78.2/main/binary-{arch}/Packages index: AMD64 3b4b091cd13da61dafb9ce1ee9deb17f6862e86b8336e035ca123d5591733b2d, ARM64 73975b863b74cd9db10d6421d7967c6192fae0dad5dce43019105216d1005c27. go version -m on each extracted published binary confirms CNI v1.9.0 and revision 4d7fbc56f3a485e733c44c9f63b36df53750d2b4.
  • Artifact delivery is verified. No deployment was performed or inferred; no source reassessment, workflow dispatch, release promotion, ticket closure, exception change, or repeat independent review was performed. OTel remains this unmerged PR; Python, GO and Retire the VEX-superseded agent CVE exceptions and repair the rest #511 holds remain as above.

Post-merge delivery verification — 2026-09-18:

Louis merged this PR at 10:11:41Z as 13451dce73328e6e9e566593cb753d6c78166475, preserving independently accepted head 85b010341b2343d8d2f29d76c00db2f1f6b5f135. The existing source review is reused. Normal push binary/cluster publication and DEB/agent publication succeeded; no additional workflow or release was dispatched. The same-SHA merge-queue lint/unit run passed. The redundant post-merge lint/unit run is still finishing its branded suite; its unbranded suite and module check passed.

Published tag 13451dce, with live registry manifest hashes independently matched to normal CI signature-verification output:

Image Multi-architecture digest
quay.io/stackstate/stackstate-k8s-agent sha256:32a25b904072a24df3c6e1d3da22e8fb7fee9e34ad98b4a73756b091acba4060
quay.io/stackstate/stackstate-k8s-cluster-agent sha256:f45f3cba492b1892fa847b0843478402f44029d904d1a44a812015881813f9d6

Agent publication/signature evidence, cluster publication/signature evidence.

Immutable Linux manifests inspected directly with Syft:

  • Agent AMD64: sha256:f2d66aa57436f932bda58c32e16bd0f8cfc9f157a3dc3dfd19cd28e728b2ca27; ARM64: sha256:a25102f00202988a68ef255b3a5c846f41a8fdda750b5eb318fc6d731bda7c4a.
  • Cluster AMD64: sha256:f78a36ce2e6b9df70649f0967a0fe7b232ebac80b2c4c9a5c9cbad5481645b2e; ARM64: sha256:513e7881ef085b68a2e1dd193345d5a3c4a22e1213cafe5e4b12060c3bf99ea6.
  • All four published images embed SDK/trace exporters v1.45.0; both agent images embed log gRPC exporter v0.21.0. No older version of those target modules appears in the published inventories.

Signed pre-release DEBs 3.78.2.git.188.13451dc-1 were published normally. Independently downloaded packages match live repository index hashes:

  • AMD64 package: SHA-256 6744e373bb70b77362a01a340d41afc1d8802d6f6f1b240c77db0fcf076fc6ae.
  • ARM64 package: SHA-256 c04856b61ec598c8dee8a0def9b958aa21932a2d0d64c823ab8469d50f36b6b1.
  • Static go version -m inspection of both downloaded binaries confirms revision 13451dce73328e6e9e566593cb753d6c78166475, SDK/trace exporters v1.45.0, log exporters v0.21.0 and otelzap v0.20.0.

Normal image scan evidence: agent AMD64 / ARM64; cluster AMD64 / ARM64. No assigned OTel findings remain, and all four separate secret scans report zero secrets. Agent images retain six Python rows (CVE-2026-15310/15806/17084/19672/87910 and CVE-2025-15367) plus GO-2026-5932; cluster images retain GO-2026-5932. Existing expired exceptions are unchanged. These inform-mode results are not whole-image clearance.

Artifact delivery is verified for later independent acceptance; deployment is neither performed nor inferred. Python source backports are proposed separately in #529 and are not yet delivered. PR511 and all applicability/GO/VEX holds remain unchanged; supervisor-owned tickets were not edited. PR522 was not modified by this task; at 10:14:11Z a Dependabot-authored update based on this merge changed its head to ff65b7538a028d0c7961b9e3dad3908aabc9b5cf (previously dbc8c8cf66137a1df47d7d7388a73af172e76df4). Its unrelated serverless scope remains outside this work.

@ai-collaboration-app
ai-collaboration-app Bot marked this pull request as ready for review September 18, 2026 08:55
@LouisLotter
LouisLotter added this pull request to the merge queue Sep 18, 2026
Merged via the queue into stackstate-7.78.2 with commit 13451dc Sep 18, 2026
42 checks passed
@LouisLotter
LouisLotter deleted the cve/otel-scan35310940870 branch September 18, 2026 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants