fix: reduce legacy Linux customData payload size - #9597
Closed
Sylvain Boily (djsly) wants to merge 1 commit into
Closed
Sylvain Boily (djsly) wants to merge 1 commit into
Sylvain Boily (djsly) wants to merge 1 commit into
Conversation
Expand gzip write_files contents before compressing the complete cloud-config so scripts share one compression stream. Preserve cloud-init metadata and leave scriptless NBC, Windows, and Ignition payloads unchanged. AB#39729730 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
|
Contributor
Windows Unit Test Results 3 files 17 suites 1m 2s ⏱️ Results for commit 3040a06. |
Contributor
|
Contributor
|
Contributor
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
It changes production Linux provisioning payload serialization across distributions, and live VM validation remains pending.
Review effort: Balanced
Findings: None
What changed in this PR
Reduces legacy Linux customData size by expanding per-file gzip content before compressing the complete cloud-config.
Changes:
- Adds shared cloud-config expansion and compression logic.
- Adds size, preservation, error, and legacy E2E regression tests.
- Documents limits and validation commands.
| File | Description |
|---|---|
README.md |
Documents customData limits and tests. |
pkg/agent/baker.go |
Implements whole-cloud-config compression. |
pkg/agent/customdata_test.go |
Tests size, fidelity, and errors. |
pkg/agent/baker_test.go |
Updates pre-provisioning assertions. |
e2e/scenario/customdata_test.go |
Tests representative legacy payloads. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Contributor
|
Addressed with this #9601 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
The legacy cloud-init path gzip/Base64-encodes each script independently and then gzips the entire cloud-config. This prevents compression across scripts and leaves AzureLinuxV2 close to ARM's 87,380-character customData limit; small script changes can make VMSS creation fail before node validation.
Expand the template's gzip
write_filescontents before compressing the complete cloud-config. Preserve file bytes, permissions, owners, extra metadata, boot commands, and other cloud-config sections. Payloads without gzipped entries remain byte-identical. Embedded templates, scriptless NBC boothooks, Windows, and Ignition retain their existing formats; no scripts or LocalDNS coverage are removed.Measured with the legacy regression fixtures:
Add size/content-preservation tests across Linux distros, cloud-config metadata and binary/newline edge cases, invalid-input diagnostics, and a regression using the full legacy E2E bootstrap configuration. Update the pre-provisioning assertion to inspect write-file paths rather than searching inside now-readable script contents.
Which issue(s) this PR fixes:
AB#39729730 — AzureLinuxV2 customData OSProfile limit
Validation:
GENERATE_TEST_DATA=true go test ./pkg/agent/...— passed; no snapshot changes produced.cd e2e && go test ./scenario -run 'CustomData|WriteFiles' -count=1— passed.cd e2e && go build ./... && go vet ./scenario— passed.write_filesand outer gzip support, not a new VHD capability.🤖 Generated by GitHub Copilot