Skip to content

Cache static User-Agent components - #3877

Merged
lukasdchang merged 6 commits into
boto:developfrom
lukasdchang:user-agent-improvements
Nov 18, 2026
Merged

lukasdchang merged 6 commits into
boto:developfrom
lukasdchang:user-agent-improvements

Conversation

@lukasdchang

Copy link
Copy Markdown
Contributor

Summary

Before this PR, on every API request, the botocore client builds the User-Agent header twice, once per client, on its first request, then again after signing which also gets repeated on every retry. This is a costly operation because it rebuilds every component from scratch and sanitizes each one character by character costing about 21 µs per call.

This PR eliminates repeated work by using a named tuple to store the static parts of a user-agent header once at the beginning of a request. Each build then reuses those cached parts and only adds the request's feature list cutting CPU per call by 13.3% across the benchmark suite with no change to the header that's sent.

Benchmark

Stubbed-HTTP boto3 calls across 71 cases and 5 protocols, on EC2 m7i.xlarge with Python 3.14.8. Each side is the median of 3 interleaved runs.

Before After CPU per call
All cases (geometric mean) 649 µs 562 µs 13.3% less
DynamoDB GetItem 422 µs 342 µs 19.1% less
S3 GetObject (REST-XML) 557 µs 467 µs 16.2% less
S3 PutObject (REST-XML) 456 µs 378 µs 17.2% less

All 71 cases improved, saving a median of 77 µs per call. The two User-Agent builds in a request dropped from 43.5 µs to 4.8 µs.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@lukasdchang lukasdchang changed the title Cache static User-Agent components [DRAFT] Cache static User-Agent components Nov 16, 2026
@codecov-commenter

codecov-commenter commented Nov 16, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.55%. Comparing base (ee748e4) to head (485f9c6).
⚠️ Report is 710 commits behind head on develop.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3877      +/-   ##
===========================================
- Coverage    92.59%   92.55%   -0.05%     
===========================================
  Files           68       69       +1     
  Lines        15634    15950     +316     
===========================================
+ Hits         14477    14763     +286     
- Misses        1157     1187      +30     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@lukasdchang lukasdchang changed the title [DRAFT] Cache static User-Agent components Cache static User-Agent components Nov 17, 2026
@lukasdchang
lukasdchang marked this pull request as ready for review November 17, 2026 16:21
@lukasdchang
lukasdchang requested a review from a team as a code owner November 17, 2026 16:21

@stobrien89 stobrien89 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just needs a changelog entry

Comment thread botocore/useragent.py Resolved
Comment thread tests/unit/test_useragent.py Resolved

@alexgromero alexgromero left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@lukasdchang
lukasdchang merged commit dc9606c into boto:develop Nov 18, 2026
49 checks passed
Alan4506 pushed a commit that referenced this pull request Nov 18, 2026
* release-1.44.6:
  Bumping version to 1.44.6
  Update endpoints model
  Update to latest models
  Cache static User-Agent components (#3877)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants