Repository navigation
feat(cpython): HTTP from Python through bashkit egress - #2568
Merged
Merged
Conversation
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
bashkit | 2e5ad2a | Commit Preview URL Branch Preview URL |
Oct 07 2026, 03:28 PM |
chaliy
force-pushed
the
claude/project-thread-io7h59
branch
from
October 7, 2026 14:37
f0e0d78 to
a87c667
Compare
urllib.request and http.client send requests through the host's HttpClient pipeline (allowlist, SSRF checks, credential injection, signing, transport, size caps) via a new bashkit.http_request import. Claude-Session: https://claude.ai/code/session_01Ke1iW2hQi9VXioZi1avPaW
chaliy
force-pushed
the
claude/project-thread-io7h59
branch
from
October 7, 2026 15:25
a87c667 to
2e5ad2a
Compare
2 tasks done
chaliy
added a commit
that referenced
this pull request
Oct 7, 2026
…2572) ## What changed `import requests`, `import httpx` and `import httpx2` work in the CPython (`cpython` feature) `python3`. They are bashkit's own compact implementations of the common API, written on top of the HTTP bridge from #2568, so every request still goes through the network allowlist, SSRF checks, credential injection and the response cap. They are preloaded in the snapshot, so importing them costs nothing. - **requests**: the verb helpers and `Session`, along with params, data, json, files, headers, cookies, basic auth, timeouts, redirects and hooks. `Response` helpers, and the upstream exception hierarchy. - **httpx** / **httpx2**: the verb helpers and `stream()`, plus `Client` and `AsyncClient` with base_url, follow_redirects, event hooks and `MockTransport`. Also `URL`, `Headers`, `QueryParams`, `Cookies`, `Timeout`, `codes`, and the upstream exceptions. - Out of scope: retries, proxies, client certificates, HTTP/2, digest auth, streaming uploads, and OPTIONS. Documented as L-CPY-010. - Response cookies are sent back only to their own host. `Authorization` is dropped on a cross-host redirect. Each redirect hop is re-checked by the host. - A http.client read timeout now surfaces from `getresponse()`, as in CPython. ## Why Upstream `requests` + urllib3 + idna + charset_normalizer import 155 modules and cost **~3.1 s per call** on Pulley (urllib3 alone 1.3 s). Every call starts from the snapshot, so that cost repeats on every call. Agents mostly use the common API surface. ## Before / After Before: `python3 -c 'import requests'` gives `ModuleNotFoundError`. Bundling upstream made the same call take ~3.1 s. After (criterion, Pulley): | Benchmark | Time | |---|---| | `python_call/cpython/print` | 4.03 ms | | `python_import/cpython/requests_httpx` (`import requests, httpx`) | 3.87 ms | Differential test vs upstream `requests` 2.34.2 / `httpx` 0.28.1 (same 9 scenarios, shared fake server, output must match line for line): ``` ok requests_basics ok requests_errors ok requests_redirects ok requests_session_and_body ok httpx_basics ok httpx_errors_and_redirects ok httpx_client_and_async ok requests_misc ok httpx_misc {"scenarios": 9, "failed": 0} ``` It caught two divergences, both fixed: `True` in requests params, and httpx's `Host` header. ## Risk - Medium - Scripts relying on features outside the subset (retries, proxies, HTTP/2) behave differently from upstream. Those options are accepted and ignored, or fail with a clear exception. - The CPython crate grows to 9.8 MiB (still under the crates.io cap; a follow-up moves the snapshot to xz). ## Checklist - [x] Tests added or updated: 11 new integration tests through the real bridge, covering gzip, charset fallback, 303/307 rules, redirect loops, cross-host auth drop, CRLF rejection, the request cap, hooks, streams and async. Also the differential test, run as a CI step. - [x] Backward compatibility considered: new modules only; the capability case that expected `import requests` to fail now checks `numpy`. --- _Generated by [Claude Code](https://claude.ai/code)_
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
urllib.requestandhttp.clientin the CPython builtin can now make HTTP and HTTPS requests. Each request goes to the host through a new wasm import,bashkit.http_request. The host runs it through the sameHttpClientpipeline ascurl: allowlist, SSRF precheck, credential injection hooks, signing, the embedder'sHttpTransport, and the response cap. TLS runs on the host, and the guest still has no sockets._bashkit.http()in C, plus an http.client adapter that replaces only the socket. http.client's own request and response code still runs, including keep-alive, HEAD and chunked uploads.HTTPSConnectionis defined so urllib registers https.builtins/cpython/http.rs): validates the request record before dispatch (method set, URL and header caps, token names, no CR/LF/NUL) and drops host-owned headers such asHostand the framing headers.CPythonLimits::max_http_requests(default 100 per call). Each request's timeout is capped by the call deadline.Context::http_clientis nowOption<&Arc<HttpClient>>so the per-call guest state can own a handle.cpython_http_fuzz. The threat model gains TM-PY-CPY-012, L-CPY-002 is narrowed, and the docs have a new HTTP section.This is step 1 of the plan in the scope doc.
requestsandhttpx2follow in a separate packages crate, behind an opt-incpython-httpfeature.Why
Python scripts could not reach the network at all, so
urlopenfailed. Routing requests at the HTTP level keeps every policy the embedder configured forcurl(allowlist, credentials, audit), with no TLS or socket code in the guest.Before
After
With
.network(allowlist):URLErrorand "access denied", and the request never reaches the transport.12 new tests (
cpython_http_tests) cover:HostretargetingAll 12 pass on both the Pulley and native builds.
Risk
Medium. This opens a new egress path from the guest. Mitigations: everything goes through
HttpClient, the host validates every request before dispatch, a request cap bounds floods, and errors are Display-only and capped at 512 bytes. The snapshot grows by about 3 KB gzipped and the stdlib zip by about 6 KB. Internal API change:Context::http_clientis now an&Arc.Checklist
cargo test -p bashkit --features cpython,http_client --lib --test integration -- cpython(112 + 12 + unit tests)cargo test -p bashkit --features cpython-native,http_client --test integration -- cpython_httpcargo clippy -p bashkit --features cpython,http_client --all-targets -- -D warnings, and the same withcpythononlycargo check --workspace --all-targets --features http_client, and the fuzz target checked withcargo checkcargo fmt --check,just check-okf,just check-doc-linksGenerated by Claude Code