Skip to content

feat(cpython): HTTP from Python through bashkit egress - #2568

Merged
chaliy merged 1 commit into
mainfrom
claude/project-thread-io7h59
Oct 7, 2026
Merged

chaliy merged 1 commit into
mainfrom
claude/project-thread-io7h59

Conversation

@chaliy

@chaliy chaliy commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

What changed

urllib.request and http.client in the CPython builtin can now make HTTP and HTTPS requests. Each request goes to the host through a new wasm import, bashkit.http_request. The host runs it through the same HttpClient pipeline as curl: allowlist, SSRF precheck, credential injection hooks, signing, the embedder's HttpTransport, and the response cap. TLS runs on the host, and the guest still has no sockets.

  • Guest side: _bashkit.http() in C, plus an http.client adapter that replaces only the socket. http.client's own request and response code still runs, including keep-alive, HEAD and chunked uploads. HTTPSConnection is defined so urllib registers https.
  • Host side (builtins/cpython/http.rs): validates the request record before dispatch (method set, URL and header caps, token names, no CR/LF/NUL) and drops host-owned headers such as Host and the framing headers.
  • New CPythonLimits::max_http_requests (default 100 per call). Each request's timeout is capped by the call deadline.
  • Context::http_client is now Option<&Arc<HttpClient>> so the per-call guest state can own a handle.
  • New fuzz target, cpython_http_fuzz. The threat model gains TM-PY-CPY-012, L-CPY-002 is narrowed, and the docs have a new HTTP section.

This is step 1 of the plan in the scope doc. requests and httpx2 follow in a separate packages crate, behind an opt-in cpython-http feature.

Why

Python scripts could not reach the network at all, so urlopen failed. Routing requests at the HTTP level keeps every policy the embedder configured for curl (allowlist, credentials, audit), with no TLS or socket code in the guest.

Before

$ python3 -c 'import urllib.request; urllib.request.urlopen("https://93.184.216.34/json")'
urllib.error.URLError: <urlopen error unknown url type: https>

After

With .network(allowlist):

python3 -c 'import json, urllib.request
with urllib.request.urlopen("https://93.184.216.34/json") as r:
    print(r.status, json.load(r))'
200 {'ok': True}
  • A denied host or private IP fails with URLError and "access denied", and the request never reaches the transport.
  • Redirects to an off-allowlist host are denied at that hop.
  • With no network configured, requests fail with "network access not configured".

12 new tests (cpython_http_tests) cover:

  • GET/POST, 404, keep-alive, HEAD and chunked upload
  • redirect re-checks
  • private IPs
  • Host retargeting
  • CRLF injection
  • placeholder credentials injected only on the host
  • the request cap, request timeout and call deadline

All 12 pass on both the Pulley and native builds.

Risk

Medium. This opens a new egress path from the guest. Mitigations: everything goes through HttpClient, the host validates every request before dispatch, a request cap bounds floods, and errors are Display-only and capped at 512 bytes. The snapshot grows by about 3 KB gzipped and the stdlib zip by about 6 KB. Internal API change: Context::http_client is now an &Arc.

Checklist

  • cargo test -p bashkit --features cpython,http_client --lib --test integration -- cpython (112 + 12 + unit tests)
  • cargo test -p bashkit --features cpython-native,http_client --test integration -- cpython_http
  • cargo clippy -p bashkit --features cpython,http_client --all-targets -- -D warnings, and the same with cpython only
  • cargo check --workspace --all-targets --features http_client, and the fuzz target checked with cargo check
  • cargo fmt --check, just check-okf, just check-doc-links
  • Knowledge: cpython-wasm (HTTP bridge), threat model, limitations, log

Generated by Claude Code

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit 2e5ad2a Commit Preview URL

Branch Preview URL
Oct 07 2026, 03:28 PM

@chaliy
chaliy force-pushed the claude/project-thread-io7h59 branch from f0e0d78 to a87c667 Compare October 7, 2026 14:37
urllib.request and http.client send requests through the host's
HttpClient pipeline (allowlist, SSRF checks, credential injection,
signing, transport, size caps) via a new bashkit.http_request import.

Claude-Session: https://claude.ai/code/session_01Ke1iW2hQi9VXioZi1avPaW
@chaliy
chaliy force-pushed the claude/project-thread-io7h59 branch from a87c667 to 2e5ad2a Compare October 7, 2026 15:25
@chaliy
chaliy merged commit 7b1745f into main Oct 7, 2026
47 checks passed
@chaliy
chaliy deleted the claude/project-thread-io7h59 branch October 7, 2026 15:50
chaliy added a commit that referenced this pull request Oct 7, 2026
…2572)

## What changed
`import requests`, `import httpx` and `import httpx2` work in the
CPython (`cpython` feature) `python3`. They are bashkit's own compact
implementations of the common API, written on top of the HTTP bridge
from #2568, so every request still goes through the network allowlist,
SSRF checks, credential injection and the response cap. They are
preloaded in the snapshot, so importing them costs nothing.

- **requests**: the verb helpers and `Session`, along with params, data,
json, files, headers, cookies, basic auth, timeouts, redirects and
hooks. `Response` helpers, and the upstream exception hierarchy.
- **httpx** / **httpx2**: the verb helpers and `stream()`, plus `Client`
and `AsyncClient` with base_url, follow_redirects, event hooks and
`MockTransport`. Also `URL`, `Headers`, `QueryParams`, `Cookies`,
`Timeout`, `codes`, and the upstream exceptions.
- Out of scope: retries, proxies, client certificates, HTTP/2, digest
auth, streaming uploads, and OPTIONS. Documented as L-CPY-010.
- Response cookies are sent back only to their own host. `Authorization`
is dropped on a cross-host redirect. Each redirect hop is re-checked by
the host.
- A http.client read timeout now surfaces from `getresponse()`, as in
CPython.

## Why
Upstream `requests` + urllib3 + idna + charset_normalizer import 155
modules and cost **~3.1 s per call** on Pulley (urllib3 alone 1.3 s).
Every call starts from the snapshot, so that cost repeats on every call.
Agents mostly use the common API surface.

## Before / After
Before: `python3 -c 'import requests'` gives `ModuleNotFoundError`.
Bundling upstream made the same call take ~3.1 s.

After (criterion, Pulley):
| Benchmark | Time |
|---|---|
| `python_call/cpython/print` | 4.03 ms |
| `python_import/cpython/requests_httpx` (`import requests, httpx`) |
3.87 ms |

Differential test vs upstream `requests` 2.34.2 / `httpx` 0.28.1 (same 9
scenarios, shared fake server, output must match line for line):
```
ok   requests_basics
ok   requests_errors
ok   requests_redirects
ok   requests_session_and_body
ok   httpx_basics
ok   httpx_errors_and_redirects
ok   httpx_client_and_async
ok   requests_misc
ok   httpx_misc
{"scenarios": 9, "failed": 0}
```
It caught two divergences, both fixed: `True` in requests params, and
httpx's `Host` header.

## Risk
- Medium
- Scripts relying on features outside the subset (retries, proxies,
HTTP/2) behave differently from upstream. Those options are accepted and
ignored, or fail with a clear exception.
- The CPython crate grows to 9.8 MiB (still under the crates.io cap; a
follow-up moves the snapshot to xz).

## Checklist
- [x] Tests added or updated: 11 new integration tests through the real
bridge, covering gzip, charset fallback, 303/307 rules, redirect loops,
cross-host auth drop, CRLF rejection, the request cap, hooks, streams
and async. Also the differential test, run as a CI step.
- [x] Backward compatibility considered: new modules only; the
capability case that expected `import requests` to fail now checks
`numpy`.


---
_Generated by [Claude Code](https://claude.ai/code)_
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant