Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions crates/bashkit-cpython-wasm/artifacts/MANIFEST
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@ wasi_sdk=24
wasmtime_wizer=49.0.1
zlib=1.3.1
sqlite=sqlite-amalgamation-3500400
ac54edd6cb27e55c00500edc995555a69e3654379c34cda437125b6ccdc694d4 python.wasm.gz
e4c1d3a9db1f8ad2f8a89900055cc4c515897b8d6d2501cfbb8fc3bc76082b29 python314.zip
6f4c6d7b93992426af97884dfb603c2ef9ddda9ad20e51e2b79e84c18eb1e140 python.wasm.gz
672dd03d9e31bee0d92c14f10f772b97c5daedbc34b9f14d5ebc21b086b77daf python314.zip
Binary file modified crates/bashkit-cpython-wasm/artifacts/python.wasm.gz
Binary file not shown.
Binary file modified crates/bashkit-cpython-wasm/artifacts/python314.zip
Binary file not shown.
66 changes: 62 additions & 4 deletions crates/bashkit-cpython-wasm/guest/_bashkit_boot.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@
Decisions:
- Tracebacks hide this driver's frames so output matches real `python3`.
- `random` is re-seeded per call: the snapshot would otherwise hand every
tenant the same sequence.
tenant the same sequence. Seeding is lazy (first use in a call), since it
costs ~0.2 ms on Pulley and most calls never touch `random`. Calling the
C base method directly (`_random.Random.random(random._inst)`) skips the
seed and sees the snapshot's state; `random` is not a CSPRNG anyway.
- The snapshot heap is frozen (`gc.freeze`) so garbage collection only
scans objects created by the current call.
- Snapshot objects are made immortal before freezing: reference counting
Expand Down Expand Up @@ -73,6 +76,7 @@ def preload():
except Exception: # pragma: no cover - optional modules
pass
_patch_asyncio()
_install_lazy_random()
# Move every object the snapshot holds into the permanent generation.
# The collector then never traverses (or writes to) snapshot memory, so
# per-call GC cost scales with what the script allocates, and snapshot
Expand All @@ -98,6 +102,63 @@ def _immortalize_snapshot():
_bashkit.immortalize(gc.get_referents(*objects))


_RANDOM_API = (
"seed", "random", "uniform", "triangular", "randint", "choice", "randrange",
"sample", "shuffle", "choices", "normalvariate", "lognormvariate",
"expovariate", "vonmisesvariate", "gammavariate", "gauss", "betavariate",
"binomialvariate", "paretovariate", "weibullvariate", "getstate",
"setstate", "getrandbits", "randbytes",
)
_LAZY_ENTRY_POINTS = ("random", "getrandbits", "getstate", "seed", "setstate")


def _install_lazy_random():
# The module-level functions are methods bound to `random._inst`. Give
# that instance a subclass whose entry points seed it from os.urandom on
# first use, then turn it back into a plain Random and rebind the module
# functions, so later calls pay nothing extra.
import random

base = random.Random

def wake(self):
# Only the overridden entry points need rebinding; the other module
# functions are base-class methods and see the class switch. Module
# attribute writes are slow on Pulley (~40 us each), so keep it to five.
self.__class__ = base
for name in _LAZY_ENTRY_POINTS:
setattr(random, name, getattr(self, name))

class _LazyRandom(base):
def random(self):
wake(self)
base.seed(self)
return self.random()

def getrandbits(self, k):
wake(self)
base.seed(self)
return self.getrandbits(k)

def getstate(self):
wake(self)
base.seed(self)
return self.getstate()

def seed(self, *args, **kwargs):
wake(self)
return self.seed(*args, **kwargs)

def setstate(self, state):
wake(self)
return self.setstate(state)

inst = random._inst
inst.__class__ = _LazyRandom
for name in _RANDOM_API:
setattr(random, name, getattr(inst, name))


def _patch_asyncio():
try:
from asyncio import selector_events
Expand Down Expand Up @@ -197,9 +258,6 @@ def _refresh_process_state():
os.chdir(pwd)
except OSError:
pass
random = sys.modules.get("random")
if random is not None:
random.seed()


def _fresh_main(filename=None):
Expand Down
2 changes: 1 addition & 1 deletion crates/bashkit/docs/cpython.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ let bash = Bash::builder()
(`FileNotFoundError`, ...) and messages are correct; code comparing
`e.errno == errno.ENOENT` works because the `errno` module matches.
- **Fixed hash seed**: `hash()` of `str`/`bytes` is the same in every call
(the seed is baked into the snapshot). `random` is re-seeded per call.
(the seed is baked into the snapshot). `random` is re-seeded per call (on first use).
- **Deep C-level recursion** (for example `repr` of a list nested 100 000
levels) ends the call with `python3: fatal error: stack overflow in the
interpreter` instead of `RecursionError`.
Expand Down
25 changes: 25 additions & 0 deletions crates/bashkit/tests/integration/cpython_integration_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -566,6 +566,31 @@ async fn random_is_reseeded_per_call() {
assert_eq!(out, "differ\n");
}

/// Seeding is lazy: every entry point into the shared `random` instance must
/// still see a fresh per-call seed, and explicit seeding stays deterministic.
#[tokio::test]
async fn lazy_random_seed_covers_every_entry_point() {
let differ = |snippet: &str| {
format!(
"a=$(python3 -c '{snippet}'); b=$(python3 -c '{snippet}'); [ \"$a\" != \"$b\" ] && echo differ"
)
};
for snippet in [
"from random import random; print(random())",
"import random; print(random.randint(0, 10**18))",
"import random; print(random.getrandbits(64))",
"import random; print(random.choice(range(10**9)))",
"import random; print(random.getstate()[1][:4])",
"import random; x = list(range(50)); random.shuffle(x); print(x)",
] {
assert_eq!(ok(&differ(snippet)).await, "differ\n", "{snippet}");
}
let out = ok("python3 -c 'import random; random.seed(42); a = random.random(); random.seed(42); print(a == random.random(), random.Random(42).random() == a)'; \
python3 -c 'import random; s = random.getstate(); a = random.random(); random.setstate(s); print(a == random.random())'")
.await;
assert_eq!(out, "True True\nTrue\n");
}

#[tokio::test]
async fn separate_bash_instances_share_nothing() {
let mut a = bash();
Expand Down
1 change: 1 addition & 0 deletions knowledge/log.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## 2026-10-07

* **Performance**: CPython `random` seeds lazily. The snapshot's `random._inst` gets a subclass whose entry points (`random`, `getrandbits`, `getstate`, `seed`, `setstate`) seed from `os.urandom` on first use, then switch it back to `Random` and rebind those five module functions. Calls that never touch `random` skip the reseed (`pass` 3.7 to 3.5 ms). See [CPython WebAssembly Runtime](runtimes/cpython-wasm.md).
* **Performance**: Recorded CPython benches after #2534/#2535 (`criterion-python-vm-linux-x86_64-1791345434.md`): first call ~19 ms, warm `print(1)` 4.4 ms (was 6.3), 64 parallel sessions 157 ms (was 291), load ~165 calls/s (was ~90). `/benches` now shows Python startup (CPython vs Monty) from these reports. See [Performance Results](operations/performance-results.md).
* **Feature**: Secret generators. `$SRANDOM` (bash 5.1, 32 bits from `getrandom`), `uuidgen` (v4 only, L-RAND-001) and `openssl rand [-hex|-base64] [-out] N` (only `rand`; capped at 1 MiB, TM-DOS-123). All draw from the OS CSPRNG like `/dev/urandom`, never the `$RANDOM` LCG.
* **Decision**: CPython stdlib ships as bytecode only. Non-preloaded imports compiled source on Pulley every call (3-7 s each); unchecked-hash `.pyc` cut that ~10x. Unusable modules (`ctypes`, `ssl`, network clients/servers, `pdb`, `pydoc`, `_pyrepl`) and pure-Python C twins dropped so the crate stays under the 10 MiB cap. Dropping preloaded modules was rejected: ~1-2 ms first-call gain, seconds per import lost. See [CPython WebAssembly Runtime](runtimes/cpython-wasm.md), [Limitations](operations/limitations.md) L-CPY-009.
Expand Down
4 changes: 3 additions & 1 deletion knowledge/runtimes/cpython-wasm.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,9 @@ option parsing (`-c`, `-m`, file, directory, `-`, `-x`, `-W`, `-V`/`-VV`,
`-h`, ignored init-only flags), fresh `__main__` per call, `sys.argv` and
`sys.path[0]` rules, tracebacks with driver frames stripped, `SystemExit`
semantics, `atexit`, then clearing `__main__` + `gc.collect()` so unclosed
files flush. `random` is re-seeded per call. asyncio's self-pipe is disabled
files flush. `random` is re-seeded per call, lazily: the module instance
seeds from `os.urandom` on first use in a call (~0.2 ms saved on calls that
never use it). asyncio's self-pipe is disabled
(no sockets, threads or signals exist to need it).

## Testing
Expand Down
Loading