Repository navigation
fix(agentos): queue concurrent WebAssembly and Python launches per VM - #2025
Conversation
|
This PR was not deployed automatically as @eersnington does not have access to the Railway project. In order to get automatic PR deploys, please add @eersnington to your workspace on Railway. |
| // A cold start awaits Pyodide warmup while holding the engine; take turns with | ||
| // other launches on this VM instead of failing with an execution conflict. | ||
| let launch_turn = execution_engines.python_launch_turn().await; |
There was a problem hiding this comment.
🔴 High · Queue before creating an unregistered process
This await happens after the duplicate-ID check and after spawn_trusted_root_process. If a queued request is cancelled, dropping KernelProcessHandle does not finish or reap that process, so it remains in the VM kernel without an active_processes entry. Two same-runtime requests with the same process_id can also both pass the check; the queue then lets both start, and the later active_processes.insert silently replaces the first. Acquire the runtime turn before creating/reserving the process and hold it through registration, or add a cancellation-safe pending reservation that claims the ID and rolls back the kernel process.
| .map_err(|_| execution_engine_conflict_error(&self.inner.vm_id, "Python", operation)) | ||
| } | ||
|
|
||
| pub(crate) async fn python_launch_turn(&self) -> tokio::sync::MutexGuard<'_, ()> { |
There was a problem hiding this comment.
🟠 Medium · Engine admission remains bypassable
The queue is independent of python()/wasm(), so existing launch paths can still take the raw RefCell and return the conflict this change is intended to remove. In production, exec_javascript_process_image_owned directly borrows the WASM/Python engines at child_process.rs:5586 and :5620 while another cold start may hold them; the legacy spawn path also queues WASM but still directly borrows Python at :4953. Encapsulate queue acquisition with the engine borrow and route every Python/WASM launch or replacement through that admission path.
There was a problem hiding this comment.
not introduced by this pr. but need to look into this deeper
f4fe86c to
249dcd9
Compare
RefCelland borrows them withtry_borrow_mut, which fails when the engine is already borrowed..await. On a cold VM the start materializes the import cache and prewarms (WebAssembly) or warms Pyodide (Python), so a second launch on the same VM fails withERR_AGENTOS_VM_EXECUTION_CONFLICT. Warm starts do not yield, so this shows up after a VM wakes, for example when an agent runs two tool calls at once.wasm_launchandpython_launchasync locks. A launch takes its turn before it borrows the engine and releases it after the start, so a second launch waits instead of failing. Processes still run in parallel once started.Covers the top-level launch in
execution/launch.rsand the child launches inexecution/child_process.rs(WebAssembly child, nested WebAssembly child, nested Python child). With only the top-level launch, a shell's own child command still hit the conflict.Adds
concurrent_launches_on_a_fresh_vm_all_runtocrates/client/tests/process_e2e.rs: three WebAssembly and three Python launches started together on a fresh VM all exit 0. Onmain, all but one launch per runtime fail withERR_AGENTOS_VM_EXECUTION_CONFLICT.