Skip to content

CORE-45: inline dependency resolution + eager tasks for async graph execution - #91

Open
LawlAoux wants to merge 1 commit into
masterfrom
CORE-45-inline-eager-async-node-execution
Open

LawlAoux wants to merge 1 commit into
masterfrom
CORE-45-inline-eager-async-node-execution

Conversation

@LawlAoux

@LawlAoux LawlAoux commented Jul 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Nodes whose dependencies are already computed now execute inline instead of each being wrapped in asyncio.create_task (~24k tasks/turn on large voice bots, none of which start executing until the topological loop ends); a node blocked on a genuinely pending dependency falls back to a task created with Python 3.14's eager_start=True, so it runs at creation and network calls fire mid-traversal.
  • Input-option priority, pruning, and exception semantics are preserved: node_to_input_inline mirrors node_to_input_async, returning _PENDING to route to the task path. get_deps_and_await is dead in the new model and removed.
  • bon_secours_voice (45,540 nodes): CG framework overhead per turn 493–504ms → 364ms (−26%); tasks/turn 24,183 → ~100 on turns without real I/O wait; wall time on network-free turns −9–12% (connect 1305→1183ms). Bot responses byte-identical across old and new runners.

Rebase onto master 78 (22 Sep)

  • Sync-chain fusion (feat: fusing chains of sync nodes that are downstream to async nodes #85) was reverted in Revert "Merge pull request #85 from hyroai/yinon/poc-fuse-chains" #98, so the fusion notes this description used to carry, and the two fusion tests this PR had adapted, are gone with it. run.py merged cleanly; version 79, uv.lock re-locked.
  • Re-benchmarked with nlu-runtime's network-free per-turn harness (model endpoints and LLM calls answered in-process, one bot per process, N=7, this branch vs the installed CG 78 back-to-back on the same nlu code; laptop numbers, so read the percentages): connect p50 −21% / −30% / −27% on atlantic / louisiana / bon_secours, greet −12% / −23% / −13%, turns with an embed + SLM call −6…−11%. Turns whose routing-classifier call suspends mid-graph are unchanged (±1%): their downstream falls back to the task path, as designed. The server's per-message gc.collect(0) drops from 12–37ms to under 1ms on connect/greet turns and to ~3ms on embed turns, since 24k fewer Task objects are created per turn. Bot responses byte-identical on all three bots, zero errors.
  • Gates on the rebased branch: CG suite 119 passed; nlu-runtime slot_filling + listen suites 587 passed against it.

Jira

https://hyro-ai.atlassian.net/browse/CORE-45

@LawlAoux LawlAoux self-assigned this Jul 28, 2026
@LawlAoux
LawlAoux force-pushed the CORE-45-inline-eager-async-node-execution branch from bb3c2ca to 05a6a9b Compare August 12, 2026 13:18
Async-region nodes were each wrapped in asyncio.create_task -- ~24k tasks
per turn on large bots, none of which start executing until the whole
topological loop finishes (the first await is the final gather). Now a
node whose dependencies are already available resolves them inline and
runs with no task at all; a node blocked on a genuinely pending
dependency falls back to the task path, created with eager_start=True
(Python 3.14) so it runs at creation and suspends only on real I/O --
network calls fire mid-traversal. Input-option priority, pruning, and
exception semantics are unchanged: node_to_input_inline mirrors
node_to_input_async, returning _PENDING to route to the task path.
get_deps_and_await is dead in the new model and removed.

bon_secours_voice (45,540 nodes): CG framework overhead per turn
493-504ms -> 364ms (-26%); tasks/turn 24,183 -> ~100 when nothing truly
suspends; network-free turn walls -9..-12%. Bot responses byte-identical
across the old and new runners.

Rebased onto sync-chain fusion (#85). The two compose: the suite passes
with COMPUTATION_GRAPH_SYNC_FUSION both on and off, and a battery of
graph shapes (skips, handled and unhandled exceptions, priority
fallback, reconvergence through async, genuinely suspending nodes)
yields identical results under master-unfused, master-fused,
inline-unfused and inline-fused. One fusion test had to change: its
unfused branch asserted that each chain node gets its own Task, which is
exactly what inline execution removes, so both tests now discriminate on
the dedicated chain Task versus the ambient one.
@LawlAoux
LawlAoux force-pushed the CORE-45-inline-eager-async-node-execution branch from 05a6a9b to 0fe81b3 Compare September 22, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant