Skip to content

Research digest: the plan-as-data bet is uncontested in Ruby, and roast just vacated the lane #23

Description

@minerva-sky

Survey of agentic's nearest neighbors: the Ruby agent gems (langchainrb, regent, sublayer), the Ruby LLM-layer gems (ruby_llm, raix), the closest conceptual competitor (Shopify's roast), the Rails one (active_agent), and the plan-and-execute frontier outside Ruby (LangGraph, OpenAI Agents SDK). Feature-level comparison against what this gem ships today, fenced by WORLD.md. Three findings, five integration candidates, four recorded non-candidates.

Finding 1: the plan-as-data bet is uncontested, and roast just vacated it

No Ruby framework ships LLM-generated inspectable plans, dependency-graph orchestration, durable journaling, or learning from execution history. langchainrb's Assistant is a conversational tool loop; regent is ReAct; sublayer is single-call generators plus triggers. The closest thing to competition was roast v0.x, whose killer feature was exactly ours: "every workflow execution automatically saved, resume from any step."

Then roast v1.0 (2026-02) rewrote workflows from YAML into a pure Ruby DSL and durable workflow replay did not survive — code search on the current repo returns zero hits for it (releases). Their plans are now code: a human can edit them, but no runtime or LLM can inspect, mutate, or replan one mid-flight. That is the precise gap WORLD.md's Direction ("the plan is the product") targets, and the best-funded neighbor just walked away from it. The lane is open.

Finding 2: the ecosystem's failure mode is provider-parity decay, and the small-core charter is the moat

Of the three Ruby agent frameworks, two are dormant (regent: no commits ~14 months; sublayer: same) and the third can't cut a release (langchainrb: active commits, but the last gem shipped 2025-05, with an open issue literally titled "Is this repo still maintained?"). The dominant open-issue theme across all three is provider-parity gaps: an 11-provider feature matrix is a treadmill that eats maintainers.

New evidence for the already-open provider question (#22, not re-raised here): raix v2.0 deleted its own provider layer and now delegates entirely to ruby_llm (changelog). Its 2.0.2–2.0.6 patch run (dropped multimodal transcripts, unenforced tool budgets, lost schema richness) is a free checklist of what breaks at a translation boundary. The Ruby ecosystem is consolidating on ruby_llm as the provider abstraction. That strengthens #22's option 3 and argues even harder against a third homegrown multi-provider matrix.

Finding 3: the frontier invests in cheap recovery, not better initial plans

LangGraph's canonical plan-and-execute is an execute→replan loop; its checkpoints support rewind, human plan-editing, and forked branches (interrupts). OpenAI's Agents SDK makes paused runs a serializable RunState that resumes with full history (HITL docs). The durable-execution camp's critique of both (diagrid): checkpoints that die with the process aren't durability. Agentic's ExecutionJournal already survives the process; what's missing is everything downstream of that fact (#21). The bet is right. The payoff is unclaimed.

Integration candidates (patterns to borrow, not dependencies)

C1: Replan-on-failure (LangGraph pattern). On task failure, feed the plan plus journal-so-far back to the LLM and regenerate the remaining plan as a new plan version. Because agentic's plans are data, this is possible here and structurally impossible in roast — the highest-leverage differentiator found. Fence: squarely Direction ("better failure recovery"); depends on graph-capable plan serialization (#20). Medium cost: plan versioning plus one orchestrator hook.

C2: Per-task failure-tolerance flags (roast pattern). continue_on_failure, tolerated-error predicates, and a succeeded?(task) accessor downstream tasks can branch on. Roast proves the Ruby ergonomics (no_abort_on_failure!). Fence: small orchestrator/task-schema change, no new deps; pairs naturally with #10's stranded-dependents fix. Cheap.

C3: Serializable paused-run state (OpenAI RunState pattern). Make "paused awaiting human approval or plan edit" a first-class persistable orchestrator state, so edit-the-plan-then-resume becomes the workflow. Extends #21's resume from crash recovery to deliberate human-in-the-loop. Fence: builds on the journal and store we already have; the diagrid survive-process-death critique is the acceptance test. Medium cost.

C4: Span read-model over the journal (regent pattern). Group journal entries into typed spans (llm_call, tool_call, task) with per-span tokens and duration, rendered hierarchically. Regent's session tracing was its one standout feature. Fence: Direction ("better execution history"); read-model only, no journal format change required. Moderate cost.

C5: Task-execution callback (langchainrb pattern). One documented hook on the orchestrator's task boundary buys user-wired tracing, redaction, and test observation without any abstraction. raix's before_completion is the same idea at the LLM boundary. Near-free.

Non-candidates (recorded so future loops don't re-raise them)

  • Multi-provider feature matrix: the documented cause of death for half this survey; Provider strategy: the researched proposal WORLD.md asks for (measured coupling, three options, one lean) #22 owns the actual decision.
  • Trigger daemons / hosted gravity (sublayer): an anti-goal (not a hosted platform), and speculative until a concrete use exists.
  • Rails MVC agent layer (active_agent): different category; complementary (they do request/response, we do planning/execution), not something to absorb.
  • MCP client (raix): integration-store gravity; capabilities keep the core small. Revisit only if a real consumer asks.

Origin: loop:research session 2026-08-23. L0 analysis only; nothing here lands without its own proposal. Full vitals: ruby_llm 1.16.0 (~11.5M downloads), raix 2.0.6, langchainrb 0.19.5, regent 0.3.4, sublayer 0.2.9, roast-ai 1.2.0 (~300K downloads), active_agent 1.3.1.

Sources: Shopify/roast · crmne/ruby_llm · OlympiaAI/raix · patterns-ai-core/langchainrb · alchaplinsky/regent · sublayerapp/sublayer · activeagents/activeagent · LangGraph interrupts · OpenAI Agents SDK HITL · diagrid on durable execution

Metadata

Metadata

Assignees

No one assigned

    Labels

    loop:researchCompetitive / related-project research loopstatus:analyzedAnalyzed, awaiting decision

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions