Skip to content

O(N²) step-cache rebuild degrades high-step-count workflows #715

Description

@kunaldevxxx

Background

StepHistory maintains a StepAttemptCache (typed as ReadonlyMap<string, StepAttempt>) that memoizes completed step results. On every step completion, addToStepAttemptCache at core/step-attempt.ts:122 is called.

Problem

The implementation:
return new Map([...cache, [attempt.stepName, attempt]])

spreads the entire existing map into a new Map on every call. For a workflow with N steps, the total entries allocated across all cache copies is 1 + 2 + 3 + … + N = N(N+1)/2. At the 1000-step limit this is ~500,000 map-entry allocations per workflow execution — purely for cache management.

Proposed Solution

Replace StepAttemptCache with a plain mutable Map<string, StepAttempt> inside StepHistory. The ReadonlyMap type annotation is a compile-time contract — callers never mutate it directly. Using a mutable internal map with mutation encapsulated inside StepHistory methods provides identical safety guarantees.

Change recordCompletion to this.cache.set(attempt.stepName, attempt). Remove addToStepAttemptCache or convert it to a void mutation method. Update findCached to read from the mutable map.

Complexity

Current: O(N²) total cache management. Target: O(N) total.

Acceptance Criteria

Benchmark: workflow with 1000 steps runs at least 2× faster (wall-clock) compared to main.
All existing step-history and execution tests pass.
Cache immutability invariant preserved: no external code can mutate the cache directly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions