Skip to content

feat(config): add arrayFunctionResultOverwritesData opt-in overwrite flag (HF-305) - #1714

Open
marcin-kordas-hoc wants to merge 11 commits into
developfrom
feat/hf-305-overwrite-flag
Open

marcin-kordas-hoc wants to merge 11 commits into
developfrom
feat/hf-305-overwrite-flag

Conversation

@marcin-kordas-hoc

@marcin-kordas-hoc marcin-kordas-hoc commented Jul 22, 2026 •

Copy link
Copy Markdown
Collaborator

Context

Adds the opt-in configuration option arrayFunctionResultOverwritesData (default false). When it is true, an array whose result spills onto occupied cells overwrites them instead of returning #SPILL!. When it is false, spills behave as before.

The overwrite reuses the step that places every spill: when the array formula is placed, it claims its whole spill range in the dependency graph, drops the occupants, and records their old values so the column index forgets them. A spill that would reach another array keeps #SPILL! and leaves that array intact, whatever the option says.

undo() restores the overwritten cells:

  • entering an array formula with setCellContents snapshots the occupants before the spill;
  • adding rows or columns inside the range an array reads makes the array grow; the cells it reaches are snapshotted, mapped back to their coordinates and references from before the added span (a formula =E1*2 comes back as =E1*2, not =F1*2), and written back right after undo removes the span.

The option does not overwrite in two places, on purpose:

  • data declared in the same buildFromArray call as the array: the build path keeps #SPILL!, because claiming cells there would leave nothing to undo;
  • moveRows and moveColumns: a move adds the moved rows or columns before it removes them, so an array that reads them grows for a moment. Overwriting in that moment cleared the cell next to the spill for good, even when the array ended at its old size. A move that makes an array grow now leaves the occupant in place and the array shows #SPILL!.

Also fixed here, because the option made it reachable more often: undoing one setCellContents call that enters an array formula and also writes a cell of its spill range brought the array formula back. This happened with the option off too. Undo now reverts the writes of one call in reverse order. It has its own CHANGELOG line under Fixed.

How did you test your changes?

Paired tests: handsontable/hyperformula-tests#45, branch feat/hf-305-overwrite-flag.

  • unit/array-function-result-overwrites-data.spec.ts: 74 tests (configuration, the default, overwriting, array-vs-array collisions, undo and redo, undo of one call that writes an array and a cell of its spill, undo of added rows and columns, column index, structural operations including row moves, cycles, unrepresentable spill ranges).
  • unit/undo-redo.spec.ts: one new test, with the option off, for undoing one call that writes an array formula and a cell of its spill range.
  • The new undo tests fail on the previous head: 10 failures. The row-move tests fail when the move does not suspend overwriting: 3 failures. Dropping the coordinate and reference mapping of the snapshot fails 2 tests.
  • Full local Jest run on this head with the paired tests: 521 suites passed, 6817 tests passed, 3 skipped.

Compatibility with Excel and Google Sheets

Not applicable to the default: with the option off, calculation results do not change. The option is opt-in and changes only what happens to cells an array spills onto.

Public API changes

  • New configuration option arrayFunctionResultOverwritesData: boolean, default false.
  • undo() of one setCellContents call reverts its writes in reverse order. The result differs only when a later write in the call lands in the spill range of an array an earlier write entered.

Out of scope and follow-ups

  • Overwriting during moveRows and moveColumns, with undo, is not implemented: moves never overwrite. No follow-up ticket yet.
  • Already on develop with the option off, not changed here: an array that shows #SPILL! keeps it after removing a column, or an undo, gives it room again; and undoing a row move inside the range an array reads changes that range (=TRANSPOSE(A3:A5) becomes =TRANSPOSE(A4:A5)). No follow-up ticket yet.

Types of changes

  • Breaking change (existing functionality no longer works as before) – agreed with the maintainers
  • New feature or improvement (a non-breaking change that adds functionality)
  • Bug fix (a non-breaking change that fixes an issue)
  • Additional language file, or a change to an existing language file (translations)
  • Change to the documentation

Related issues

  1. HF-305. No public issue.

Checklist


Note

Medium Risk
Opt-in destructive cell clearing on array spill affects dependency graph, evaluation, structural ops, and undo; default false preserves Excel-compatible behavior.

Overview
Adds an opt-in engine flag arrayFunctionResultOverwritesData (default false). When enabled, array formulas that would normally hit #SPILL! on occupied non-array cells clear those occupants and spill instead; collisions with another array still return #SPILL!.

DependencyGraph wires the config in, gates overwrite via canOverwriteArrayResult / isOverwritingDataAllowed, suspends overwriting during row/column moves (withoutOverwritingData), and records content changes for cleared spill cells so the column index stays consistent. Initial buildFromArray placement is unchanged—occupants declared in the same build still win.

Operations snapshots spill victims before placing formulas, returns them from setCellContent and from addRows / addColumns when arrays grow over static data, and updates column search on formula restore.

Undo/redo restores snapshotted cells after reverting anchors or undoing row/column inserts that triggered growth overwrites; setCellContents undo order is fixed so a formula plus writes into its spill range no longer leaves a stray array.

Docs and changelog document the option and the “Spilling onto occupied cells” behavior.

Reviewed by Cursor Bugbot for commit 7aecb9f. Bugbot is set up for automated code reviews on this repo. Configure here.

@qunabu

qunabu commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

@netlify

netlify Bot commented Jul 22, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for hyperformula-dev-docs ready!

Name Link
🔨 Latest commit 467c186
🔍 Latest deploy log https://app.netlify.com/projects/hyperformula-dev-docs/deploys/6a6602deeedd020008331aa9
😎 Deploy Preview https://deploy-preview-1714--hyperformula-dev-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch from 9009463 to b2da92e Compare July 22, 2026 16:18
@github-actions

github-actions Bot commented Jul 22, 2026 •

Copy link
Copy Markdown

Performance comparison of head (7aecb9f) vs base (d143456)

                                     testName |    base |   head |  change
--------------------------------------------------------------------------
                                      Sheet A |  330.28 | 332.44 |  +0.65%
                                      Sheet B |   108.1 | 104.88 |  -2.98%
                                      Sheet T |   98.48 |  94.09 |  -4.46%
                                Column ranges |  486.79 | 501.55 |  +3.03%
                                Sorted lookup | 15813.4 |  17468 | +10.46%
Sheet A:  change value, add/remove row/column |   10.61 |   9.71 |  -8.48%
 Sheet B: change value, add/remove row/column |    94.2 |  84.93 |  -9.84%
                   Column ranges - add column |  122.62 |  107.8 | -12.09%
                Column ranges - without batch |  385.34 | 357.73 |  -7.17%
                        Column ranges - batch |   97.22 |     91 |  -6.40%

@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread hyperformula/src/DependencyGraph/DependencyGraph.ts
@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch 2 times, most recently from 4c02791 to d2a2671 Compare July 23, 2026 03:53
@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread hyperformula/src/Operations.ts
Comment thread src/DependencyGraph/DependencyGraph.ts Outdated
@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch from d2a2671 to 5e397d1 Compare July 24, 2026 11:14
@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

bugbot run

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@marcin-kordas-hoc
marcin-kordas-hoc marked this pull request as ready for review July 25, 2026 10:58
@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch from 5e397d1 to 2f48fba Compare July 26, 2026 02:31
Comment thread hyperformula/src/UndoRedo.ts
@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch from 2f48fba to b877ed4 Compare July 26, 2026 02:43
@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread hyperformula/src/DependencyGraph/DependencyGraph.ts
Comment thread hyperformula/src/Operations.ts
@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch from b877ed4 to 467c186 Compare July 26, 2026 12:51
@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

bugbot run

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 28, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
hyperformula-docs 7aecb9f Commit Preview URL

Branch Preview URL
Oct 23 2026, 07:27 AM

marcin-kordas-hoc and others added 3 commits August 28, 2026 06:23
…flag (HF-305)

Adds a public boolean Config option `arrayFunctionResultOverwritesData`
(default `false`). When `false`, behavior is unchanged: an array spill
onto an occupied cell yields `#SPILL!` and leaves the occupant intact.
When `true`, the spill overwrites the occupied cells (clears occupants,
spills the array, reroutes dependents) instead of emitting `#SPILL!`.

Implementation reuses the existing spill-placement exchange primitive:
- ConfigParams/Config: new option, mirrored on `useArrayArithmetic`.
- DependencyGraph gains `config` + two guarded early-returns in
  exchangeOrAddFormulaVertex / setAddressMappingForArrayVertex.
- Evaluator: recompute no longer emits `#SPILL!` when overwrite is allowed.

Array-vs-array safety: a spill colliding with ANOTHER array always keeps
`#SPILL!` and leaves that array intact, even in overwrite mode
(canOverwriteArrayResult / overwriteWouldHitArray). This matches Excel and
avoids corrupting a pre-existing array; overwrite only clears static data.

Default `false` guarantees no existing embedder loses data silently.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 53 cases now live in hyperformula-tests on the same-name branch
(feat/hf-305-overwrite-flag), which is what test/fetch-tests.sh pairs on. They
arrived here as test/hf-305-overwrite.spec.ts, and #1612 took every spec out of
this repository: on develop, test/ carries only the setup files, fetch-tests.sh
and smoke.spec.ts. Jest's testMatch (test/**/*spec.(ts|js)) did pick the file up,
so the suite was running - it was running from the wrong repository, where it is
invisible to reviewers of the tests repo and ships in the public tree.

Coverage is not lost by the move: measured against this commit, every executable
line this PR adds to Operations.ts (29) and DependencyGraph.ts (17) is exercised
by the ported spec alone, which is what the red codecov/patch was about.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rebase onto develop surfaced this: the entry was written when the branch's
base still had 3.4.0 open, so replaying it landed the line inside the released
3.4.0 section. Moved to [Unreleased], and given the PR link every neighbouring
entry carries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@marcin-kordas-hoc
marcin-kordas-hoc force-pushed the feat/hf-305-overwrite-flag branch from f051ab3 to 63e2137 Compare August 28, 2026 06:27
Comment thread hyperformula/src/Operations.ts
The overwrite snapshot guarded on `width * height <= 1`, but placement
(FormulaVertex.fromAst) treats every ref-flavored size as a non-spilling
scalar via ArraySize.isScalar(). A bare range formula such as =C1:C3
therefore snapshotted neighbours it never overwrites, and undoing that
edit spuriously restored them (re-entering formulas, re-rolling
volatiles). Guard on size.isScalar() so the snapshot scope matches the
placement scope, as the JSDoc already described.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
marcin-kordas-hoc added a commit that referenced this pull request Aug 28, 2026
This morning's develop merge landed the TEXT/numberFormat entry inside
the already-published 3.4.0 Fixed list, putting CHANGELOG.md out of sync
with docs/guide/release-notes.md for that released version. The fix is
unreleased, so the entry belongs under [Unreleased]; the 3.4.0 lists in
the two files match again. Same correction as 63e2137 on #1714.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

Known-gap disclosure from writing the paired suite (tests #45), re-verified today on this head (3e2a099): with the flag ON, an array whose spill range cannot be represented silently loses #SPILL! and truncates to its anchor.

Repro: useArrayArithmetic: true, =2*(B:B) in A2 — flag OFF: #SPILL! (pinned by arrays.spec.ts); flag ON: A2 = 2, no error indicator anywhere. Same silent truncation on the buildFromArray / updateConfig path.

Root cause: overwriteWouldHitArray returns false for an undefined range, so canOverwriteArrayResult returns true and Evaluator skips setNoSpace(). Overwrite mode should bypass occupancy collisions only, not a range that cannot be spanned at all — suggested one-line fix: canOverwriteArrayResult returns false when arrayVertex.getRangeOrUndef() is undefined.

The paired suite deliberately pins only the stable half (occupants below are not clobbered), so it stays green whichever way this is resolved.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread hyperformula/src/DependencyGraph/DependencyGraph.ts Outdated
…ill ranges

The overwrite feature resolves a spill collision when the array formula is
placed: exchangeOrAddFormulaVertex claims the whole spill range in the
address mapping before evaluation, so isThereSpaceForArray is already true
when the array is recomputed. The canOverwriteArrayResult bypass added to
the Evaluator (and to the build-path setAddressMappingForArrayVertex) could
therefore only ever fire for arrays whose overwrite never happened - an
array blocked by an occupant at buildFromArray/updateConfig time, or an
array whose spill range cannot be represented on the sheet (e.g. =2*(B:B)
anchored outside row 1, where getRangeOrUndef() is undefined and
overwriteWouldHitArray vacuously returns false). In both cases the array
silently truncated to its anchor value instead of yielding #SPILL!.

Remove the bypass from both sites so an array that does not own its spill
range keeps #SPILL! exactly like with the flag off, and make
canOverwriteArrayResult itself refuse unrepresentable ranges - overwrite
mode bypasses occupancy collisions only, never a range that cannot be
spanned at all.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

Fixed in e410ed0f2 (tests 1116587, +4 cases → 60). The operative insight: every legitimate overwrite claims its spill range in the address mapping (exchangeOrAddFormulaVertex) before evaluation, so isThereSpaceForArray is already true when the array recomputes — the canOverwriteArrayResult bypass in Evaluator could only ever fire for arrays whose overwrite never happened, which is exactly the two silent-truncation paths. The bypass is removed; canOverwriteArrayResult additionally refuses an unrepresentable range as a defensive guard, and the build path keeps #SPILL! under the flag exactly as with it off. Fail-then-pass: the 3 new error assertions fail on the previous head, 60/60 after; changelog clause added. Verified independently on the pushed head: =2*(B:B) and the build-time collision both show #SPILL! with the flag on.

@Tobiadefami Tobiadefami left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed the structural-undo issue noted in the description. With overwrite enabled, A1 contains =TRANSPOSE(D1:E1), D1:E1 contains 10, 20, and A3 contains 99.

State Formula in A1 A1 A2 A3
Calculated sheet before insertion =TRANSPOSE(D1:E1) 10 20 99
After inserting a column before E =TRANSPOSE(D1:F1) 10 blank 20
After undo =TRANSPOSE(D1:E1) 10 20 blank — expected 99

Could we preserve and restore the overwritten cells in the column-insertion undo path?

marcin-kordas-hoc and others added 2 commits October 22, 2026 17:35
…shorten the changelog entry

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread hyperformula/src/UndoRedo.ts
marcin-kordas-hoc and others added 3 commits October 22, 2026 17:52
… re-checking it

canOverwriteArrayResult already returns false for an unrepresentable range, so
the second undefined check in overwriteWouldHitArray could never be reached.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ns are undone

With arrayFunctionResultOverwritesData on, an array that grows because rows or
columns were added inside the range it reads overwrites the cells it reaches.
Undo now restores them: the add operation snapshots those cells, maps them back
to the coordinates and references from before the added span, and undo writes
them back right after removing the span.

Undo of one setCellContents call now reverts its writes in reverse order, so a
write into the spill range of an array entered earlier in the same call no
longer brings that array formula back. This also applies with the option off.

Row and column moves no longer overwrite data: a move adds the moved rows or
columns before it removes them, and the array that grew for that moment cleared
cells it does not cover once the move ends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cupied cells

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7aecb9f. Configure here.

*/
private restoreCellsOverwrittenBySpan(overwrittenCells: [SimpleCellAddress, ClipboardCell][] = []) {
for (const [address, clipboardCell] of overwrittenCells) {
this.operations.restoreCell(address, clipboardCell)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Undo leaves stale column-index values

Medium Severity

Undoing addRows or addColumns that grew an array over a cell restores the occupant, but the vacated spill value can stay in the column index. With useColumnIndex on, MATCH and VLOOKUP can still hit that spill after undo().

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 7aecb9f. Configure here.

@codecov

codecov Bot commented Oct 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.13043% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 97.30%. Comparing base (d143456) to head (7aecb9f).

Files with missing lines Patch % Lines
hyperformula/src/Operations.ts 98.55% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1714      +/-   ##
===========================================
+ Coverage    97.28%   97.30%   +0.02%     
===========================================
  Files          207      207              
  Lines        16218    16304      +86     
  Branches      3556     3610      +54     
===========================================
+ Hits         15777    15865      +88     
+ Misses         433      432       -1     
+ Partials         8        7       -1     
Files with missing lines Coverage Δ
hyperformula/src/Config.ts 94.69% <100.00%> (+0.04%) ⬆️
hyperformula/src/CrudOperations.ts 99.20% <100.00%> (+<0.01%) ⬆️
...yperformula/src/DependencyGraph/DependencyGraph.ts 98.84% <100.00%> (+0.03%) ⬆️
hyperformula/src/Evaluator.ts 100.00% <ø> (ø)
hyperformula/src/UndoRedo.ts 100.00% <100.00%> (ø)
hyperformula/src/Operations.ts 98.86% <98.55%> (-0.08%) ⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants