Skip to content

Share a single cache for all type relationships and propagate results hierarchically - #64808

Open
Daniel Rosenwasser (DanielRosenwasser) wants to merge 4 commits into
mainfrom
sharedRelationCache
Open

Share a single cache for all type relationships and propagate results hierarchically#64808
Daniel Rosenwasser (DanielRosenwasser) wants to merge 4 commits into
mainfrom
sharedRelationCache

Conversation

@DanielRosenwasser

@DanielRosenwasser Daniel Rosenwasser (DanielRosenwasser) commented Oct 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

This change brings back an optimization I was really hoping to explore a while back, but I implemented it in a slightly naive way in the Strada codebase. The inspiration is hierarchical caching, but it is now applied in a manner where cache results are all packed into the same uint32 in the same shared map.

Today we have a separate cache for every type relationship. This keeps things clean, but it comes with a few downsides. One is that there is some amount of overlap between the caches. For example, when I had an agent analyze the overlap between caches, it found the following results.

VS Code cache overlap analysis

Ranking all ten pairs on the VS Code src workload, using Jaccard overlap (shared keys / unique keys across both caches) as the primary metric:

Rank Relation pair Shared keys Jaccard Left cache covered Right cache covered Potential entry reduction Same result
1 subtype / strict-subtype 90,772 20.178% 32.655% 34.559% 16.790% 99.633%
2 subtype / assignable 28,182 2.065% 10.138% 2.527% 2.023% 88.979%
3 strict-subtype / assignable 15,583 1.144% 5.933% 1.397% 1.131% 91.747%
4 subtype / comparable 3,519 0.897% 1.266% 2.981% 0.889% 96.135%
5 assignable / comparable 7,856 0.641% 0.704% 6.655% 0.637% 97.123%
6 strict-subtype / comparable 2,364 0.625% 0.900% 2.003% 0.621% 91.497%
7 subtype / identity 162 0.051% 0.058% 0.378% 0.050% 70.370%
8 strict-subtype / identity 94 0.031% 0.036% 0.219% 0.031% 81.915%
9 assignable / identity 172 0.015% 0.015% 0.402% 0.015% 64.535%
10 comparable / identity 21 0.013% 0.018% 0.049% 0.013% 28.571%

“Potential entry reduction” is shared / (left size + right size): the percentage of existing map entries eliminated if the two maps shared one key while retaining separate relation results.

Ranking by raw duplicate count

If you care only about the absolute number of duplicated keys rather than normalized similarity, the order is:

  1. subtype / strict-subtype — 90,772
  2. subtype / assignable — 28,182
  3. strict-subtype / assignable — 15,583
  4. assignable / comparable — 7,856
  5. subtype / comparable — 3,519
  6. strict-subtype / comparable — 2,364
  7. assignable / identity — 172
  8. subtype / identity — 162
  9. strict-subtype / identity — 94
  10. comparable / identity — 21

The practical conclusion is that subtype/strict-subtype is the only major overlap pair. Subtype/assignable is a distant second, and every other normalized overlap is below 1%. Identity comparisons are additionally imperfect here because identity keys normalize source/target order, while the other relations are directional.

The other downside to these distinct caches is that there's additional bookkeeping and waste for each separate type relationship - every result takes up 3 more bytes than is required to represent the actual comparison result. In fact, each result only needs 6 bits to represent the result, and we only have 5 type relationships: comparable, assignable, subtype, strict-subtype, and identity.

With this change, we use a single shared cache for every type relationship. Cache results share the same underlying uint32, and type relationships keep track of where in the uint32 they live. Stricter cache values live in the lower bits (i.e. identity starts at the 0th bit), while weaker cache values live in the higher bits.

Additionally, cached positive results are proactively propagated for weaker type relationships, while cached negative results are propagated for stronger type relationships. This propagation is cheap to propagate and requires no extra memory if no other type relationship is ever interested in the result.

For example, let's say we have types Foo and Bar. If we check whether Foo is comparable to Bar, the result will be cached. If the result is negative, we can propagate that information to stronger type relationships. We know that if something is not comparable, it is not assignable, nor a subtype, nor a strict subtype, and certainly not identical.

I tested a few different variations of this. One of them tried to limit sharing between only the subtype and strict subtype relationships. That was good, but I could restore the special case there; but the current implementation has some more optimizations that make the code cleaner and still more efficient overall. Turning off propagation entirely resulted in slightly better perf (0.4% faster), while turning it back on was slightly more memory efficient (0.6% less memory), but both of these amounts technically fell in the noise threshold.

But full propagation between all type relationships also created fewer objects in the checker on VS Code: 124 fewer symbols, 98 fewer types, 2976 fewer instantiations.

In compiling VS Code with --noEmit --checkers 1, this nets a 1.2% speedup plus a 35MiB reduction in memory usage. When I ran on multiple checkers, here's what I got:

Checkers Variant Wall time Peak RSS Change
8 Before 11.024 ± 0.146 s 7251.1 ± 148.8 MiB —
8 Shared packed 10.863 ± 0.090 s 7263.2 ± 156.1 MiB 1.5% faster, 12.1 MiB heavier
16 Before 7.836 ± 0.093 s 8292.1 ± 175.6 MiB —
16 Shared packed 7.873 ± 0.132 s 8205.8 ± 104.0 MiB 0.5% slower, 86.2 MiB leaner

I might still want to do some refactoring, but we might as well see how this goes in a perf run.

@github-project-automation github-project-automation Bot moved this to Not started in PR Backlog Oct 14, 2026
Copilot AI balanced review requested due to automatic review settings October 14, 2026 00:05
@typescript-automation typescript-automation Bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Oct 14, 2026
@DanielRosenwasser

Daniel Rosenwasser (DanielRosenwasser) commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) test top1000
TypeScript Bot (@typescript-bot) test dt
TypeScript Bot (@typescript-bot) perf test this

@typescript-automation

typescript-automation Bot commented Oct 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top1000 ✅ Started ✅ Results
perf test this ✅ Started 👀 Results

@DanielRosenwasser Daniel Rosenwasser (DanielRosenwasser) changed the title Share a single relationship checking cache, propagate results hierarchically Share a single cache for all type relationships and propagate results hierarchically Oct 14, 2026
Copilot AI reviewed Oct 14, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Directional failure propagation can create unreachable identity entries that incorrectly consume the identity comparison budget.

1 open finding
What changed in this PR

Shares and packs checker relationship caches, with hierarchical result propagation to reduce memory and redundant comparisons.

Changes:

  • Packs five relationship results into one shared cache entry.
  • Propagates successful and failed relationship implications.
  • Adds unit tests and a storage benchmark.
File Description
tsc/​internal/​checker/​relater.go Implements packed shared caching and propagation.
tsc/​internal/​checker/​relater_test.go Tests packing, propagation, precedence, and storage.
tsc/​internal/​checker/​checker.go Initializes the shared relationship cache.

🧠 Review effort: Balanced

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread tsc/internal/checker/relater.go Outdated
Comment on lines +154 to +157
if result&RelationComparisonResultFailed != 0 && result&RelationComparisonResultOverflow == 0 {
for implied, shift := shiftedResult>>packedRelationResultBits, int(r.packedShift)-packedRelationResultBits; implied != 0; implied, shift = implied>>packedRelationResultBits, shift-packedRelationResultBits {
packed = r.packedResults.setPackedResult(packed, implied, uint8(shift), true)
}

@DanielRosenwasser Daniel Rosenwasser (DanielRosenwasser) Oct 14, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is pretty annoying, and all of this does make it harder to report results. I'm not sure how accurately address this in a performant way other than distinguishing between filled-in entries and lookups that initially had no value but caused work to actually occur. I would actually say the latter is more what we're interested in in the checker anyway though.

@DanielRosenwasser Daniel Rosenwasser (DanielRosenwasser) Oct 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, what we could do is not propagate until we actually care. Basically

  1. Fetch a cache entry - if it's 0, nothing's available. Relate the types, fill in the blank, update the count.
  2. Check your own flags. Stop if you've got anything.
  3. If not, something else has checked the current types.
  4. Clear all bits below, and while bits are set, shift right by 6, check if the lower 6 are set, and if so check for a (usable!) false result.
    1. If one is set, propagate.
    2. Otherwise do work
    3. Update the count
  5. Basically do the right corresponding thing for positive results in the lower bits.

With this we won't update counts unnecessarily, but we will sort of "miscount" how much work happens. But I think that is okay.

We could also just keep the values completely disjoint when fetching and avoid this issue.

@typescript-automation

typescript-automation Bot commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Daniel Rosenwasser (@DanielRosenwasser)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - native
Errors 41 41 ~ ~ ~ p=1.000 n=12
Symbols 115,276 115,276 ~ ~ ~ p=1.000 n=12
Types 79,613 79,613 ~ ~ ~ p=1.000 n=12
Memory Used 133,217k (± 0.05%) 130,416k (± 0.07%) -2,801k (- 2.10%) 130,207k 130,609k p=0.000 n=12
Memory Allocs 2,190,299 (± 0.01%) 2,188,567 (± 0.01%) -1,732 (- 0.08%) 2,188,142 2,188,969 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.051s (± 3.91%) 0.050s (± 3.46%) ~ 0.046s 0.054s p=0.366 n=12
Bind Time 0.017s (±13.97%) 0.015s (±12.17%) ~ 0.012s 0.021s p=0.398 n=12
Check Time 0.595s (± 0.81%) 0.595s (± 0.92%) ~ 0.580s 0.612s p=0.920 n=12
Emit Time 0.327s (± 3.39%) 0.324s (± 1.66%) ~ 0.306s 0.336s p=0.660 n=12
Total Time 0.998s (± 0.98%) 0.992s (± 0.49%) ~ 0.982s 1.005s p=0.271 n=12
angular-1 - native
Errors 3 3 ~ ~ ~ p=1.000 n=12
Symbols 846,527 (± 0.07%) 847,260 (± 0.06%) ~ 846,347 848,853 p=0.089 n=12
Types 247,032 (± 0.00%) 247,029 (± 0.00%) -3 (- 0.00%) 247,027 247,032 p=0.001 n=12
Memory Used 743,825k (± 0.01%) 743,726k (± 0.01%) ~ 743,460k 744,080k p=0.128 n=12
Memory Allocs 12,919,353 (± 0.22%) 12,898,261 (± 0.01%) -21,092 (- 0.16%) 12,895,751 12,901,939 p=0.001 n=12
Config Time 0.020s (± 1.52%) 0.020s (± 1.52%) ~ 0.020s 0.021s p=1.000 n=12
Parse Time 0.329s (± 2.26%) 0.317s (± 2.57%) 🟩-0.013s (- 3.85%) 0.286s 0.335s p=0.016 n=12
Bind Time 0.075s (± 7.63%) 0.078s (± 9.34%) ~ 0.071s 0.112s p=0.118 n=12
Check Time 0s 0s ~ ~ ~ p=1.000 n=12
Emit Time 1.836s (± 0.84%) 1.836s (± 0.62%) ~ 1.807s 1.871s p=0.639 n=12
Total Time 2.276s (± 0.81%) 2.268s (± 0.59%) ~ 2.241s 2.304s p=0.580 n=12
mui-docs - native
Errors 11,405 11,405 ~ ~ ~ p=1.000 n=12
Symbols 4,331,835 4,331,835 ~ ~ ~ p=1.000 n=12
Types 1,370,520 1,370,499 -21 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 2,771,325k (± 0.04%) 2,774,505k (± 0.06%) +3,180k (+ 0.11%) 2,770,009k 2,779,139k p=0.004 n=12
Memory Allocs 32,649,570 (± 0.04%) 32,663,683 (± 0.07%) ~ 32,623,006 32,754,922 p=0.478 n=12
Config Time 0.021s (± 1.37%) 0.021s (± 1.57%) ~ 0.020s 0.021s p=0.667 n=12
Parse Time 0.613s (± 3.09%) 0.612s (± 2.20%) ~ 0.564s 0.637s p=1.000 n=12
Bind Time 0.003s (±13.14%) 0.002s (±13.27%) ~ 0.002s 0.003s p=0.680 n=12
Check Time 9.652s (± 0.54%) 9.642s (± 0.33%) ~ 9.599s 9.759s p=0.766 n=12
Emit Time 0.597s (±12.55%) 0.548s (± 9.56%) ~ 0.501s 0.737s p=0.173 n=12
Total Time 11.844s (± 0.55%) 11.772s (± 0.55%) ~ 11.680s 11.984s p=0.178 n=12
strada-build-src - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,388,669 1,388,618 -51 (- 0.00%) ~ ~ p=0.000 n=12
Types 441,862 441,852 -10 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 1,603,276k (± 0.49%) 1,598,187k (± 0.65%) ~ 1,578,311k 1,627,934k p=0.319 n=12
Memory Allocs 82,425,050 (± 0.05%) 82,369,603 (± 0.09%) ~ 82,023,937 82,481,645 p=0.242 n=12
Config Time 0.004s (±10.30%) 0.004s (±16.90%) ~ 0.002s 0.005s p=0.639 n=12
Parse Time 0.319s (± 1.63%) 0.309s (± 1.72%) 🟩-0.010s (- 3.06%) 0.297s 0.324s p=0.014 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 2.145s (± 0.51%) 2.163s (± 0.68%) ~ 2.130s 2.202s p=0.086 n=12
Emit Time 0.333s (± 2.21%) 0.351s (± 5.24%) ~ 0.322s 0.392s p=0.147 n=12
Total Time 24.877s (± 0.67%) 25.174s (± 0.81%) +0.297s (+ 1.19%) 24.617s 25.604s p=0.020 n=12
strada-compiler - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 334,953 334,922 -31 (- 0.01%) ~ ~ p=0.000 n=12
Types 197,616 197,610 -6 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 296,048k (± 0.02%) 295,313k (± 0.02%) -736k (- 0.25%) 295,217k 295,448k p=0.000 n=12
Memory Allocs 4,621,127 (± 0.01%) 4,618,762 (± 0.01%) -2,365 (- 0.05%) 4,617,360 4,619,310 p=0.000 n=12
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=12
Parse Time 0.107s (± 3.84%) 0.115s (± 5.17%) 🔻+0.008s (+ 7.45%) 0.101s 0.132s p=0.043 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 1.032s (± 0.52%) 1.043s (± 0.43%) +0.012s (+ 1.13%) 1.032s 1.060s p=0.002 n=12
Emit Time 0.149s (± 4.49%) 0.143s (± 7.09%) ~ 0.114s 0.173s p=0.297 n=12
Total Time 1.341s (± 0.68%) 1.352s (± 1.08%) ~ 1.320s 1.394s p=0.385 n=12
ts-pre-modules - native
Errors 87 87 ~ ~ ~ p=1.000 n=12
Symbols 303,834 303,802 -32 (- 0.01%) ~ ~ p=0.000 n=12
Types 181,663 181,658 -5 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 256,435k (± 0.02%) 256,062k (± 0.03%) -373k (- 0.15%) 255,910k 256,263k p=0.000 n=12
Memory Allocs 1,603,613 (± 0.01%) 1,601,735 (± 0.02%) -1,878 (- 0.12%) 1,601,002 1,602,813 p=0.000 n=12
Config Time 0.000s (±217.90%) 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.098s (± 5.43%) 0.099s (± 5.36%) ~ 0.082s 0.111s p=0.723 n=12
Bind Time 0.040s (±11.82%) 0.036s (±10.00%) ~ 0.030s 0.048s p=0.110 n=12
Check Time 0.815s (± 0.82%) 0.822s (± 0.76%) ~ 0.804s 0.838s p=0.065 n=12
Emit Time 0.000s 0.000s (±217.90%) ~ 0.000s 0.001s p=1.000 n=12
Total Time 0.967s (± 0.89%) 0.971s (± 0.95%) ~ 0.944s 0.998s p=0.659 n=12
vscode - native
Errors 396 396 ~ ~ ~ p=1.000 n=12
Symbols 11,539,689 11,539,571 -118 (- 0.00%) ~ ~ p=0.000 n=12
Types 3,819,845 3,819,735 -110 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 6,579,737k (± 0.03%) 6,571,480k (± 0.01%) -8,256k (- 0.13%) 6,569,313k 6,573,758k p=0.000 n=12
Memory Allocs 56,588,004 (± 0.03%) 56,559,549 (± 0.04%) -28,455 (- 0.05%) 56,529,466 56,659,673 p=0.020 n=12
Config Time 0.075s (± 0.33%) 0.075s (± 0.49%) ~ 0.074s 0.076s p=0.230 n=12
Parse Time 1.891s (± 3.16%) 1.894s (± 3.19%) ~ 1.742s 2.009s p=0.768 n=12
Bind Time 0.598s (±20.66%) 0.577s (±16.32%) ~ 0.441s 0.840s p=0.619 n=12
Check Time 14.183s (± 1.18%) 14.206s (± 1.31%) ~ 13.769s 14.508s p=0.855 n=12
Emit Time 5.065s (±10.33%) 4.822s (± 9.84%) ~ 4.244s 6.102s p=0.713 n=12
Total Time 21.903s (± 1.91%) 21.661s (± 1.61%) ~ 21.223s 22.633s p=0.418 n=12
webpack - native
Errors 585 585 ~ ~ ~ p=1.000 n=12
Symbols 1,325,855 1,325,764 -91 (- 0.01%) ~ ~ p=0.000 n=12
Types 627,988 627,966 -22 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 951,187k (± 0.01%) 951,493k (± 0.01%) +306k (+ 0.03%) 951,181k 951,776k p=0.001 n=12
Memory Allocs 6,739,708 (± 0.03%) 6,734,918 (± 0.01%) -4,791 (- 0.07%) 6,731,978 6,737,703 p=0.000 n=12
Config Time 0.008s 0.008s ~ ~ ~ p=1.000 n=12
Parse Time 0.245s (± 2.28%) 0.241s (± 3.02%) ~ 0.227s 0.258s p=0.504 n=12
Bind Time 0.057s (± 5.82%) 0.068s (±19.04%) ~ 0.050s 0.112s p=0.722 n=12
Check Time 1.639s (± 0.59%) 1.633s (± 0.65%) ~ 1.612s 1.669s p=0.325 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.978s (± 0.41%) 1.979s (± 0.53%) ~ 1.944s 2.007s p=0.766 n=12
xstate-main - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,062,854 1,062,736 -118 (- 0.01%) ~ ~ p=0.000 n=12
Types 390,845 390,840 -5 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 548,104k (± 0.02%) 547,291k (± 0.02%) -813k (- 0.15%) 547,072k 547,700k p=0.000 n=12
Memory Allocs 4,893,594 (± 0.07%) 4,890,688 (± 0.06%) ~ 4,882,795 4,897,723 p=0.319 n=12
Config Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Parse Time 0.113s (± 2.72%) 0.115s (± 3.25%) ~ 0.105s 0.126s p=0.384 n=12
Bind Time 0.034s (±14.53%) 0.038s (±16.02%) ~ 0.026s 0.053s p=0.465 n=12
Check Time 0.859s (± 1.02%) 0.852s (± 0.66%) ~ 0.833s 0.862s p=0.701 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.017s (± 0.86%) 1.017s (± 0.98%) ~ 0.986s 1.034s p=0.560 n=12
System info unknown
Hosts
  • native
Scenarios
  • Compiler-Unions - native
  • angular-1 - native
  • mui-docs - native
  • strada-build-src - native
  • strada-compiler - native
  • ts-pre-modules - native
  • vscode - native
  • webpack - native
  • xstate-main - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

lsp

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsLSP - native
Req 1 - updateOpen 12ms (±12.45%) 11ms (±12.67%) ~ 9ms 15ms p=0.599 n=12
Req 2 - geterr 566ms (± 0.39%) 561ms (± 0.53%) -5ms (- 0.80%) 554ms 568ms p=0.015 n=12
Req 3 - references 42ms (± 4.90%) 43ms (± 6.85%) ~ 37ms 50ms p=0.578 n=12
Req 4 - navto 16ms (± 1.90%) 16ms (± 1.75%) ~ 16ms 17ms p=1.000 n=12
Req 5 - completionInfo count 1,363 1,363 ~ ~ ~ p=1.000 n=12
Req 5 - completionInfo 17ms (± 6.05%) 16ms (± 5.20%) ~ 14ms 18ms p=0.073 n=12
CompilerLSP - native
Req 1 - updateOpen 12ms (±12.94%) 11ms (± 8.95%) ~ 10ms 15ms p=0.227 n=12
Req 2 - geterr 297ms (± 0.63%) 297ms (± 0.67%) ~ 292ms 300ms p=0.808 n=12
Req 3 - references 48ms (± 2.60%) 50ms (± 3.45%) +2ms (+ 3.98%) 46ms 54ms p=0.046 n=12
Req 4 - navto 17ms (± 6.63%) 17ms (± 2.33%) ~ 16ms 18ms p=0.793 n=12
Req 5 - completionInfo count 1,525 1,525 ~ ~ ~ p=1.000 n=12
Req 5 - completionInfo 18ms (± 9.64%) 17ms (± 8.53%) ~ 15ms 22ms p=0.529 n=12
System info unknown
Hosts
  • native
Scenarios
  • CompilerLSP - native
  • Compiler-UnionsLSP - native
  • xstate-main-1-LSP - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
lsp-startup - native
Execution time 4.80ms (±365.09%) 4.82ms (±365.07%) +0.02ms (+ 0.33%) 0.00ms 483.93ms p=0.000 n=1200
tsgo-startup - native
Execution time 5.86ms (±365.10%) 5.86ms (±365.09%) +0.00ms (+ 0.03%) 0.00ms 593.17ms p=0.000 n=1200
System info unknown
Hosts
  • native
Scenarios
  • lsp-startup - native
  • tsgo-startup - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

Developer Information:

Download Benchmarks

@DanielRosenwasser
Daniel Rosenwasser (DanielRosenwasser) marked this pull request as ready for review October 14, 2026 00:41
@typescript-automation

typescript-automation Bot commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Daniel Rosenwasser (@DanielRosenwasser) Here are the results of running the top 1000 repos with tsc comparing baseline and pr:

Everything looks good!

@DanielRosenwasser

Daniel Rosenwasser (DanielRosenwasser) commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) perf test this

@typescript-automation

typescript-automation Bot commented Oct 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-automation

typescript-automation Bot commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Daniel Rosenwasser (@DanielRosenwasser)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - native
Errors 41 41 ~ ~ ~ p=1.000 n=12
Symbols 115,276 115,276 ~ ~ ~ p=1.000 n=12
Types 79,613 79,613 ~ ~ ~ p=1.000 n=12
Memory Used 132,752k (± 0.05%) 129,847k (± 0.05%) -2,905k (- 2.19%) 129,729k 130,032k p=0.000 n=12
Memory Allocs 2,092,426 (± 0.01%) 2,090,436 (± 0.01%) -1,990 (- 0.10%) 2,090,081 2,091,023 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.049s (± 2.68%) 0.048s (± 3.30%) ~ 0.045s 0.052s p=0.444 n=12
Bind Time 0.016s (±12.66%) 0.016s (±10.88%) ~ 0.013s 0.023s p=0.756 n=12
Check Time 0.592s (± 0.81%) 0.566s (± 0.80%) 🟩-0.026s (- 4.32%) 0.555s 0.576s p=0.000 n=12
Emit Time 0.329s (± 1.92%) 0.327s (± 2.93%) ~ 0.295s 0.349s p=0.832 n=12
Total Time 0.995s (± 0.70%) 0.965s (± 0.99%) -0.030s (- 2.97%) 0.931s 0.990s p=0.000 n=12
angular-1 - native
Errors 3 3 ~ ~ ~ p=1.000 n=12
Symbols 846,413 (± 0.10%) 847,077 (± 0.06%) ~ 845,557 848,569 p=0.114 n=12
Types 247,032 (± 0.00%) 247,031 (± 0.00%) -0 (- 0.00%) 247,028 247,041 p=0.038 n=12
Memory Used 741,364k (± 0.02%) 741,292k (± 0.02%) ~ 740,891k 741,677k p=0.590 n=12
Memory Allocs 12,665,932 (± 0.12%) 12,654,793 (± 0.02%) ~ 12,651,937 12,663,811 p=0.101 n=12
Config Time 0.020s (± 0.90%) 0.020s ~ ~ ~ p=1.000 n=12
Parse Time 0.315s (± 2.92%) 0.320s (± 2.47%) ~ 0.285s 0.334s p=0.271 n=12
Bind Time 0.083s (±16.09%) 0.079s (±13.99%) ~ 0.072s 0.134s p=0.946 n=12
Check Time 0s 0s ~ ~ ~ p=1.000 n=12
Emit Time 1.865s (± 1.35%) 1.869s (± 1.19%) ~ 1.830s 1.948s p=0.660 n=12
Total Time 2.300s (± 1.44%) 2.304s (± 1.17%) ~ 2.261s 2.390s p=0.370 n=12
mui-docs - native
Errors 11,405 11,405 ~ ~ ~ p=1.000 n=12
Symbols 4,331,835 4,331,835 ~ ~ ~ p=1.000 n=12
Types 1,370,520 1,370,499 -21 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 2,909,190k (± 0.06%) 2,912,402k (± 0.06%) +3,212k (+ 0.11%) 2,908,542k 2,917,299k p=0.020 n=12
Memory Allocs 31,249,624 (± 0.09%) 31,228,975 (± 0.08%) ~ 31,170,853 31,314,187 p=0.266 n=12
Config Time 0.020s (± 1.40%) 0.020s (± 1.40%) ~ 0.020s 0.021s p=1.000 n=12
Parse Time 0.623s (± 3.21%) 0.608s (± 2.80%) ~ 0.563s 0.646s p=0.311 n=12
Bind Time 0.003s (±13.14%) 0.002s (±13.14%) ~ 0.002s 0.003s p=1.000 n=12
Check Time 9.446s (± 0.29%) 9.488s (± 0.32%) +0.042s (+ 0.45%) 9.440s 9.617s p=0.043 n=12
Emit Time 0.539s (±10.04%) 0.509s (± 0.84%) ~ 0.500s 0.521s p=0.502 n=12
Total Time 11.577s (± 0.46%) 11.575s (± 0.36%) ~ 11.485s 11.736s p=0.944 n=12
strada-build-src - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,388,669 1,388,618 -51 (- 0.00%) ~ ~ p=0.000 n=12
Types 441,862 441,852 -10 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 1,601,192k (± 0.68%) 1,601,619k (± 0.62%) ~ 1,583,663k 1,627,878k p=0.932 n=12
Memory Allocs 78,816,150 (± 0.05%) 78,749,740 (± 0.10%) ~ 78,445,183 78,904,087 p=0.178 n=12
Config Time 0.004s (±20.12%) 0.004s (±16.85%) ~ 0.002s 0.006s p=0.673 n=12
Parse Time 0.308s (± 2.24%) 0.312s (± 1.71%) ~ 0.298s 0.324s p=0.504 n=12
Bind Time 0.000s 0.000s (±217.90%) ~ 0.000s 0.005s p=1.000 n=12
Check Time 2.134s (± 0.64%) 2.115s (± 0.46%) -0.019s (- 0.90%) 2.097s 2.141s p=0.032 n=12
Emit Time 0.353s (± 6.92%) 0.367s (± 5.68%) ~ 0.329s 0.421s p=0.225 n=12
Total Time 24.806s (± 1.00%) 24.783s (± 1.01%) ~ 24.269s 25.538s p=0.843 n=12
strada-compiler - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 334,953 334,922 -31 (- 0.01%) ~ ~ p=0.000 n=12
Types 197,616 197,610 -6 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 293,424k (± 0.03%) 292,728k (± 0.02%) -697k (- 0.24%) 292,563k 292,921k p=0.000 n=12
Memory Allocs 4,412,597 (± 0.01%) 4,410,371 (± 0.01%) -2,227 (- 0.05%) 4,408,894 4,411,159 p=0.000 n=12
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=12
Parse Time 0.115s (± 5.32%) 0.111s (± 4.13%) ~ 0.099s 0.125s p=0.182 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 1.021s (± 0.37%) 1.018s (± 0.47%) ~ 1.009s 1.031s p=0.296 n=12
Emit Time 0.144s (± 5.54%) 0.147s (± 6.49%) ~ 0.113s 0.168s p=0.485 n=12
Total Time 1.333s (± 0.73%) 1.331s (± 0.77%) ~ 1.301s 1.349s p=0.755 n=12
ts-pre-modules - native
Errors 87 87 ~ ~ ~ p=1.000 n=12
Symbols 303,834 303,802 -32 (- 0.01%) ~ ~ p=0.000 n=12
Types 181,663 181,658 -5 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 253,985k (± 0.02%) 253,562k (± 0.02%) -423k (- 0.17%) 253,431k 253,688k p=0.000 n=12
Memory Allocs 1,410,653 (± 0.01%) 1,408,743 (± 0.02%) -1,910 (- 0.14%) 1,408,062 1,410,209 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.097s (± 4.61%) 0.096s (± 5.10%) ~ 0.081s 0.105s p=0.744 n=12
Bind Time 0.035s (± 7.16%) 0.040s (±12.08%) ~ 0.027s 0.051s p=0.246 n=12
Check Time 0.809s (± 1.02%) 0.809s (± 0.99%) ~ 0.789s 0.825s p=0.921 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 0.957s (± 0.92%) 0.959s (± 0.93%) ~ 0.937s 0.979s p=0.810 n=12
vscode - native
Errors 396 396 ~ ~ ~ p=1.000 n=12
Symbols 11,608,806 11,608,628 -178 (- 0.00%) ~ ~ p=0.000 n=12
Types 3,842,875 3,842,784 -91 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 6,665,743k (± 0.02%) 6,666,591k (± 0.02%) ~ 6,663,921k 6,670,842k p=0.443 n=12
Memory Allocs 53,003,467 (± 0.03%) 52,968,991 (± 0.01%) -34,476 (- 0.07%) 52,957,226 53,003,803 p=0.000 n=12
Config Time 0.061s (± 0.40%) 0.061s (± 0.46%) ~ 0.061s 0.062s p=1.000 n=12
Parse Time 1.469s (± 3.80%) 1.498s (± 3.94%) ~ 1.372s 1.628s p=0.551 n=12
Bind Time 0.536s (±15.93%) 0.495s (±19.37%) ~ 0.374s 0.788s p=0.386 n=12
Check Time 11.416s (± 1.23%) 11.486s (± 1.41%) ~ 11.124s 11.760s p=0.671 n=12
Emit Time 4.430s (± 9.54%) 4.258s (± 8.95%) ~ 3.629s 5.224s p=0.619 n=12
Total Time 17.994s (± 1.79%) 17.871s (± 1.56%) ~ 17.408s 18.655s p=0.887 n=12
webpack - native
Errors 585 585 ~ ~ ~ p=1.000 n=12
Symbols 1,325,855 1,325,764 -91 (- 0.01%) ~ ~ p=0.000 n=12
Types 627,988 627,966 -22 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 943,782k (± 0.01%) 944,013k (± 0.01%) +231k (+ 0.02%) 943,702k 944,268k p=0.017 n=12
Memory Allocs 6,111,990 (± 0.02%) 6,108,870 (± 0.02%) -3,119 (- 0.05%) 6,106,146 6,112,240 p=0.001 n=12
Config Time 0.007s (± 3.42%) 0.007s (± 3.42%) ~ 0.007s 0.008s p=1.000 n=12
Parse Time 0.241s (± 3.58%) 0.238s (± 2.79%) ~ 0.215s 0.253s p=0.788 n=12
Bind Time 0.058s (± 8.31%) 0.072s (±19.08%) ~ 0.052s 0.115s p=0.190 n=12
Check Time 1.609s (± 0.30%) 1.608s (± 0.40%) ~ 1.592s 1.620s p=0.854 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.946s (± 0.33%) 1.956s (± 0.52%) ~ 1.928s 1.975s p=0.062 n=12
xstate-main - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,062,854 1,062,736 -118 (- 0.01%) ~ ~ p=0.000 n=12
Types 390,845 390,840 -5 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 550,380k (± 0.03%) 549,544k (± 0.02%) -835k (- 0.15%) 549,322k 550,027k p=0.000 n=12
Memory Allocs 4,507,148 (± 0.09%) 4,502,995 (± 0.08%) ~ 4,494,968 4,516,186 p=0.101 n=12
Config Time 0.003s (±10.34%) 0.003s (± 8.64%) ~ 0.002s 0.003s p=1.000 n=12
Parse Time 0.140s (± 3.71%) 0.139s (± 3.96%) ~ 0.129s 0.157s p=0.680 n=12
Bind Time 0.052s (±18.27%) 0.052s (±15.83%) ~ 0.033s 0.067s p=0.787 n=12
Check Time 0.999s (± 0.65%) 1.001s (± 0.42%) ~ 0.993s 1.013s p=0.400 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.202s (± 0.60%) 1.204s (± 0.49%) ~ 1.188s 1.218s p=0.966 n=12
System info unknown
Hosts
  • native
Scenarios
  • Compiler-Unions - native
  • angular-1 - native
  • mui-docs - native
  • strada-build-src - native
  • strada-compiler - native
  • ts-pre-modules - native
  • vscode - native
  • webpack - native
  • xstate-main - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

lsp

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsLSP - native
Req 1 - updateOpen 12ms (±14.84%) 11ms (±14.94%) ~ 9ms 17ms p=1.000 n=12
Req 2 - geterr 561ms (± 0.28%) 541ms (± 0.47%) 🟩-20ms (- 3.49%) 535ms 548ms p=0.000 n=12
Req 3 - references 43ms (± 5.72%) 45ms (± 5.72%) ~ 39ms 52ms p=0.161 n=12
Req 4 - navto 16ms (± 2.51%) 16ms (± 2.41%) ~ 15ms 17ms p=0.880 n=12
Req 5 - completionInfo count 1,363 1,363 ~ ~ ~ p=1.000 n=12
Req 5 - completionInfo 17ms (± 6.43%) 16ms (± 8.08%) ~ 14ms 21ms p=0.671 n=12
CompilerLSP - native
Req 1 - updateOpen 11ms (±11.21%) 12ms (±15.51%) ~ 10ms 18ms p=0.733 n=12
Req 2 - geterr 294ms (± 0.63%) 293ms (± 0.79%) ~ 286ms 300ms p=0.991 n=12
Req 3 - references 51ms (± 3.77%) 50ms (± 4.35%) ~ 44ms 54ms p=0.483 n=12
Req 4 - navto 17ms (± 4.64%) 18ms (± 6.59%) ~ 16ms 23ms p=0.641 n=12
Req 5 - completionInfo count 1,525 1,525 ~ ~ ~ p=1.000 n=12
Req 5 - completionInfo 16ms (± 5.45%) 17ms (± 6.54%) ~ 15ms 20ms p=0.240 n=12
System info unknown
Hosts
  • native
Scenarios
  • CompilerLSP - native
  • Compiler-UnionsLSP - native
  • xstate-main-1-LSP - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
lsp-startup - native
Execution time 4.72ms (±365.08%) 4.72ms (±365.08%) +0.01ms (+ 0.13%) 0.00ms 476.74ms p=0.042 n=1200
tsgo-startup - native
Execution time 5.78ms (±365.09%) 5.75ms (±365.09%) -0.03ms (- 0.52%) 0.00ms 580.42ms p=0.000 n=1200
System info unknown
Hosts
  • native
Scenarios
  • lsp-startup - native
  • tsgo-startup - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

Developer Information:

Download Benchmarks

@DanielRosenwasser

Daniel Rosenwasser (DanielRosenwasser) commented Oct 14, 2026

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) perf test this

@typescript-automation

typescript-automation Bot commented Oct 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-automation

typescript-automation Bot commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Daniel Rosenwasser (@DanielRosenwasser)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - native
Errors 41 41 ~ ~ ~ p=1.000 n=12
Symbols 115,276 115,276 ~ ~ ~ p=1.000 n=12
Types 79,613 79,613 ~ ~ ~ p=1.000 n=12
Memory Used 132,813k (± 0.05%) 129,797k (± 0.04%) -3,016k (- 2.27%) 129,711k 130,022k p=0.000 n=12
Memory Allocs 2,092,365 (± 0.01%) 2,090,431 (± 0.01%) -1,934 (- 0.09%) 2,090,142 2,090,765 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.048s (± 4.23%) 0.050s (± 2.32%) ~ 0.047s 0.053s p=0.256 n=12
Bind Time 0.015s (±13.45%) 0.015s (±11.58%) ~ 0.013s 0.022s p=1.000 n=12
Check Time 0.591s (± 0.72%) 0.570s (± 0.75%) 🟩-0.021s (- 3.50%) 0.560s 0.578s p=0.000 n=12
Emit Time 0.331s (± 2.11%) 0.321s (± 1.69%) -0.010s (- 3.00%) 0.307s 0.332s p=0.016 n=12
Total Time 0.993s (± 0.77%) 0.964s (± 0.68%) -0.030s (- 2.99%) 0.945s 0.981s p=0.000 n=12
angular-1 - native
Errors 3 3 ~ ~ ~ p=1.000 n=12
Symbols 846,738 (± 0.05%) 846,163 (± 0.07%) ~ 844,240 847,311 p=0.160 n=12
Types 247,032 (± 0.00%) 247,030 (± 0.00%) -2 (- 0.00%) 247,027 247,032 p=0.002 n=12
Memory Used 741,318k (± 0.01%) 741,218k (± 0.01%) ~ 741,036k 741,493k p=0.174 n=12
Memory Allocs 12,670,511 (± 0.10%) 12,666,298 (± 0.10%) ~ 12,652,531 12,720,162 p=0.160 n=12
Config Time 0.020s 0.021s (± 4.37%) ~ 0.020s 0.025s p=0.093 n=12
Parse Time 0.323s (± 2.76%) 0.335s (± 1.37%) +0.012s (+ 3.74%) 0.325s 0.346s p=0.011 n=12
Bind Time 0.077s (± 6.27%) 0.075s (± 2.05%) ~ 0.071s 0.079s p=0.991 n=12
Check Time 0s 0s ~ ~ ~ p=1.000 n=12
Emit Time 1.873s (± 1.00%) 1.898s (± 0.97%) ~ 1.855s 1.954s p=0.058 n=12
Total Time 2.310s (± 0.88%) 2.342s (± 0.83%) +0.033s (+ 1.42%) 2.296s 2.393s p=0.012 n=12
mui-docs - native
Errors 11,405 11,405 ~ ~ ~ p=1.000 n=12
Symbols 4,331,835 4,331,835 ~ ~ ~ p=1.000 n=12
Types 1,370,520 1,370,499 -21 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 2,910,107k (± 0.04%) 2,912,961k (± 0.05%) +2,855k (+ 0.10%) 2,908,271k 2,916,481k p=0.005 n=12
Memory Allocs 31,228,695 (± 0.06%) 31,248,620 (± 0.09%) ~ 31,194,292 31,347,862 p=0.291 n=12
Config Time 0.020s 0.020s ~ ~ ~ p=1.000 n=12
Parse Time 0.616s (± 3.57%) 0.612s (± 2.97%) ~ 0.545s 0.639s p=0.832 n=12
Bind Time 0.002s (±12.64%) 0.003s (±13.14%) ~ 0.002s 0.003s p=0.400 n=12
Check Time 9.381s (± 0.18%) 9.463s (± 0.59%) +0.082s (+ 0.87%) 9.386s 9.677s p=0.000 n=12
Emit Time 0.514s (± 1.65%) 0.513s (± 3.22%) ~ 0.500s 0.595s p=0.295 n=12
Total Time 11.465s (± 0.23%) 11.557s (± 0.54%) +0.092s (+ 0.80%) 11.445s 11.780s p=0.005 n=12
strada-build-src - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,388,669 1,388,618 -51 (- 0.00%) ~ ~ p=0.000 n=12
Types 441,862 441,852 -10 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 1,597,735k (± 0.64%) 1,607,090k (± 0.57%) ~ 1,587,828k 1,625,321k p=0.089 n=12
Memory Allocs 78,804,630 (± 0.08%) 78,743,686 (± 0.08%) ~ 78,505,455 78,853,192 p=0.078 n=12
Config Time 0.004s (±12.74%) 0.004s (±12.74%) ~ 0.002s 0.005s p=1.000 n=12
Parse Time 0.311s (± 1.55%) 0.307s (± 1.40%) ~ 0.298s 0.318s p=0.285 n=12
Bind Time 0.000s 0.001s (±129.56%) ~ 0.000s 0.004s p=0.217 n=12
Check Time 2.144s (± 0.58%) 2.121s (± 0.34%) -0.023s (- 1.09%) 2.102s 2.141s p=0.004 n=12
Emit Time 0.341s (± 4.24%) 0.359s (± 4.70%) ~ 0.324s 0.392s p=0.101 n=12
Total Time 24.723s (± 0.94%) 24.646s (± 0.91%) ~ 24.212s 25.075s p=0.671 n=12
strada-compiler - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 334,953 334,922 -31 (- 0.01%) ~ ~ p=0.000 n=12
Types 197,616 197,610 -6 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 293,453k (± 0.02%) 292,697k (± 0.02%) -756k (- 0.26%) 292,549k 292,837k p=0.000 n=12
Memory Allocs 4,413,169 (± 0.01%) 4,410,208 (± 0.01%) -2,961 (- 0.07%) 4,408,545 4,411,026 p=0.000 n=12
Config Time 0.001s 0.001s ~ ~ ~ p=1.000 n=12
Parse Time 0.108s (± 4.69%) 0.113s (± 3.79%) ~ 0.101s 0.126s p=0.084 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 1.022s (± 0.61%) 1.020s (± 0.61%) ~ 1.011s 1.042s p=0.639 n=12
Emit Time 0.148s (± 4.91%) 0.146s (± 5.63%) ~ 0.124s 0.164s p=0.702 n=12
Total Time 1.332s (± 0.65%) 1.335s (± 0.96%) ~ 1.307s 1.364s p=0.619 n=12
ts-pre-modules - native
Errors 87 87 ~ ~ ~ p=1.000 n=12
Symbols 303,834 303,802 -32 (- 0.01%) ~ ~ p=0.000 n=12
Types 181,663 181,658 -5 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 253,974k (± 0.01%) 253,568k (± 0.02%) -406k (- 0.16%) 253,485k 253,722k p=0.000 n=12
Memory Allocs 1,410,942 (± 0.02%) 1,408,531 (± 0.01%) -2,411 (- 0.17%) 1,407,814 1,409,098 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.098s (± 2.38%) 0.095s (± 2.82%) ~ 0.089s 0.103s p=0.181 n=12
Bind Time 0.040s (± 9.68%) 0.037s (±12.88%) ~ 0.028s 0.050s p=0.155 n=12
Check Time 0.803s (± 0.79%) 0.807s (± 0.78%) ~ 0.788s 0.824s p=0.401 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 0.956s (± 0.55%) 0.954s (± 0.65%) ~ 0.939s 0.971s p=0.418 n=12
vscode - native
Errors 396 396 ~ ~ ~ p=1.000 n=12
Symbols 11,578,142 11,577,970 -172 (- 0.00%) ~ ~ p=0.000 n=12
Types 3,822,747 3,822,660 -87 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 6,650,490k (± 0.01%) 6,644,528k (± 0.01%) -5,962k (- 0.09%) 6,642,923k 6,648,612k p=0.000 n=12
Memory Allocs 52,944,645 (± 0.09%) 52,898,423 (± 0.02%) -46,222 (- 0.09%) 52,886,046 52,930,166 p=0.003 n=12
Config Time 0.075s (± 0.36%) 0.075s (± 0.43%) ~ 0.074s 0.076s p=0.832 n=12
Parse Time 1.863s (± 4.42%) 1.858s (± 3.46%) ~ 1.701s 2.036s p=0.944 n=12
Bind Time 0.661s (±21.87%) 0.624s (±18.50%) ~ 0.463s 0.953s p=0.660 n=12
Check Time 14.080s (± 1.05%) 14.090s (± 1.03%) ~ 13.705s 14.355s p=0.755 n=12
Emit Time 5.461s (± 8.21%) 5.246s (± 8.71%) ~ 4.507s 6.214s p=0.724 n=12
Total Time 22.226s (± 1.72%) 21.981s (± 1.85%) ~ 21.302s 22.858s p=0.590 n=12
webpack - native
Errors 585 585 ~ ~ ~ p=1.000 n=12
Symbols 1,325,855 1,325,764 -91 (- 0.01%) ~ ~ p=0.000 n=12
Types 627,988 627,966 -22 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 943,681k (± 0.02%) 944,003k (± 0.01%) +321k (+ 0.03%) 943,780k 944,264k p=0.001 n=12
Memory Allocs 6,111,294 (± 0.02%) 6,108,627 (± 0.02%) -2,667 (- 0.04%) 6,105,038 6,111,264 p=0.001 n=12
Config Time 0.007s (± 3.92%) 0.007s (± 3.42%) ~ 0.007s 0.008s p=1.000 n=12
Parse Time 0.238s (± 3.46%) 0.237s (± 3.86%) ~ 0.218s 0.263s p=0.898 n=12
Bind Time 0.067s (±21.58%) 0.077s (±21.47%) ~ 0.051s 0.108s p=0.540 n=12
Check Time 1.603s (± 0.74%) 1.601s (± 0.50%) ~ 1.585s 1.621s p=0.766 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.945s (± 0.65%) 1.953s (± 0.43%) ~ 1.931s 1.972s p=0.192 n=12
xstate-main - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,062,854 1,062,736 -118 (- 0.01%) ~ ~ p=0.000 n=12
Types 390,845 390,840 -5 (- 0.00%) ~ ~ p=0.000 n=12
Memory Used 550,368k (± 0.03%) 549,475k (± 0.02%) -893k (- 0.16%) 549,229k 549,779k p=0.000 n=12
Memory Allocs 4,504,661 (± 0.13%) 4,502,519 (± 0.08%) ~ 4,493,235 4,512,532 p=0.713 n=12
Config Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Parse Time 0.113s (± 3.53%) 0.112s (± 3.88%) ~ 0.102s 0.123s p=0.599 n=12
Bind Time 0.032s (±19.47%) 0.038s (±19.65%) ~ 0.027s 0.057s p=0.050 n=12
Check Time 0.829s (± 0.69%) 0.837s (± 0.92%) ~ 0.825s 0.865s p=0.146 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 0.985s (± 0.89%) 0.998s (± 0.59%) +0.013s (+ 1.31%) 0.979s 1.013s p=0.029 n=12
System info unknown
Hosts
  • native
Scenarios
  • Compiler-Unions - native
  • angular-1 - native
  • mui-docs - native
  • strada-build-src - native
  • strada-compiler - native
  • ts-pre-modules - native
  • vscode - native
  • webpack - native
  • xstate-main - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

lsp

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsLSP - native
Req 1 - updateOpen 11ms (± 9.52%) 11ms (±12.64%) ~ 9ms 15ms p=0.822 n=12
Req 2 - geterr 559ms (± 0.43%) 541ms (± 0.28%) 🟩-18ms (- 3.28%) 535ms 543ms p=0.000 n=12
Req 3 - references 44ms (± 4.84%) 45ms (± 4.75%) ~ 41ms 50ms p=0.613 n=12
Req 4 - navto 16ms (± 1.90%) 16ms (± 1.13%) ~ 16ms 17ms p=0.317 n=12
Req 5 - completionInfo count 1,363 1,363 ~ ~ ~ p=1.000 n=12
Req 5 - completionInfo 17ms (± 7.94%) 16ms (± 6.84%) 🟩-2ms (- 9.13%) 14ms 20ms p=0.044 n=12
CompilerLSP - native
Req 1 - updateOpen 11ms (±14.33%) 12ms (±12.79%) ~ 10ms 15ms p=0.595 n=12
Req 2 - geterr 293ms (± 0.52%) 294ms (± 1.14%) ~ 288ms 307ms p=0.943 n=12
Req 3 - references 50ms (± 3.89%) 49ms (± 4.83%) ~ 44ms 55ms p=0.294 n=12
Req 4 - navto 19ms (± 8.52%) 17ms (± 2.46%) ~ 16ms 19ms p=0.208 n=12
Req 5 - completionInfo count 1,525 1,525 ~ ~ ~ p=1.000 n=12
Req 5 - completionInfo 16ms (± 5.77%) 18ms (± 8.16%) ~ 15ms 22ms p=0.083 n=12
System info unknown
Hosts
  • native
Scenarios
  • CompilerLSP - native
  • Compiler-UnionsLSP - native
  • xstate-main-1-LSP - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
lsp-startup - native
Execution time 4.73ms (±365.09%) 4.73ms (±365.08%) +0.01ms (+ 0.20%) 0.00ms 479.16ms p=0.000 n=1200
tsgo-startup - native
Execution time 5.77ms (±365.09%) 5.76ms (±365.10%) -0.01ms (- 0.11%) 0.00ms 586.26ms p=0.000 n=1200
System info unknown
Hosts
  • native
Scenarios
  • lsp-startup - native
  • tsgo-startup - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

Developer Information:

Download Benchmarks

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments

At least 1 approving review is required to merge this pull request.

Assignees

@DanielRosenwasser Daniel Rosenwasser (DanielRosenwasser)

Labels

Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug

Projects

PR Backlog
Status: Not started

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@DanielRosenwasser