From c63e3320472d591742e4b2731fb72abfdcfafb3c Mon Sep 17 00:00:00 2001 From: Christian Findlay <16697547+MelbourneDeveloper@users.noreply.github.com> Date: Sun, 18 Oct 2026 17:22:15 +1100 Subject: [PATCH 01/24] Fix the open v0.22.0 review findings: parenthesized C# project names, framework-switch order, Windows-only selections that run nothing - A single-target C# project whose name ends in (tfm) is no longer read as a framework context (#382) - A framework switch re-publishes open documents' diagnostics before announcing itself and asking for refreshes (#383) - Off Windows, a MOCHA_FILES pattern naming only windowsOnly suites fails instead of passing a run of nothing (#384) --- docs/reviews/V0.22.0-STATIC-REVIEW.md | 12 +++ docs/specs/DISTRIBUTION-SPEC.md | 4 + docs/specs/DOTNET-FRAMEWORK-SPEC.md | 15 ++-- src/editors/vscode/src/test/suite/index.ts | 23 +++-- .../test/suite/test-runner-selection.test.ts | 58 ++++++++++++ src/editors/vscode/test-chunks.json | 3 +- src/sharplsp/src/diagnostics.rs | 88 ++++++++++++++++--- src/sharplsp/src/main.rs | 46 +++++----- src/sharplsp/src/target_framework.rs | 32 +++++-- .../tests/e2e_modules/target_framework.rs | 56 +++++++++++- .../WorkspaceManagerTargetFrameworkTests.cs | 31 ++++++- .../Workspace/TargetFrameworks.cs | 17 +++- 12 files changed, 325 insertions(+), 60 deletions(-) create mode 100644 src/editors/vscode/src/test/suite/test-runner-selection.test.ts diff --git a/docs/reviews/V0.22.0-STATIC-REVIEW.md b/docs/reviews/V0.22.0-STATIC-REVIEW.md index ce5e418e..c61ce01b 100644 --- a/docs/reviews/V0.22.0-STATIC-REVIEW.md +++ b/docs/reviews/V0.22.0-STATIC-REVIEW.md @@ -8,6 +8,18 @@ The .NET Framework and multi-targeting feature has been implemented across the R The spec explicitly excludes debugging the desktop .NET Framework CLR: Debug of a Framework-only target is refused, while Run uses the desktop CLR on Windows. “Finished” must be judged against that stated scope, not against availability of a .NET Framework debugger. +## Resolution + +Re-checked against `main` on 2026-10-18 (`817d359f`): + +| Finding | Status | +|---|---| +| Parenthesized C# project name read as a framework | Fixed (#382). `FrameworkOf` accepts only the `(tfm)` suffix past the project file's stem. Covered by a real single-target `Utility(net48).csproj` and a multi-targeted `Tool(net10.0).csproj` in `WorkspaceManagerTargetFrameworkTests`. | +| Switch announced before invalidation and diagnostics | Fixed (#383). The host clears `nav_cache`, publishes every open document's diagnostics and waits for each, then announces and asks for refreshes. `test_full_stack_target_framework_switch` asserts that order, using a warning only `net10.0` compiles. | +| Run/F5 project paths compared with the wrong case rule | Already fixed before this re-check: `chosenFramework` uses `samePath` from `paths.ts`. | +| Explicit Windows-only selection passes with zero tests | Fixed (#384). Off Windows, a `MOCHA_FILES` pattern that names only `windowsOnly` suites fails with the platform reason (`test-runner-selection.test.ts`). | +| Dependabot skips `detect-changes` | Intended: Dependabot PRs run no CI by design and are tested once at the `dependabot-upgrades → main` consolidation PR ([DIST-CI-DEPENDABOT]). | + ## Findings, in priority order ### [P2] A single-target C# project name can be mistaken for a framework diff --git a/docs/specs/DISTRIBUTION-SPEC.md b/docs/specs/DISTRIBUTION-SPEC.md index 5af0f875..17c877ad 100644 --- a/docs/specs/DISTRIBUTION-SPEC.md +++ b/docs/specs/DISTRIBUTION-SPEC.md @@ -1107,6 +1107,10 @@ Invariants: distinction the manifest carries is `"linuxOnly": true` or `"windowsOnly": true`, which drops a chunk from the other platform's matrix. A chunk MUST NOT be declared in CI YAML. +- **A selection never runs nothing.** The in-host runner fails a run whose + `MOCHA_FILES` pattern matches no compiled suite, and, off Windows, one whose + pattern names only suites of `windowsOnly` chunks: dropping every one would + report a green run of zero tests (`test-runner-selection.test.ts`). - **One runner, one shard target.** `make _run-vsix-suite` is the single recipe; `CHUNK` selects the slice (empty runs every suite) and `VSIX_SUITE_PREBUILT` says the suite is already compiled. Coverage is NOT a knob — the runner always diff --git a/docs/specs/DOTNET-FRAMEWORK-SPEC.md b/docs/specs/DOTNET-FRAMEWORK-SPEC.md index a2f72951..9c10a616 100644 --- a/docs/specs/DOTNET-FRAMEWORK-SPEC.md +++ b/docs/specs/DOTNET-FRAMEWORK-SPEC.md @@ -64,13 +64,16 @@ built at load or at the switch that needs them - `sharplsp/targetFramework` `{ textDocument }` → `{ active, available, project }`; `sharplsp/setTargetFramework` `{ textDocument, targetFramework }` switches the whole PROJECT in both languages and answers the same shape. A single-target project answers - `{ active: null, available: [] }`; a document no loaded project compiles answers - `{ active: null, available: [], project: null }`, and switching it is an error. + `{ active: null, available: [] }`, even one whose name ends in `(tfm)`: a framework is + only the suffix `MSBuildWorkspace` appends to the project FILE's stem. A document no + loaded project compiles answers `{ active: null, available: [], project: null }`, and + switching it is an error. - After a switch the host clears its navigation cache, re-publishes every open document's - diagnostics, sends `sharplsp/targetFrameworkChanged` `{ textDocument, active, available, - project }`, then each of `workspace/semanticTokens/refresh`, `workspace/inlayHint/refresh`, - `workspace/codeLens/refresh` and `workspace/diagnostic/refresh` whose `refreshSupport` the - client declared. + diagnostics and waits for each first publication (a failed or provisional fetch goes on + retrying in the background), sends `sharplsp/targetFrameworkChanged` `{ textDocument, + active, available, project }`, then each of `workspace/semanticTokens/refresh`, + `workspace/inlayHint/refresh`, `workspace/codeLens/refresh` and + `workspace/diagnostic/refresh` whose `refreshSupport` the client declared. - VS Code: a status-bar item `$(versions) ` shows the focused document's active framework when its project is multi-targeted, and runs `sharplsp.selectTargetFramework [tfm]`. Without an argument the command opens a pick placeheld `Target framework for ` (file name, no diff --git a/src/editors/vscode/src/test/suite/index.ts b/src/editors/vscode/src/test/suite/index.ts index feb44bb3..b2928f9b 100644 --- a/src/editors/vscode/src/test/suite/index.ts +++ b/src/editors/vscode/src/test/suite/index.ts @@ -47,8 +47,8 @@ interface ChunkManifest { * Suites of `windowsOnly` chunks: .NET Framework test hosts run only on Windows * ([NETFX-SCOPE]), so they never load elsewhere — not even in a local full run. */ -function windowsOnlySuites(): ReadonlySet { - if (process.platform === 'win32') return new Set(); +function windowsOnlySuites(platform: NodeJS.Platform): ReadonlySet { + if (platform === 'win32') return new Set(); const manifest = path.resolve(__dirname, '..', '..', '..', 'test-chunks.json'); const { chunks } = JSON.parse(fs.readFileSync(manifest, 'utf8')) as ChunkManifest; return new Set( @@ -58,17 +58,28 @@ function windowsOnlySuites(): ReadonlySet { /** * A chunk that selects nothing must fail loudly: a mistyped file list would - * otherwise report a green run that executed zero assertions. + * otherwise report a green run that executed zero assertions. So must a pattern + * that, off Windows, names only Windows-only suites: dropping every one of them + * would pass a run of nothing. */ -function resolveSuiteFiles(testsRoot: string): string[] { +export function resolveSuiteFiles( + testsRoot: string, + globs: readonly string[] = requestedGlobs(), + platform: NodeJS.Platform = process.platform, +): string[] { const selected = new Set(); - const windowsOnly = windowsOnlySuites(); - for (const pattern of requestedGlobs()) { + const windowsOnly = windowsOnlySuites(platform); + for (const pattern of globs) { const matches = globSync(pattern, { cwd: testsRoot }); if (matches.length === 0) { throw new Error(`MOCHA_FILES pattern matched no compiled suite: ${pattern}`); } const runnable = matches.filter((match) => !windowsOnly.has(match.split(path.sep).join('/'))); + if (runnable.length === 0) { + throw new Error( + `MOCHA_FILES pattern selects only Windows-only suites, which run on Windows alone ([NETFX-SCOPE]): ${pattern}`, + ); + } for (const match of runnable.filter(matchesShape)) { selected.add(match); } diff --git a/src/editors/vscode/src/test/suite/test-runner-selection.test.ts b/src/editors/vscode/src/test/suite/test-runner-selection.test.ts new file mode 100644 index 00000000..9c0f768d --- /dev/null +++ b/src/editors/vscode/src/test/suite/test-runner-selection.test.ts @@ -0,0 +1,58 @@ +// Tests for [DIST-CI-WIN-VSIX] and [NETFX-SCOPE]: which suites a run executes. +// +// The runner (./index) turns MOCHA_FILES into the suites Mocha loads. A request +// that resolves to nothing must fail, never pass a run of zero tests: a pattern +// matching no compiled suite, and, off Windows, a pattern naming only suites of +// `windowsOnly` chunks, whose .NET Framework test hosts need the desktop CLR. +// The Windows-only names are the real test-chunks.json's. +import * as assert from 'node:assert/strict'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { removeDirRecursive } from '../../utils'; +import { resolveSuiteFiles } from './index'; + +const NETFX_LANGUAGE = 'netfx-language.test.js'; +const NETFX_RUN = 'netfx-run-fsharp.test.js'; +const EVERYWHERE = 'everywhere.test.js'; +const COMPILED = [EVERYWHERE, NETFX_LANGUAGE, NETFX_RUN]; + +suite('Test runner — suite selection', () => { + let root = ''; + + suiteSetup(() => { + root = fs.mkdtempSync(path.join(os.tmpdir(), 'slsp-select-')); + for (const suiteFile of COMPILED) { + fs.writeFileSync(path.join(root, suiteFile), ''); + } + }); + + suiteTeardown(() => removeDirRecursive(root)); + + test('off Windows: a pattern naming only Windows-only suites fails with the platform reason → a broad pattern runs the rest', () => { + assert.throws( + () => resolveSuiteFiles(root, [NETFX_LANGUAGE], 'linux'), + /selects only Windows-only suites, which run on Windows alone \(\[NETFX-SCOPE\]\): netfx-language\.test\.js$/, + ); + assert.throws( + () => resolveSuiteFiles(root, [EVERYWHERE, 'netfx-*.test.js'], 'darwin'), + /Windows alone .*: netfx-\*\.test\.js$/, + 'one Windows-only pattern fails the whole request', + ); + assert.deepEqual(resolveSuiteFiles(root, ['*.test.js'], 'linux'), [EVERYWHERE]); + assert.deepEqual(resolveSuiteFiles(root, [EVERYWHERE], 'darwin'), [EVERYWHERE]); + }); + + test('on Windows: the same patterns run their suites → a pattern matching nothing still fails', () => { + assert.deepEqual(resolveSuiteFiles(root, [NETFX_LANGUAGE], 'win32'), [NETFX_LANGUAGE]); + assert.deepEqual(resolveSuiteFiles(root, ['netfx-*.test.js'], 'win32'), [ + NETFX_LANGUAGE, + NETFX_RUN, + ]); + assert.deepEqual(resolveSuiteFiles(root, ['*.test.js'], 'win32'), [...COMPILED].sort()); + assert.throws( + () => resolveSuiteFiles(root, ['nowhere.test.js'], 'win32'), + /matched no compiled suite: nowhere\.test\.js$/, + ); + }); +}); diff --git a/src/editors/vscode/test-chunks.json b/src/editors/vscode/test-chunks.json index f5c37a2b..4a2c60fb 100644 --- a/src/editors/vscode/test-chunks.json +++ b/src/editors/vscode/test-chunks.json @@ -191,7 +191,8 @@ "nuget-browser.test.js", "nuget-deps-e2e.test.js", "fsi-build-output-e2e.test.js", - "utils.test.js" + "utils.test.js", + "test-runner-selection.test.js" ] }, "profiler": { diff --git a/src/sharplsp/src/diagnostics.rs b/src/sharplsp/src/diagnostics.rs index d07386bd..e59eb6e8 100644 --- a/src/sharplsp/src/diagnostics.rs +++ b/src/sharplsp/src/diagnostics.rs @@ -85,6 +85,9 @@ struct SolutionDiagnosticsRequest { project_filter: Vec, } +/// A document to push diagnostics for: its owning sidecar, URI and native path. +pub type PushTarget = (Arc, Uri, String); + /// Spawn a background task to fetch diagnostics and publish them. /// /// Non-blocking: the main loop continues processing messages while @@ -103,6 +106,57 @@ pub fn request_in_background( }); } +/// Publish every target's diagnostics NOW, concurrently, and return once each +/// first answer is out, so whatever the caller sends next follows them. A fetch +/// that failed, or answered a provisional set, goes on as a background push. +/// Implements [NETFX-CONTEXT] and [DIAG-PUSH-GATE]. +pub fn publish_now( + runtime: &tokio::runtime::Runtime, + targets: Vec, + sender: &crossbeam_channel::Sender, +) { + let unfinished = runtime.block_on(async { + let mut pushes = tokio::task::JoinSet::new(); + for target in targets { + let generation = next_generation(&target.1); + let _push = pushes.spawn(first_push(target, generation, sender.clone())); + } + still_pushing(pushes).await + }); + for (sidecar, uri, file_path) in unfinished { + request_in_background(runtime, sidecar, sender.clone(), uri, file_path); + } +} + +/// A target's first fetch and publish; the target back when its push must go on. +async fn first_push( + target: PushTarget, + generation: u64, + sender: crossbeam_channel::Sender, +) -> Option { + let (sidecar, uri, file_path) = ⌖ + let source_tag = source_tag_for_uri(uri); + let again = publish_once(sidecar, &sender, uri, file_path, &source_tag, generation) + .await + .unwrap_or_else(|err| { + warn!(uri = %uri.as_str(), "Sidecar diagnostics unavailable: {err:#}"); + true + }); + again.then_some(target) +} + +/// Every first push finished: the targets whose push must go on. +async fn still_pushing(mut pushes: tokio::task::JoinSet>) -> Vec { + let mut unfinished = Vec::new(); + while let Some(joined) = pushes.join_next().await { + match joined { + Ok(again) => unfinished.extend(again), + Err(err) => warn!("Diagnostics push task failed: {err}"), + } + } + unfinished +} + /// Register a new push generation for `uri`, superseding in-flight fetches. /// [DIAG-PUSH-GATE] fn next_generation(uri: &Uri) -> u64 { @@ -188,18 +242,10 @@ async fn fetch_and_publish_gated( generation: u64, ) { for attempt in 1..=MAX_PUSH_ATTEMPTS { - if sidecar.is_shutting_down() || !is_current(uri, generation) { - return; - } - match fetch(sidecar, file_path, source_tag).await { - Ok(diagnostics) => { - if !publish_provisional(sender, uri, generation, diagnostics) { - return; - } - } - Err(err) => { - warn!("Sidecar diagnostics unavailable (attempt {attempt}): {err:#}"); - } + match publish_once(sidecar, sender, uri, file_path, source_tag, generation).await { + Ok(false) => return, + Ok(true) => {} + Err(err) => warn!("Sidecar diagnostics unavailable (attempt {attempt}): {err:#}"), } tokio::time::sleep(PUSH_RETRY_DELAY).await; } @@ -209,6 +255,24 @@ async fn fetch_and_publish_gated( ); } +/// One gated fetch and publish. `Ok(true)` when the published set is +/// provisional and the push must go on, `Ok(false)` when it is done or +/// superseded, and the fetch's error when it must retry. [DIAG-PUSH-GATE] +async fn publish_once( + sidecar: &SidecarManager, + sender: &crossbeam_channel::Sender, + uri: &Uri, + file_path: &str, + source_tag: &str, + generation: u64, +) -> Result { + if sidecar.is_shutting_down() || !is_current(uri, generation) { + return Ok(false); + } + let diagnostics = fetch(sidecar, file_path, source_tag).await?; + Ok(publish_provisional(sender, uri, generation, diagnostics)) +} + /// Determine the diagnostic source tag based on the document language. fn source_tag_for_uri(uri: &Uri) -> String { match crate::tree_sitter_parse::LangId::from_uri(uri) { diff --git a/src/sharplsp/src/main.rs b/src/sharplsp/src/main.rs index a31a5dc1..f409ef15 100644 --- a/src/sharplsp/src/main.rs +++ b/src/sharplsp/src/main.rs @@ -950,8 +950,7 @@ fn handle_request( } "sharplsp/setTargetFramework" => { let sidecar = pick_sidecar(&req, csharp_sidecar, fsharp_sidecar); - let switched = target_framework::handle_set(req, runtime, sidecar, &connection.sender); - if switched.is_ok() { + target_framework::handle_set(req, runtime, sidecar).and_then(|switched| { nav_cache.clear(); republish_open_diagnostics( vfs, @@ -960,8 +959,8 @@ fn handle_request( fsharp_sidecar, connection, ); - } - switched + switched.announce(&connection.sender) + }) } // Solution loading "sharplsp/loadSolution" => handle_load_solution( @@ -1611,8 +1610,9 @@ fn handle_notification( } } -/// Re-publish every open document's diagnostics: a framework switch changes -/// which `#if` branches compile. [NETFX-CONTEXT] +/// Re-publish every open document's diagnostics, returning once each is out: a +/// framework switch changes which `#if` branches compile, and the editor hears +/// of the switch only after them. [NETFX-CONTEXT] fn republish_open_diagnostics( vfs: &Vfs, runtime: &tokio::runtime::Runtime, @@ -1620,10 +1620,23 @@ fn republish_open_diagnostics( fsharp_sidecar: Option<&Arc>, connection: &Connection, ) { - let open: Vec = vfs.iter().map(|entry| entry.key().clone()).collect(); - for uri in &open { - trigger_diagnostics(uri, runtime, csharp_sidecar, fsharp_sidecar, connection); - } + let open = vfs + .iter() + .filter_map(|entry| push_target(entry.key(), csharp_sidecar, fsharp_sidecar)) + .collect(); + diagnostics::publish_now(runtime, open, &connection.sender); +} + +/// Where a document's diagnostics come from: the sidecar its language routes +/// to, and its native path. +fn push_target( + uri: &Uri, + csharp_sidecar: Option<&Arc>, + fsharp_sidecar: Option<&Arc>, +) -> Option { + let sidecar = sidecar_for_uri(uri, csharp_sidecar, fsharp_sidecar)?; + let file_path = paths::uri_to_path(uri.as_str()).ok()?; + Some((Arc::clone(sidecar), uri.clone(), file_path)) } /// Spawn a background diagnostic request for the given URI. @@ -1636,19 +1649,10 @@ fn trigger_diagnostics( fsharp_sidecar: Option<&Arc>, connection: &Connection, ) { - let Some(sidecar) = sidecar_for_uri(uri, csharp_sidecar, fsharp_sidecar) else { - return; - }; - let Ok(file_path) = paths::uri_to_path(uri.as_str()) else { + let Some((sidecar, uri, file_path)) = push_target(uri, csharp_sidecar, fsharp_sidecar) else { return; }; - diagnostics::request_in_background( - runtime, - Arc::clone(sidecar), - connection.sender.clone(), - uri.clone(), - file_path, - ); + diagnostics::request_in_background(runtime, sidecar, connection.sender.clone(), uri, file_path); } /// Re-parse a document with tree-sitter and cache the result. diff --git a/src/sharplsp/src/target_framework.rs b/src/sharplsp/src/target_framework.rs index e7fbb202..27688449 100644 --- a/src/sharplsp/src/target_framework.rs +++ b/src/sharplsp/src/target_framework.rs @@ -110,13 +110,33 @@ pub fn handle_get( Ok(serde_json::to_value(context)?) } -/// `sharplsp/setTargetFramework`: switch the project, then tell the editor. +/// A project the sidecar has switched, not yet announced: the editor hears of it +/// only after the host has invalidated what the switch made stale. [NETFX-CONTEXT] +pub struct Switched { + /// The document the switch was asked from. + document: TextDocumentIdentifier, + /// The project's framework context after the switch. + context: FrameworkContext, +} + +impl Switched { + /// Announce the switch and ask for the refreshes, then answer the request. + pub fn announce( + self, + sender: &crossbeam_channel::Sender, + ) -> Result { + announce(sender, &self.document, &self.context)?; + Ok(serde_json::to_value(self.context)?) + } +} + +/// `sharplsp/setTargetFramework`: switch the project. The caller clears its +/// navigation cache and re-publishes diagnostics, THEN announces. [NETFX-CONTEXT] pub fn handle_set( req: Request, runtime: &tokio::runtime::Runtime, sidecar: Option<&Arc>, - sender: &crossbeam_channel::Sender, -) -> Result { +) -> Result { let params: FrameworkParams = serde_json::from_value(req.params)?; let framework = params .target_framework @@ -124,8 +144,10 @@ pub fn handle_set( .context("sharplsp/setTargetFramework requires targetFramework")?; let context = forward(runtime, sidecar, "workspace/setTargetFramework", ¶ms)?; info!(uri = %params.text_document.uri.as_str(), framework, "Target framework switched"); - announce(sender, ¶ms.text_document, &context)?; - Ok(serde_json::to_value(context)?) + Ok(Switched { + document: params.text_document, + context, + }) } /// Ask the sidecar that owns the document. diff --git a/src/sharplsp/tests/e2e_modules/target_framework.rs b/src/sharplsp/tests/e2e_modules/target_framework.rs index 8556a06e..d448d8aa 100644 --- a/src/sharplsp/tests/e2e_modules/target_framework.rs +++ b/src/sharplsp/tests/e2e_modules/target_framework.rs @@ -22,6 +22,7 @@ public static class Probe public static string OnFramework => "framework"; #else public static string OnModern => "modern"; + public static int Modern() { int unused; return 0; } #endif } "#; @@ -57,8 +58,16 @@ fn methods(messages: &[Value]) -> Vec<&str> { .collect() } -/// The project answers from its first framework until switched; a switch announces -/// itself and asks for exactly the refreshes the client declared; a bad switch fails. +/// Where `method` first arrived, if it did. +fn position(messages: &[Value], method: &str) -> Option { + messages + .iter() + .position(|message| message["method"] == method) +} + +/// The project answers from its first framework until switched; a switch re-publishes +/// the open document's diagnostics for the new framework, THEN announces itself, THEN +/// asks for exactly the refreshes the client declared; a bad switch fails. #[test] fn test_full_stack_target_framework_switch() { require_dotnet(); @@ -108,6 +117,7 @@ fn test_full_stack_target_framework_switch() { !asked.contains(&"workspace/codeLens/refresh"), "not declared: {asked:?}" ); + assert_switch_order(&arrived, &file_uri); let after = client.request("sharplsp/targetFramework", document); assert_eq!(after["result"]["active"], "net10.0"); @@ -128,3 +138,45 @@ fn test_full_stack_target_framework_switch() { client.shutdown_and_exit(); client.wait_with_timeout(); } + +/// [NETFX-CONTEXT]'s order: the new framework's diagnostics (net10.0 alone compiles +/// `Modern`'s unused local), then `sharplsp/targetFrameworkChanged`, then each refresh. +fn assert_switch_order(arrived: &[Value], file_uri: &str) { + let republished = arrived + .iter() + .position(|message| { + message["method"] == "textDocument/publishDiagnostics" + && message["params"]["uri"] == json!(file_uri) + && message["params"]["diagnostics"] + .as_array() + .is_some_and(|all| all.iter().any(|d| d["code"] == "CS0168")) + }) + .unwrap_or_else(|| panic!("net10.0's diagnostics before the answer: {arrived:?}")); + let warning = &arrived[republished]["params"]["diagnostics"] + .as_array() + .unwrap() + .iter() + .find(|d| d["code"] == "CS0168") + .unwrap()["range"]["start"]; + assert_eq!( + (&warning["line"], &warning["character"]), + (&json!(8), &json!(37)) + ); + let changed = position(arrived, "sharplsp/targetFrameworkChanged").unwrap(); + assert!( + republished < changed, + "diagnostics are re-published BEFORE the switch is announced: {:?}", + methods(arrived) + ); + for refresh in [ + "workspace/semanticTokens/refresh", + "workspace/inlayHint/refresh", + ] { + let asked = position(arrived, refresh).unwrap(); + assert!( + changed < asked, + "{refresh} follows the announcement: {:?}", + methods(arrived) + ); + } +} diff --git a/src/sidecars/SharpLsp.Sidecar.CSharp.Tests/WorkspaceManagerTargetFrameworkTests.cs b/src/sidecars/SharpLsp.Sidecar.CSharp.Tests/WorkspaceManagerTargetFrameworkTests.cs index e9c48717..0906fec7 100644 --- a/src/sidecars/SharpLsp.Sidecar.CSharp.Tests/WorkspaceManagerTargetFrameworkTests.cs +++ b/src/sidecars/SharpLsp.Sidecar.CSharp.Tests/WorkspaceManagerTargetFrameworkTests.cs @@ -119,11 +119,14 @@ public async Task An_undeclared_framework_fails_by_name_and_a_file_no_project_co ); } - [Fact] - public async Task A_single_target_project_has_no_framework_to_choose() + /// A name ending in (tfm) is the project's own name, not a framework context. + [Theory] + [InlineData("Single")] + [InlineData("Utility(net48)")] + public async Task A_single_target_project_has_no_framework_to_choose(string name) { - var single = WriteProject("Single", "net10.0"); - var source = NativePaths.Join(_root, "Single", "Single.cs"); + var single = WriteProject(name, "net10.0"); + var source = NativePaths.Join(_root, name, "Single.cs"); await File.WriteAllTextAsync(source, "namespace Fx;\n\npublic static class Single { }\n"); Restore(single); using var manager = await OpenAsync(single); @@ -131,6 +134,26 @@ public async Task A_single_target_project_has_no_framework_to_choose() Assert.Null(context.Active); Assert.Empty(context.Available); Assert.Equal(single, context.Project); + var refused = await manager.SetTargetFrameworkAsync(source, "net48", default); + Assert.True(refused.IsError, "a single-target project has no framework to switch to"); + } + + [Fact] + public async Task A_multi_targeted_project_named_like_a_framework_answers_its_declared_frameworks() + { + var name = "Tool(net10.0)"; + var tool = WriteProject(name, "net48;net10.0"); + var source = NativePaths.Join(_root, name, "Tool.cs"); + await File.WriteAllTextAsync(source, "namespace Fx;\n\npublic static class Tool { }\n"); + Restore(tool); + using var manager = await OpenAsync(tool); + var context = AssertOk(await manager.GetTargetFrameworksAsync(source, default)); + Assert.Equal("net48", context.Active); + Assert.Equal(Declared, context.Available); + Assert.Equal(tool, context.Project); + var switched = AssertOk(await manager.SetTargetFrameworkAsync(source, "net10.0", default)); + Assert.Equal(("net10.0", tool), (switched.Active, switched.Project)); + Assert.Equal(Declared, switched.Available); } private const string SharedSource = """ diff --git a/src/sidecars/SharpLsp.Sidecar.CSharp/Workspace/TargetFrameworks.cs b/src/sidecars/SharpLsp.Sidecar.CSharp/Workspace/TargetFrameworks.cs index 7fd047d2..5810f9f7 100644 --- a/src/sidecars/SharpLsp.Sidecar.CSharp/Workspace/TargetFrameworks.cs +++ b/src/sidecars/SharpLsp.Sidecar.CSharp/Workspace/TargetFrameworks.cs @@ -14,12 +14,23 @@ namespace SharpLsp.Sidecar.CSharp.Workspace; /// internal static class TargetFrameworks { - /// The framework of a per-framework project; null for a single-target one. + /// + /// The framework of a per-framework project; null for a single-target one. + /// MSBuildWorkspace appends (tfm) to the project FILE's stem, so only a + /// suffix past that stem is a framework: a single-target Utility(net48).csproj + /// is named Utility(net48) and has none. + /// internal static string? FrameworkOf(Project project) { var name = project.Name; - var open = name.LastIndexOf('('); - return open > 0 && name.EndsWith(')') ? name[(open + 1)..^1] : null; + var stem = project.FilePath is null ? name : NativePaths.StemOf(project.FilePath); + return + name.Length > stem.Length + 2 + && name.StartsWith(stem, StringComparison.Ordinal) + && name[stem.Length] == '(' + && name.EndsWith(')') + ? name[(stem.Length + 1)..^1] + : null; } /// The per-framework projects built from 's file, in load order. From 238067d541710a6b6dc38287d355b1fd331e29fe Mon Sep 17 00:00:00 2001 From: Christian Findlay <16697547+MelbourneDeveloper@users.noreply.github.com> Date: Sun, 18 Oct 2026 17:29:09 +1100 Subject: [PATCH 02/24] CI: fail a Dependabot PR to main, whose skipped pipeline ran nothing The whole PR pipeline skips Dependabot, and the terminal gate passed the skipped run, so a security bump the sweep did not retire showed a green required check and could merge into main untested. The gate now fails it; bumps reach main only through the tested dependabot-upgrades consolidation PR. --- .github/workflows/ci.yml | 15 ++++++++++++--- docs/reviews/V0.22.0-STATIC-REVIEW.md | 4 ++-- docs/specs/DISTRIBUTION-SPEC.md | 12 +++++++----- tools/ci/security-gates.test.mjs | 14 ++++++++++++++ 4 files changed, 35 insertions(+), 10 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b45e17eb..0dd9df86 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -296,9 +296,9 @@ jobs: # # `always()` so it reports a real conclusion rather than inheriting a skip: # on a docs-only PR every upstream job skips and this passes; on a Dependabot - # PR the whole pipeline skips and this passes; any failure or cancellation - # anywhere upstream fails it, and so does a skipped build or test job when - # code changed. Either way the merge is blocked. + # PR the whole pipeline skips and this FAILS, since it ran nothing; any + # failure or cancellation anywhere upstream fails it, and so does a skipped + # build or test job when code changed. Either way the merge is blocked. ci: name: CI if: ${{ always() }} @@ -338,6 +338,15 @@ jobs: run: |- echo "::error::Code changed but a build or test job was skipped. Check its if: condition." exit 1 + # The whole pipeline skipped this Dependabot PR, so a pass would be a + # pass of nothing. Its bump reaches `main` only through the tested + # consolidation PR; the sweep retires this one. [DIST-CI-DEPENDABOT] + - name: Fail a Dependabot PR to main, which ran no job + if: ${{ github.actor == 'dependabot[bot]' }} + shell: bash + run: |- + echo "::error::Dependabot PRs run no CI. dependabot-automerge.yml sweeps this bump into dependabot-upgrades; it reaches main through the tested dependabot-upgrades -> main PR." + exit 1 - name: Report shell: bash run: echo "All CI jobs passed or were skipped." diff --git a/docs/reviews/V0.22.0-STATIC-REVIEW.md b/docs/reviews/V0.22.0-STATIC-REVIEW.md index c61ce01b..840a40c1 100644 --- a/docs/reviews/V0.22.0-STATIC-REVIEW.md +++ b/docs/reviews/V0.22.0-STATIC-REVIEW.md @@ -16,9 +16,9 @@ Re-checked against `main` on 2026-10-18 (`817d359f`): |---|---| | Parenthesized C# project name read as a framework | Fixed (#382). `FrameworkOf` accepts only the `(tfm)` suffix past the project file's stem. Covered by a real single-target `Utility(net48).csproj` and a multi-targeted `Tool(net10.0).csproj` in `WorkspaceManagerTargetFrameworkTests`. | | Switch announced before invalidation and diagnostics | Fixed (#383). The host clears `nav_cache`, publishes every open document's diagnostics and waits for each, then announces and asks for refreshes. `test_full_stack_target_framework_switch` asserts that order, using a warning only `net10.0` compiles. | -| Run/F5 project paths compared with the wrong case rule | Already fixed before this re-check: `chosenFramework` uses `samePath` from `paths.ts`. | +| Run/F5 project paths compared with the wrong case rule | Already fixed before this re-check: `chosenFramework` uses `samePath` from `paths.ts`, and `run-debug-target.test.ts` covers the explicit-project, differently cased focused project on Linux for C# and F#. | | Explicit Windows-only selection passes with zero tests | Fixed (#384). Off Windows, a `MOCHA_FILES` pattern that names only `windowsOnly` suites fails with the platform reason (`test-runner-selection.test.ts`). | -| Dependabot skips `detect-changes` | Intended: Dependabot PRs run no CI by design and are tested once at the `dependabot-upgrades → main` consolidation PR ([DIST-CI-DEPENDABOT]). | +| Dependabot skips `detect-changes`, and the terminal gate passes the skipped run | Fixed. The skip is intended: bumps are tested once, at the `dependabot-upgrades → main` consolidation PR. But `CI` now FAILS a Dependabot PR to `main`, so a security bump the sweep did not retire cannot merge untested ([DIST-CI-DEPENDABOT], `security-gates.test.mjs`). | ## Findings, in priority order diff --git a/docs/specs/DISTRIBUTION-SPEC.md b/docs/specs/DISTRIBUTION-SPEC.md index 17c877ad..2524895f 100644 --- a/docs/specs/DISTRIBUTION-SPEC.md +++ b/docs/specs/DISTRIBUTION-SPEC.md @@ -867,10 +867,10 @@ inheriting a skip: 1. Any upstream job that FAILED or was CANCELLED fails `CI`, and with it the merge. 2. A SKIPPED upstream job satisfies it only when no code changed. That is how a - docs-only PR (every job skipped by [DIST-CI-CLASSIFICATION]) and a Dependabot - PR (the whole pipeline skipped, [DIST-CI-DEPENDABOT]) pass. When code changed, - `build`, every test leg and `coverage` MUST have succeeded; a skip among them - fails `CI` (#365). + docs-only PR (every job skipped by [DIST-CI-CLASSIFICATION]) passes. When code + changed, `build`, every test leg and `coverage` MUST have succeeded; a skip + among them fails `CI` (#365). A Dependabot PR, whose whole pipeline skipped + ([DIST-CI-DEPENDABOT]), always fails `CI`: it ran nothing. 3. Every PHASE 4 leg and `coverage` states its condition explicitly: `!cancelled()`, code changed, and success of the jobs it consumes. GitHub's implicit `success()` spans the whole dependency chain, so a legitimately @@ -929,7 +929,9 @@ superseded, and MUST NOT merge anything into `main` unattended. staged batch is verified once, by the full pipeline and CodeQL, on the `dependabot-upgrades → main` consolidation PR — the only route a dependency change takes into `main`. A staged security fix reaches `main` when that PR - is opened, so it is opened promptly. + is opened, so it is opened promptly. `CI` enforces the route: it FAILS a + Dependabot PR to `main`, so a security bump whose sweep did not retire it + cannot merge untested. 4. `dependabot-automerge.yml` MUST exist on both branches: for `pull_request`, GitHub runs the workflow from the PR's BASE branch. diff --git a/tools/ci/security-gates.test.mjs b/tools/ci/security-gates.test.mjs index 3348b228..d8c1028f 100644 --- a/tools/ci/security-gates.test.mjs +++ b/tools/ci/security-gates.test.mjs @@ -79,6 +79,20 @@ test("the terminal gate fails when code changed but the build or test phase did assert.match(gate.run, /exit 1/); }); +// [DIST-CI-DEPENDABOT]: the whole pipeline skips a Dependabot PR, so a green CI +// would be a pass of nothing. Every one CI sees targets `main`, where the only +// route in is the tested `dependabot-upgrades` consolidation PR. +test("the terminal gate fails a Dependabot PR to main, whose pipeline ran nothing", () => { + assert.deepEqual(ci.on.pull_request.branches, ["main"], "every PR CI sees targets main"); + assert.equal(ci.jobs["detect-changes"].if, "github.actor != 'dependabot[bot]'"); + const steps = ci.jobs.ci.steps; + const gate = steps.find((step) => step.if?.includes("github.actor == 'dependabot[bot]'")); + assert.ok(gate, "a Dependabot PR must not report green"); + assert.ok(steps.indexOf(gate) < steps.findIndex((step) => step.name === "Report")); + assert.match(gate.run, /dependabot-upgrades/, "the failure names the route that IS tested"); + assert.match(gate.run, /exit 1/); +}); + test("all compiled test consumers opt into their non-building runners", () => { for (const job of Object.values(workflow("ci-test-dotnet").jobs)) { const run = job.steps.find((step) => step.run?.startsWith("make _test-dotnet")); From fc0982734f103104222cb022b0b340c05c06e210 Mon Sep 17 00:00:00 2001 From: Christian Findlay <16697547+MelbourneDeveloper@users.noreply.github.com> Date: Sun, 18 Oct 2026 17:37:12 +1100 Subject: [PATCH 03/24] fixes --- .agents/skills/submit-pr/SKILL.md | 2 +- .../test/suite/memory-autopilot-e2e.test.ts | 7 +- .../suite/profiler-fsharp-cpu-e2e.test.ts | 307 +++++++++++ .../src/test/suite/profiler-fsharp-kit.ts | 492 ++++++++++++++++++ .../suite/profiler-fsharp-memory-e2e.test.ts | 414 +++++++++++++++ .../profiler-fsharp-retention-e2e.test.ts | 335 ++++++++++++ .../vscode/src/test/suite/profiler-kit.ts | 89 +++- .../src/test/suite/profiler-ui-helpers.ts | 38 +- .../fsharp-profiling/Directory.Build.props | 5 + .../fsharp-profiling/FsOrderLedger/Audit.fs | 68 +++ .../fsharp-profiling/FsOrderLedger/Bus.fs | 21 + .../fsharp-profiling/FsOrderLedger/Domain.fs | 41 ++ .../FsOrderLedger/FsOrderLedger.fsproj | 25 + .../fsharp-profiling/FsOrderLedger/Gate.fs | 10 + .../fsharp-profiling/FsOrderLedger/History.fs | 15 + .../fsharp-profiling/FsOrderLedger/Ledger.fs | 70 +++ .../fsharp-profiling/FsOrderLedger/Program.fs | 188 +++++++ .../fsharp-profiling/FsOrderLedger/Quotes.fs | 35 ++ .../FsOrderLedger/Rejections.fs | 14 + .../fsharp-profiling/FsPricingEngine/Csv.fs | 54 ++ .../FsPricingEngine/Domain.fs | 28 + .../FsPricingEngine/FsPricingEngine.fsproj | 23 + .../FsPricingEngine/Pricing.fs | 68 +++ .../FsPricingEngine/Program.fs | 147 ++++++ .../FsPricingEngine/Reports.fs | 47 ++ .../FsPricingEngine/Routing.fs | 47 ++ .../FsPricingEngine/Search.fs | 28 + 27 files changed, 2589 insertions(+), 29 deletions(-) create mode 100644 src/editors/vscode/src/test/suite/profiler-fsharp-cpu-e2e.test.ts create mode 100644 src/editors/vscode/src/test/suite/profiler-fsharp-kit.ts create mode 100644 src/editors/vscode/src/test/suite/profiler-fsharp-memory-e2e.test.ts create mode 100644 src/editors/vscode/src/test/suite/profiler-fsharp-retention-e2e.test.ts create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/Directory.Build.props create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsOrderLedger/Audit.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsOrderLedger/Bus.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsOrderLedger/Domain.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsOrderLedger/FsOrderLedger.fsproj create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsOrderLedger/Gate.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsOrderLedger/History.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsOrderLedger/Ledger.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsOrderLedger/Program.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsOrderLedger/Quotes.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsOrderLedger/Rejections.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsPricingEngine/Csv.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsPricingEngine/Domain.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsPricingEngine/FsPricingEngine.fsproj create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsPricingEngine/Pricing.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsPricingEngine/Program.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsPricingEngine/Reports.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsPricingEngine/Routing.fs create mode 100644 src/sharplsp/tests/fixtures/fsharp-profiling/FsPricingEngine/Search.fs diff --git a/.agents/skills/submit-pr/SKILL.md b/.agents/skills/submit-pr/SKILL.md index 700a404d..a21c83da 100644 --- a/.agents/skills/submit-pr/SKILL.md +++ b/.agents/skills/submit-pr/SKILL.md @@ -1,7 +1,7 @@ --- name: submit-pr description: Creates a pull request with a well-structured description after verifying CI passes. Use when the user asks to submit, create, or open a pull request. -disable-model-invocation: true +disable-model-invocation: false --- diff --git a/src/editors/vscode/src/test/suite/memory-autopilot-e2e.test.ts b/src/editors/vscode/src/test/suite/memory-autopilot-e2e.test.ts index 24008f04..b9ec41dd 100644 --- a/src/editors/vscode/src/test/suite/memory-autopilot-e2e.test.ts +++ b/src/editors/vscode/src/test/suite/memory-autopilot-e2e.test.ts @@ -24,6 +24,7 @@ import { clearMemoryDecorations, } from '@nimblesite/lsp-profiler-ui/out/src/memory-decorations'; import { comparablePath } from './test-helpers'; +import { setProfilerSetting } from './profiler-kit'; import { launchConfigOf, pollUntilResult } from './profiler-ui-helpers'; import { LEAK_LINES, @@ -50,11 +51,7 @@ const AUTO_CAPTURE_WAIT_MS = 40_000; /** Profiler config keys this suite toggles; reset to default after each test. */ const CONFIG_KEYS = ['autoSnapshotOnPause', 'autoSnapshot', 'autoSnapshotInterval']; -async function setProfilerConfig(key: string, value: unknown): Promise { - await vscode.workspace - .getConfiguration('sharplsp.profiler') - .update(key, value, vscode.ConfigurationTarget.Global); -} +const setProfilerConfig = setProfilerSetting; async function resetProfilerConfig(): Promise { for (const key of CONFIG_KEYS) { diff --git a/src/editors/vscode/src/test/suite/profiler-fsharp-cpu-e2e.test.ts b/src/editors/vscode/src/test/suite/profiler-fsharp-cpu-e2e.test.ts new file mode 100644 index 00000000..32f37246 --- /dev/null +++ b/src/editors/vscode/src/test/suite/profiler-fsharp-cpu-e2e.test.ts @@ -0,0 +1,307 @@ +// Tests for [PROFILER-SHARED-LIBRARY-VSCODE] on F#. See docs/specs/PROFILER-SPEC.md#PROFILER-SHARED-LIBRARY-VSCODE +// +// CPU profiling of a real F# program, FsPricingEngine: a retail checkout backend +// whose workloads are the F# shapes a profiler must name and place — +// - active patterns over CSV rows; +// - a recursive discount-rule engine that matches every union case; +// - Levenshtein search over a 2D array, ranked through a List.sortBy closure; +// - Dijkstra over an immutable Set and Map with an inner `let rec`; +// - seq expressions and a task {} builder. +// Every journey starts where a user starts: the program's row in the processes +// panel. It then reads back everything the editor shows: the live chrome, the +// gating keys, the heat map and its tiers, the results panel, and the three +// exported artifacts. lspkit's [PROF-CPU-START], [PROF-CPU-SNAPSHOT], +// [PROF-CPU-PROGRESS], [PROF-UI-CPU], [PROF-UI-HEATMAP], [PROF-EXPORT]. +import * as assert from 'node:assert/strict'; +import * as vscode from 'vscode'; +import { lastProfileResult, recordedOperations } from '@nimblesite/lsp-profiler-ui'; +import { + disposeFlamegraphPanel, + flamegraphPanelOpen, +} from '@nimblesite/lsp-profiler-ui/out/src/profiler-flamegraph-html'; +import type { ProfileSessionList } from '@nimblesite/lsp-profiler-ui/out/src/wire-generated'; +import { + execute, + executeError, + profilerStore, + profilingSuite, + setProfilerSetting, + stopProfiling, + uiCommand, + waitFor, +} from './profiler-kit'; +import { + assertArtifacts, + assertCpuIdle, + assertCpuLive, + assertHeatMap, + assertIdleRow, + assertRanked, + assertResultsPanel, + awaitLiveProgress, + command, + count, + ENGINE, + heatOn, + hotFunction, + processRow, + profileRow, + showSources, + type Span, + within, +} from './profiler-fsharp-kit'; + +const SAMPLE_MS = 3_000; +const STOP_MS = 90_000; + +/** One F# workload of FsPricingEngine and where its time must land. */ +interface Workload { + readonly command: string; + readonly shape: string; + /** Functions that must be hot, in `span.file`. The first names the artifacts. */ + readonly identifiers: readonly string[]; + /** The lines its hot lines fall in. */ + readonly span: Span; +} + +const WORKLOADS: readonly Workload[] = [ + { + command: 'parse', + shape: 'active patterns over 2,000 CSV rows', + identifiers: ['parseAll', 'parseLine'], + span: { file: 'Csv.fs', first: 14, last: 39 }, + }, + { + command: 'price', + shape: 'a recursive discount-rule engine matching every union case', + identifiers: ['basketTotal', 'applyRules'], + span: { file: 'Pricing.fs', first: 10, last: 68 }, + }, + { + command: 'search', + shape: 'Levenshtein over a 2D array, ranked through a List.sortBy closure', + identifiers: ['rank', 'distance'], + span: { file: 'Search.fs', first: 10, last: 28 }, + }, + { + command: 'route', + shape: 'Dijkstra over an immutable Set and Map with an inner let rec', + identifiers: ['shortestPath'], + span: { file: 'Routing.fs', first: 29, last: 47 }, + }, + { + command: 'report', + shape: 'seq expressions grouped and summed, and a task {} builder', + identifiers: ['dailyTotals'], + span: { file: 'Reports.fs', first: 13, last: 47 }, + }, +]; + +async function sessions(): Promise { + return (await execute('cpu.list', {})).sessions; +} + +async function delay(ms: number): Promise { + await new Promise((resolve) => setTimeout(resolve, ms)); +} + +/** Both progress notifications opened and closed in order since `from`. */ +function assertProgressClosed(from: number): void { + const ops = recordedOperations().slice(from); + for (const title of ['SharpLsp: Starting CPU profiler', 'SharpLsp: Stopping profiler']) { + const begin = ops.indexOf(`begin:${title}`); + assert.ok(begin !== -1, `"${title}" shows progress: ${ops.join(' | ')}`); + assert.ok(ops.indexOf(`end:${title}`, begin) > begin, `"${title}" closes`); + } +} + +suite('F# CPU profiling — FsPricingEngine, one F# shape at a time', () => { + const live = profilingSuite('profiler-fs-cpu-', ENGINE, { + stopping: async () => { + if (profilerStore().profiler.value.cpu !== 'idle') await stopProfiling(STOP_MS); + await setProfilerSetting('showInlineHeatMap', undefined); + disposeFlamegraphPanel(); + }, + }); + + for (const workload of WORKLOADS) { + test(`${workload.shape}: Profile CPU from the F# program's row → live chrome → UI snapshot → stop → heat map, results panel and artifacts name ${workload.identifiers.join(', ')}`, async function () { + this.timeout(STOP_MS * 3); + const { target } = live; + const started = await command(target, `run ${workload.command}`); + await showSources(ENGINE, [workload.span.file, 'Program.fs']); + + // The row the user clicks, before anything runs. + const row = await processRow(target.pid); + assertIdleRow(row, target.pid, ENGINE); + assert.equal(profilerStore().profiler.value.cpu, 'idle', 'nothing profiles yet'); + + // Click Profile CPU: the chrome, the keys and the wire agree on one session. + const opsBefore = recordedOperations().length; + const sessionId = await profileRow(row); + await assertCpuLive(target.pid); + const top = await awaitLiveProgress(); + assert.ok(top.length > 0, 'the live readout names a top function'); + const listed = (await sessions()).filter((s) => s.pid === target.pid); + assert.equal(listed.length, 1, 'exactly one server session for the program'); + const [mine] = listed; + assert.ok(mine); + assert.equal(mine.session_id, sessionId, 'the UI drives the listed session'); + assert.equal(mine.runtime, 'dotnet'); + assert.ok(mine.runtime_version.length > 0, 'the runtime version is reported'); + assert.ok(!Number.isNaN(Date.parse(mine.started_at)), `RFC 3339 start: ${mine.started_at}`); + assert.ok(mine.sample_count > 0, `the live count is real: ${mine.sample_count}`); + + // A second click and a second wire start are both refused; the session survives. + await vscode.commands.executeCommand(uiCommand('profileProcess'), row); + assert.equal(profilerStore().profiler.value.cpuSessionId, sessionId, 'the click is a no-op'); + const refused = await executeError('cpu.start', { pid: target.pid }); + assert.equal(refused?.code, -32004, `ALREADY_PROFILING: ${refused?.message}`); + assert.equal((await sessions()).filter((s) => s.pid === target.pid).length, 1); + + // A UI snapshot reports the F# code so far and the session keeps running. + await vscode.commands.executeCommand(uiCommand('profileSnapshot')); + const snapshot = await waitFor( + () => lastProfileResult(), + (result) => result?.sessionId === sessionId, + STOP_MS, + 'the snapshot result of this session', + ); + assert.ok(snapshot); + assertRanked(snapshot); + for (const identifier of workload.identifiers) { + assert.ok( + hotFunction(snapshot, identifier, workload.span.file), + `the snapshot already names ${identifier} in ${workload.span.file}: ${snapshot.hotFunctions.map((f) => f.name).join(', ')}`, + ); + } + assert.equal(profilerStore().profiler.value.cpu, 'active', 'a snapshot never ends it'); + await delay(SAMPLE_MS); + + // Stop: the session ends everywhere, and the result covers the whole run. + await stopProfiling(STOP_MS); + await assertCpuIdle(target.pid); + assert.deepEqual( + (await sessions()).filter((s) => s.pid === target.pid), + [], + 'the stopped session leaves the list', + ); + const result = lastProfileResult(); + assert.ok(result); + assert.equal(result.sessionId, sessionId, 'the stop result is this session’s'); + assert.ok( + result.totalSamples > snapshot.totalSamples, + `sampling went on past the snapshot: ${result.totalSamples} > ${snapshot.totalSamples}`, + ); + assertRanked(result); + for (const identifier of workload.identifiers) { + const fn = hotFunction(result, identifier, workload.span.file); + assert.ok(fn, `${identifier} is hot in ${workload.span.file}`); + assert.ok(fn.line >= 1, `${identifier} carries its F# line`); + } + assert.ok( + result.hotLines.some((line) => within(workload.span, line.file, line.line)), + `a hot line sits in ${workload.span.file} ${workload.span.first}-${workload.span.last}`, + ); + assert.equal(heatOn('Program.fs').length > 0, result.hotLines.some((l) => l.file.endsWith('Program.fs')), + 'Program.fs is painted exactly when it has hot lines'); + assertHeatMap(result, workload.span); + assertResultsPanel(result, workload.identifiers); + assertArtifacts(result, workload.identifiers[0] ?? '', workload.span.file); + assertProgressClosed(opsBefore); + + // Closing the panel never strands the result: Show Profile Results re-opens it. + disposeFlamegraphPanel(); + assert.equal(flamegraphPanelOpen(), false); + await vscode.commands.executeCommand(uiCommand('profileShowResults')); + await waitFor(() => flamegraphPanelOpen(), (open) => open, 5_000, 'the panel re-opens'); + assertResultsPanel(result, workload.identifiers); + + // Attaching was never invasive: the program kept working throughout. + const after = await command(target, 'stats'); + assert.ok( + count(after, 'iterations') > count(started, 'iterations'), + `the F# program kept running under the profiler: ${JSON.stringify(after)}`, + ); + }); + } + + test('one session, two phases: parse → snapshot → switch to search → snapshot → stop paints both files; heat map off, then a mixed run paints every module', async function () { + this.timeout(STOP_MS * 5); + const { target } = live; + const [parse, search, route] = WORKLOADS.filter((w) => + ['parse', 'search', 'route'].includes(w.command), + ); + assert.ok(parse && search && route); + await command(target, 'run parse'); + await showSources(ENGINE, ['Csv.fs', 'Search.fs']); + const sessionId = await profileRow(await processRow(target.pid)); + await awaitLiveProgress(); + + // Phase one: only CSV parsing has run, so search cannot be hot yet. + await vscode.commands.executeCommand(uiCommand('profileSnapshot')); + const first = await waitFor( + () => lastProfileResult(), + (r) => r?.sessionId === sessionId, + STOP_MS, + 'phase-one snapshot', + ); + assert.ok(first && hotFunction(first, 'parseAll', 'Csv.fs'), 'parsing is hot in phase one'); + assert.equal(hotFunction(first, 'distance', 'Search.fs'), undefined, 'search never ran yet'); + + // Phase two: the same session sees the switch. + await command(target, 'run search'); + await delay(SAMPLE_MS * 2); + await vscode.commands.executeCommand(uiCommand('profileSnapshot')); + const second = await waitFor( + () => lastProfileResult(), + (r) => r !== first && r?.sessionId === sessionId && hotFunction(r, 'distance', 'Search.fs') !== undefined, + STOP_MS, + 'phase-two snapshot naming distance', + ); + assert.ok(second); + assert.ok(second.totalSamples > first.totalSamples, 'the second snapshot covers more'); + assert.ok(hotFunction(second, 'parseAll', 'Csv.fs'), 'phase one is still in the session'); + await stopProfiling(STOP_MS); + const both = lastProfileResult(); + assert.ok(both && both.totalSamples >= second.totalSamples, 'the stop covers both phases'); + assertHeatMap(both, parse.span); + assertHeatMap(both, search.span); + + // Heat map off: the result still lands in the panel, but no tint is painted. + await setProfilerSetting('showInlineHeatMap', false); + await command(target, 'run route'); + await showSources(ENGINE, ['Routing.fs']); + const routed = await profileRow(await processRow(target.pid)); + await awaitLiveProgress(); + await stopProfiling(STOP_MS); + const quiet = lastProfileResult(); + assert.ok(quiet && quiet.sessionId === routed, 'a second session on the same program'); + assert.ok(hotFunction(quiet, 'shortestPath', 'Routing.fs'), 'routing is hot'); + assert.deepEqual(heatOn('Routing.fs'), [], 'the setting turns the tint off'); + assertResultsPanel(quiet, ['shortestPath']); + + // Heat map back on, every workload at once: each module is hot and painted. + await setProfilerSetting('showInlineHeatMap', true); + await command(target, 'run mixed'); + const files = WORKLOADS.map((w) => w.span.file); + await showSources(ENGINE, files); + await profileRow(await processRow(target.pid)); + await awaitLiveProgress(); + await delay(SAMPLE_MS * 2); + await stopProfiling(STOP_MS); + const mixed = lastProfileResult(); + assert.ok(mixed); + assertRanked(mixed); + for (const workload of WORKLOADS) { + const identifier = workload.identifiers[0] ?? ''; + assert.ok(hotFunction(mixed, identifier, workload.span.file), `mixed: ${identifier} is hot`); + assertHeatMap(mixed, workload.span); + } + assertResultsPanel( + mixed, + WORKLOADS.map((w) => w.identifiers[0] ?? ''), + ); + await command(target, 'idle'); + }); +}); diff --git a/src/editors/vscode/src/test/suite/profiler-fsharp-kit.ts b/src/editors/vscode/src/test/suite/profiler-fsharp-kit.ts new file mode 100644 index 00000000..0b7d7ea4 --- /dev/null +++ b/src/editors/vscode/src/test/suite/profiler-fsharp-kit.ts @@ -0,0 +1,492 @@ +// Tests for [PROFILER-SHARED-LIBRARY-VSCODE] on F#. See docs/specs/PROFILER-SPEC.md#PROFILER-SHARED-LIBRARY-VSCODE +// +// The F# profiler suites drive SharpLsp's two real-world F# programs +// (src/sharplsp/tests/fixtures/fsharp-profiling) through the extension's own UI +// over the real LSP. This kit owns what every F# suite shares: +// - the fixtures' stdin protocol and the exact live counts they report; +// - the exact source lines each F# shape lands on; +// - the spec checks every suite repeats on what the editor shows: the live +// chrome, the heat map, the results panel, the exported artifacts and the +// memory track (lspkit's [PROF-UI-CPU], [PROF-UI-HEATMAP], [PROF-EXPORT], +// [PROF-UI-MEMORY], [PROF-AGGREGATION-THRESHOLD]). +import * as assert from 'node:assert/strict'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import * as vscode from 'vscode'; +import { + appliedMemoryDecorations, + appliedProfileDecorations, + contextKeyValues, + flamegraphPanelState, + processesViewState, + profilerStatusText, + type ProcessTreeItem, + type ProfileResult, +} from '@nimblesite/lsp-profiler-ui'; +import type { AppliedDecoration } from '@nimblesite/lsp-profiler-ui/out/src/profiler-decorations'; +import type { MemoryDashboardSnapshot } from '@nimblesite/lsp-profiler-ui/out/src/memory-dashboard'; +import type { + HeapDiff, + MemoryStep, + SuspectedLeak, +} from '@nimblesite/lsp-profiler-ui/out/src/wire-generated'; +import { + completed, + execute, + fixtureFile, + profilerStore, + rowOf, + type Target, + tooltipText, + uiCommand, + waitFor, +} from './profiler-kit'; +import { comparablePath } from './test-helpers'; +import { LSP_RESPONSE_MS, PROCESS_START_MS } from './test-timeouts'; + +export const ENGINE = 'FsPricingEngine'; +export const LEDGER = 'FsOrderLedger'; + +/** The CPU heat palette, critical to cool ([PROF-UI-WEBVIEW]: orange/amber for CPU). */ +export const CPU_PALETTE = ['#e8500a', '#f97316', '#fbbf24', '#4a5468']; +/** The purple memory-allocation palette ([PROF-UI-WEBVIEW]: purple for memory). */ +export const MEMORY_PALETTE = ['#c084fc', '#a78bfa', '#8b5cf6', '#7c3aed']; +/** The leak-confidence palette ([PROF-UI-WEBVIEW]: red for leaks). */ +export const LEAK_PALETTE = ['#ef4444', '#f87171', '#fb923c', '#a78bfa']; +/** A leak badge names its confidence in its after-line text. */ +const BADGE = /\b(LOW|MEDIUM|HIGH|DEFINITE)\b/; + +// ── The fixtures' stdin protocol ───────────────────────────────────────────── + +/** The exact live counts a fixture reports after every command. */ +export type Counts = Readonly>; + +/** Send `line`; assert the fixture accepted it; return the integer counts it reported. */ +export async function command(target: Target, line: string): Promise { + const reply = await target.send(line); + assert.ok(reply.startsWith(`OK ${line}`), `the fixture must accept "${line}": ${reply}`); + const pairs = reply + .split(' ') + .filter((word) => word.includes('=')) + .map((word) => word.split('=') as [string, string]) + .filter(([, value]) => Number.isInteger(Number(value))); + return Object.fromEntries(pairs.map(([key, value]) => [key, Number(value)])); +} + +/** One count from a reply; fails when the fixture did not report it. */ +export function count(counts: Counts, key: string): number { + const value = counts[key]; + assert.ok(value !== undefined, `the reply carries ${key}: ${JSON.stringify(counts)}`); + return value; +} + +// ── Where each F# shape lands ──────────────────────────────────────────────── + +/** A 1-based line range inside one fixture file. */ +export interface Span { + readonly file: string; + readonly first: number; + readonly last: number; +} + +/** True when `line` of `file` falls inside `span`. */ +export function within(span: Span, file: string | undefined, line: number): boolean { + return isFile(file, span.file) && line >= span.first && line <= span.last; +} + +/** True when `file` is the fixture file named `name` (any fixture). */ +export function isFile(file: string | undefined, name: string): boolean { + return file !== undefined && path.basename(comparablePath(file)) === comparablePath(name); +} + +/** The absolute path of a fixture file, as the editor and the server name it. */ +export function sourceOf(fixture: string, file: string): string { + return fixtureFile(fixture, file); +} + +/** Open fixture files side by side, so decorations have editors to land on. */ +export async function showSources(fixture: string, files: readonly string[]): Promise { + await vscode.commands.executeCommand('workbench.action.closeAllEditors'); + for (const [index, file] of files.entries()) { + const document = await vscode.workspace.openTextDocument(sourceOf(fixture, file)); + await vscode.window.showTextDocument(document, { + preview: false, + viewColumn: index === 0 ? vscode.ViewColumn.One : vscode.ViewColumn.Beside, + }); + } + const visible = vscode.window.visibleTextEditors.map((e) => path.basename(e.document.fileName)); + for (const file of files) { + assert.ok(visible.includes(file), `${file} is visible: ${visible.join(', ')}`); + } +} + +// ── The processes panel row of a live F# program ───────────────────────────── + +/** Refresh the panel and return the real row the user would click for `pid`. */ +export async function processRow(pid: number): Promise { + await vscode.commands.executeCommand(uiCommand('refreshProcesses')); + const row = await waitFor( + () => rowOf(pid), + (item) => item !== undefined, + LSP_RESPONSE_MS, + `the processes panel row of PID ${pid}`, + ); + assert.ok(row && 'process' in row, `PID ${pid} renders as a process row`); + return row as ProcessTreeItem; +} + +/** [PROF-UI-PROCESSES] + [PROF-UI-CAPABILITIES]: what an idle F# program's row shows. */ +export function assertIdleRow(row: ProcessTreeItem, pid: number, name: string): void { + assert.equal(row.id, `process:${pid}`, 'a stable row id'); + const tip = tooltipText(row); + for (const needle of [`PID ${pid}`, '.NET: ', 'CPU: ', 'Memory: ', 'Runtime: ']) { + assert.ok(tip.includes(needle), `the tooltip carries "${needle}": ${tip}`); + } + assert.equal(row.process.pid, pid); + assert.equal(row.process.runtime, 'dotnet', 'an F# program is classified as .NET'); + assert.ok(row.process.name.includes(name), `the row names the program: ${row.process.name}`); + assert.equal(row.process.debuggable, true, 'a live F# program is attachable'); + assert.ok(row.process.runtimeVersion, 'the runtime version is resolved'); + assert.equal(row.contextValue, 'processDebuggee', 'direct memory: Track Memory is offered'); + assert.equal(row.command, undefined, 'an assembly entry is never opened as source'); + assert.ok( + !(row.iconPath instanceof vscode.ThemeIcon && row.iconPath.id === 'flame'), + 'an idle row does not flame', + ); +} + +/** Profile CPU from the real row, exactly as its inline button does. */ +export async function profileRow(row: ProcessTreeItem): Promise { + await vscode.commands.executeCommand(uiCommand('profileProcess'), row); + await waitFor( + () => profilerStore().profiler.value.cpu, + (cpu) => cpu === 'active', + PROCESS_START_MS, + `CPU session on PID ${row.process.pid}`, + ); + const id = profilerStore().profiler.value.cpuSessionId; + assert.ok(id, 'a live session id'); + return id; +} + +// ── Live chrome ([PROF-UI-STATE], [PROF-UI-PROCESSES]) ─────────────────────── + +/** The `sharplsp.profiler.*` gating keys the library last wrote. */ +export function keys(): Record { + const values = contextKeyValues(); + const read = (name: string) => values.get(`sharplsp.profiler.${name}`); + return { + profiling: read('profiling'), + cpuBusy: read('cpuBusy'), + memoryTracking: read('memoryTracking'), + memoryBusy: read('memoryBusy'), + profilerBusy: read('profilerBusy'), + }; +} + +/** Everything a live CPU session on `pid` paints, with no memory session running. */ +export async function assertCpuLive(pid: number): Promise { + const session = profilerStore().profiler.value; + assert.equal(session.cpuPid, pid, 'the session profiles the clicked program'); + assert.ok(profilerStatusText()?.includes('Profiling'), `status: ${profilerStatusText()}`); + assert.deepEqual( + { profiling: keys().profiling, cpuBusy: keys().cpuBusy, profilerBusy: keys().profilerBusy }, + { profiling: true, cpuBusy: true, profilerBusy: true }, + 'the CPU gating keys follow the session', + ); + assert.equal(processesViewState().badge, 1, 'a badge while busy'); + assert.ok( + processesViewState().message?.startsWith(`🔥 Profiling PID ${pid}`) === true, + `the panel message names the PID: ${processesViewState().message}`, + ); + const row = await waitFor( + () => rowOf(pid), + (item) => item?.contextValue === 'processProfiling', + LSP_RESPONSE_MS, + 'the profiled row to swap to its Stop action', + ); + assert.ok( + row?.iconPath instanceof vscode.ThemeIcon && row.iconPath.id === 'flame', + 'the profiled row flames', + ); +} + +/** Wait for live progress: elapsed time, a sample count and a top function. */ +export async function awaitLiveProgress(): Promise { + const store = profilerStore(); + const live = await waitFor( + () => store.profiler.value, + (s) => s.durationSecs >= 1 && s.sampleCount > 0 && (s.topFunction ?? '').length > 0, + LSP_RESPONSE_MS * 2, + 'live progress: elapsed time, samples and a top function', + ); + await waitFor( + () => profilerStatusText() ?? '', + (text) => /[1-9][\d.]* ?K? samples/.test(text), + LSP_RESPONSE_MS, + 'the live sample count in the status bar', + ); + return live.topFunction ?? ''; +} + +/** Nothing is profiling CPU: store, status bar, keys and the row agree. */ +export async function assertCpuIdle(pid: number): Promise { + assert.equal(profilerStore().profiler.value.cpu, 'idle'); + assert.equal(profilerStore().profiler.value.cpuSessionId, undefined, 'no stale session id'); + assert.equal(profilerStatusText(), undefined, 'the CPU status bar hides'); + assert.equal(keys().profiling, false); + assert.equal(keys().cpuBusy, false); + const row = await waitFor( + () => rowOf(pid), + (item) => item !== undefined && item.contextValue !== 'processProfiling', + LSP_RESPONSE_MS, + 'the row to drop its profiling marker', + ); + assert.equal(row?.contextValue, 'processDebuggee'); +} + +// ── The CPU result as the editor shows it ──────────────────────────────────── + +/** The heat tier a share paints with; `undefined` near a tier boundary. */ +export function cpuTier(percentage: number): string | undefined { + const tiers: readonly [number, string][] = [ + [20, '#e8500a'], + [10, '#f97316'], + [5, '#fbbf24'], + [0, '#4a5468'], + ]; + if ([20, 10, 5].some((edge) => Math.abs(percentage - edge) < 0.5)) return undefined; + return tiers.find(([floor]) => percentage > floor)?.[1]; +} + +/** [PROF-AGGREGATION-LOGIC] + [PROF-AGGREGATION-THRESHOLD]: shares are sane and ranked. */ +export function assertRanked(result: ProfileResult): void { + assert.ok(result.totalSamples > 0, `samples: ${result.totalSamples}`); + const shares = result.hotFunctions.map((fn) => fn.percentage); + assert.deepEqual(shares, [...shares].sort((a, b) => b - a), 'hot functions arrive ranked'); + for (const fn of result.hotFunctions) { + assert.ok(fn.percentage <= 100 && fn.percentage >= 2, `${fn.name}: ${fn.percentage}%`); + assert.ok(fn.selfPercentage <= fn.percentage + 1e-9, `${fn.name}: self within inclusive`); + assert.ok(fn.samples <= result.totalSamples, `${fn.name}: never more samples than ticks`); + } + for (const line of result.hotLines) { + assert.ok(line.percentage <= 100 && line.percentage >= 1, `line ${line.line}: ${line.percentage}%`); + assert.ok(line.line >= 1, 'lines are 1-based in the editor'); + } +} + +/** A hot function whose name contains `identifier` and that resolves to `file`. */ +export function hotFunction(result: ProfileResult, identifier: string, file: string) { + return result.hotFunctions.find((fn) => fn.name.includes(identifier) && isFile(fn.file, file)); +} + +/** The heat map as painted on `file`. */ +export function heatOn(file: string): AppliedDecoration[] { + return appliedProfileDecorations().filter((d) => isFile(d.file, file)); +} + +/** [PROF-UI-HEATMAP]: `span` is painted, in the CPU palette, graded by each line's share. */ +export function assertHeatMap(result: ProfileResult, span: Span): void { + const painted = heatOn(span.file); + assert.ok( + painted.some((d) => within(span, d.file, d.line)), + `${span.file} ${span.first}-${span.last} is painted: ${JSON.stringify(painted.map((d) => d.line))}`, + ); + for (const decoration of painted) { + assert.ok(CPU_PALETTE.includes(decoration.color), `CPU track colour: ${decoration.color}`); + assert.ok(/\d%/.test(decoration.contentText), `a share is shown: ${decoration.contentText}`); + const hot = result.hotLines.find((l) => isFile(l.file, span.file) && l.line === decoration.line); + assert.ok(hot, `line ${decoration.line} is painted only because it is a hot line`); + const tier = cpuTier(hot.percentage); + if (tier !== undefined) { + assert.equal(decoration.color, tier, `line ${hot.line} at ${hot.percentage}% wears its tier`); + } + } +} + +/** [PROF-UI-CPU]: the branded results panel lists `identifiers` and links to source. */ +export function assertResultsPanel(result: ProfileResult, identifiers: readonly string[]): void { + const panel = flamegraphPanelState(); + assert.ok(panel, 'the results panel is open'); + assert.equal(panel.title, 'SharpLsp Profiler'); + assert.ok(panel.html.includes('SHARPLSP PROFILER'), 'brand'); + assert.ok(!/basilisk/i.test(panel.html), 'no other product named'); + assert.ok(panel.html.includes('`); } + +// ── Process-row fixtures ([PROF-UI-PROCESSES]) ───────────────────── + +const MB = 1024 * 1024; + +/** + * A debuggable process row; each caller overrides what it exercises. The + * one fixture every Processes-panel test builds its table from. + */ +export function processFixture( + pid: number, + overrides: Partial = {}, +): ProcessInfo { + return { + pid, + ppid: 1, + name: 'python3.12', + interpreterPath: '/usr/bin/python3.12', + script: null, + runtime: 'python', + runtimeVersion: '3.12.1', + cpuPercent: 0, + memoryBytes: MB, + runtimeSecs: 1, + user: null, + requiresElevation: false, + inWorkspace: false, + launcher: null, + debuggable: true, + undebuggableReason: null, + ...overrides, + }; +} + +/** Read the PID a process row carries (the arg passed to inline commands). */ +export function rowPid(item: vscode.TreeItem): number | undefined { + // `process` is attached to the row by the provider under test, not by + // `vscode.TreeItem`, so it is read back by name rather than asserted on. + return numberField(rawField(item, 'process'), 'pid'); +} + +/** The row of `pid`, failing the test when the panel dropped it. */ +export function rowByPid(rows: readonly vscode.TreeItem[], pid: number, what: string): vscode.TreeItem { + const row = rows.find((item) => rowPid(item) === pid); + assert.ok(row !== undefined, `${what}: the PID ${pid} row must exist`); + return row!; +} + +// ── Decoration ledgers ───────────────────────────────────────────── + +/** What one applied decoration must show: its line, a colour, text it carries. */ +export interface PaintedExpectation { + readonly line?: number; + readonly color?: string; + /** A substring the decoration's after-line text must include. */ + readonly includes?: string; +} + +/** Assert a decoration was applied and shows exactly what `expect` names. */ +export function assertPainted( + painted: + | { line: number; color: string; contentText: string } + | undefined, + expect: PaintedExpectation, + what: string, +): void { + assert.ok(painted !== undefined, `${what}: a decoration must be applied`); + if (expect.line !== undefined) { + assert.equal(painted.line, expect.line, `${what}: it lands on its line`); + } + if (expect.color !== undefined) { + assert.equal(painted.color, expect.color, `${what}: it wears its colour`); + } + if (expect.includes !== undefined) { + assert.ok(painted.contentText.includes(expect.includes), painted.contentText); + } +} + +// ── Dashboard snapshots ──────────────────────────────────────────── + +/** A minimal dashboard snapshot (fresh session: one capture, no diff yet). */ +export function dashboardSnapshot( + overrides: Partial = {}, +): MemoryDashboardSnapshot { + return { + memorySessionId: 'mem-1', + snapshotId: 'snap-1', + currentMemory: 1_048_576, + peakMemory: 2_097_152, + gcObjects: 1200, + gcCounts: [700, 12, 3], + topAllocations: [{ file: '/app/Program.cs', line: 10, size: 4096, count: 8 }], + timeline: [], + heapProfilePath: '', + ...overrides, + }; +} diff --git a/src/website/tools/profiler-film/capture.mjs b/src/website/tools/profiler-film/capture.mjs index 297a94dc..fa19fcce 100644 --- a/src/website/tools/profiler-film/capture.mjs +++ b/src/website/tools/profiler-film/capture.mjs @@ -14,21 +14,22 @@ import { launchVsCode, runCommand } from './vscode.mjs'; import { captureFlameGraph } from './flamegraph.mjs'; import { PARK, + STOREFRONT, clickSharpLspIcon, + cpuStills, cueLog, + demoApp, + demoRow, headerButton, openFile, record, rowButton, scrollDashboard, sleep, - stageStorefront, + stageDemo, still, - storefront, - storefrontRow, - toggleSideBar, trackMemory, - waitForStorefrontRow, + waitForDemoRow, } from './studio.mjs'; const here = path.dirname(fileURLToPath(import.meta.url)); @@ -38,15 +39,6 @@ const clips = path.join(out, 'clips'); const PROFILE_SECONDS = 10; const MEMORY_SECONDS = 26; -/** - * Toggle `title`'s editor group maximised by double-clicking its tab — no - * palette, which a refreshing webview would close by stealing focus. - */ -async function toggleMaximised(page, title) { - await page.locator('.tab', { hasText: title }).first().dblclick(); - await sleep(1200); -} - async function cpuAct(page, cues) { await cues.click(page, await rowButton(page, 'Profile CPU'), 'Profile CPU'); await sleep(7000); @@ -57,17 +49,6 @@ async function cpuAct(page, cues) { await sleep(3500); } -async function cpuStills(page) { - await toggleMaximised(page, 'SharpLsp Profiler'); - await still(page, stills, 'profiler-results'); - await toggleMaximised(page, 'SharpLsp Profiler'); - await toggleMaximised(page, 'CheckoutPipeline.cs'); - await toggleSideBar(page); - await still(page, stills, 'profiler-heatmap'); - await toggleSideBar(page); - await toggleMaximised(page, 'CheckoutPipeline.cs'); -} - async function memoryAct(page, cues) { await trackMemory(page, (locator, label) => cues.click(page, locator, label)); await sleep(MEMORY_SECONDS * 1000); @@ -86,9 +67,9 @@ async function prepare(page) { await clickSharpLspIcon(page); await openFile(page, 'CheckoutPipeline.cs'); await sleep(15_000); - await waitForStorefrontRow(page); + await waitForDemoRow(page); await page.mouse.move(PARK.x, PARK.y); - await storefrontRow(page).hover(); + await demoRow(page).hover(); await sleep(600); await page.screenshot({ path: path.join(stills, 'profiler-processes.png') }); } @@ -97,12 +78,12 @@ async function shoot(page, cdp, app) { const set = { cdp, cues: cueLog(clips), frames: path.join(out, 'frames'), clips }; await prepare(page); await record(set, 'cpu', () => cpuAct(page, set.cues)); - await cpuStills(page); + await cpuStills(page, stills, 'CheckoutPipeline.cs', 'profiler'); await runCommand(page, 'View: Close All Editors'); await openFile(page, 'SessionCache.cs'); app.stop(); await app.start(); - await waitForStorefrontRow(page); + await waitForDemoRow(page); await record(set, 'memory', () => memoryAct(page, set.cues)); await memoryStills(page); } @@ -110,9 +91,9 @@ async function shoot(page, cdp, app) { async function main() { fs.mkdirSync(stills, { recursive: true }); fs.mkdirSync(clips, { recursive: true }); - const { root, workspace } = stageStorefront(); + const { root, workspace } = stageDemo(STOREFRONT); const editor = await launchVsCode({ workspace }); - const app = storefront(workspace); + const app = demoApp(workspace); await app.start(); try { await shoot(editor.page, editor.cdp, app); diff --git a/src/website/tools/profiler-film/encode.mjs b/src/website/tools/profiler-film/encode.mjs index 9cd7df3f..b83008d1 100644 --- a/src/website/tools/profiler-film/encode.mjs +++ b/src/website/tools/profiler-film/encode.mjs @@ -1,12 +1,14 @@ // Implements [WEB-DESIGN-MEDIA]. How every profiler video reaches the site: one // H.264 and one VP9 setting, a WebP poster per video, WebP stills. import { spawnSync } from 'node:child_process'; +import fs from 'node:fs'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; const here = path.dirname(fileURLToPath(import.meta.url)); export const out = path.join(here, 'out'); export const videos = path.resolve(here, '../../src/assets/videos'); +export const shots = path.resolve(here, '../../src/assets/screenshots/profiler'); /** H.264 for reach: every site video has an MP4. */ export const H264 = ['-c:v', 'libx264', '-preset', 'veryslow', '-crf', '26', '-pix_fmt', 'yuv420p', '-movflags', '+faststart', '-an']; @@ -29,3 +31,11 @@ export function poster(video, at, name) { ffmpeg(['-ss', String(at), '-i', video, '-frames:v', '1', frame], `${name} poster`); webp(frame, path.join(videos, `${name}-poster.webp`), 1920); } + +/** Every PNG still under `dir`, as the site's 2400px WebP of the same name. */ +export function publishStills(dir) { + fs.mkdirSync(shots, { recursive: true }); + for (const file of fs.readdirSync(dir).filter((name) => name.endsWith('.png'))) { + webp(path.join(dir, file), path.join(shots, file.replace(/\.png$/, '.webp')), 2400); + } +} diff --git a/src/website/tools/profiler-film/fs-storefront/Catalog.fs b/src/website/tools/profiler-film/fs-storefront/Catalog.fs new file mode 100644 index 00000000..a1552a90 --- /dev/null +++ b/src/website/tools/profiler-film/fs-storefront/Catalog.fs @@ -0,0 +1,14 @@ +module FsStorefront.Catalog + +open System.Globalization + +let private adjectives = [| "Ergonomic"; "Rustic"; "Sleek"; "Vintage" |] +let private nouns = [| "Keyboard"; "Lamp"; "Backpack"; "Kettle"; "Monitor" |] + +let private row index = + let name = $"{adjectives[index % adjectives.Length]} {nouns[index % nouns.Length]}" + let price = (9.99m + decimal (index % 90)).ToString CultureInfo.InvariantCulture + $"SKU-%05d{index},{name},{price}" + +/// The catalog as CSV, one product per line. +let generate itemCount = List.init itemCount row |> String.concat "\n" diff --git a/src/website/tools/profiler-film/fs-storefront/Checkout.fs b/src/website/tools/profiler-film/fs-storefront/Checkout.fs new file mode 100644 index 00000000..7465d3b8 --- /dev/null +++ b/src/website/tools/profiler-film/fs-storefront/Checkout.fs @@ -0,0 +1,30 @@ +module FsStorefront.Checkout + +open System +open System.Globalization + +let private toProduct (line: string) = + match line.Split ',' with + | [| sku; name; price |] -> Some { Sku = sku; Name = name; Price = Decimal.Parse(price, CultureInfo.InvariantCulture) } + | _ -> None + +let parseCatalog (csv: string) = csv.Split '\n' |> Array.choose toProduct |> List.ofArray + +let pickBasket seed products = + let random = Random seed + products |> List.sortBy (fun _ -> random.Next()) |> List.truncate 12 + +let private applyDiscounts product = + let hash = Seq.fold (fun hash round -> (hash * 31) ^^^ int product.Sku[round % product.Sku.Length]) 17 (seq { 0..4_999 }) + if hash % 7 = 0 then product.Price * 0.9m else product.Price + +let priceBasket basket = basket |> List.sumBy applyDiscounts + +let renderInvoice orderId basket total = + let lines = basket |> List.map (fun p -> sprintf "%-12s%-28s%10.2f" p.Sku p.Name p.Price) + sprintf "INVOICE #%d\n%s\nTOTAL %.2f" orderId (String.concat "\n" lines) total + +let checkout catalog orderId = + let basket = parseCatalog catalog |> pickBasket orderId + let total = priceBasket basket + { Id = orderId; Items = basket; Total = total; Invoice = renderInvoice orderId basket total } diff --git a/src/website/tools/profiler-film/fs-storefront/Directory.Build.props b/src/website/tools/profiler-film/fs-storefront/Directory.Build.props new file mode 100644 index 00000000..37390751 --- /dev/null +++ b/src/website/tools/profiler-film/fs-storefront/Directory.Build.props @@ -0,0 +1,3 @@ + + + diff --git a/src/website/tools/profiler-film/fs-storefront/Domain.fs b/src/website/tools/profiler-film/fs-storefront/Domain.fs new file mode 100644 index 00000000..5546f827 --- /dev/null +++ b/src/website/tools/profiler-film/fs-storefront/Domain.fs @@ -0,0 +1,11 @@ +namespace FsStorefront + +type Product = { Sku: string; Name: string; Price: decimal } + +type Order = { Id: int; Items: Product list; Total: decimal; Invoice: string } + +type Stage = + | Browsing + | CheckedOut of total: decimal + +type CustomerSession = { Order: Order; Stage: Stage; Snapshot: byte array } diff --git a/src/website/tools/profiler-film/fs-storefront/FsStorefront.fsproj b/src/website/tools/profiler-film/fs-storefront/FsStorefront.fsproj new file mode 100644 index 00000000..9fc17acf --- /dev/null +++ b/src/website/tools/profiler-film/fs-storefront/FsStorefront.fsproj @@ -0,0 +1,13 @@ + + + Exe + net10.0 + + + + + + + + + diff --git a/src/website/tools/profiler-film/fs-storefront/FsStorefront.slnx b/src/website/tools/profiler-film/fs-storefront/FsStorefront.slnx new file mode 100644 index 00000000..fe762db4 --- /dev/null +++ b/src/website/tools/profiler-film/fs-storefront/FsStorefront.slnx @@ -0,0 +1,3 @@ + + + diff --git a/src/website/tools/profiler-film/fs-storefront/Program.fs b/src/website/tools/profiler-film/fs-storefront/Program.fs new file mode 100644 index 00000000..788b3d50 --- /dev/null +++ b/src/website/tools/profiler-film/fs-storefront/Program.fs @@ -0,0 +1,12 @@ +// FsStorefront: the F# demo workload the website's profiler film records. +// It checks out orders forever, with real hot paths and a cache that never evicts. +module FsStorefront.Program + +open System + +[] +let main _ = + let catalog = Catalog.generate 2_000 + printfn $"FsStorefront running (PID {Environment.ProcessId})" + Seq.initInfinite (Checkout.checkout catalog) |> Seq.iter Sessions.remember + 0 diff --git a/src/website/tools/profiler-film/fs-storefront/Sessions.fs b/src/website/tools/profiler-film/fs-storefront/Sessions.fs new file mode 100644 index 00000000..06afa375 --- /dev/null +++ b/src/website/tools/profiler-film/fs-storefront/Sessions.fs @@ -0,0 +1,10 @@ +module FsStorefront.Sessions + +open System.Collections.Generic + +// Remembers every order "for analytics", and never forgets one. +let private sessions = Dictionary() + +let remember order = + let stage = CheckedOut order.Total + sessions[order.Id] <- { Order = order; Stage = stage; Snapshot = Array.zeroCreate 4096 } diff --git a/src/website/tools/profiler-film/leak.mjs b/src/website/tools/profiler-film/leak.mjs index 77eca62d..f7adc0c8 100644 --- a/src/website/tools/profiler-film/leak.mjs +++ b/src/website/tools/profiler-film/leak.mjs @@ -9,59 +9,13 @@ // // Clips land in tools/profiler-film/out/leak/ (gitignored). Every stretch the // published video fast-forwards carries a speed badge on screen. -import fs from 'node:fs'; import path from 'node:path'; -import { launchVsCode, runCommand } from './vscode.mjs'; -import { caption, ringClick } from './overlay.mjs'; -import { H264, VP9, ffmpeg, out as filmOut, poster, videos } from './encode.mjs'; -import { - awaitDashboard, - buildStorefront, - clickSharpLspIcon, - cueLog, - headerButton, - openFile, - record, - rowButton, - scrollDashboard, - sleep, - stageStorefront, - storefront, - storefrontRow, - waitForStorefrontRow, -} from './studio.mjs'; +import { out as filmOut } from './encode.mjs'; +import { STOREFRONT } from './studio.mjs'; +import { memoryLoop, prepare, publishWalkthrough, shootWalkthrough } from './walkthrough.mjs'; const out = path.join(filmOut, 'leak'); -const clips = path.join(out, 'clips'); -const NAME = 'profiler-fix-memory-leak'; -const GROW_SECONDS = 6; -const VERIFY_SECONDS = 45; - -/** Each clip, in order, and its playback speed in the published video. */ -const SCENES = [ - ['track', 1], ['grow', 8], ['suspect', 1], ['ask', 1], ['walk', 8], ['graph', 1], - ['fix', 1], ['retrack', 1], ['verify', 8], ['steady', 1], -]; -const POSTER = { scene: 'graph', at: 5 }; - -/** - * Workspace settings: snapshots every 10 seconds, the interval the guide - * recommends, so a reference walk never overlaps an auto-capture; and an editor - * that lets the fix be typed key by key, exactly as written. - */ -const WORKSPACE = { - 'sharplsp.profiler.autoSnapshotInterval': 10, - 'editor.autoClosingBrackets': 'never', - 'editor.autoClosingQuotes': 'never', - 'editor.autoSurround': 'never', - 'editor.autoIndent': 'none', - 'editor.formatOnType': false, - 'editor.quickSuggestions': { other: 'off', comments: 'off', strings: 'off' }, - 'editor.suggestOnTriggerCharacters': false, - 'editor.parameterHints.enabled': false, - 'editor.inlineSuggest.enabled': false, - 'editor.hover.enabled': false, -}; +const FILE = 'SessionCache.cs'; /** The fix: each entry replaces one line of SessionCache.cs. */ const FIX = [ @@ -78,209 +32,26 @@ const FIX = [ }, ]; -const dashboard = (page) => page.frameLocator('iframe.webview').first().frameLocator('#active-frame'); - -/** - * Hover the live Storefront row so its memory shows, then click Track Memory. - * Other processes come and go in the list, so a click that started nothing - * fails the take. - */ -async function startTracking(page, cues) { - await storefrontRow(page).hover(); - await sleep(2500); - await ringClick(page, cues, await rowButton(page, 'Track Memory'), 'Track Memory'); - await page.locator('.statusbar', { hasText: 'Memory: tracking' }).waitFor({ timeout: 15_000 }); - await sleep(1000); -} - -async function trackScene(page, cues) { - await caption(page, 1, 'Track Memory on the process whose memory keeps climbing'); - await startTracking(page, cues); -} - -/** Runs until the autopilot ranks a suspect High (three growths in a row), and long enough to watch the climb. */ -async function growScene(page) { - await caption(page, 2, 'Let it run: SharpLsp snapshots the heap every 10 seconds', '8×'); - await awaitDashboard(page); - const high = dashboard(page).locator('.leak-row .badge', { hasText: /high/i }).first(); - await Promise.all([high.waitFor({ state: 'attached', timeout: 120_000 }), sleep(GROW_SECONDS * 1000)]); -} - -async function suspectScene(page, cues) { - await caption(page, 3, 'Suspects rank by consistent growth. Click one to jump to its code'); - await scrollDashboard(page, 8000); - await sleep(2500); - await ringClick(page, cues, dashboard(page).locator('.leak-row', { hasText: 'SessionCache.cs' }).first(), 'Suspect'); - await sleep(3500); -} - -/** Pick `type` from the types the latest snapshot reported; a take without it is shot again. */ -async function pickType(page, type) { - await page.locator('.quick-input-widget input').fill(type.split('.').at(-1)); - await sleep(700); - if ((await page.locator('.quick-input-list .monaco-list-row', { hasText: type }).count()) === 0) { - await page.keyboard.press('Escape'); - throw new Error(`${type} is not in the latest snapshot yet`); - } - await page.keyboard.press('Enter'); -} - -const GRAPH_CAPTION = 'Ask what keeps it alive: SharpLsp: Show Reference Graph'; - -async function askScene(page) { - await caption(page, 4, GRAPH_CAPTION); - await capturing(page).waitFor({ state: 'detached', timeout: 60_000 }); - await sleep(600); - await runCommand(page, 'SharpLsp: Show Reference Graph'); - await pickType(page, 'Storefront.CustomerSession'); -} - -/** The walk over a live heap of a hundred megabytes takes a while: fast-forwarded. */ -async function walkScene(page) { - await caption(page, 4, GRAPH_CAPTION, '8×'); - await page.locator('.tab', { hasText: 'Retention Graph' }).waitFor({ timeout: 240_000 }); - await sleep(1000); -} - -async function graphScene(page) { - await caption(page, 4, GRAPH_CAPTION); - await sleep(7000); -} - -/** Go to `line`, select it, and type `text` over it. */ -async function typeOver(page, line, text) { - await page.keyboard.press('Control+G'); - await page.locator('.quick-input-widget input').fill(`:${line}`); - await page.keyboard.press('Enter'); - await page.keyboard.press('Shift+End'); - await page.keyboard.type(text, { delay: 28 }); -} - -async function fixScene(page) { - await caption(page, 5, 'Fix it: keep a bounded window and evict the rest'); - await sleep(2000); - for (const { line, text } of FIX) await typeOver(page, line, text); - await page.keyboard.press('Meta+S'); - await sleep(3000); -} - -async function retrackScene(page, cues) { - await caption(page, 6, 'Rebuild, run, and track memory again'); - await startTracking(page, cues); -} - -async function verifyScene(page) { - await caption(page, 6, 'Rebuild, run, and track memory again', '8×'); - await awaitDashboard(page); - await sleep(VERIFY_SECONDS * 1000); -} - -async function steadyScene(page) { - await caption(page, 7, 'The heap holds steady, and no suspect climbs past Low'); - await sleep(2500); - await scrollDashboard(page, 8000); - await sleep(4000); -} - -/** The status-bar progress of an auto-capture still talking to the target. */ -const capturing = (page) => page.locator('.statusbar', { hasText: 'Auto-capturing memory' }); - -/** - * Off camera: stop tracking, let any in-flight capture finish before the - * leaking app exits, and reopen a clean SessionCache.cs. - */ -async function stopAndReopen(page, app) { - await clickSharpLspIcon(page); - await (await headerButton(page, 'Stop Memory Tracking')).click(); - await capturing(page).waitFor({ state: 'detached', timeout: 60_000 }); - app.stop(); - await openFile(page, 'SessionCache.cs'); - await runCommand(page, 'File: Revert File'); - await runCommand(page, 'View: Close All Editors'); - await openFile(page, 'SessionCache.cs'); - await clickSharpLspIcon(page); - await page.locator('.monaco-editor .view-lines').first().click(); -} - -/** Off camera: build the fixed code and start it with the Processes view open. */ -async function relaunch(page, app, workspace) { - buildStorefront(workspace); - await app.start(); - await clickSharpLspIcon(page); - await waitForStorefrontRow(page); -} - -/** Shoot a scene again when a refreshing dashboard closes its palette, or the type is not listed yet. */ -async function retried(attempts, shootScene) { - for (let attempt = 1; ; attempt++) { - try { - return await shootScene(); - } catch (error) { - if (attempt >= attempts) throw error; - process.stdout.write(`retrying a take: ${error.message.split('\n')[0]}\n`); - await sleep(3500); - } - } -} - -async function prepare(page, app) { - await runCommand(page, 'Notifications: Toggle Do Not Disturb Mode'); - await clickSharpLspIcon(page); - await openFile(page, 'SessionCache.cs'); - await sleep(15_000); - app.stop(); - await app.start(); - await waitForStorefrontRow(page); -} - -async function shoot(page, set, app, workspace) { - await prepare(page, app); - await retried(3, () => record(set, 'track', () => trackScene(page, set.cues))); - await record(set, 'grow', () => growScene(page)); - await record(set, 'suspect', () => suspectScene(page, set.cues)); - await retried(5, () => record(set, 'ask', () => askScene(page))); - await record(set, 'walk', () => walkScene(page)); - await record(set, 'graph', () => graphScene(page)); - await stopAndReopen(page, app); - await record(set, 'fix', () => fixScene(page)); - await relaunch(page, app, workspace); - await retried(3, () => record(set, 'retrack', () => retrackScene(page, set.cues))); - await record(set, 'verify', () => verifyScene(page)); - await record(set, 'steady', () => steadyScene(page)); -} - -async function shootAll() { - fs.mkdirSync(clips, { recursive: true }); - const { root, workspace } = stageStorefront(); - fs.mkdirSync(path.join(workspace, '.vscode'), { recursive: true }); - fs.writeFileSync(path.join(workspace, '.vscode/settings.json'), JSON.stringify(WORKSPACE)); - const editor = await launchVsCode({ workspace }); - const app = storefront(workspace); - await app.start(); - try { - await shoot(editor.page, { cdp: editor.cdp, cues: cueLog(clips), frames: path.join(out, 'frames'), clips }, app, workspace); - } catch (error) { - await editor.page.screenshot({ path: path.join(out, 'failure.png') }).catch(() => undefined); - throw error; - } finally { - app.stop(); - await editor.close(); - fs.rmSync(root, { recursive: true, force: true }); - } -} - -/** Cut the scenes together at their speeds, then publish the MP4, the WebM and the poster. */ -function publish() { - const inputs = SCENES.flatMap(([name]) => ['-i', path.join(clips, `${name}.mp4`)]); - const chains = SCENES.map(([, speed], i) => `[${i}:v]setpts=PTS/${speed},fps=30,scale=1920:-2:flags=lanczos[v${i}]`); - const joined = `${SCENES.map((_, i) => `[v${i}]`).join('')}concat=n=${SCENES.length}:v=1:a=0[cut]`; - const master = path.join(out, `${NAME}-master.mp4`); - const lossless = ['-c:v', 'libx264', '-preset', 'slow', '-crf', '12', '-pix_fmt', 'yuv420p']; - ffmpeg([...inputs, '-filter_complex', `${chains.join(';')};${joined}`, '-map', '[cut]', ...lossless, master], 'leak master'); - ffmpeg(['-i', master, ...H264, path.join(videos, `${NAME}.mp4`)], 'leak mp4'); - ffmpeg(['-i', master, ...VP9, path.join(videos, `${NAME}.webm`)], 'leak webm'); - poster(path.join(clips, `${POSTER.scene}.mp4`), POSTER.at, NAME); -} +const SCENES = [ + { offCamera: (shot) => prepare(shot, FILE) }, + ...memoryLoop( + { + file: FILE, + type: 'Storefront.CustomerSession', + fix: FIX, + captions: { + track: 'Track Memory on the process whose memory keeps climbing', + grow: 'Let it run: SharpLsp snapshots the heap every 10 seconds', + suspect: 'Suspects rank by consistent growth. Click one to jump to its code', + graph: 'Ask what keeps it alive: SharpLsp: Show Reference Graph', + fix: 'Fix it: keep a bounded window and evict the rest', + retrack: 'Rebuild, run, and track memory again', + steady: 'The heap holds steady, and no suspect climbs past Low', + }, + }, + 1, + ), +]; -if (!process.argv.includes('--publish')) await shootAll(); -publish(); +if (!process.argv.includes('--publish')) await shootWalkthrough({ demo: STOREFRONT, out, scenes: SCENES }); +publishWalkthrough({ name: 'profiler-fix-memory-leak', out, scenes: SCENES, posterAt: { scene: 'graph', at: 5 } }); diff --git a/src/website/tools/profiler-film/render.mjs b/src/website/tools/profiler-film/render.mjs index 3df62909..e30f9558 100644 --- a/src/website/tools/profiler-film/render.mjs +++ b/src/website/tools/profiler-film/render.mjs @@ -10,10 +10,9 @@ import fs from 'node:fs'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; import { chromium } from '@playwright/test'; -import { H264, VP9, ffmpeg, out, poster, videos, webp } from './encode.mjs'; +import { H264, VP9, ffmpeg, out, poster, publishStills, videos } from './encode.mjs'; const here = path.dirname(fileURLToPath(import.meta.url)); -const shots = path.resolve(here, '../../src/assets/screenshots/profiler'); const FPS = 30; const POSTER_AT = 25.5; @@ -93,21 +92,15 @@ function publishRecordings() { poster(path.join(videos, 'profiler-memory-timelapse.mp4'), 14, 'profiler-memory-timelapse'); } -function publishStills() { - for (const file of fs.readdirSync(path.join(out, 'stills')).filter((name) => name.endsWith('.png'))) { - webp(path.join(out, 'stills', file), path.join(shots, file.replace(/\.png$/, '.webp')), 2400); - } -} - async function main() { const flag = process.argv.indexOf('--preview'); if (flag > 0) return preview(process.argv[flag + 1].split(',').map(Number)); - for (const dir of [videos, shots]) fs.mkdirSync(dir, { recursive: true }); + fs.mkdirSync(videos, { recursive: true }); const master = path.join(out, 'film-master.mp4'); const duration = await renderMaster(master); publishFilm(master); publishRecordings(); - publishStills(); + publishStills(path.join(out, 'stills')); process.stdout.write(`rendered ${duration.toFixed(1)}s film and the profiler media\n`); } diff --git a/src/website/tools/profiler-film/studio.mjs b/src/website/tools/profiler-film/studio.mjs index b47cb967..70bd71f7 100644 --- a/src/website/tools/profiler-film/studio.mjs +++ b/src/website/tools/profiler-film/studio.mjs @@ -1,7 +1,8 @@ -// Implements [WEB-DESIGN-MEDIA]. The set every profiler film is shot on: the -// Storefront demo staged and built at a neutral temp path, the isolated VS Code's -// Processes view and editor moves, and the real-time screencast recorder. -// capture.mjs (the profiler film) and leak.mjs (the leak walkthrough) both shoot here. +// Implements [WEB-DESIGN-MEDIA]. The set every profiler film is shot on: a demo +// app staged and built at a neutral temp path, the isolated VS Code's Processes +// view and editor moves, and the real-time screencast recorder. capture.mjs (the +// profiler film), leak.mjs (the C# leak walkthrough) and fsharp.mjs (the F# +// walkthrough) all shoot here. import { spawn, spawnSync } from 'node:child_process'; import fs from 'node:fs'; import os from 'node:os'; @@ -19,55 +20,64 @@ export const DASHBOARD = 'SharpLsp Memory Dashboard'; export const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); -/** Build the staged Storefront; a failed build stops the shoot. */ -export function buildStorefront(workspace) { +/** + * The demo apps the films are shot on: a folder here holding one project. The + * C# Storefront and its F# twin check out orders forever, with real hot paths + * and a cache that never evicts. + */ +export const STOREFRONT = { folder: 'storefront', project: 'Storefront' }; +export const FS_STOREFRONT = { folder: 'fs-storefront', project: 'FsStorefront' }; + +/** Build the staged demo; a failed build stops the shoot. */ +export function buildDemo(workspace) { const build = spawnSync('dotnet', ['build', '-c', 'Debug', '--nologo', '-v', 'q'], { cwd: workspace, encoding: 'utf8', }); - if (build.status !== 0) throw new Error(`Storefront build failed: ${build.stdout}`); + if (build.status !== 0) throw new Error(`${path.basename(workspace)} build failed: ${build.stdout}`); } /** - * Copy the demo to a neutral temp path and build it there, so no home-directory + * Copy `demo` to a neutral temp path and build it there, so no home-directory * path (the user's name) can reach a frame name or tooltip on camera. */ -export function stageStorefront() { +export function stageDemo(demo) { // The real path: macOS's tmpdir is a symlink the .NET runtime resolves, and // heat-map decorations only land when both sides spell the file the same way. const root = fs.mkdtempSync(path.join(fs.realpathSync(os.tmpdir()), 'film-')); - const workspace = path.join(root, 'Storefront'); - fs.cpSync(path.join(here, 'storefront'), workspace, { + const workspace = path.join(root, demo.project); + fs.cpSync(path.join(here, demo.folder), workspace, { recursive: true, filter: (source) => !['bin', 'obj', '.sharplsp'].includes(path.basename(source)), }); - buildStorefront(workspace); + buildDemo(workspace); return { root, workspace }; } -/** Start Storefront by absolute path (discovery needs it) and wait for its banner. */ -async function startStorefront(workspace) { - const app = spawn(path.join(workspace, 'bin/Debug/net10.0/Storefront'), [], { +/** Start the demo by absolute path (discovery needs it) and wait for its banner. */ +async function startDemo(workspace) { + const project = path.basename(workspace); + const app = spawn(path.join(workspace, `bin/Debug/net10.0/${project}`), [], { cwd: workspace, stdio: ['ignore', 'pipe', 'ignore'], }); await new Promise((resolve, reject) => { app.stdout.once('data', resolve); - app.once('exit', (code) => reject(new Error(`Storefront exited with ${code}`))); + app.once('exit', (code) => reject(new Error(`${project} exited with ${code}`))); }); return app; } -/** The live Storefront's PID: its row, never a stale one from a restarted run. */ -let storefrontPid = 0; +/** The live demo's PID: its row, never a stale one from a restarted run. */ +let demoPid = 0; -/** The demo app, restartable so a memory act starts from a fresh, small heap. */ -export function storefront(workspace) { +/** The staged demo app, restartable so a memory act starts from a fresh, small heap. */ +export function demoApp(workspace) { let app; return { start: async () => { - app = await startStorefront(workspace); - storefrontPid = app.pid; + app = await startDemo(workspace); + demoPid = app.pid; }, stop: () => app?.kill(), }; @@ -113,8 +123,8 @@ export async function record({ cdp, cues, frames, clips }, name, act) { const processesHeader = (page) => page.locator('.pane-header', { hasText: 'Processes' }); -export const storefrontRow = (page) => - page.getByRole('treeitem').filter({ hasText: `Storefront — StorefrontPID ${storefrontPid}` }).first(); +export const demoRow = (page) => + page.getByRole('treeitem').filter({ hasText: new RegExp(`PID ${demoPid}(?!\\d)`) }).first(); export async function headerButton(page, name) { await processesHeader(page).hover(); @@ -122,19 +132,19 @@ export async function headerButton(page, name) { } export async function rowButton(page, name) { - await storefrontRow(page).hover(); + await demoRow(page).hover(); await sleep(250); - return storefrontRow(page).getByRole('button', { name: new RegExp(`^${name}`) }); + return demoRow(page).getByRole('button', { name: new RegExp(`^${name}`) }); } -/** Wait until the Processes view lists the live Storefront row. */ -export async function waitForStorefrontRow(page) { +/** Wait until the Processes view lists the live demo's row. */ +export async function waitForDemoRow(page) { for (let attempt = 0; attempt < 30; attempt++) { - if (await storefrontRow(page).isVisible().catch(() => false)) return; + if (await demoRow(page).isVisible().catch(() => false)) return; await (await headerButton(page, 'Refresh Processes')).click(); await sleep(1000); } - throw new Error('Storefront never appeared in the Processes view'); + throw new Error(`PID ${demoPid} never appeared in the Processes view`); } /** A clean frame: no hover (Do Not Disturb keeps toasts off screen). */ @@ -170,7 +180,7 @@ export async function scrollDashboard(page, pixels) { } /** - * Start memory tracking on the Storefront row and fold the side bar once the + * Start memory tracking on the demo's row and fold the side bar once the * dashboard opens. `click(locator, label)` is the act's own way of clicking. */ export async function trackMemory(page, click) { @@ -186,7 +196,31 @@ export async function awaitDashboard(page) { await page.mouse.move(PARK.x, PARK.y); } +/** + * Toggle `title`'s editor group maximised by double-clicking its tab — no + * palette, which a refreshing webview would close by stealing focus. + */ +export async function toggleMaximised(page, title) { + await page.locator('.tab', { hasText: title }).first().dblclick(); + await sleep(1200); +} + export async function toggleSideBar(page) { await runCommand(page, 'View: Toggle Primary Side Bar Visibility'); await sleep(800); } + +/** + * After a CPU stop: `-results`, the results panel maximised, and + * `-heatmap`, `file` maximised with its heat bars and no side bar. + */ +export async function cpuStills(page, dir, file, prefix) { + await toggleMaximised(page, 'SharpLsp Profiler'); + await still(page, dir, `${prefix}-results`); + await toggleMaximised(page, 'SharpLsp Profiler'); + await toggleMaximised(page, file); + await toggleSideBar(page); + await still(page, dir, `${prefix}-heatmap`); + await toggleSideBar(page); + await toggleMaximised(page, file); +} diff --git a/src/website/tools/profiler-film/walkthrough.mjs b/src/website/tools/profiler-film/walkthrough.mjs new file mode 100644 index 00000000..4c602fc9 --- /dev/null +++ b/src/website/tools/profiler-film/walkthrough.mjs @@ -0,0 +1,265 @@ +// Implements [WEB-DESIGN-MEDIA]. The walkthrough films: a list of scenes, each a +// real-time clip with its playback speed, shot in one isolated VS Code on a staged +// demo app, then cut together and published. leak.mjs (C#) and fsharp.mjs (F#) +// share the memory loop below: track, watch the heap grow, open a suspect, ask +// what keeps it alive, type the fix, rebuild, and track the fixed build again. +import fs from 'node:fs'; +import path from 'node:path'; +import { launchVsCode, runCommand } from './vscode.mjs'; +import { caption, ringClick } from './overlay.mjs'; +import { H264, VP9, ffmpeg, poster, videos } from './encode.mjs'; +import { + awaitDashboard, + buildDemo, + clickSharpLspIcon, + cueLog, + demoApp, + demoRow, + headerButton, + openFile, + record, + rowButton, + scrollDashboard, + sleep, + stageDemo, + waitForDemoRow, +} from './studio.mjs'; + +const GROW_SECONDS = 6; +const VERIFY_SECONDS = 45; + +/** + * Workspace settings: snapshots every 10 seconds, the interval the guide + * recommends, so a reference walk never overlaps an auto-capture; and an editor + * that lets the fix be typed key by key, exactly as written. + */ +const WORKSPACE = { + 'sharplsp.profiler.autoSnapshotInterval': 10, + 'editor.autoClosingBrackets': 'never', + 'editor.autoClosingQuotes': 'never', + 'editor.autoSurround': 'never', + 'editor.autoIndent': 'none', + 'editor.formatOnType': false, + 'editor.quickSuggestions': { other: 'off', comments: 'off', strings: 'off' }, + 'editor.suggestOnTriggerCharacters': false, + 'editor.parameterHints.enabled': false, + 'editor.inlineSuggest.enabled': false, + 'editor.hover.enabled': false, +}; + +export const dashboard = (page) => page.frameLocator('iframe.webview').first().frameLocator('#active-frame'); + +/** The status-bar progress of an auto-capture still talking to the target. */ +const capturing = (page) => page.locator('.statusbar', { hasText: 'Auto-capturing memory' }); + +/** + * Hover the live demo's row so its memory shows, then click `action`. Other + * processes come and go in the list, so a click that started nothing fails the take. + */ +export async function startOnRow(page, cues, action, status) { + await demoRow(page).hover(); + await sleep(2500); + await ringClick(page, cues, await rowButton(page, action), action); + await page.locator('.statusbar', { hasText: status }).waitFor({ timeout: 15_000 }); + await sleep(1000); +} + +const startTracking = (page, cues) => startOnRow(page, cues, 'Track Memory', 'Memory: tracking'); + +/** Pick `type` from the types the latest snapshot reported; a take without it is shot again. */ +async function pickType(page, type) { + await page.locator('.quick-input-widget input').fill(type.split('.').at(-1)); + await sleep(700); + if ((await page.locator('.quick-input-list .monaco-list-row', { hasText: type }).count()) === 0) { + await page.keyboard.press('Escape'); + throw new Error(`${type} is not in the latest snapshot yet`); + } + await page.keyboard.press('Enter'); +} + +/** Go to `line` and type `text` over it, or after its end when the edit appends. */ +async function typeAt(page, { line, text, append = false }) { + await page.keyboard.press('Control+G'); + await page.locator('.quick-input-widget input').fill(`:${line}`); + await page.keyboard.press('Enter'); + await page.keyboard.press(append ? 'End' : 'Shift+End'); + await page.keyboard.type(text, { delay: 28 }); +} + +/** + * Off camera: stop tracking, let any in-flight capture finish before the + * leaking app exits, and reopen a clean `file`. + */ +async function stopAndReopen({ page, app }, file) { + await clickSharpLspIcon(page); + await (await headerButton(page, 'Stop Memory Tracking')).click(); + await capturing(page).waitFor({ state: 'detached', timeout: 60_000 }); + app.stop(); + await openFile(page, file); + await runCommand(page, 'File: Revert File'); + await runCommand(page, 'View: Close All Editors'); + await openFile(page, file); + await clickSharpLspIcon(page); + await page.locator('.monaco-editor .view-lines').first().click(); +} + +/** Off camera: build the fixed code and start it with the Processes view open. */ +async function relaunch({ page, app, workspace }) { + buildDemo(workspace); + await app.start(); + await clickSharpLspIcon(page); + await waitForDemoRow(page); +} + +/** Off camera: the editor on `file`, loaded, beside a freshly started demo. */ +export async function prepare({ page, app }, file) { + await runCommand(page, 'Notifications: Toggle Do Not Disturb Mode'); + await clickSharpLspIcon(page); + await openFile(page, file); + await sleep(15_000); + await restart({ page, app }); +} + +/** Off camera: a fresh process, so a memory act starts from a small heap. */ +export async function restart({ page, app }) { + app.stop(); + await app.start(); + await waitForDemoRow(page); +} + +/** + * The memory loop as scenes, numbered from `first`. `story` names the file the + * fix goes in, the suspect's type, the fix's edits and each step's caption + * (`rooted`, over the drawn graph, defaults to the `graph` question), plus any + * off-camera `afterSuspect` and `afterGraph` steps. + */ +export function memoryLoop(story, first) { + const step = (offset) => first + offset; + const graph = story.captions.graph; + return [ + { clip: 'track', retries: 3, act: ({ page, set }) => caption(page, step(0), story.captions.track).then(() => startTracking(page, set.cues)) }, + { clip: 'grow', speed: 8, act: ({ page }) => growScene(page, step(1), story.captions.grow) }, + { clip: 'suspect', act: (shot) => suspectScene(shot, step(2), story) }, + ...(story.afterSuspect ? [{ offCamera: story.afterSuspect }] : []), + { clip: 'ask', retries: 5, act: ({ page }) => askScene(page, step(3), graph, story.type) }, + { clip: 'walk', speed: 8, act: ({ page }) => walkScene(page, step(3), graph) }, + { clip: 'graph', act: ({ page }) => caption(page, step(3), story.captions.rooted ?? graph).then(() => sleep(7000)) }, + ...(story.afterGraph ? [{ offCamera: story.afterGraph }] : []), + { offCamera: (shot) => stopAndReopen(shot, story.file) }, + { clip: 'fix', act: ({ page }) => fixScene(page, step(4), story) }, + { offCamera: relaunch }, + { clip: 'retrack', retries: 3, act: ({ page, set }) => caption(page, step(5), story.captions.retrack).then(() => startTracking(page, set.cues)) }, + { clip: 'verify', speed: 8, act: ({ page }) => verifyScene(page, step(5), story.captions.retrack) }, + { clip: 'steady', act: ({ page }) => steadyScene(page, step(6), story.captions.steady) }, + ]; +} + +/** Runs until the autopilot ranks a suspect High (three growths in a row), and long enough to watch the climb. */ +async function growScene(page, step, text) { + await caption(page, step, text, '8×'); + await awaitDashboard(page); + const high = dashboard(page).locator('.leak-row .badge', { hasText: /high/i }).first(); + await Promise.all([high.waitFor({ state: 'attached', timeout: 120_000 }), sleep(GROW_SECONDS * 1000)]); +} + +async function suspectScene({ page, set }, step, story) { + await caption(page, step, story.captions.suspect); + await scrollDashboard(page, 8000); + await sleep(2500); + await ringClick(page, set.cues, dashboard(page).locator('.leak-row', { hasText: story.file }).first(), 'Suspect'); + await sleep(3500); +} + +async function askScene(page, step, text, type) { + await caption(page, step, text); + await capturing(page).waitFor({ state: 'detached', timeout: 60_000 }); + await sleep(600); + await runCommand(page, 'SharpLsp: Show Reference Graph'); + await pickType(page, type); +} + +/** The walk over a live heap of a hundred megabytes takes a while: fast-forwarded. */ +async function walkScene(page, step, text) { + await caption(page, step, text, '8×'); + await page.locator('.tab', { hasText: 'Retention Graph' }).waitFor({ timeout: 240_000 }); + await sleep(1000); +} + +async function fixScene(page, step, story) { + await caption(page, step, story.captions.fix); + await sleep(2000); + for (const edit of story.fix) await typeAt(page, edit); + await page.keyboard.press('Meta+S'); + await sleep(3000); +} + +async function verifyScene(page, step, text) { + await caption(page, step, text, '8×'); + await awaitDashboard(page); + await sleep(VERIFY_SECONDS * 1000); +} + +async function steadyScene(page, step, text) { + await caption(page, step, text); + await sleep(2500); + await scrollDashboard(page, 8000); + await sleep(4000); +} + +/** Shoot a scene again when a refreshing dashboard closes its palette, or the type is not listed yet. */ +async function retried(attempts, shootScene) { + for (let attempt = 1; ; attempt++) { + try { + return await shootScene(); + } catch (error) { + if (attempt >= attempts) throw error; + process.stdout.write(`retrying a take: ${error.message.split('\n')[0]}\n`); + await sleep(3500); + } + } +} + +async function shootScenes(shot, scenes) { + for (const scene of scenes) { + if (scene.offCamera) await scene.offCamera(shot); + else await retried(scene.retries ?? 1, () => record(shot.set, scene.clip, () => scene.act(shot))); + } +} + +/** Stage `demo`, open it in an isolated VS Code, and shoot `scenes` into `/clips`. */ +export async function shootWalkthrough({ demo, out, scenes }) { + const clips = path.join(out, 'clips'); + fs.mkdirSync(clips, { recursive: true }); + const { root, workspace } = stageDemo(demo); + fs.mkdirSync(path.join(workspace, '.vscode'), { recursive: true }); + fs.writeFileSync(path.join(workspace, '.vscode/settings.json'), JSON.stringify(WORKSPACE)); + const editor = await launchVsCode({ workspace }); + const app = demoApp(workspace); + await app.start(); + const set = { cdp: editor.cdp, cues: cueLog(clips), frames: path.join(out, 'frames'), clips }; + try { + await shootScenes({ page: editor.page, set, app, workspace }, scenes); + } catch (error) { + await editor.page.screenshot({ path: path.join(out, 'failure.png') }).catch(() => undefined); + throw error; + } finally { + app.stop(); + await editor.close(); + fs.rmSync(root, { recursive: true, force: true }); + } +} + +/** Cut the scenes together at their speeds, then publish the MP4, the WebM and the poster. */ +export function publishWalkthrough({ name, out, scenes, posterAt }) { + const shot = scenes.filter((scene) => scene.clip); + const clip = (name) => path.join(out, 'clips', `${name}.mp4`); + const inputs = shot.flatMap((scene) => ['-i', clip(scene.clip)]); + const chains = shot.map((scene, i) => `[${i}:v]setpts=PTS/${scene.speed ?? 1},fps=30,scale=1920:-2:flags=lanczos[v${i}]`); + const joined = `${shot.map((_, i) => `[v${i}]`).join('')}concat=n=${shot.length}:v=1:a=0[cut]`; + const master = path.join(out, `${name}-master.mp4`); + const lossless = ['-c:v', 'libx264', '-preset', 'slow', '-crf', '12', '-pix_fmt', 'yuv420p']; + ffmpeg([...inputs, '-filter_complex', `${chains.join(';')};${joined}`, '-map', '[cut]', ...lossless, master], `${name} master`); + ffmpeg(['-i', master, ...H264, path.join(videos, `${name}.mp4`)], `${name} mp4`); + ffmpeg(['-i', master, ...VP9, path.join(videos, `${name}.webm`)], `${name} webm`); + poster(clip(posterAt.scene), posterAt.at, name); +} From fb3a882ff241acf85c3e032ab5054c904fa699ab Mon Sep 17 00:00:00 2001 From: Christian Findlay <16697547+MelbourneDeveloper@users.noreply.github.com> Date: Sun, 18 Oct 2026 18:14:46 +1100 Subject: [PATCH 08/24] Pin lspkit at the objects-summary and FSharp.Core anchor fixes The objects step keeps the requested type's own row when it is too small to rank among the twenty largest, so a union case such as OrderStatus+Paid agrees with its totals. Types whose PDB names a build server's checkout, as FSharp.Core's does, have no location instead of a path that does not exist here. --- lspkit | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lspkit b/lspkit index e079bcf4..b9066b19 160000 --- a/lspkit +++ b/lspkit @@ -1 +1 @@ -Subproject commit e079bcf46aae5380c2e361aeb59d912b9905808c +Subproject commit b9066b192da4b71a2a1452a9bc487c2e5bad50c8 From b2f89486cb6421b2a6b5ff8bf13d31538cd8cd50 Mon Sep 17 00:00:00 2001 From: Christian Findlay <16697547+MelbourneDeveloper@users.noreply.github.com> Date: Sun, 18 Oct 2026 18:17:01 +1100 Subject: [PATCH 09/24] Format the profiler test kits and drop non-null assertions their asserts already narrow --- .../vscode/src/test/suite/profiler-kit.ts | 6 ++---- .../src/test/suite/profiler-ui-helpers.ts | 21 ++++++++----------- 2 files changed, 11 insertions(+), 16 deletions(-) diff --git a/src/editors/vscode/src/test/suite/profiler-kit.ts b/src/editors/vscode/src/test/suite/profiler-kit.ts index 1e75d57e..8f11d4b0 100644 --- a/src/editors/vscode/src/test/suite/profiler-kit.ts +++ b/src/editors/vscode/src/test/suite/profiler-kit.ts @@ -485,9 +485,7 @@ export async function trackedMemorySession(pid: number): Promise { * Start a memory session on `pid` over the wire and return the handle every * memory step takes. */ -export async function startMemorySession( - pid: number, -): Promise<{ memory_session_id: string }> { +export async function startMemorySession(pid: number): Promise<{ memory_session_id: string }> { const started = await execute('memory.start', { pid }); assert.ok(started.memory_session_id.length > 0, 'a memory session must be minted'); return { memory_session_id: started.memory_session_id }; @@ -506,7 +504,7 @@ export async function assertWireError( const error = await executeError(name, params); assert.notEqual(error, undefined, `${what}: the call must be refused, not answered`); assert.equal(error?.code, code, `${what}: ${error?.message ?? 'no error'}`); - return error!; + return error; } /** diff --git a/src/editors/vscode/src/test/suite/profiler-ui-helpers.ts b/src/editors/vscode/src/test/suite/profiler-ui-helpers.ts index 4191cded..6941196e 100644 --- a/src/editors/vscode/src/test/suite/profiler-ui-helpers.ts +++ b/src/editors/vscode/src/test/suite/profiler-ui-helpers.ts @@ -16,9 +16,7 @@ import type { ProcessInfo } from '@nimblesite/lsp-profiler-ui/out/src/process-ex import type { ProcessInfo as WireProcess } from '@nimblesite/lsp-profiler-ui/out/src/wire-generated'; import { delay } from '@nimblesite/lsp-profiler-ui/out/src/timeouts'; import type { ProfileResult } from '@nimblesite/lsp-profiler-ui/out/src/profiler-decorations'; -import type { - MemoryDashboardSnapshot, -} from '@nimblesite/lsp-profiler-ui/out/src/memory-dashboard'; +import type { MemoryDashboardSnapshot } from '@nimblesite/lsp-profiler-ui/out/src/memory-dashboard'; import type { MemoryDiffResult, MemorySnapshotResult, @@ -387,10 +385,7 @@ const MB = 1024 * 1024; * A debuggable process row; each caller overrides what it exercises. The * one fixture every Processes-panel test builds its table from. */ -export function processFixture( - pid: number, - overrides: Partial = {}, -): ProcessInfo { +export function processFixture(pid: number, overrides: Partial = {}): ProcessInfo { return { pid, ppid: 1, @@ -420,10 +415,14 @@ export function rowPid(item: vscode.TreeItem): number | undefined { } /** The row of `pid`, failing the test when the panel dropped it. */ -export function rowByPid(rows: readonly vscode.TreeItem[], pid: number, what: string): vscode.TreeItem { +export function rowByPid( + rows: readonly vscode.TreeItem[], + pid: number, + what: string, +): vscode.TreeItem { const row = rows.find((item) => rowPid(item) === pid); assert.ok(row !== undefined, `${what}: the PID ${pid} row must exist`); - return row!; + return row; } // ── Decoration ledgers ───────────────────────────────────────────── @@ -438,9 +437,7 @@ export interface PaintedExpectation { /** Assert a decoration was applied and shows exactly what `expect` names. */ export function assertPainted( - painted: - | { line: number; color: string; contentText: string } - | undefined, + painted: { line: number; color: string; contentText: string } | undefined, expect: PaintedExpectation, what: string, ): void { From 211b3a9dac8822641e180c1e03646f9da89f9b33 Mon Sep 17 00:00:00 2001 From: Christian Findlay <16697547+MelbourneDeveloper@users.noreply.github.com> Date: Sun, 18 Oct 2026 18:21:30 +1100 Subject: [PATCH 10/24] Share the branded-panel and dashboard-snapshot test helpers The CPU results panel and the memory dashboard checked the same SharpLsp brand, foreign-product and nonce rules in two copies, and the webviews suite kept its own copy of the kit's dashboard snapshot. One helper each brings duplication back under the Deslop ratchet; every assertion is unchanged. --- .../src/test/suite/profiler-fsharp-kit.ts | 30 +++++++++++++------ .../suite/profiler-fsharp-memory-e2e.test.ts | 12 +++----- .../src/test/suite/profiler-webviews.test.ts | 19 +----------- 3 files changed, 26 insertions(+), 35 deletions(-) diff --git a/src/editors/vscode/src/test/suite/profiler-fsharp-kit.ts b/src/editors/vscode/src/test/suite/profiler-fsharp-kit.ts index 5ed8f74d..fa1bf61f 100644 --- a/src/editors/vscode/src/test/suite/profiler-fsharp-kit.ts +++ b/src/editors/vscode/src/test/suite/profiler-fsharp-kit.ts @@ -318,18 +318,30 @@ export function assertHeatMap(result: ProfileResult, span: Span): void { } } -/** [PROF-UI-CPU]: the branded results panel lists `identifiers` and links to source. */ -export function assertResultsPanel(result: ProfileResult, identifiers: readonly string[]): void { - const panel = flamegraphPanelState(); - assert.ok(panel, 'the results panel is open'); - assert.equal(panel.title, 'SharpLsp Profiler'); - assert.ok(panel.html.includes('SHARPLSP PROFILER'), 'brand'); +/** + * [PROF-UI-WEBVIEW]: an open SharpLsp panel titled `title`, branded `brand`, + * naming no other product, behind a nonce-gated script. Returns its HTML. + */ +export function assertBrandedPanel( + panel: { readonly title: string; readonly html: string } | undefined, + title: string, + brand: string, +): string { + assert.ok(panel, `the ${title} panel is open`); + assert.equal(panel.title, title); + assert.ok(panel.html.includes(`SHARPLSP ${brand}`), 'brand'); assert.ok(!/basilisk/i.test(panel.html), 'no other product named'); assert.ok(panel.html.includes(''; const SOURCE = '/app/Program.cs'; -function dashboardSnapshot( - overrides: Partial = {}, -): MemoryDashboardSnapshot { - return { - memorySessionId: 'mem-1', - snapshotId: 'snap-1', - currentMemory: 1_048_576, - peakMemory: 2_097_152, - gcObjects: 1200, - gcCounts: [700, 12, 3], - topAllocations: [{ file: SOURCE, line: 10, size: 4096, count: 8 }], - timeline: [], - heapProfilePath: '', - ...overrides, - }; -} - function profileResult(overrides: Partial = {}): ProfileResult { return baseResult({ hotFunctions: [ From 2ec7b5f29f318ab56f9878c887c65a7f822896ab Mon Sep 17 00:00:00 2001 From: Christian Findlay <16697547+MelbourneDeveloper@users.noreply.github.com> Date: Sun, 18 Oct 2026 18:32:54 +1100 Subject: [PATCH 11/24] Format the F# soak suites and the process-explorer test with Prettier --- .../suite/profiler-fsharp-soak-e2e.test.ts | 23 +++++++++++++++---- .../test/suite/profiler-fsharp-soak-kit.ts | 18 +++++++++------ .../suite/profiler-process-explorer.test.ts | 8 ++++++- 3 files changed, 37 insertions(+), 12 deletions(-) diff --git a/src/editors/vscode/src/test/suite/profiler-fsharp-soak-e2e.test.ts b/src/editors/vscode/src/test/suite/profiler-fsharp-soak-e2e.test.ts index 1009c6ec..96c89d73 100644 --- a/src/editors/vscode/src/test/suite/profiler-fsharp-soak-e2e.test.ts +++ b/src/editors/vscode/src/test/suite/profiler-fsharp-soak-e2e.test.ts @@ -232,7 +232,11 @@ suite('F# soak — FsOrderLedger: the leak-confidence ladder over one long sessi assert.equal(count(freed, 'blobs'), 0, 'release all drops the blob'); assert.equal(count(freed, 'handlers'), 2_000, 'release all never clears the Event handlers'); const afterRelease = await compareMemory(wire); - assert.equal(leakBadge('Ledger.fs', SHAPES.order.line), undefined, 'the old Order badge clears'); + assert.equal( + leakBadge('Ledger.fs', SHAPES.order.line), + undefined, + 'the old Order badge clears', + ); const resetSnap = await uiSnapshot(bloomSnap); assert.ok(resetSnap.currentMemory < bloomSnap.currentMemory, 'the heap shrank'); assertSitesCoherent(resetSnap); @@ -304,7 +308,11 @@ suite('F# soak — FsOrderLedger: the leak-confidence ladder over one long sessi const first = await uiSnapshot(); const wire = await snapshotSession(target.pid); assertAnchored(first, SHAPES.audit, 400); - assert.equal(handlerSite(first).count, count(queued, 'handlers'), 'all retained handlers are counted'); + assert.equal( + handlerSite(first).count, + count(queued, 'handlers'), + 'all retained handlers are counted', + ); assertSitesCoherent(first); assertTimelineCoherent(first); assertPaletteDiscipline(); @@ -408,7 +416,11 @@ suite('F# soak — FsOrderLedger: the leak-confidence ladder over one long sessi const sixth = await uiSnapshot(fifth); assertAnchored(sixth, SHAPES.order, 400); assert.equal(count(released, 'snapshots'), 2, 'history still retains the first 400 orders'); - assert.equal(handlerSite(sixth).count, count(released, 'handlers'), 'the closure leak survives'); + assert.equal( + handlerSite(sixth).count, + count(released, 'handlers'), + 'the closure leak survives', + ); assertAnchored(sixth, SHAPES.quote, 800); assert.ok(sixth.currentMemory <= fifth.currentMemory, 'releasing orders shrank the heap'); assertNoPhantomMemoryPaint(sixth); @@ -420,7 +432,10 @@ suite('F# soak — FsOrderLedger: the leak-confidence ladder over one long sessi // Only dropping history too makes the Orders unreachable ([PROF-DOTNET-HEAP-MARK]). await command(target, 'release all'); const unrooted = await uiSnapshot(sixth); - assert.equal(unrooted.topAllocations.some((site) => site.typeName === SHAPES.order.type), false); + assert.equal( + unrooted.topAllocations.some((site) => site.typeName === SHAPES.order.type), + false, + ); assertAnchored(unrooted, SHAPES.quote, 800); // ── Interaction 8: stop. diff --git a/src/editors/vscode/src/test/suite/profiler-fsharp-soak-kit.ts b/src/editors/vscode/src/test/suite/profiler-fsharp-soak-kit.ts index d4f292d9..be7eceec 100644 --- a/src/editors/vscode/src/test/suite/profiler-fsharp-soak-kit.ts +++ b/src/editors/vscode/src/test/suite/profiler-fsharp-soak-kit.ts @@ -143,7 +143,10 @@ export async function compareMemory(wire: { memory_session_id: string }) { const toasts = await captureToasts(() => vscode.commands.executeCommand(uiCommand('memoryDiff'))); assert.deepEqual(toasts.error, [], 'the UI comparison succeeds'); assert.deepEqual(toasts.warning, [], 'the UI comparison is not refused'); - assert.ok(memoryDashboardState()?.html.includes('const leaks = '), 'the dashboard renders the diff'); + assert.ok( + memoryDashboardState()?.html.includes('const leaks = '), + 'the dashboard renders the diff', + ); return wireDiff(wire); } @@ -252,15 +255,16 @@ export function assertPaletteDiscipline(): void { } /** [PROF-DOTNET-SOURCE] + [PROF-DOTNET-HEAP-MARK]: check shapes the fixture actually retains. */ -export function assertEveryShape( - snapshot: MemoryDashboardSnapshot, - live: Counts, -): void { +export function assertEveryShape(snapshot: MemoryDashboardSnapshot, live: Counts): void { for (const shape of Object.values(SHAPES)) { const expected = count(live, shape.live); if (expected > 0) assertAnchored(snapshot, shape, expected); - else assert.equal(snapshot.topAllocations.some((site) => site.typeName === shape.type), false, - `${shape.type} has no reachable instances`); + else + assert.equal( + snapshot.topAllocations.some((site) => site.typeName === shape.type), + false, + `${shape.type} has no reachable instances`, + ); } } diff --git a/src/editors/vscode/src/test/suite/profiler-process-explorer.test.ts b/src/editors/vscode/src/test/suite/profiler-process-explorer.test.ts index ab42bbec..745f3a08 100644 --- a/src/editors/vscode/src/test/suite/profiler-process-explorer.test.ts +++ b/src/editors/vscode/src/test/suite/profiler-process-explorer.test.ts @@ -22,7 +22,13 @@ import { createProfilerStore, type ProfilerStore as Store, } from '@nimblesite/lsp-profiler-ui/out/src/store'; -import { seedSignal, fakeHost, toWireProcess, ungroup, rowPid as pidOf } from './profiler-ui-helpers'; +import { + seedSignal, + fakeHost, + toWireProcess, + ungroup, + rowPid as pidOf, +} from './profiler-ui-helpers'; import { numberField, rawField, From 70703968ad4f8cdc9fd5e1f5e5ca58f043c7367d Mon Sep 17 00:00:00 2001 From: Christian Findlay <16697547+MelbourneDeveloper@users.noreply.github.com> Date: Sun, 18 Oct 2026 18:34:27 +1100 Subject: [PATCH 12/24] Land the F# profiler suites' final shape, under 500 lines a file The corner suite releases the ledger before counting 300 live orders and no longer claims the dashboard keeps a stopped session's snapshot, which memory-flows and the F# memory suite rule out. The memory-step helpers and error codes move into the soak kit, the wire suite splits into protocol, CPU and memory suites, and the fixture locations and builds move from profiler-kit into profiler-fixtures, so no suite or kit passes 500 lines. --- .../src/test/suite/profiler-fixtures.ts | 86 +++++ .../test/suite/profiler-fsharp-artifacts.ts | 72 ++++ .../suite/profiler-fsharp-corner-e2e.test.ts | 200 ++++------- .../suite/profiler-fsharp-cpu-e2e.test.ts | 24 +- .../test/suite/profiler-fsharp-e2e.test.ts | 103 ++++++ .../src/test/suite/profiler-fsharp-kit.ts | 85 +---- .../profiler-fsharp-retention-e2e.test.ts | 47 ++- .../profiler-fsharp-soak-cpu-e2e.test.ts | 38 ++- .../suite/profiler-fsharp-soak-e2e.test.ts | 12 +- .../test/suite/profiler-fsharp-soak-kit.ts | 164 ++++++++- .../vscode/src/test/suite/profiler-kit.ts | 87 +---- .../test/suite/profiler-lsp-cpu-e2e.test.ts | 149 ++++++++ .../suite/profiler-lsp-memory-e2e.test.ts | 139 ++++++++ .../suite/profiler-lsp-protocol-e2e.test.ts | 321 +++--------------- .../suite/profiler-process-explorer.test.ts | 24 +- 15 files changed, 952 insertions(+), 599 deletions(-) create mode 100644 src/editors/vscode/src/test/suite/profiler-fixtures.ts create mode 100644 src/editors/vscode/src/test/suite/profiler-fsharp-artifacts.ts create mode 100644 src/editors/vscode/src/test/suite/profiler-lsp-cpu-e2e.test.ts create mode 100644 src/editors/vscode/src/test/suite/profiler-lsp-memory-e2e.test.ts diff --git a/src/editors/vscode/src/test/suite/profiler-fixtures.ts b/src/editors/vscode/src/test/suite/profiler-fixtures.ts new file mode 100644 index 00000000..2bc5f610 --- /dev/null +++ b/src/editors/vscode/src/test/suite/profiler-fixtures.ts @@ -0,0 +1,86 @@ +// Tests for [PROFILER-SHARED-LIBRARY-VSCODE]. The .NET programs the profiler +// suites profile: where each fixture project lives, and its Release build. +import * as assert from 'node:assert/strict'; +import { spawnSync } from 'node:child_process'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; + +const FIXTURES = path.resolve(__dirname, '../../../../../sharplsp/tests/fixtures'); +/** lspkit's own targets, which prove the engine on every .NET language. */ +const LSPKIT_FIXTURES = path.resolve(__dirname, '../../../../../../lspkit/dotnet/fixtures'); +/** SharpLsp's real-world F# programs: a pricing engine and an event-sourced order ledger. */ +const FSHARP_FIXTURES = path.join(FIXTURES, 'fsharp-profiling'); + +/** A profiling target: its project folder, target framework and source file. */ +interface FixtureProject { + readonly dir: string; + readonly framework: string; + readonly source: string; +} + +const FIXTURE_PROJECTS: Readonly> = { + ProfileTarget: { dir: FIXTURES, framework: 'net10.0', source: 'Program.cs' }, + ProfileLeakTarget: { dir: FIXTURES, framework: 'net10.0', source: 'Program.cs' }, + // F#: records holding union cases leak into a module value ([PROF-TESTING]). + HotLeakAppFs: { dir: LSPKIT_FIXTURES, framework: 'net8.0', source: 'Program.fs' }, + // F# CPU: CSV parsing, recursive discount rules, fuzzy search, route planning, reports. + FsPricingEngine: { dir: FSHARP_FIXTURES, framework: 'net10.0', source: 'Program.fs' }, + // F# memory: event, memoize, mailbox and undo-history leaks in an order ledger. + FsOrderLedger: { dir: FSHARP_FIXTURES, framework: 'net10.0', source: 'Program.fs' }, +}; + +function fixtureProject(name: string): FixtureProject { + const project = FIXTURE_PROJECTS[name]; + assert.ok(project, `unknown profiling fixture ${name}`); + return project; +} + +/** `dotnet` from PATH, else the user-local install the test host's PATH may lack. */ +function dotnetExecutable(): string { + const candidates = [ + 'dotnet', + path.join(os.homedir(), '.dotnet', process.platform === 'win32' ? 'dotnet.exe' : 'dotnet'), + ]; + for (const candidate of candidates) { + if (spawnSync(candidate, ['--version'], { encoding: 'utf8' }).status === 0) return candidate; + } + throw new Error('dotnet SDK not found: install it or put it on PATH'); +} + +/** Build a fixture project once (Release) and return its binary. */ +export function buildFixture(name: string): string { + const { dir, framework } = fixtureProject(name); + const project = path.join(dir, name); + const binary = path.join( + project, + 'bin', + 'Release', + framework, + process.platform === 'win32' ? `${name}.exe` : name, + ); + if (!fs.existsSync(binary)) { + const build = spawnSync(dotnetExecutable(), ['build', '-c', 'Release', '--nologo', '-v', 'q'], { + cwd: project, + encoding: 'utf8', + }); + assert.equal( + build.status, + 0, + `building ${name} failed: ${build.stderr ?? build.error?.message}`, + ); + } + assert.ok(fs.existsSync(binary), `${binary} missing after build`); + return binary; +} + +/** The source file of a fixture, where hot lines and allocation sites land. */ +export function fixtureSource(name: string): string { + const { dir, source } = fixtureProject(name); + return path.join(dir, name, source); +} + +/** Another source file of a multi-file fixture project. */ +export function fixtureFile(name: string, file: string): string { + return path.join(fixtureProject(name).dir, name, file); +} diff --git a/src/editors/vscode/src/test/suite/profiler-fsharp-artifacts.ts b/src/editors/vscode/src/test/suite/profiler-fsharp-artifacts.ts new file mode 100644 index 00000000..778ae182 --- /dev/null +++ b/src/editors/vscode/src/test/suite/profiler-fsharp-artifacts.ts @@ -0,0 +1,72 @@ +// Tests for [PROF-EXPORT] and [PROF-DOTNET-CPU-SYMBOLS]: real F# profile artifacts. +import * as assert from 'node:assert/strict'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import type { ProfileResult } from '@nimblesite/lsp-profiler-ui'; +import { comparablePath } from './test-helpers'; + +/** True when `file` is the fixture file named `name` (any fixture). */ +export function isFile(file: string | undefined, name: string): boolean { + return file !== undefined && path.basename(comparablePath(file)) === comparablePath(name); +} + +interface Speedscope { + shared: { frames: { name: string; file?: string; line?: number }[] }; + profiles: { samples: number[][]; weights: number[] }[]; +} + +interface CpuProfile { + nodes: { id: number; callFrame: { functionName: string; url: string } }[]; + samples: number[]; + timeDeltas: number[]; + startTime: number; + endTime: number; +} + +function readJson(file: string | undefined, what: string): unknown { + assert.ok(file !== undefined && fs.existsSync(file), `${what} is written: ${String(file)}`); + return JSON.parse(fs.readFileSync(file, 'utf8')); +} + +/** [PROF-EXPORT]: all three artifacts exist, validate, and carry the F# function. */ +export function assertArtifacts(result: ProfileResult, identifier: string, file: string): void { + assert.equal(result.exportError, undefined, `no export error: ${result.exportError}`); + const speedscope = readJson(result.outputFile, 'the speedscope profile') as Speedscope; + const { frames } = speedscope.shared; + const keysSeen = frames.map((f) => `${f.name}|${f.file ?? ''}|${f.line ?? ''}`); + assert.equal(new Set(keysSeen).size, keysSeen.length, 'frames are deduplicated'); + assert.ok( + frames.some((f) => f.name.includes(identifier) && isFile(f.file, file)), + `a speedscope frame names ${identifier} in ${file}`, + ); + assert.ok(speedscope.profiles.length > 0, 'one profile per thread'); + for (const profile of speedscope.profiles) { + assert.equal(profile.samples.length, profile.weights.length, 'samples match weights'); + assert.ok( + profile.weights.every((w) => Number.isFinite(w) && w >= 0), + 'weights are sane', + ); + assert.ok( + profile.samples.flat().every((i) => i >= 0 && i < frames.length), + 'indices in range', + ); + } + const v8 = readJson(result.cpuProfilePath, 'the V8 .cpuprofile') as CpuProfile; + const ids = new Set(v8.nodes.map((n) => n.id)); + assert.equal(v8.samples.length, v8.timeDeltas.length, 'one time delta per sample'); + assert.ok( + v8.samples.every((id) => ids.has(id)), + 'every sample names a node', + ); + assert.ok(v8.endTime >= v8.startTime, 'time runs forward'); + assert.ok( + v8.nodes.some((n) => n.callFrame.functionName.includes(identifier)), + `a .cpuprofile node names ${identifier}`, + ); + assert.ok(result.flamegraphPath !== undefined, 'a flame graph is exported'); + const svg = fs.readFileSync(result.flamegraphPath, 'utf8'); + assert.ok( + svg.includes('): Promise { - const result = await execute(name, params); - assert.equal(result.kind, 'Completed', `[PROF-MEMORY-STEP] ${name} completes in Direct mode`); - assert.ok(result.kind === 'Completed', 'narrowed'); - return result.result.outcome; -} - -async function snapshotOutcome(id: string): Promise> { - const outcome = await step('memory.snapshot', { memory_session_id: id }); - assert.equal(outcome.kind, 'Snapshot', '[PROF-MEMORY-STEP] snapshot yields a Snapshot outcome'); - assert.ok(outcome.kind === 'Snapshot', 'narrowed'); - return outcome; -} - -async function diffOutcome(id: string): Promise> { - const outcome = await step('memory.diff', { memory_session_id: id }); - assert.equal(outcome.kind, 'Diff', '[PROF-MEMORY-STEP] diff yields a Diff outcome'); - assert.ok(outcome.kind === 'Diff', 'narrowed'); - return outcome; -} - -/** The UI snapshot the different `previous` snapshot. */ -async function uiSnapshot(previous?: MemoryDashboardSnapshot): Promise { - await vscode.commands.executeCommand(uiCommand('memorySnapshot')); - const snapshot = await waitFor( - () => lastMemorySnapshot(), - (snap) => snap !== undefined && snap.snapshotId !== previous?.snapshotId, - STOP_MS, - 'a newer snapshot in the dashboard', - ); - assert.ok(snapshot, 'the dashboard holds a snapshot'); - return snapshot; -} - suite('F# profiler corner cases — FsOrderLedger, wire first', () => { const live = profilingSuite('profiler-fs-corner-', LEDGER, { stopping: endProfiling }); @@ -217,9 +151,10 @@ suite('F# profiler corner cases — FsOrderLedger, wire first', () => { ); }); - test('memory steps: traceback-capped sites, objectsByType, references with cycles, gcCollect frees bytes, and ingest quotes garbage back', async function () { + test('memory steps: traceback-capped F# sites, exact objectsByType counts, retention roots, growth and release, and Direct mode refuses courier ingest', async function () { this.timeout(STOP_MS * 6); const { target } = live; + await command(target, 'release all'); const seeded = await command(target, 'place 300'); assert.equal(count(seeded, 'orders'), 300, '300 live orders'); @@ -248,12 +183,8 @@ suite('F# profiler corner cases — FsOrderLedger, wire first', () => { '[PROF-MEMORY-TIMELINE] the first snapshot is the first point', ); assert.equal(first.trending_upward, false, 'one point cannot trend upward'); - if (first.heapprofile_path !== undefined) { - assert.ok( - fs.existsSync(first.heapprofile_path), - `[PROF-MEMORY-HEAPPROFILE] ${first.heapprofile_path}`, - ); - } + assert.ok(first.heapprofile_path, '[PROF-MEMORY-HEAPPROFILE] every snapshot exports'); + assert.ok(fs.existsSync(first.heapprofile_path), 'the exported heap profile exists'); // ── [PROF-MEMORY-STEP] objectsByType names the type and summarises the heap. const objects = (await step('memory.objectsByType', { @@ -263,7 +194,11 @@ suite('F# profiler corner cases — FsOrderLedger, wire first', () => { assert.equal(objects.kind, 'Objects', 'objectsByType yields an Objects outcome'); const listed: ObjectList = objects.objects; assert.equal(listed.type_name, SHAPES.order.type, 'the requested type is echoed'); - assert.ok(listed.total_count >= 300, `at least the 300 live orders: ${listed.total_count}`); + assert.equal( + listed.total_count, + count(seeded, 'orders'), + 'every live F# Order, counted exactly', + ); assert.ok(listed.total_bytes > 0, 'the type has live bytes'); assert.ok( listed.objects.length > 0 && listed.objects.length <= listed.total_count, @@ -275,14 +210,14 @@ suite('F# profiler corner cases — FsOrderLedger, wire first', () => { assert.ok(object.label.length > 0, 'a sample has a label'); } assert.ok(listed.type_summary.length > 0, 'a per-type summary is offered'); - const counts = listed.type_summary.map((stat) => stat.count); + const sizes = listed.type_summary.map((stat) => stat.size_bytes); assert.deepEqual( - counts, - [...counts].sort((a, b) => b - a), - '[PROF-INVARIANTS] largest types first', + sizes, + [...sizes].sort((a, b) => b - a), + '[PROF-INVARIANTS] types ranked by live bytes, not instance count', ); - for (const shape of Object.values(SHAPE_TABLE)) { + for (const shape of Object.values(SHAPES)) { const listedType = (await step('memory.objectsByType', { memory_session_id: id, type_name: shape.type, @@ -298,6 +233,11 @@ suite('F# profiler corner cases — FsOrderLedger, wire first', () => { `only ${shape.type} samples`, ); assert.ok(listedType.objects.total_bytes >= 0, `${shape.type} bytes are never negative`); + assert.equal( + listedType.objects.total_count, + count(seeded, shape.live), + `[PROF-DOTNET-HEAP-MARK] exactly the retained ${shape.type} instances`, + ); assert.ok( listedType.objects.objects.length <= listedType.objects.total_count, `${shape.type} samples are a subset`, @@ -306,7 +246,7 @@ suite('F# profiler corner cases — FsOrderLedger, wire first', () => { listedType.objects.type_summary.length > 0, `${shape.type} still summarises the heap`, ); - const summary = listedType.objects.type_summary.map((stat) => stat.count); + const summary = listedType.objects.type_summary.map((stat) => stat.size_bytes); assert.deepEqual( summary, [...summary].sort((a, b) => b - a), @@ -361,23 +301,27 @@ suite('F# profiler corner cases — FsOrderLedger, wire first', () => { >; assert.equal(gc.kind, 'Gc', 'gcCollect yields a Gc outcome'); const collected: GcResult = gc.gc; - assert.ok(collected.collected >= 0, 'a collected count'); + assert.equal(collected.collected, 1, '[PROF-DOTNET-GC] one induced collection was observed'); assert.ok(collected.freed_bytes >= 0, 'a freed-bytes total'); - assert.ok(collected.uncollectable_count >= 0, 'an uncollectable count'); + assert.equal(collected.uncollectable_count, 0, '.NET reports no uncollectable objects'); assert.equal( collected.uncollectable.length, collected.uncollectable_count, 'the uncollectable details match their count', ); - // ── [PROF-MEMORY-INGEST] an unrecognised payload quotes the delivered text back. + // ── [PROF-MEMORY-MODE] .NET is Direct: it never parses a courier's payload. const marker = 'not-a-payload-7fd1'; const refused = await executeError('memory.ingest', { memory_session_id: id, output: marker }); assert.ok(refused, 'garbage is refused'); - assert.ok(refused.code !== 0, `an error code is returned: ${JSON.stringify(refused)}`); + assert.equal( + refused.code, + CODE.OTHER, + '[PROF-ERRORS] a recognised runtime refuses an unsupported backend operation', + ); assert.ok( - refused.message.includes(marker), - `[PROF-MEMORY-INGEST] the error quotes the delivered text: ${refused.message}`, + refused.message.includes('captures directly'), + `[PROF-MEMORY-MODE] the refusal explains Direct capture: ${refused.message}`, ); // ── Ingesting into an unknown session is an error, never a new session. @@ -386,11 +330,11 @@ suite('F# profiler corner cases — FsOrderLedger, wire first', () => { output: '{}', }); assert.ok(unknown, 'an unknown session fails'); + assert.equal(unknown.code, CODE.OTHER, 'ingest never invents an unknown session'); assert.equal((await snapshotOutcome(id)).kind, 'Snapshot', 'the real session still answers'); // ── [PROF-MEMORY-COMPARE] growth after the baseline, then release, then freed > 0. - await command(target, 'place 300'); - const grown = await snapshotOutcome(id); + const more = await command(target, 'place 300'); const diff = await diffOutcome(id); assert.ok(diff.diff.total_growth > 0, 'the growth is measured'); assert.equal(diff.diff.net_growth, diff.diff.total_growth - diff.diff.total_freed); @@ -399,13 +343,16 @@ suite('F# profiler corner cases — FsOrderLedger, wire first', () => { orderSite, `[PROF-MEMORY-COMPARE] the order site grew: ${JSON.stringify(diff.diff.grown.map((s) => s.type_name))}`, ); - assert.ok(orderSite.size_diff > 0 && orderSite.count_diff > 0, 'growth is bytes and count'); - assert.ok((orderSite.line ?? 0) >= 1, '[PROF-WIRE] site lines are 1-based'); + assert.equal(orderSite.count_diff, count(more, 'orders') - count(seeded, 'orders')); + assert.ok(orderSite.size_diff > 0, 'the additional F# records grew in bytes'); + assert.ok(isFile(orderSite.file, SHAPES.order.file), 'Order growth resolves to Ledger.fs'); + assert.equal(orderSite.line, SHAPES.order.line, 'Order growth anchors where constructed'); + const grown = await snapshotOutcome(id); assert.ok(grown.snapshot.current_bytes >= first.snapshot.current_bytes, 'the heap grew'); assert.equal( - grown.trending_upward, - true, - '[PROF-MEMORY-TIMELINE] two rising points trend upward', + grown.snapshot.sites.find((site) => site.type_name === SHAPES.order.type)?.count, + count(more, 'orders'), + 'the post-diff snapshot retains the exact new F# count', ); for (let index = 1; index < grown.timeline.length; index += 1) { const before = grown.timeline[index - 1]; @@ -415,8 +362,8 @@ suite('F# profiler corner cases — FsOrderLedger, wire first', () => { } await command(target, 'release all'); - const shrunk = await snapshotOutcome(id); const freed: HeapDiff = (await diffOutcome(id)).diff; + const shrunk = await snapshotOutcome(id); assert.ok(shrunk.snapshot.current_bytes < grown.snapshot.current_bytes, 'the heap shrank'); assert.ok( freed.grown.every((site) => site.size_diff > 0), @@ -426,6 +373,12 @@ suite('F# profiler corner cases — FsOrderLedger, wire first', () => { freed.freed.every((site) => site.size_diff < 0), 'only shrinking sites are listed as freed', ); + assert.equal( + freed.freed.find((site) => site.type_name === SHAPES.order.type)?.count_diff, + -count(more, 'orders'), + 'every unrooted F# Order is freed', + ); + assert.ok(freed.total_freed > 0, 'the released F# records contribute freed bytes'); completed(await execute('memory.stop', { memory_session_id: id })); }); }); @@ -437,14 +390,13 @@ suite('F# profiler corner cases — FsOrderLedger through the dashboard', () => this.timeout(STOP_MS * 6); const { target } = live; await showSources(LEDGER, ['Ledger.fs', 'Quotes.fs']); - await command(target, 'quote 250'); + const seeded = await command(target, 'quote 250'); await trackMemory(target.pid); assert.equal( profilerStore().profiler.value.memory, 'active', '[PROF-UI-STATE] the store says tracking', ); - const opsBefore = recordedOperations().length; // ── [PROF-MEMORY-HEAPPROFILE] the .heapprofile is real V8 JSON. const first = await uiSnapshot(); @@ -458,18 +410,19 @@ suite('F# profiler corner cases — FsOrderLedger through the dashboard', () => }; assert.ok(profile.head !== undefined, 'the profile has a call-tree root'); assert.ok(Array.isArray(profile.samples), 'the profile has samples'); - assert.ok((profile.samples as unknown[]).length >= 0, 'the sample array parses from disk'); + assert.ok((profile.samples as unknown[]).length > 0, 'the live F# heap exports samples'); // ── [PROF-UI-MEMORY] generation bytes are not collection counts. assert.deepEqual(first.gcCounts, [], 'counts are never presented as generation bytes'); - if (first.generationSizes !== undefined) { - assert.ok(first.generationSizes.length > 0, 'the runtime named its heap regions'); - const total = first.generationSizes.reduce((sum, region) => sum + region.size, 0); - assert.ok(total <= first.currentMemory + MIB, `regions fit in the live heap: ${total}`); - for (const region of first.generationSizes) { - assert.ok(region.name.length > 0, 'a region has a runtime label'); - assert.ok(region.size >= 0, 'a region size is a byte count'); - } + assert.ok(first.generationSizes, 'the .NET runtime reports generation bytes'); + assert.deepEqual( + first.generationSizes.map((region) => region.name), + ['Gen 0', 'Gen 1', 'Gen 2', 'LOH', 'POH'], + 'all five .NET heap regions are named', + ); + for (const region of first.generationSizes) { + assert.ok(region.name.length > 0, 'a region has a runtime label'); + assert.ok(region.size >= 0, 'a region size is a byte count'); } assert.equal(memoryDashboardState()?.title, 'SharpLsp Memory Dashboard'); assertAnchored(first, SHAPES.quote, 250); @@ -477,7 +430,7 @@ suite('F# profiler corner cases — FsOrderLedger through the dashboard', () => assertTimelineCoherent(first); assertPaletteDiscipline(); assertStoreCoherent(); - assertEveryShape(first, { [SHAPES.quote.type]: 250 }); + assertEveryShape(first, seeded); for (const site of first.topAllocations) { assert.ok((site.typeName ?? '').length > 0, 'every top site names its runtime type'); assert.ok( @@ -501,10 +454,7 @@ suite('F# profiler corner cases — FsOrderLedger through the dashboard', () => 1, 'one site per type, never a duplicate row', ); - assert.ok( - recordedOperations().length >= opsBefore, - '[PROF-UI-STATE] the store is the source of truth for what was rendered', - ); + assertAnchored(second, SHAPES.quote, count(seeded, 'quotes')); assert.equal( lastMemorySnapshot()?.snapshotId, second.snapshotId, @@ -528,11 +478,11 @@ suite('F# profiler corner cases — FsOrderLedger through the dashboard', () => profilerBusy: false, }); - // ── [PROF-UI-STATE] the last snapshot is a report, not a live session: it survives. - assert.equal( - lastMemorySnapshot()?.memorySessionId, - second.memorySessionId, - 'the report is kept for reading', + // ── [PROF-UI-STATE] the session clears; [PROF-UI-MEMORY] the rendered report stays readable. + assert.equal(lastMemorySnapshot(), undefined, 'no stale session snapshot survives stop'); + assert.ok( + memoryDashboardState()?.html.includes(SHAPES.quote.type), + 'the F# report stays readable', ); const stopped = await executeError('memory.snapshot', { memory_session_id: second.memorySessionId, diff --git a/src/editors/vscode/src/test/suite/profiler-fsharp-cpu-e2e.test.ts b/src/editors/vscode/src/test/suite/profiler-fsharp-cpu-e2e.test.ts index e17dfdf5..16acfe81 100644 --- a/src/editors/vscode/src/test/suite/profiler-fsharp-cpu-e2e.test.ts +++ b/src/editors/vscode/src/test/suite/profiler-fsharp-cpu-e2e.test.ts @@ -316,14 +316,18 @@ suite('F# CPU profiling — FsPricingEngine, one F# shape at a time', () => { assert.deepEqual(heatOn('Routing.fs'), [], 'the setting turns the tint off'); assertResultsPanel(quiet, ['shortestPath']); - // Heat map back on, every workload at once: each module is hot and painted. + // Give every workload a full sampling window in ONE session. A mixed loop's + // stages have unequal cost; merely executing a stage does not make it a 2% hotspot. await setProfilerSetting('showInlineHeatMap', true); await command(target, 'run mixed'); const files = WORKLOADS.map((w) => w.span.file); await showSources(ENGINE, files); await profileRow(await processRow(target.pid)); await awaitLiveProgress(); - await delay(SAMPLE_MS * 2); + for (const workload of WORKLOADS) { + await command(target, `run ${workload.command}`); + await delay(SAMPLE_MS); + } await stopProfiling(STOP_MS); const mixed = lastProfileResult(); assert.ok(mixed); @@ -340,15 +344,18 @@ suite('F# CPU profiling — FsPricingEngine, one F# shape at a time', () => { await command(target, 'idle'); }); - test('the profiler tells the truth about where F# time goes: on a mixed run, each stage’s share of the samples matches the share FsPricingEngine measured with its own Stopwatch, and the stages rank in the same order', async function () { + test('the profiler tells the truth about four sustained F# workloads: sampled shares match the fixture’s Stopwatch and materially different stages rank in the same order', async function () { this.timeout(STOP_MS * 3); const { target } = live; - await command(target, 'run mixed'); + await command(target, 'run parse'); await showSources(ENGINE, ['Csv.fs', 'Search.fs', 'Routing.fs']); const before = await command(target, 'stats'); await profileRow(await processRow(target.pid)); await awaitLiveProgress(); - await delay(SAMPLE_MS * 3); + for (const stage of GROUND_TRUTH) { + await command(target, `run ${stage.stage}`); + await delay(SAMPLE_MS); + } await stopProfiling(STOP_MS); const after = await command(target, 'stats'); const result = lastProfileResult(); @@ -372,6 +379,13 @@ suite('F# CPU profiling — FsPricingEngine, one F# shape at a time', () => { })); const major = shares.filter((s) => s.truth >= MAJOR_SHARE); assert.ok(major.length >= 2, `at least two stages carry real time: ${JSON.stringify(shares)}`); + for (const stage of shares) { + assert.ok(stage.micros > 0, `the fixture actually exercised ${stage.identifier}`); + assert.ok( + Math.abs(stage.seen - stage.truth) <= SHARE_TOLERANCE, + `${stage.identifier}: sampled ${pct(stage.seen)}, measured ${pct(stage.truth)}`, + ); + } for (const stage of major) { assert.ok(stage.sampled > 0, `${stage.identifier} took ${pct(stage.truth)} and must be hot`); assert.ok( diff --git a/src/editors/vscode/src/test/suite/profiler-fsharp-e2e.test.ts b/src/editors/vscode/src/test/suite/profiler-fsharp-e2e.test.ts index 66b5b803..30f1ecfb 100644 --- a/src/editors/vscode/src/test/suite/profiler-fsharp-e2e.test.ts +++ b/src/editors/vscode/src/test/suite/profiler-fsharp-e2e.test.ts @@ -46,6 +46,16 @@ const SPIN_LINES = { first: 26, last: 31 }; const RECORD = { type: 'HotLeakAppFs.LeakedPayload', line: 23 }; const UNION_CASE = { type: 'HotLeakAppFs.PayloadKind+Small', line: 21 }; const GENERIC_RECORD = { type: 'HotLeakAppFs.Tagged`1[System.Int32]', line: 22 }; +/** The rest of lspkit's F# heap zoo, each at the line that constructs it. */ +const ORDER = { type: 'HotLeakAppFs.Order', line: 77 }; +const ORDER_LINE = { type: 'HotLeakAppFs.OrderLine', line: 75 }; +const CUSTOMER = { type: 'HotLeakAppFs.CustomerId', line: 73 }; +const EXPRESS = { type: 'HotLeakAppFs.Priority+Express', line: 76 }; +const QUOTE = { type: 'HotLeakAppFs.Quote', line: 82 }; +const NODE = { type: 'HotLeakAppFs.Node', line: 88 }; +/** The subscription closure F# builds for `Event.Publish.Add`. */ +const CLOSURE_LINE = 97; +type DashboardSite = NonNullable>['topAllocations'][number]; const inSpin = (line: number): boolean => line >= SPIN_LINES.first && line <= SPIN_LINES.last; const isProgramFs = (file: string | undefined): boolean => file?.endsWith('Program.fs') ?? false; @@ -140,6 +150,99 @@ suite('F# profiling — real end-to-end', () => { assert.equal(memoryStatusText(), undefined, 'the status bar hides once idle'); }); + test('the F# heap zoo: grow orders, quotes, closures, a cycle and arrays → every shape anchors at its constructing line, arrays never do → the growing record reaches Low → stop clears the track', async function () { + this.timeout(STEP_MS * 3); + const { target } = live; + // A fresh, deterministic heap: no background growth, then one shape at a time. + for (const line of ['work off', 'release all', 'payload 100', 'grow 200', 'blob 4']) { + const reply = await target.send(line); + assert.ok(reply.startsWith(`OK ${line}`), `the fixture accepts "${line}": ${reply}`); + } + await openSource(fixtureSource(FIXTURE)); + await trackMemory(target.pid); + await vscode.commands.executeCommand(uiCommand('memorySnapshot')); + const snapshot = await waitFor( + () => lastMemorySnapshot(), + (snap) => snap?.topAllocations.some((site) => site.typeName === ORDER.type) ?? false, + STEP_MS, + 'a snapshot naming the F# order record', + ); + assert.ok(snapshot, 'the dashboard holds the snapshot'); + for (const { type, line } of [ + RECORD, + UNION_CASE, + GENERIC_RECORD, + ORDER, + ORDER_LINE, + CUSTOMER, + EXPRESS, + QUOTE, + NODE, + ]) { + const foundSite: DashboardSite | undefined = snapshot.topAllocations.find( + (entry) => entry.typeName === type, + ); + assert.ok( + foundSite, + `${type} is a site: ${snapshot.topAllocations.map((entry) => entry.typeName).join(', ')}`, + ); + assert.equal(foundSite.line, line, `${type} anchors where the program constructs it`); + assert.ok(isProgramFs(foundSite.file), `${type} resolves to Program.fs: ${foundSite.file}`); + assert.ok(foundSite.size > 0, `${type} has bytes`); + assert.ok(foundSite.count > 0, `${type} has live instances`); + } + assert.equal( + snapshot.topAllocations.find((site) => site.typeName === ORDER.type)?.count, + 200, + 'every live order is counted', + ); + assert.equal( + snapshot.topAllocations.find((site) => site.typeName === NODE.type)?.count, + 202, + 'the cycle plus one node per order', + ); + assert.ok( + snapshot.topAllocations.some( + (site) => site.line === CLOSURE_LINE && site.typeName?.startsWith('HotLeakAppFs.Shapes'), + ), + 'the event-handler closure is a site at its own line', + ); + // [PROF-DOTNET-SOURCE] an array is the runtime's own type and is never anchored. + for (const type of ['System.Byte[]', 'HotLeakAppFs.Order[]', 'HotLeakAppFs.LeakedPayload[]']) { + const foundSite: DashboardSite | undefined = snapshot.topAllocations.find( + (entry) => entry.typeName === type, + ); + assert.ok(foundSite, `${type} is a site`); + assert.equal(foundSite.file, undefined, `${type} has no source file`); + assert.equal(foundSite.line, undefined, `${type} has no line`); + } + const painted = appliedMemoryDecorations().filter((entry) => isProgramFs(entry.file)); + for (const { line } of [ORDER, ORDER_LINE, CUSTOMER, EXPRESS, QUOTE, NODE]) { + assert.ok( + painted.some((entry) => entry.line === line), + `the memory track paints line ${line}`, + ); + } + assert.ok( + memoryDashboardState()?.html.includes('HotLeakAppFs.Order'), + 'the dashboard names the F# record', + ); + + await target.send('payload 100'); + await vscode.commands.executeCommand(uiCommand('memoryDiff')); + const leaks = await waitFor( + () => embeddedLeaks(memoryDashboardState()?.html ?? ''), + (found) => found.length > 0, + STEP_MS, + 'suspected leaks after the growth', + ); + const record = leaks.find((leak) => leak.typeName === RECORD.type); + assert.ok(record, `the growing F# record is a leak: ${JSON.stringify(leaks)}`); + assert.equal(record.confidence, 'low', '[PROF-MEMORY-CONFIDENCE] one growth is Low'); + await stopTracking(); + assert.deepEqual(appliedMemoryDecorations(), [], 'stop clears the memory track'); + }); + test('the leaked F# record is held by the module value `static items`, and its union case is found by its full name', async function () { this.timeout(STEP_MS * 3); const { target } = live; diff --git a/src/editors/vscode/src/test/suite/profiler-fsharp-kit.ts b/src/editors/vscode/src/test/suite/profiler-fsharp-kit.ts index fa1bf61f..aae3377c 100644 --- a/src/editors/vscode/src/test/suite/profiler-fsharp-kit.ts +++ b/src/editors/vscode/src/test/suite/profiler-fsharp-kit.ts @@ -10,7 +10,6 @@ // memory track (lspkit's [PROF-UI-CPU], [PROF-UI-HEATMAP], [PROF-EXPORT], // [PROF-UI-MEMORY], [PROF-AGGREGATION-THRESHOLD]). import * as assert from 'node:assert/strict'; -import * as fs from 'node:fs'; import * as path from 'node:path'; import * as vscode from 'vscode'; import { @@ -44,7 +43,8 @@ import { uiCommand, waitFor, } from './profiler-kit'; -import { comparablePath } from './test-helpers'; +import { isFile } from './profiler-fsharp-artifacts'; +export { isFile, assertArtifacts } from './profiler-fsharp-artifacts'; import { LSP_RESPONSE_MS, PROCESS_START_MS } from './test-timeouts'; export const ENGINE = 'FsPricingEngine'; @@ -99,11 +99,6 @@ export function within(span: Span, file: string | undefined, line: number): bool return isFile(file, span.file) && line >= span.first && line <= span.last; } -/** True when `file` is the fixture file named `name` (any fixture). */ -export function isFile(file: string | undefined, name: string): boolean { - return file !== undefined && path.basename(comparablePath(file)) === comparablePath(name); -} - /** The absolute path of a fixture file, as the editor and the server name it. */ export function sourceOf(fixture: string, file: string): string { return fixtureFile(fixture, file); @@ -252,7 +247,7 @@ export async function assertCpuIdle(pid: number): Promise { // ── The CPU result as the editor shows it ──────────────────────────────────── -/** The heat tier a share paints with; `undefined` near a tier boundary. */ +/** The heat tier a share paints with, including its exact threshold. */ export function cpuTier(percentage: number): string | undefined { const tiers: readonly [number, string][] = [ [20, '#e8500a'], @@ -260,8 +255,7 @@ export function cpuTier(percentage: number): string | undefined { [5, '#fbbf24'], [0, '#4a5468'], ]; - if ([20, 10, 5].some((edge) => Math.abs(percentage - edge) < 0.5)) return undefined; - return tiers.find(([floor]) => percentage > floor)?.[1]; + return tiers.find(([floor]) => percentage >= floor)?.[1]; } /** [PROF-AGGREGATION-LOGIC] + [PROF-AGGREGATION-THRESHOLD]: shares are sane and ranked. */ @@ -307,10 +301,14 @@ export function assertHeatMap(result: ProfileResult, span: Span): void { for (const decoration of painted) { assert.ok(CPU_PALETTE.includes(decoration.color), `CPU track colour: ${decoration.color}`); assert.ok(/\d%/.test(decoration.contentText), `a share is shown: ${decoration.contentText}`); - const hot = result.hotLines.find( - (l) => isFile(l.file, span.file) && l.line === decoration.line, + const hot = + result.hotLines.find((l) => isFile(l.file, span.file) && l.line === decoration.line) ?? + result.hotFunctions.find((fn) => isFile(fn.file, span.file) && fn.line === decoration.line); + assert.ok(hot, `line ${decoration.line} is painted from a reported hot line or function`); + assert.ok( + decoration.contentText.includes(`${hot.percentage.toFixed(1)}%`), + 'the decoration displays the reported share at that source location', ); - assert.ok(hot, `line ${decoration.line} is painted only because it is a hot line`); const tier = cpuTier(hot.percentage); if (tier !== undefined) { assert.equal(decoration.color, tier, `line ${hot.line} at ${hot.percentage}% wears its tier`); @@ -345,67 +343,6 @@ export function assertResultsPanel(result: ProfileResult, identifiers: readonly } } -interface Speedscope { - shared: { frames: { name: string; file?: string; line?: number }[] }; - profiles: { samples: number[][]; weights: number[] }[]; -} - -interface CpuProfile { - nodes: { id: number; callFrame: { functionName: string; url: string } }[]; - samples: number[]; - timeDeltas: number[]; - startTime: number; - endTime: number; -} - -function readJson(file: string | undefined, what: string): unknown { - assert.ok(file !== undefined && fs.existsSync(file), `${what} is written: ${String(file)}`); - return JSON.parse(fs.readFileSync(file, 'utf8')); -} - -/** [PROF-EXPORT]: all three artifacts exist, validate, and carry the F# function. */ -export function assertArtifacts(result: ProfileResult, identifier: string, file: string): void { - assert.equal(result.exportError, undefined, `no export error: ${result.exportError}`); - const speedscope = readJson(result.outputFile, 'the speedscope profile') as Speedscope; - const { frames } = speedscope.shared; - const keysSeen = frames.map((f) => `${f.name}|${f.file ?? ''}|${f.line ?? ''}`); - assert.equal(new Set(keysSeen).size, keysSeen.length, 'frames are deduplicated'); - assert.ok( - frames.some((f) => f.name.includes(identifier) && isFile(f.file, file)), - `a speedscope frame names ${identifier} in ${file}`, - ); - assert.ok(speedscope.profiles.length > 0, 'one profile per thread'); - for (const profile of speedscope.profiles) { - assert.equal(profile.samples.length, profile.weights.length, 'samples match weights'); - assert.ok( - profile.weights.every((w) => Number.isFinite(w) && w >= 0), - 'weights are sane', - ); - assert.ok( - profile.samples.flat().every((i) => i >= 0 && i < frames.length), - 'indices in range', - ); - } - const v8 = readJson(result.cpuProfilePath, 'the V8 .cpuprofile') as CpuProfile; - const ids = new Set(v8.nodes.map((n) => n.id)); - assert.equal(v8.samples.length, v8.timeDeltas.length, 'one time delta per sample'); - assert.ok( - v8.samples.every((id) => ids.has(id)), - 'every sample names a node', - ); - assert.ok(v8.endTime >= v8.startTime, 'time runs forward'); - assert.ok( - v8.nodes.some((n) => n.callFrame.functionName.includes(identifier)), - `a .cpuprofile node names ${identifier}`, - ); - assert.ok(result.flamegraphPath !== undefined, 'a flame graph is exported'); - const svg = fs.readFileSync(result.flamegraphPath, 'utf8'); - assert.ok( - svg.includes('= sizes.reduce((a, b) => a + b, 0), 'the total covers the listed'); if (live > 0) { - const summary = objects.type_summary.find((t) => t.type_name === type); - assert.ok(summary, `${type} is in the summary`); - assert.equal(summary.count, live, 'the summary agrees with the total'); + const own = objects.type_summary.find((entry) => entry.type_name === type); + assert.ok(own, '[PROF-DOTNET-MEMORY] the summary always includes the requested live type'); + assert.equal(own.count, live, 'the requested summary row counts every live F# instance'); + assert.equal( + own.size_bytes, + objects.total_bytes, + 'its summary bytes match the full type total', + ); } + if (live <= limit) + assert.equal( + objects.total_bytes, + sizes.reduce((sum, size) => sum + size, 0), + 'an exhaustive object list accounts for every byte of the requested F# type', + ); + const ranked = objects.type_summary.map((entry) => entry.size_bytes); + assert.deepEqual( + ranked, + [...ranked].sort((a, b) => b - a), + 'the heap summary ranks by bytes', + ); } /** Seed one of every leak, deterministically. */ @@ -185,7 +202,13 @@ suite('F# retention, objects, GC and refusals — FsOrderLedger', () => { } const tight = await retentionGraph(session, type, 2, 8); assertGraphRules(tight, type, 2, 8); - assert.equal(tight.truncated, true, `${type}: thousands of instances overflow 8 nodes`); + const exhausted = await retentionGraph(session, type, 2, 1); + assertGraphRules(exhausted, type, 2, 1); + assert.equal( + exhausted.truncated, + true, + `${type}: one node cannot hold the target and its holder`, + ); } // The never-removed handler: an F# closure, rooted at the event's module value. @@ -244,6 +267,22 @@ suite('F# retention, objects, GC and refusals — FsOrderLedger', () => { assertObjects(await objectsOf(session, type, limit), type, liveCount, limit); } } + // [PROF-DOTNET-MEMORY] every summary row agrees with that type's exhaustive totals. + const summary = (await objectsOf(session, SHAPES.order.type, 1)).type_summary; + assert.ok(summary.length > 0, 'the real heap produces a ranked type summary'); + for (const entry of summary) { + const actual = await objectsOf(session, entry.type_name, 1); + assert.equal( + entry.count, + actual.total_count, + `${entry.type_name}: summary count matches objects`, + ); + assert.equal( + entry.size_bytes, + actual.total_bytes, + `${entry.type_name}: summary bytes match objects`, + ); + } for (const absent of [ 'FsOrderLedger.OrderStatus+Pending', 'FsOrderLedger.Address', diff --git a/src/editors/vscode/src/test/suite/profiler-fsharp-soak-cpu-e2e.test.ts b/src/editors/vscode/src/test/suite/profiler-fsharp-soak-cpu-e2e.test.ts index a61c5476..17b2117f 100644 --- a/src/editors/vscode/src/test/suite/profiler-fsharp-soak-cpu-e2e.test.ts +++ b/src/editors/vscode/src/test/suite/profiler-fsharp-soak-cpu-e2e.test.ts @@ -31,9 +31,11 @@ import { import { execute, executeError, + completed, profilerStore, profilingSuite, setProfilerSetting, + snapshotSession, stopProfiling, stopTracking, trackMemory, @@ -68,7 +70,9 @@ import { showSources, STOP_MS, within, + wireLeak, } from './profiler-fsharp-kit'; +import type { MemoryStep } from '@nimblesite/lsp-profiler-ui/out/src/wire-generated'; import { assertConfidence, assertDashboard, @@ -78,6 +82,7 @@ import { assertSitesCoherent, assertStoreCoherent, assertTimelineCoherent, + compareMemory, delay, SAMPLE_MS, uiSnapshot, @@ -114,7 +119,6 @@ suite('F# soak — FsPricingEngine through five workloads in one long CPU sessio stages.map((stage) => stage.file), ); - let previousSamples = 0; for (const stage of stages) { // ── Interaction: switch the workload, then profile this stage from the real row. const started = await command(target, stage.command); @@ -205,8 +209,10 @@ suite('F# soak — FsPricingEngine through five workloads in one long CPU sessio const result = lastProfileResult(); assert.ok(result, `a result for ${stage.command}`); assert.equal(result.sessionId, sessionId, 'the result belongs to its own session'); - assert.ok(result.totalSamples >= previousSamples, 'samples accumulate within a session'); - previousSamples = result.totalSamples; + assert.ok( + result.totalSamples >= snapshot.totalSamples, + '[PROF-CPU-STOP] stop includes every tick from this session’s snapshot', + ); assertRanked(result); const fn = hotFunction(result, stage.identifier, stage.file); assert.ok( @@ -275,7 +281,10 @@ suite('F# soak — FsPricingEngine through five workloads in one long CPU sessio await command(target, 'run mixed'); await profileRow(await processRow(target.pid)); await awaitLiveProgress(); - await delay(SAMPLE_MS * 2); + for (const stage of stages) { + await command(target, stage.command); + await delay(SAMPLE_MS); + } await stopProfiling(STOP_MS); const loud = lastProfileResult(); assert.ok(loud, 'a result with the heat map on'); @@ -348,6 +357,7 @@ suite('F# soak — FsOrderLedger with CPU and memory tracking at once', () => { // ── Interaction 4: [PROF-UI-HEATMAP] the memory track paints during the CPU session too. const snap = await uiSnapshot(); + const wire = await snapshotSession(target.pid); assertAnchored(snap, SHAPES.order, 300); assert.equal(handlerSite(snap).count, 300, 'the handler closures are all reachable'); assertDashboard(snap, 1, [SHAPES.order.type]); @@ -355,7 +365,7 @@ suite('F# soak — FsOrderLedger with CPU and memory tracking at once', () => { assertSitesCoherent(snap); assertTimelineCoherent(snap); assertPaletteDiscipline(); - assertEveryShape(snap, { [SHAPES.order.type]: 300 }); + assertEveryShape(snap, batch); assert.ok(memoryStatusText()?.includes('tracking'), 'memory still tracks during the burn'); assert.ok(heatOn('Ledger.fs').length > 0, 'the CPU heat map paints Ledger.fs'); assert.ok(memoryTrackOn('Ledger.fs').length > 0, 'the memory track paints Ledger.fs'); @@ -369,6 +379,13 @@ suite('F# soak — FsOrderLedger with CPU and memory tracking at once', () => { // ── Interaction 5: more work in the same session grows the heap, not the sample count alone. const grew = await command(target, 'place 200'); assert.equal(count(grew, 'orders'), 500, 'the second batch lands in the same session'); + const growth = await compareMemory(wire); + assertConfidence(growth.leaks, SHAPES.order.type, 'Low', ['Ledger.fs']); + assert.equal( + wireLeak(growth.leaks, SHAPES.order.type)?.count_growth, + 200, + '[PROF-MEMORY-COMPARE] exactly the new F# orders grew', + ); const after = await uiSnapshot(snap); assert.ok(after.currentMemory >= snap.currentMemory, 'the heap grew under the CPU session'); assertAnchored(after, SHAPES.order, 500); @@ -376,7 +393,7 @@ suite('F# soak — FsOrderLedger with CPU and memory tracking at once', () => { assertSitesCoherent(after); assertTimelineCoherent(after); assertPaletteDiscipline(); - assertEveryShape(after, { [SHAPES.order.type]: 500 }); + assertEveryShape(after, grew); // ── Interaction 6: stopping CPU leaves memory tracking and clears only the heat map. await stopProfiling(STOP_MS); @@ -387,8 +404,8 @@ suite('F# soak — FsOrderLedger with CPU and memory tracking at once', () => { assert.equal(profilerStatusText(), undefined, 'the CPU chrome is gone'); assert.ok(memoryTrackOn('Ledger.fs').length > 0, 'the memory track stays painted'); - // ── Interaction 7: the diff is taken on a program that is still running. - await assertConfidence(SHAPES.order.type, 'Low', ['Ledger.fs']); + // ── Interaction 7: the process still answers after CPU stop, while memory tracks it. + assert.equal(count(await command(target, 'burn off'), 'orders'), 500); // ── Interaction 8: stop everything; nothing survives. await command(target, 'burn off'); @@ -396,7 +413,7 @@ suite('F# soak — FsOrderLedger with CPU and memory tracking at once', () => { await stopTracking(); assert.equal(memoryStatusText(), undefined, 'the memory chrome hides'); assert.deepEqual(appliedMemoryDecorations(), [], 'the memory track clears'); - assert.deepEqual(appliedProfileDecorations(), [], 'the heat map clears'); + assert.ok(heatOn('Ledger.fs').length > 0, 'the completed CPU report remains painted'); assert.equal( flamegraphPanelState() !== undefined, true, @@ -409,7 +426,8 @@ suite('F# soak — FsOrderLedger with CPU and memory tracking at once', () => { memoryBusy: false, profilerBusy: false, }); - assert.equal(processesViewState().badge, 0, 'no badge once idle'); + assert.equal(processesViewState().badge, undefined, 'no badge once idle'); + completed(await execute('memory.stop', wire)); assert.deepEqual(await sessions(), [], 'no CPU session is left running'); assertStoreCoherent(); assertPaletteDiscipline(); diff --git a/src/editors/vscode/src/test/suite/profiler-fsharp-soak-e2e.test.ts b/src/editors/vscode/src/test/suite/profiler-fsharp-soak-e2e.test.ts index 96c89d73..e78538bf 100644 --- a/src/editors/vscode/src/test/suite/profiler-fsharp-soak-e2e.test.ts +++ b/src/editors/vscode/src/test/suite/profiler-fsharp-soak-e2e.test.ts @@ -19,7 +19,6 @@ // [PROF-DOTNET-SOURCE] F# records, union cases and closures anchor where they are built. // [PROF-DOTNET-HEAP-GRAPH] a closed generic FSharpMap record is one site. import * as assert from 'node:assert/strict'; -import * as vscode from 'vscode'; import { appliedMemoryDecorations, memoryStatusText, @@ -35,7 +34,6 @@ import { snapshotSession, stopTracking, trackMemory, - uiCommand, } from './profiler-kit'; import { assertAnchored, @@ -293,6 +291,10 @@ suite('F# soak — FsOrderLedger: the leak-confidence ladder over one long sessi '[PROF-MEMORY-SESSION] the memory session never created a CPU session', ); }); +}); + +suite('F# soak — FsOrderLedger: mailbox, cache and immutable history retention', () => { + const live = profilingSuite('profiler-fs-soak-retention-', LEDGER, { stopping: endProfiling }); test('a paused mailbox and a memoize cache in one session: the queue is a leak while paused and gone after resume, the cache never evicts, and the undo history is a closed generic record over FSharpMap', async function () { this.timeout(STOP_MS * 8); @@ -411,7 +413,11 @@ suite('F# soak — FsOrderLedger: the leak-confidence ladder over one long sessi const released = await command(target, 'release orders'); assert.equal(count(released, 'orders'), 0, 'the ledger holds no order'); assert.equal(count(released, 'quotes'), 800, 'the memoize cache still holds every SKU'); - assert.equal(count(released, 'handlers'), 400, 'the handlers are never removed'); + assert.equal( + count(released, 'handlers'), + count(more, 'handlers'), + 'the handlers are never removed', + ); assert.equal(count(released, 'audits'), 0, 'the mailbox is still drained'); const sixth = await uiSnapshot(fifth); assertAnchored(sixth, SHAPES.order, 400); diff --git a/src/editors/vscode/src/test/suite/profiler-fsharp-soak-kit.ts b/src/editors/vscode/src/test/suite/profiler-fsharp-soak-kit.ts index be7eceec..f9e60e3d 100644 --- a/src/editors/vscode/src/test/suite/profiler-fsharp-soak-kit.ts +++ b/src/editors/vscode/src/test/suite/profiler-fsharp-soak-kit.ts @@ -16,7 +16,14 @@ import * as assert from 'node:assert/strict'; import * as fs from 'node:fs'; import * as vscode from 'vscode'; -import { embeddedLeaks, profilerStore, uiCommand, waitFor } from './profiler-kit'; +import { + embeddedLeaks, + execute, + lspClient, + profilerStore, + uiCommand, + waitFor, +} from './profiler-kit'; import { captureToasts } from './profiler-ui-helpers'; import { lastMemorySnapshot, @@ -38,11 +45,24 @@ import { STOP_MS, wireDiff, } from './profiler-fsharp-kit'; -import type { SuspectedLeak } from '@nimblesite/lsp-profiler-ui/out/src/wire-generated'; +import type { + MemoryOutcome, + MemoryStep, + ProfileReport, + SuspectedLeak, +} from '@nimblesite/lsp-profiler-ui/out/src/wire-generated'; import type { MemoryDashboardSnapshot } from '@nimblesite/lsp-profiler-ui/out/src/memory-dashboard'; export const SAMPLE_MS = 3_000; export const MIB = 1024 * 1024; +/** [PROF-ERRORS] classified protocol failures. */ +export const CODE = { + OTHER: -32_000, + PROCESS_NOT_FOUND: -32_001, + UNSUPPORTED_RUNTIME: -32_002, + ALREADY_PROFILING: -32_004, + INVALID_PARAMS: -32_602, +} as const; /** [PROF-MEMORY-CONFIDENCE] grades as the wire spells them ([PROF-WIRE-GEN]). */ export const BANDS = ['Low', 'Medium', 'High', 'Definite'] as const; export type Band = (typeof BANDS)[number]; @@ -58,10 +78,17 @@ export async function delay(ms: number): Promise { export async function uiSnapshot( previous?: MemoryDashboardSnapshot, ): Promise { - await vscode.commands.executeCommand(uiCommand('memorySnapshot')); + const before = previous ?? lastMemorySnapshot(); + const session = trackedSession(); + const toasts = await captureToasts(() => + vscode.commands.executeCommand(uiCommand('memorySnapshot')), + ); + assert.deepEqual(toasts.error, [], 'the F# snapshot succeeds'); + assert.deepEqual(toasts.warning, [], 'the F# snapshot is not refused'); const snapshot = await waitFor( () => lastMemorySnapshot(), - (snap) => snap !== undefined && snap.snapshotId !== previous?.snapshotId, + (snap) => + snap?.memorySessionId === session.memory_session_id && snap.snapshotId !== before?.snapshotId, STOP_MS, 'a newer snapshot in the dashboard', ); @@ -147,7 +174,10 @@ export async function compareMemory(wire: { memory_session_id: string }) { memoryDashboardState()?.html.includes('const leaks = '), 'the dashboard renders the diff', ); - return wireDiff(wire); + const compared = await wireDiff(wire); + for (const site of compared.diff.grown) assert.ok(site.size_diff > 0, 'grown sites gained bytes'); + for (const site of compared.diff.freed) assert.ok(site.size_diff < 0, 'freed sites lost bytes'); + return compared; } /** [PROF-MEMORY-CONFIDENCE] + [PROF-UI-MEMORY]: inspect one comparison without advancing it. */ @@ -170,6 +200,8 @@ export function assertConfidence( assert.equal(displayed.confidence, band.toLowerCase(), 'the dashboard shows the exact grade'); assert.equal(displayed.countGrowth, leak.count_growth, 'the same F# count growth reaches the UI'); assert.equal(displayed.currentCount, leak.current_count, 'the same live F# count reaches the UI'); + assert.equal(displayed.sizeGrowth, leak.size_growth, 'the same F# byte growth reaches the UI'); + assert.equal(displayed.currentSize, leak.current_size, 'the same F# live bytes reach the UI'); for (const file of lines) { assert.ok(isFile(leak.file, file), `${type} has its F# source file`); assert.ok(leak.line !== undefined, `${type} has a source line`); @@ -197,6 +229,10 @@ export function assertSitesCoherent(snapshot: MemoryDashboardSnapshot): void { assert.ok((site.typeName ?? '').length > 0, 'a row names its runtime type'); assert.ok(site.size > 0, `${key} has bytes`); assert.ok(site.count > 0, `${key} has objects`); + if (site.typeName?.endsWith('[]') || site.typeName?.endsWith('[,]')) { + assert.equal(site.file, undefined, '[PROF-DOTNET-SOURCE] an array has no source file'); + assert.equal(site.line, undefined, '[PROF-DOTNET-SOURCE] an array has no source line'); + } assert.ok(site.line === undefined || site.line >= 1, `[PROF-WIRE] 1-based line: ${site.line}`); if (site.file !== undefined) { assert.ok(fs.existsSync(site.file), `[PROF-DOTNET-SOURCE] the anchor exists: ${site.file}`); @@ -293,3 +329,121 @@ export function assertStoreCoherent(): void { if (values.profilerBusy) assert.ok((badge ?? 0) >= 1, 'a running session badges the panel'); else assert.equal(badge ?? 0, 0, 'no badge when nothing runs'); } + +/** The outcome of one memory step; a Courier Evaluate step is a failure for a .NET target. */ +export async function step(name: string, params: Record): Promise { + const result = await execute(name, params); + assert.equal(result.kind, 'Completed', `[PROF-MEMORY-STEP] ${name} completes in Direct mode`); + assert.ok(result.kind === 'Completed', 'narrowed'); + return result.result.outcome; +} + +export async function snapshotOutcome( + id: string, +): Promise> { + const outcome = await step('memory.snapshot', { memory_session_id: id }); + assert.equal(outcome.kind, 'Snapshot', '[PROF-MEMORY-STEP] snapshot yields a Snapshot outcome'); + assert.ok(outcome.kind === 'Snapshot', 'narrowed'); + return outcome; +} + +export async function diffOutcome(id: string): Promise> { + const outcome = await step('memory.diff', { memory_session_id: id }); + assert.equal(outcome.kind, 'Diff', '[PROF-MEMORY-STEP] diff yields a Diff outcome'); + assert.ok(outcome.kind === 'Diff', 'narrowed'); + return outcome; +} + +/** `workspace/executeCommand` exactly as the spec shapes it, with caller-chosen arguments. */ +export function raw(name: string, args: unknown[]): Promise { + return lspClient().sendRequest('workspace/executeCommand', { + command: name, + arguments: args, + }); +} + +/** The error a raw call rejects with. */ +export async function rawError( + name: string, + args: unknown[], +): Promise<{ code: number; message: string }> { + try { + await raw(name, args); + } catch (error: unknown) { + const failure = error as { code?: unknown; message?: unknown }; + return { + code: typeof failure.code === 'number' ? failure.code : 0, + message: String(failure.message ?? error), + }; + } + assert.fail(`${name} with ${JSON.stringify(args)} should have failed`); +} + +/** [PROF-CPU-REPORT] + [PROF-AGGREGATION-LOGIC] + [PROF-SAMPLE] + [PROF-EXPORT]. */ +export function assertProfileShape(report: ProfileReport): void { + assert.ok(Array.isArray(report.hot_functions), 'hot functions are a list'); + assert.ok(Array.isArray(report.hot_lines), 'hot lines are a list'); + const percentages = report.hot_functions.map((fn) => fn.percentage); + assert.deepEqual( + percentages, + [...percentages].sort((a, b) => b - a), + 'functions are ranked hottest first', + ); + for (const fn of report.hot_functions) { + assert.ok(fn.name.length > 0, 'a function has a name'); + assert.ok( + fn.percentage >= 0 && fn.percentage <= 100, + `inclusive share 0–100: ${fn.percentage}`, + ); + assert.ok( + fn.self_percentage <= fn.percentage, + `self never exceeds inclusive: ${fn.self_percentage} > ${fn.percentage}`, + ); + assert.ok(fn.samples >= fn.self_samples, 'inclusive samples cover the self samples'); + assert.ok(fn.samples <= report.user_samples, 'a function cannot exceed retained ticks'); + assert.equal( + fn.percentage, + (fn.samples / report.user_samples) * 100, + '[PROF-AGGREGATION-LOGIC] inclusive share uses retained ticks as its denominator', + ); + assert.equal( + fn.self_percentage, + (fn.self_samples / report.user_samples) * 100, + '[PROF-AGGREGATION-LOGIC] self share uses the same retained-tick denominator', + ); + if (fn.line !== undefined) { + assert.ok(fn.line >= 1, `[PROF-SAMPLE] 1-based definition line: ${fn.line}`); + assert.ok((fn.file ?? '').length > 0, 'a located function names its file'); + } + } + const linePercentages = report.hot_lines.map((line) => line.percentage); + assert.deepEqual( + linePercentages, + [...linePercentages].sort((a, b) => b - a), + 'lines are ranked hottest first', + ); + for (const line of report.hot_lines) { + assert.ok(line.line >= 1, `[PROF-SAMPLE] 1-based hot line: ${line.line}`); + assert.ok(line.file.length > 0, 'a hot line names its file'); + assert.ok( + line.percentage > 0 && line.percentage <= 100, + `a hot line share: ${line.percentage}`, + ); + assert.ok(line.samples <= report.user_samples, 'a hot line cannot exceed retained ticks'); + assert.equal( + line.percentage, + (line.samples / report.user_samples) * 100, + '[PROF-AGGREGATION-LOGIC] a hot line’s share agrees with its tick count', + ); + } + const artifacts = report.artifacts; + for (const artifact of [ + artifacts.speedscope_path, + artifacts.flamegraph_path, + artifacts.cpuprofile_path, + ]) { + assert.ok(artifact, '[PROF-EXPORT] every required CPU artifact is returned'); + assert.ok(fs.existsSync(artifact), `[PROF-EXPORT] the artifact exists: ${artifact}`); + } + assert.ok(Array.isArray(report.diagnostics), 'diagnostics travel with the result'); +} diff --git a/src/editors/vscode/src/test/suite/profiler-kit.ts b/src/editors/vscode/src/test/suite/profiler-kit.ts index 8f11d4b0..026ad5c0 100644 --- a/src/editors/vscode/src/test/suite/profiler-kit.ts +++ b/src/editors/vscode/src/test/suite/profiler-kit.ts @@ -2,10 +2,7 @@ // suites: the library's store through the extension API, the .NET fixture // targets, and the wire commands the extension drives. import * as assert from 'node:assert/strict'; -import { spawn, spawnSync, type ChildProcess } from 'node:child_process'; -import * as fs from 'node:fs'; -import * as os from 'node:os'; -import * as path from 'node:path'; +import { spawn, type ChildProcess } from 'node:child_process'; import * as vscode from 'vscode'; import type { LanguageClient } from 'vscode-languageclient/node'; import { @@ -30,6 +27,9 @@ import { teardownLspTestSuite, } from './test-helpers'; import { ACTIVATION_MS, PROCESS_START_MS } from './test-timeouts'; +import { buildFixture } from './profiler-fixtures'; + +export { buildFixture, fixtureFile, fixtureSource } from './profiler-fixtures'; /** The profiler's commands as the server advertises them. */ export function lspCommand(name: string): string { @@ -120,85 +120,6 @@ export async function executeError( } } -const FIXTURES = path.resolve(__dirname, '../../../../../sharplsp/tests/fixtures'); -/** lspkit's own targets, which prove the engine on every .NET language. */ -const LSPKIT_FIXTURES = path.resolve(__dirname, '../../../../../../lspkit/dotnet/fixtures'); -/** SharpLsp's real-world F# programs: a pricing engine and an event-sourced order ledger. */ -const FSHARP_FIXTURES = path.join(FIXTURES, 'fsharp-profiling'); - -/** A profiling target: its project folder, target framework and source file. */ -interface FixtureProject { - readonly dir: string; - readonly framework: string; - readonly source: string; -} - -const FIXTURE_PROJECTS: Readonly> = { - ProfileTarget: { dir: FIXTURES, framework: 'net10.0', source: 'Program.cs' }, - ProfileLeakTarget: { dir: FIXTURES, framework: 'net10.0', source: 'Program.cs' }, - // F#: records holding union cases leak into a module value ([PROF-TESTING]). - HotLeakAppFs: { dir: LSPKIT_FIXTURES, framework: 'net8.0', source: 'Program.fs' }, - // F# CPU: CSV parsing, recursive discount rules, fuzzy search, route planning, reports. - FsPricingEngine: { dir: FSHARP_FIXTURES, framework: 'net10.0', source: 'Program.fs' }, - // F# memory: event, memoize, mailbox and undo-history leaks in an order ledger. - FsOrderLedger: { dir: FSHARP_FIXTURES, framework: 'net10.0', source: 'Program.fs' }, -}; - -function fixtureProject(name: string): FixtureProject { - const project = FIXTURE_PROJECTS[name]; - assert.ok(project, `unknown profiling fixture ${name}`); - return project; -} - -/** `dotnet` from PATH, else the user-local install the test host's PATH may lack. */ -function dotnetExecutable(): string { - const candidates = [ - 'dotnet', - path.join(os.homedir(), '.dotnet', process.platform === 'win32' ? 'dotnet.exe' : 'dotnet'), - ]; - for (const candidate of candidates) { - if (spawnSync(candidate, ['--version'], { encoding: 'utf8' }).status === 0) return candidate; - } - throw new Error('dotnet SDK not found: install it or put it on PATH'); -} - -/** Build a fixture project once (Release) and return its binary. */ -export function buildFixture(name: string): string { - const { dir, framework } = fixtureProject(name); - const project = path.join(dir, name); - const binary = path.join( - project, - 'bin', - 'Release', - framework, - process.platform === 'win32' ? `${name}.exe` : name, - ); - if (!fs.existsSync(binary)) { - const build = spawnSync(dotnetExecutable(), ['build', '-c', 'Release', '--nologo', '-v', 'q'], { - cwd: project, - encoding: 'utf8', - }); - assert.equal( - build.status, - 0, - `building ${name} failed: ${build.stderr ?? build.error?.message}`, - ); - } - assert.ok(fs.existsSync(binary), `${binary} missing after build`); - return binary; -} - -/** The source file of a fixture, where hot lines and allocation sites land. */ -export function fixtureSource(name: string): string { - const { dir, source } = fixtureProject(name); - return path.join(dir, name, source); -} - -/** Another source file of a multi-file fixture project. */ -export function fixtureFile(name: string, file: string): string { - return path.join(fixtureProject(name).dir, name, file); -} - /** How long a fixture may take to answer one stdin command. */ const REPLY_MS = 60_000; diff --git a/src/editors/vscode/src/test/suite/profiler-lsp-cpu-e2e.test.ts b/src/editors/vscode/src/test/suite/profiler-lsp-cpu-e2e.test.ts new file mode 100644 index 00000000..a0744264 --- /dev/null +++ b/src/editors/vscode/src/test/suite/profiler-lsp-cpu-e2e.test.ts @@ -0,0 +1,149 @@ +// Tests for [PROFILER-SHARED-LIBRARY-VSCODE], [PROF-CPU-SESSIONS], [PROF-CPU-SNAPSHOT] +// and [PROF-CPU-STOP] over the real F# LSP: one CPU session's whole life on the wire. +import * as assert from 'node:assert/strict'; +import type { + ProfileReport, + ProfileSessionList, +} from '@nimblesite/lsp-profiler-ui/out/src/wire-generated'; +import { execute, executeError, profilingSuite } from './profiler-kit'; +import { assertProfileShape } from './profiler-fsharp-soak-kit'; +import { + command, + count, + endProfiling, + isFile, + LEDGER, + PRICE_ALL, + STOP_MS, + within, +} from './profiler-fsharp-kit'; + +suite('LSP-level F# CPU protocol — FsOrderLedger', () => { + const live = profilingSuite('profiler-lsp-cpu-', LEDGER, { stopping: endProfiling }); + + test('CPU over the wire on F#: identity on start, samples and versions in list, a snapshot that keeps the session, ranked 1-based positions, a report with artifacts, stop, and no session afterwards', async function () { + this.timeout(STOP_MS * 4); + const { target } = live; + const seeded = await command(target, 'place 1000'); + await command(target, 'burn on'); + + // ── [PROF-CPU-START] returns the session's identity. + const started = await execute<{ + session_id: string; + pid: number; + runtime: string; + runtime_version: string; + started_at: string; + }>('cpu.start', { pid: target.pid, include_native: false }); + assert.ok(started.session_id.length > 0, 'a session id'); + assert.equal(started.pid, target.pid, 'the session names its PID'); + assert.equal(started.runtime, 'dotnet', '[PROF-RUNTIME] runtime identity travels as data'); + assert.ok(started.runtime_version.length > 0, '[PROF-CPU-START] a runtime version is reported'); + assert.match(started.started_at, /^\d{4}-\d{2}-\d{2}T/, 'an RFC 3339 start time'); + + // ── [PROF-CPU-SESSIONS] the live session carries samples and a duration. + await new Promise((resolve) => setTimeout(resolve, 2_500)); + const listed = (await execute('cpu.list', {})).sessions; + const now = listed.find((entry) => entry.session_id === started.session_id); + assert.ok(now, 'the session is listed while it runs'); + assert.equal(now.pid, target.pid); + assert.equal(now.runtime, 'dotnet'); + assert.ok(now.runtime_version.length > 0, 'the version survives the listing'); + assert.ok(now.sample_count > 0, `samples accumulate: ${now.sample_count}`); + assert.ok(now.duration_secs > 0, `the duration runs: ${now.duration_secs}`); + assert.ok(now.started_at.length > 0, 'the start time is listed'); + + // ── [PROF-CPU-SNAPSHOT] a snapshot is a report and the session survives it. + const snapshot = await execute('cpu.snapshot', { + session_id: started.session_id, + }); + assert.equal(snapshot.session_id, started.session_id, 'the report names its session'); + assert.equal(snapshot.pid, target.pid); + assert.equal(snapshot.runtime, 'dotnet'); + assert.ok(snapshot.total_samples > 0, 'the snapshot samples something'); + assert.ok( + snapshot.user_samples >= 0 && snapshot.user_samples <= snapshot.total_samples, + 'user samples are a subset', + ); + assert.ok(snapshot.sample_rate >= 1, `a positive sample rate: ${snapshot.sample_rate}`); + assert.ok(snapshot.duration_secs >= 0, 'a duration'); + assert.ok( + (await execute('cpu.list', {})).sessions.some( + (entry) => entry.session_id === started.session_id, + ), + '[PROF-CPU-SNAPSHOT] the snapshot did not end the session', + ); + assertProfileShape(snapshot); + + // ── [PROF-CPU-STOP] stop drains and returns the final report. + const stopped = await execute('cpu.stop', { session_id: started.session_id }); + assert.equal(stopped.session_id, started.session_id, 'the stopped report is the session'); + assert.ok(stopped.total_samples >= snapshot.total_samples, 'samples never shrink on stop'); + assert.deepEqual(stopped.artifacts.export_error, undefined, '[PROF-EXPORT] no export error'); + assertProfileShape(stopped); + assert.ok( + stopped.hot_functions.some( + (fn) => fn.name.includes('priceAll') && isFile(fn.file, 'Ledger.fs'), + ), + '[PROF-TESTING] the burning F# function is named in its own source file', + ); + assert.ok( + stopped.hot_lines.some((line) => within(PRICE_ALL, line.file, line.line)), + '[PROF-DOTNET-CPU-SYMBOLS] the report samples inside the F# burn loop', + ); + assert.equal( + (await execute('cpu.list', {})).sessions.some( + (entry) => entry.session_id === started.session_id, + ), + false, + '[PROF-CPU-SESSIONS] a stopped session is no longer active', + ); + + // ── A stopped session cannot be stopped or snapshotted again. + const restop = await executeError('cpu.stop', { session_id: started.session_id }); + assert.ok(restop, 'a second stop fails'); + assert.ok(restop.code !== 0, `[PROF-ERRORS] ${JSON.stringify(restop)}`); + const resnap = await executeError('cpu.snapshot', { session_id: started.session_id }); + assert.ok(resnap, 'a snapshot after stop fails'); + assert.ok(resnap.code !== 0, 'the ended session rejects snapshots'); + + // ── The program was never disturbed: attaching is not invasive. + const alive = await command(target, 'place 1'); + assert.equal( + count(alive, 'orders'), + count(seeded, 'orders') + 1, + 'the F# program still accepts and retains work after profiling', + ); + await command(target, 'burn off'); + }); + + test('a duration on start stops the session by itself and keeps its report for the next stop', async function () { + this.timeout(STOP_MS * 4); + const { target } = live; + await command(target, 'burn on'); + // ── [PROF-CPU-SESSIONS] `duration` ends the session automatically. + const started = await execute<{ session_id: string }>('cpu.start', { + pid: target.pid, + duration_secs: 2, + }); + assert.ok(started.session_id.length > 0, 'a session id with a duration'); + await new Promise((resolve) => setTimeout(resolve, 5_000)); + const active = (await execute('cpu.list', {})).sessions; + assert.equal( + active.some((entry) => entry.session_id === started.session_id), + false, + 'the duration elapsed and the session ended by itself', + ); + // ── Its report is kept for the next stop. + const kept = await execute('cpu.stop', { session_id: started.session_id }); + assert.equal(kept.session_id, started.session_id, 'the kept report belongs to the session'); + assert.ok(kept.total_samples > 0, 'the timed session sampled the running F# workload'); + assert.ok(kept.duration_secs >= 2, 'the timed report covers its configured duration'); + assertProfileShape(kept); + assert.ok( + kept.hot_lines.some((line) => within(PRICE_ALL, line.file, line.line)), + 'the timed report contains the F# workload, not just runtime frames', + ); + await command(target, 'burn off'); + }); +}); diff --git a/src/editors/vscode/src/test/suite/profiler-lsp-memory-e2e.test.ts b/src/editors/vscode/src/test/suite/profiler-lsp-memory-e2e.test.ts new file mode 100644 index 00000000..a8bd9ee8 --- /dev/null +++ b/src/editors/vscode/src/test/suite/profiler-lsp-memory-e2e.test.ts @@ -0,0 +1,139 @@ +// Tests for [PROFILER-SHARED-LIBRARY-VSCODE], [PROF-MEMORY-COMPARE], +// [PROF-DOTNET-SOURCE] and [PROF-DOTNET-HEAP-GRAPH] over the real F# LSP. +import * as assert from 'node:assert/strict'; +import type { MemoryStep } from '@nimblesite/lsp-profiler-ui/out/src/wire-generated'; +import { completed, execute, executeError, profilingSuite } from './profiler-kit'; +import { command, count, endProfiling, isFile, LEDGER, STOP_MS } from './profiler-fsharp-kit'; +import { CODE } from './profiler-fsharp-soak-kit'; +suite('LSP-level F# memory protocol — FsOrderLedger', () => { + const live = profilingSuite('profiler-lsp-memory-', LEDGER, { stopping: endProfiling }); + + test('memory over the wire on F#: the baseline anchors every shape, objects respect the limit, refs reach the module root, a growth is Low, GC is observed, and a stopped session refuses every step', async function () { + this.timeout(STOP_MS * 5); + const { target } = live; + await command(target, 'release all'); + const seeded = await command(target, 'place 250'); + assert.equal(count(seeded, 'orders'), 250, 'a known live heap'); + + // ── [PROF-MEMORY-START] + [PROF-MEMORY-STEP] the baseline completes in Direct mode. + const started = await execute<{ memory_session_id: string; mode: { kind: string } }>( + 'memory.start', + { pid: target.pid, traceback_depth: 4 }, + ); + assert.ok(started.memory_session_id.length > 0, 'a memory session id'); + assert.equal(started.mode.kind, 'Direct', '[PROF-MEMORY-MODE] .NET reads its own heap'); + const session = { memory_session_id: started.memory_session_id }; + const baseline = completed(await execute('memory.snapshot', session)); + assert.equal(baseline.outcome.kind, 'Snapshot', '[PROF-MEMORY-OUTCOME] snapshot'); + assert.ok(baseline.outcome.kind === 'Snapshot', 'narrowed'); + const snapshot = baseline.outcome.snapshot; + assert.ok(snapshot.current_bytes > 0, 'live bytes are measured'); + assert.ok(snapshot.object_count > 0, 'live objects are counted'); + assert.equal(snapshot.gc_generations.length, 0, '[PROF-UI-MEMORY] counts are not bytes'); + for (const [type, line, live] of [ + ['FsOrderLedger.Order', 43, 250], + ['FsOrderLedger.OrderLine', 41, 250], + ['FsOrderLedger.CustomerId', 33, 250], + ['FsOrderLedger.OrderStatus+Paid', 37, count(seeded, 'paid')], + ['FsOrderLedger.OrderStatus+Shipped', 38, count(seeded, 'shipped')], + ['FsOrderLedger.OrderStatus+Cancelled', 39, count(seeded, 'cancelled')], + ] as const) { + const site = snapshot.sites.find((entry) => entry.type_name === type); + assert.ok(site, `${type} is a site: ${snapshot.sites.map((s) => s.type_name).join(', ')}`); + assert.ok(isFile(site.file, 'Ledger.fs'), `${type} anchors in Ledger.fs: ${site.file}`); + assert.equal(site.line, line, `${type} anchors where it is built`); + assert.equal(site.count, live, `${type}: every live instance`); + assert.ok(site.size_bytes > 0, `${type} has bytes`); + assert.ok((site.traceback ?? []).length <= 4, 'traceback_depth caps the stack'); + } + + // ── [PROF-DOTNET-MEMORY] objectsByType honours its limit and ranks the heap. + const objects = completed( + await execute('memory.objectsByType', { + ...session, + type_name: 'FsOrderLedger.Order', + limit: 3, + }), + ); + assert.equal(objects.outcome.kind, 'Objects', 'objectsByType'); + assert.ok(objects.outcome.kind === 'Objects', 'narrowed'); + assert.equal(objects.outcome.objects.total_count, 250, 'the snapshot count is echoed'); + assert.equal(objects.outcome.objects.objects.length, 3, 'the limit is honoured'); + assert.ok( + objects.outcome.objects.objects.every( + (entry) => entry.type_name === 'FsOrderLedger.Order' && entry.size_bytes > 0, + ), + 'every sample is the requested type', + ); + const summary = objects.outcome.objects.type_summary.map((stat) => stat.size_bytes); + assert.deepEqual( + summary, + [...summary].sort((a, b) => b - a), + 'largest types first', + ); + + // ── [PROF-DOTNET-HEAP-GRAPH] the retention path ends at the module value `orders`. + const refs = completed( + await execute('memory.references', { + ...session, + type_name: 'FsOrderLedger.Order', + limit: 30, + max_depth: 6, + }), + ); + assert.equal(refs.outcome.kind, 'Refs', 'references'); + assert.ok(refs.outcome.kind === 'Refs', 'narrowed'); + assert.ok( + refs.outcome.graph.nodes.some( + (node) => node.is_target && node.type_name === 'FsOrderLedger.Order', + ), + 'the requested type is the target', + ); + assert.ok( + refs.outcome.graph.nodes.some( + (node) => node.is_root && node.root_kind === 'Static' && node.label === 'static orders', + ), + `the module value roots the leak: ${JSON.stringify(refs.outcome.graph.nodes.filter((n) => n.is_root))}`, + ); + assert.ok(refs.outcome.graph.edges.length > 0, 'the graph holds edges'); + assert.ok(refs.outcome.graph.nodes.length <= 30, 'the node budget is honoured'); + + // ── [PROF-MEMORY-CONFIDENCE] one growth is Low, and [PROF-DOTNET-GC] a collection is observed. + await command(target, 'place 100'); + const diff = completed(await execute('memory.diff', session)); + assert.equal(diff.outcome.kind, 'Diff', 'a diff answers with a diff'); + assert.ok(diff.outcome.kind === 'Diff', 'narrowed'); + assert.equal( + diff.outcome.diff.net_growth, + diff.outcome.diff.total_growth - diff.outcome.diff.total_freed, + 'net = grown - freed', + ); + const leak = diff.outcome.leaks.find((entry) => entry.type_name === 'FsOrderLedger.Order'); + assert.ok(leak, `the order growth is a suspected leak: ${JSON.stringify(diff.outcome.leaks)}`); + assert.equal(leak.confidence.kind, 'Low', '[PROF-MEMORY-CONFIDENCE] one growth is Low'); + assert.equal(leak.count_growth, 100, '[PROF-MEMORY-COMPARE] exactly the new F# Orders grew'); + assert.equal(leak.current_count, 350, 'every retained Order is counted after the growth'); + assert.ok(leak.size_growth > 0, 'the new F# records grew in bytes'); + const gc = completed(await execute('memory.gcCollect', session)); + assert.equal(gc.outcome.kind, 'Gc', 'gcCollect'); + assert.ok(gc.outcome.kind === 'Gc', 'narrowed'); + assert.equal(gc.outcome.gc.collected, 1, '[PROF-DOTNET-GC] a collection is observed'); + + // ── [PROF-ERRORS] stop, then every step on the stopped session is OTHER. + const stopped = completed(await execute('memory.stop', session)); + assert.equal(stopped.outcome.kind, 'Ack', 'stop acknowledges'); + for (const name of [ + 'memory.snapshot', + 'memory.diff', + 'memory.references', + 'memory.objectsByType', + 'memory.gcCollect', + 'memory.stop', + ]) { + const refused = await executeError(name, { ...session, type_name: 'FsOrderLedger.Order' }); + assert.ok(refused, `${name} on a stopped session fails`); + assert.equal(refused.code, CODE.OTHER, `[PROF-ERRORS] ${name} is OTHER: ${refused.message}`); + } + await command(target, 'release all'); + }); +}); diff --git a/src/editors/vscode/src/test/suite/profiler-lsp-protocol-e2e.test.ts b/src/editors/vscode/src/test/suite/profiler-lsp-protocol-e2e.test.ts index 289b7d34..659f93ad 100644 --- a/src/editors/vscode/src/test/suite/profiler-lsp-protocol-e2e.test.ts +++ b/src/editors/vscode/src/test/suite/profiler-lsp-protocol-e2e.test.ts @@ -1,50 +1,18 @@ -// Tests for [PROFILER-SHARED-LIBRARY-VSCODE] at the LSP level. See -// docs/specs/PROFILER-SPEC.md#PROFILER-SHARED-LIBRARY-VSCODE and -// lspkit/docs/specs/LSP-PROFILER-SPEC.md. -// -// These go one layer below the VS Code commands: the JSON-RPC surface an editor without the UI -// library uses. Every assertion names the spec section that forces it: -// -// [PROF-WIRE-METHODS] every operation is `workspace/executeCommand` with exactly one argument, -// the typed params object; commands are advertised in -// `executeCommandProvider.commands`. -// [PROF-HOST-ROUTING] `initialize` adds the profiler's commands and `experimental.lspProfiler`; -// profiler requests never block the host's message loop. -// [PROF-UI-IDS] UI ids are `.profiler.*` and never collide with server commands. -// [PROF-ERRORS] OTHER, PROCESS_NOT_FOUND, UNSUPPORTED_RUNTIME, ALREADY_PROFILING, -// INVALID_PARAMS — one code per failure kind. -// [PROF-PROCESSES] one ProcessInfo per recognised process; attachable ones sort first. -// [PROF-PROCESSES-KILL] `kill` only reaches a process a registered runtime recognises. -// [PROF-CPU-START] returns `{ session_id, pid, runtime, runtime_version, started_at }`; -// a missing PID is invalid, a refused start leaves nothing behind. -// [PROF-CPU-SESSIONS] `cpu.list` carries id, pid, runtime, version, start time, samples, duration; -// `duration` on start stops the session and keeps its report. -// [PROF-CPU-SNAPSHOT] a snapshot returns the report shape without ending the session. -// [PROF-CPU-STOP] stop drains, aggregates and returns the report. -// [PROF-CPU-REPORT] hot functions with inclusive/self share, hot lines, artifacts. -// [PROF-AGGREGATION-LOGIC] shares are 0–100, self ≤ inclusive, ranked hottest first. -// [PROF-SAMPLE] positions are LSP `Location`s: a file and a 1-based line. +// Tests for [PROFILER-SHARED-LIBRARY-VSCODE] on real F# processes. +// SharpLsp contract: docs/specs/PROFILER-SPEC.md. +// Shared contract: lspkit/docs/specs/LSP-PROFILER-SPEC.md. +// Each journey cites the specific protocol, CPU, memory and UI requirements it enforces. import * as assert from 'node:assert/strict'; import { spawn, type ChildProcess } from 'node:child_process'; -import * as fs from 'node:fs'; import * as vscode from 'vscode'; import type { InitializeResult } from 'vscode-languageclient/node'; import type { - ProfileReport, ProfileSessionList, ProcessList, } from '@nimblesite/lsp-profiler-ui/out/src/wire-generated'; import { execute, executeError, lspClient, profilingSuite } from './profiler-kit'; -import { command, count, endProfiling, LEDGER, STOP_MS } from './profiler-fsharp-kit'; - -/** [PROF-ERRORS] codes, as `docs/models/profiler-methods.json` spells them. */ -const CODE = { - OTHER: -32_000, - PROCESS_NOT_FOUND: -32_001, - UNSUPPORTED_RUNTIME: -32_002, - ALREADY_PROFILING: -32_004, - INVALID_PARAMS: -32_602, -} as const; +import { CODE, raw, rawError } from './profiler-fsharp-soak-kit'; +import { endProfiling, LEDGER, STOP_MS } from './profiler-fsharp-kit'; /** Every operation the wire registry names, under SharpLsp's prefix. */ const COMMANDS = [ @@ -57,7 +25,6 @@ const COMMANDS = [ 'cpu.list', 'cpu.cooperativeScript', 'cpu.cooperativeAttach', - 'cpu.progress', 'memory.start', 'memory.snapshot', 'memory.diff', @@ -68,57 +35,6 @@ const COMMANDS = [ 'memory.ingest', ].map((name) => `sharplsp.lspProfiler.${name}`); -/** `workspace/executeCommand` exactly as the spec shapes it, with caller-chosen arguments. */ -function raw(name: string, args: unknown[]): Promise { - return lspClient().sendRequest('workspace/executeCommand', { - command: name, - arguments: args, - }); -} - -/** The error a raw call rejects with. */ -async function rawError(name: string, args: unknown[]): Promise<{ code: number; message: string }> { - try { - await raw(name, args); - } catch (error: unknown) { - const failure = error as { code?: unknown; message?: unknown }; - return { - code: typeof failure.code === 'number' ? failure.code : 0, - message: String(failure.message ?? error), - }; - } - assert.fail(`${name} with ${JSON.stringify(args)} should have failed`); -} - -/** A refusal the registry defines, or a success: never anything else. */ -const REGISTRY_CODES: ReadonlySet = new Set([ - CODE.OTHER, - CODE.PROCESS_NOT_FOUND, - CODE.UNSUPPORTED_RUNTIME, - -32_003, - CODE.ALREADY_PROFILING, - -32_005, - -32_006, - CODE.INVALID_PARAMS, -]); - -/** The outcome of one call: it succeeded, or it refused with a registry code. */ -async function outcomeOf(name: string, args: unknown[]): Promise { - try { - await raw(name, args); - return 'ok'; - } catch (error: unknown) { - const failure = error as { code?: unknown; message?: unknown }; - const code = typeof failure.code === 'number' ? failure.code : 0; - assert.ok( - REGISTRY_CODES.has(code), - `[PROF-ERRORS] ${name} refuses with a registry code, not ${code}: ${String(failure.message)}`, - ); - assert.ok(String(failure.message ?? '').length > 0, `${name} explains its refusal`); - return 'refused'; - } -} - /** A running process no profiler runtime recognises: a plain `sleep`. */ function sleeper(): ChildProcess { const child = spawn('sleep', ['240'], { stdio: 'ignore' }); @@ -146,7 +62,12 @@ suite('LSP-level profiler protocol — FsOrderLedger behind the same server', () const capabilities = initialized.capabilities; const advertised = capabilities.executeCommandProvider?.commands ?? []; - // ── [PROF-WIRE-METHODS] every registry operation is advertised, exactly once. + // ── [PROF-WIRE-METHODS] every request is advertised; progress is a server notification. + assert.equal( + advertised.includes('sharplsp.lspProfiler.cpu.progress'), + false, + '[PROF-CPU-PROGRESS] progress is not an executable client command', + ); for (const commandId of COMMANDS) { const hits = advertised.filter((entry) => entry === commandId).length; assert.equal(hits, 1, `${commandId} is advertised exactly once (${hits})`); @@ -179,11 +100,11 @@ suite('LSP-level profiler protocol — FsOrderLedger behind the same server', () assert.ok(Object.keys(profiler).length > 0, 'the profiler capabilities are not empty'); } - // ── Every advertised id is a well-formed `.lspProfiler..`. + // ── [PROF-WIRE-METHODS] top-level capabilities/processes/kill and area operations share a prefix. for (const commandId of COMMANDS) { assert.match( commandId, - /^sharplsp\.lspProfiler\.[A-Za-z]+\.[A-Za-z]+$/, + /^sharplsp\.lspProfiler\.[A-Za-z]+(?:\.[A-Za-z]+)?$/, `the id composes from prefix, area and action: ${commandId}`, ); assert.ok(!/\s/.test(commandId), `no whitespace in ${commandId}`); @@ -205,7 +126,7 @@ suite('LSP-level profiler protocol — FsOrderLedger behind the same server', () // ── [PROF-WIRE-METHODS] exactly one argument: the typed params object. const malformed: readonly (readonly [string, unknown[]])[] = [ ['zero arguments', []], - ['two arguments', [{}]], + ['two arguments', [{}, {}]], ['a number', [42]], ['a string', ['pid=1']], ['null', [null]], @@ -255,15 +176,24 @@ suite('LSP-level profiler protocol — FsOrderLedger behind the same server', () ); }); - test('all eighteen operations answer a malformed call without wedging the loop, and none of them leaves a session behind', async function () { + test('all seventeen requests enforce their typed arguments with INVALID_PARAMS and leave the server responsive', async function () { this.timeout(STOP_MS * 4); for (const commandId of COMMANDS) { - // ── [PROF-ERRORS] refuse-or-succeed, never an unclassified failure. - const outcome = await outcomeOf(commandId, [{}]); - assert.ok(outcome === 'ok' || outcome === 'refused', `${commandId} answered: ${outcome}`); - if (outcome === 'refused') { - const again = await outcomeOf(commandId, [{}]); - assert.equal(again, 'refused', `${commandId} refuses the same malformed call twice`); + // [PROF-WIRE-METHODS] only these two requests have no required object fields. + const empty = ['capabilities', 'cpu.list'].some( + (name) => commandId === `sharplsp.lspProfiler.${name}`, + ); + if (empty) { + const result = await raw(commandId, [{}]); + assert.ok( + result !== null && typeof result === 'object', + `${commandId} accepts its empty params object`, + ); + } else { + for (const args of [[{}], [{}]]) { + const failure = await rawError(commandId, args); + assert.equal(failure.code, CODE.INVALID_PARAMS, `${commandId} requires its typed fields`); + } } // ── [PROF-HOST-ROUTING] the loop still serves the next request. const sessions = await execute('cpu.list', {}); @@ -271,9 +201,16 @@ suite('LSP-level profiler protocol — FsOrderLedger behind the same server', () // ── [PROF-CPU-START] a refused call leaves nothing running. assert.deepEqual(sessions.sessions, [], `${commandId} created no CPU session`); } - // ── The same sweep with no argument at all: still classified, still no session. + // [PROF-WIRE-METHODS] exactly one typed object, for EVERY operation. for (const commandId of COMMANDS) { - await outcomeOf(commandId, []); + for (const args of [[], [{}, {}], [42], ['not-an-object']]) { + const failure = await rawError(commandId, args); + assert.equal( + failure.code, + CODE.INVALID_PARAMS, + `${commandId} rejects ${JSON.stringify(args)}`, + ); + } assert.deepEqual( (await execute('cpu.list', {})).sessions, [], @@ -380,13 +317,22 @@ suite('LSP-level profiler protocol — FsOrderLedger behind the same server', () // the same processes come back with a root as without one. const roots = (vscode.workspace.workspaceFolders ?? []).map((folder) => folder.uri.fsPath); const scoped = await execute('processes', { workspace_roots: roots }); - assert.deepEqual( - scoped.processes.map((entry) => entry.pid).sort((a, b) => a - b), - processes.map((entry) => entry.pid).sort((a, b) => a - b), - 'a workspace root never filters the process table', - ); const scopedTarget = scoped.processes.find((entry) => entry.pid === target.pid); assert.ok(scopedTarget, 'the F# target survives scoping'); + assert.equal( + fsharp.in_workspace, + false, + 'without roots the fixture is outside the workspace', + ); + const fixtureRoot = fsharp.entry!; + const containing = await execute('processes', { + workspace_roots: [fixtureRoot], + }); + assert.equal( + containing.processes.find((entry) => entry.pid === target.pid)?.in_workspace, + true, + '[PROF-PROCESSES-SCOPE] membership follows the supplied root, without hiding the target', + ); assert.equal(typeof scopedTarget.in_workspace, 'boolean', 'membership is a boolean flag'); assert.deepEqual( (await execute('cpu.list', {})).sessions, @@ -397,163 +343,4 @@ suite('LSP-level profiler protocol — FsOrderLedger behind the same server', () decoy.kill(); } }); - - test('CPU over the wire on F#: identity on start, samples and versions in list, a snapshot that keeps the session, ranked 1-based positions, a report with artifacts, stop, and no session afterwards', async function () { - this.timeout(STOP_MS * 4); - const { target } = live; - await command(target, 'burn on'); - - // ── [PROF-CPU-START] returns the session's identity. - const started = await execute<{ - session_id: string; - pid: number; - runtime: string; - runtime_version: string; - started_at: string; - }>('cpu.start', { pid: target.pid, include_native: false }); - assert.ok(started.session_id.length > 0, 'a session id'); - assert.equal(started.pid, target.pid, 'the session names its PID'); - assert.equal(started.runtime, 'dotnet', '[PROF-RUNTIME] runtime identity travels as data'); - assert.ok(started.runtime_version.length > 0, '[PROF-CPU-START] a runtime version is reported'); - assert.match(started.started_at, /^\d{4}-\d{2}-\d{2}T/, 'an RFC 3339 start time'); - - // ── [PROF-CPU-SESSIONS] the live session carries samples and a duration. - await new Promise((resolve) => setTimeout(resolve, 2_500)); - const listed = (await execute('cpu.list', {})).sessions; - const now = listed.find((entry) => entry.session_id === started.session_id); - assert.ok(now, 'the session is listed while it runs'); - assert.equal(now.pid, target.pid); - assert.equal(now.runtime, 'dotnet'); - assert.ok(now.runtime_version.length > 0, 'the version survives the listing'); - assert.ok(now.sample_count > 0, `samples accumulate: ${now.sample_count}`); - assert.ok(now.duration_secs > 0, `the duration runs: ${now.duration_secs}`); - assert.ok(now.started_at.length > 0, 'the start time is listed'); - - // ── [PROF-CPU-SNAPSHOT] a snapshot is a report and the session survives it. - const snapshot = await execute('cpu.snapshot', { - session_id: started.session_id, - }); - assert.equal(snapshot.session_id, started.session_id, 'the report names its session'); - assert.equal(snapshot.pid, target.pid); - assert.equal(snapshot.runtime, 'dotnet'); - assert.ok(snapshot.total_samples > 0, 'the snapshot samples something'); - assert.ok( - snapshot.user_samples >= 0 && snapshot.user_samples <= snapshot.total_samples, - 'user samples are a subset', - ); - assert.ok(snapshot.sample_rate >= 1, `a positive sample rate: ${snapshot.sample_rate}`); - assert.ok(snapshot.duration_secs >= 0, 'a duration'); - assert.ok( - (await execute('cpu.list', {})).sessions.some( - (entry) => entry.session_id === started.session_id, - ), - '[PROF-CPU-SNAPSHOT] the snapshot did not end the session', - ); - assertProfileShape(snapshot); - - // ── [PROF-CPU-STOP] stop drains and returns the final report. - const stopped = await execute('cpu.stop', { session_id: started.session_id }); - assert.equal(stopped.session_id, started.session_id, 'the stopped report is the session'); - assert.ok(stopped.total_samples >= snapshot.total_samples, 'samples never shrink on stop'); - assert.deepEqual(stopped.artifacts.export_error, undefined, '[PROF-EXPORT] no export error'); - assertProfileShape(stopped); - assert.equal( - (await execute('cpu.list', {})).sessions.some( - (entry) => entry.session_id === started.session_id, - ), - false, - '[PROF-CPU-SESSIONS] a stopped session is no longer active', - ); - - // ── A stopped session cannot be stopped or snapshotted again. - const restop = await executeError('cpu.stop', { session_id: started.session_id }); - assert.ok(restop, 'a second stop fails'); - assert.ok(restop.code !== 0, `[PROF-ERRORS] ${JSON.stringify(restop)}`); - const resnap = await executeError('cpu.snapshot', { session_id: started.session_id }); - assert.ok(resnap, 'a snapshot after stop fails'); - assert.ok(resnap.code !== 0, 'the ended session rejects snapshots'); - - // ── The program was never disturbed: attaching is not invasive. - const alive = await command(target, 'stats'); - assert.ok(count(alive, 'iterations') >= 0, 'the F# program still answers on stdin'); - }); - - test('a duration on start stops the session by itself and keeps its report for the next stop', async function () { - this.timeout(STOP_MS * 4); - const { target } = live; - // ── [PROF-CPU-SESSIONS] `duration` ends the session automatically. - const started = await execute<{ session_id: string }>('cpu.start', { - pid: target.pid, - duration: 2, - }); - assert.ok(started.session_id.length > 0, 'a session id with a duration'); - await new Promise((resolve) => setTimeout(resolve, 5_000)); - const active = (await execute('cpu.list', {})).sessions; - assert.equal( - active.some((entry) => entry.session_id === started.session_id), - false, - 'the duration elapsed and the session ended by itself', - ); - // ── Its report is kept for the next stop. - const kept = await execute('cpu.stop', { session_id: started.session_id }); - assert.equal(kept.session_id, started.session_id, 'the kept report belongs to the session'); - assert.ok(kept.total_samples >= 0, 'the kept report has a sample count'); - assert.ok(kept.duration_secs >= 0, 'the kept report has a duration'); - assertProfileShape(kept); - }); }); - -/** [PROF-CPU-REPORT] + [PROF-AGGREGATION-LOGIC] + [PROF-SAMPLE] + [PROF-EXPORT]. */ -function assertProfileShape(report: ProfileReport): void { - assert.ok(Array.isArray(report.hot_functions), 'hot functions are a list'); - assert.ok(Array.isArray(report.hot_lines), 'hot lines are a list'); - const percentages = report.hot_functions.map((fn) => fn.percentage); - assert.deepEqual( - percentages, - [...percentages].sort((a, b) => b - a), - 'functions are ranked hottest first', - ); - for (const fn of report.hot_functions) { - assert.ok(fn.name.length > 0, 'a function has a name'); - assert.ok( - fn.percentage >= 0 && fn.percentage <= 100, - `inclusive share 0–100: ${fn.percentage}`, - ); - assert.ok( - fn.self_percentage <= fn.percentage, - `self never exceeds inclusive: ${fn.self_percentage} > ${fn.percentage}`, - ); - assert.ok(fn.samples >= fn.self_samples, 'inclusive samples cover the self samples'); - assert.ok(fn.samples <= report.total_samples, 'a function cannot exceed the total'); - if (fn.line !== undefined) { - assert.ok(fn.line >= 1, `[PROF-SAMPLE] 1-based definition line: ${fn.line}`); - assert.ok((fn.file ?? '').length > 0, 'a located function names its file'); - } - } - const linePercentages = report.hot_lines.map((line) => line.percentage); - assert.deepEqual( - linePercentages, - [...linePercentages].sort((a, b) => b - a), - 'lines are ranked hottest first', - ); - for (const line of report.hot_lines) { - assert.ok(line.line >= 1, `[PROF-SAMPLE] 1-based hot line: ${line.line}`); - assert.ok(line.file.length > 0, 'a hot line names its file'); - assert.ok( - line.percentage > 0 && line.percentage <= 100, - `a hot line share: ${line.percentage}`, - ); - assert.ok(line.samples <= report.total_samples, 'a hot line cannot exceed the total'); - } - const artifacts = report.artifacts; - for (const artifact of [ - artifacts.speedscope_path, - artifacts.flamegraph_path, - artifacts.cpuprofile_path, - ]) { - if (artifact !== undefined) { - assert.ok(fs.existsSync(artifact), `[PROF-EXPORT] the artifact exists: ${artifact}`); - } - } - assert.ok(Array.isArray(report.diagnostics), 'diagnostics travel with the result'); -} diff --git a/src/editors/vscode/src/test/suite/profiler-process-explorer.test.ts b/src/editors/vscode/src/test/suite/profiler-process-explorer.test.ts index 745f3a08..b0901180 100644 --- a/src/editors/vscode/src/test/suite/profiler-process-explorer.test.ts +++ b/src/editors/vscode/src/test/suite/profiler-process-explorer.test.ts @@ -28,6 +28,7 @@ import { toWireProcess, ungroup, rowPid as pidOf, + processFixture, } from './profiler-ui-helpers'; import { numberField, @@ -37,29 +38,6 @@ import { const MB = 1024 * 1024; -/** A debuggable process row; each test overrides what it exercises. */ -function processFixture(pid: number, overrides: Partial = {}): ProcessInfo { - return { - pid, - ppid: 1, - name: 'python3.12', - interpreterPath: '/usr/bin/python3.12', - script: null, - runtime: 'python', - runtimeVersion: '3.12.1', - cpuPercent: 0, - memoryBytes: MB, - runtimeSecs: 1, - user: null, - requiresElevation: false, - inWorkspace: false, - launcher: null, - debuggable: true, - undebuggableReason: null, - ...overrides, - }; -} - /** A representative process table covering launchers, users, and versions. */ const STUB_PROCESSES: readonly ProcessInfo[] = [ processFixture(100, { From 4c29e9cb96b54f277a29d7d200b71a0318a186de Mon Sep 17 00:00:00 2001 From: Christian Findlay <16697547+MelbourneDeveloper@users.noreply.github.com> Date: Sun, 18 Oct 2026 18:38:10 +1100 Subject: [PATCH 13/24] Pin lspkit at main's F# memory fixes (Nimblesite/lspkit#13, #14) FSharp.Core types no longer anchor at the build server's paths, sites with no location stay unlocated in the UI, the objects summary keeps the requested type's row, and the retention graph draws each root by its label. --- lspkit | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lspkit b/lspkit index b9066b19..7e0a0c56 160000 --- a/lspkit +++ b/lspkit @@ -1 +1 @@ -Subproject commit b9066b192da4b71a2a1452a9bc487c2e5bad50c8 +Subproject commit 7e0a0c56c68c1d5f31721f66db33466ac3bd252f From c4f4d2763286da1f1dd2bf639f9a368e2415a22a Mon Sep 17 00:00:00 2001 From: Christian Findlay <16697547+MelbourneDeveloper@users.noreply.github.com> Date: Sun, 18 Oct 2026 18:49:07 +1100 Subject: [PATCH 14/24] Refuse a profiler command whose arguments are not one params object Every profiler operation takes exactly one argument, its typed params object ([PROF-WIRE-METHODS]). The host passed the first argument on and ignored the rest, so `capabilities` answered `[]`, `[42]` and two objects alike. Anything but one JSON object is now INVALID_PARAMS before the service sees it; an absent argument list still means empty params. --- src/sharplsp/src/profiler/host.rs | 72 +++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/src/sharplsp/src/profiler/host.rs b/src/sharplsp/src/profiler/host.rs index 004b23fb..18328494 100644 --- a/src/sharplsp/src/profiler/host.rs +++ b/src/sharplsp/src/profiler/host.rs @@ -112,6 +112,10 @@ impl ProfilerHost { let Some((method, params)) = profiler_call(&req.params) else { return false; }; + if let Some(reason) = malformed_arguments(&req.params) { + refuse(req.id.clone(), method, reason, sender); + return true; + } self.pending.retain(|_, cancel| !cancel.is_closed()); let (cancel, cancelled) = oneshot::channel(); let _ = self.pending.insert(req.id.clone(), cancel); @@ -161,6 +165,42 @@ fn profiler_call(params: &Value) -> Option<(&'static str, Value)> { Some((method, arguments)) } +/// Why a profiler command's arguments are not exactly one params object +/// ([PROF-WIRE-METHODS]); `None` when they are, or when the list is absent. +fn malformed_arguments(params: &Value) -> Option<&'static str> { + match params.get("arguments") { + None | Some(Value::Null) => None, + Some(Value::Array(list)) => match list.as_slice() { + [Value::Object(_)] => None, + [_] => Some("the profiler params argument must be a JSON object"), + _ => Some("a profiler command takes exactly one argument, its params object"), + }, + Some(_) => Some("a profiler command's arguments must be a list of one params object"), + } +} + +/// Answer a profiler command whose arguments are malformed with `INVALID_PARAMS`. +fn refuse( + id: RequestId, + method: &'static str, + reason: &'static str, + sender: &crossbeam_channel::Sender, +) { + warn!(method, reason, "profiler request refused"); + let error = lsp_profiler::ProfilerError::InvalidParams(reason.to_owned()); + let code = i32::try_from(error.code()).unwrap_or(-32000); + if sender + .send(Message::Response(Response::new_err( + id, + code, + error.to_string(), + ))) + .is_err() + { + warn!(method, "profiler response dropped: connection closed"); + } +} + /// Emit exactly one response after completion or cancellation. async fn complete( service: Arc, @@ -213,6 +253,38 @@ mod tests { ); } + #[test] + fn anything_but_one_params_object_is_malformed() { + let refused = [ + serde_json::json!([]), + serde_json::json!([{}, {}]), + serde_json::json!([42]), + serde_json::json!(["pid=1"]), + serde_json::json!([null]), + serde_json::json!("pid=1"), + ]; + for arguments in refused { + let params = serde_json::json!({ "command": "sharplsp.lspProfiler.capabilities", "arguments": arguments }); + assert!( + malformed_arguments(¶ms).is_some(), + "{arguments} is not one params object" + ); + } + } + + #[test] + fn one_params_object_or_no_argument_list_is_well_formed() { + let accepted = [ + serde_json::json!({ "command": "sharplsp.lspProfiler.cpu.list" }), + serde_json::json!({ "command": "sharplsp.lspProfiler.cpu.list", "arguments": null }), + serde_json::json!({ "command": "sharplsp.lspProfiler.cpu.list", "arguments": [{}] }), + serde_json::json!({ "command": "sharplsp.lspProfiler.cpu.start", "arguments": [{ "pid": 7 }] }), + ]; + for params in accepted { + assert_eq!(malformed_arguments(¶ms), None, "{params}"); + } + } + #[test] fn a_missing_argument_list_becomes_empty_params() { let params = serde_json::json!({ "command": "sharplsp.lspProfiler.capabilities" }); From 4da2a5a6cda8e7a090c7e2c6096e2baab4b6afe3 Mon Sep 17 00:00:00 2001 From: Christian Findlay <16697547+MelbourneDeveloper@users.noreply.github.com> Date: Sun, 18 Oct 2026 18:52:26 +1100 Subject: [PATCH 15/24] Profile F# in the docs with a narrated walkthrough, and make the film tools reusable A new Profiling F# page (en, ja, zh) covers what the profiler does with F#'s compiled code: closures and pipeline stages by name, records and union cases painted on the line that builds them, module values named as written in the retention graph, and the fix. The profiler, F# and memory-leak pages link to it, and the F# pages play profiler-fsharp: FsStorefront's CPU and memory, narrated, scored and captioned in every site language. The film tools become a kit any film can use. A film is one file in films/: its demo, its poster and its scenes, told through kit/acts/cpu.mjs and kit/acts/memory.mjs. film.mjs (or make _film FILM=) shoots and publishes it. kit/set.mjs stages a demo from demos/ beside an isolated VS Code; kit/workbench.mjs, camera.mjs and overlay.mjs are the moves, the recorder and the captions; kit/sound/ speaks a reviewed script locally with Kokoro (kokoro-js), synthesises an original score, ducks it under the voice and normalises to -16 LUFS. The README says how to add a film. {% video %} plays a narrated film unmuted with caption tracks; only autoplaying loops stay muted. --- Makefile | 14 +- docs/specs/DESIGN-SYSTEM.md | 7 +- docs/specs/PROFILER-SPEC.md | 7 +- src/website/src/_data/i18n.json | 3 + src/website/src/_data/navigation.json | 2 +- src/website/src/_data/videos.json | 22 + .../profiler/profiler-fsharp-graph.webp | Bin 0 -> 87808 bytes .../profiler/profiler-fsharp-heatmap.webp | Bin 0 -> 149806 bytes .../profiler/profiler-fsharp-leaks.webp | Bin 0 -> 144878 bytes .../profiler/profiler-fsharp-results.webp | Bin 0 -> 253418 bytes .../assets/videos/profiler-fsharp-poster.webp | Bin 0 -> 115408 bytes .../src/assets/videos/profiler-fsharp.en.vtt | 32 + .../src/assets/videos/profiler-fsharp.ja.vtt | 32 + .../src/assets/videos/profiler-fsharp.mp4 | 3 + .../src/assets/videos/profiler-fsharp.webm | 3 + .../src/assets/videos/profiler-fsharp.zh.vtt | 32 + src/website/src/docs/fsharp-profiling.md | 102 ++ src/website/src/docs/fsharp.md | 7 + src/website/src/docs/memory-leaks.md | 8 +- src/website/src/docs/profiler.md | 4 + src/website/src/ja/docs/fsharp-profiling.md | 103 ++ src/website/src/ja/docs/fsharp.md | 7 + src/website/src/ja/docs/memory-leaks.md | 8 +- src/website/src/ja/docs/profiler.md | 4 + src/website/src/zh/docs/fsharp-profiling.md | 103 ++ src/website/src/zh/docs/fsharp.md | 7 + src/website/src/zh/docs/memory-leaks.md | 8 +- src/website/src/zh/docs/profiler.md | 4 + src/website/tests/docs-nav.spec.js | 2 +- src/website/tests/profiler-media.spec.js | 46 +- src/website/tests/site-videos.spec.js | 24 +- src/website/tools/profiler-film/.gitignore | 1 + src/website/tools/profiler-film/README.md | 93 ++ .../{ => demos}/fs-storefront/Catalog.fs | 0 .../{ => demos}/fs-storefront/Checkout.fs | 0 .../fs-storefront/Directory.Build.props | 0 .../{ => demos}/fs-storefront/Domain.fs | 0 .../fs-storefront/FsStorefront.fsproj | 0 .../fs-storefront/FsStorefront.slnx | 0 .../{ => demos}/fs-storefront/Program.fs | 0 .../{ => demos}/fs-storefront/Sessions.fs | 0 .../{ => demos}/storefront/Catalog.cs | 0 .../storefront/CheckoutPipeline.cs | 0 .../storefront/Directory.Build.props | 0 .../{ => demos}/storefront/Program.cs | 0 .../{ => demos}/storefront/SessionCache.cs | 0 .../{ => demos}/storefront/Storefront.csproj | 0 .../{ => demos}/storefront/Storefront.slnx | 0 src/website/tools/profiler-film/film.mjs | 28 + .../tools/profiler-film/films/fsharp.mjs | 61 + .../profiler-film/films/fsharp.narration.json | 67 + .../tools/profiler-film/films/leak.mjs | 50 + .../{ => films/profiler}/capture.mjs | 42 +- .../{ => films/profiler}/film.html | 8 +- .../{ => films/profiler}/film.js | 4 +- .../{ => films/profiler}/flamegraph.mjs | 0 .../{ => films/profiler}/render.mjs | 6 +- .../tools/profiler-film/kit/acts/cpu.mjs | 61 + .../tools/profiler-film/kit/acts/memory.mjs | 153 +++ .../tools/profiler-film/kit/camera.mjs | 64 + .../{vscode.mjs => kit/editor.mjs} | 2 +- .../tools/profiler-film/{ => kit}/encode.mjs | 23 +- .../tools/profiler-film/{ => kit}/overlay.mjs | 2 +- .../tools/profiler-film/kit/scenes.mjs | 158 +++ .../profiler-film/{ => kit}/screencast.mjs | 0 src/website/tools/profiler-film/kit/set.mjs | 98 ++ .../tools/profiler-film/kit/sound/score.mjs | 137 ++ .../profiler-film/kit/sound/soundtrack.mjs | 61 + .../tools/profiler-film/kit/sound/voice.mjs | 52 + .../tools/profiler-film/kit/workbench.mjs | 123 ++ src/website/tools/profiler-film/leak.mjs | 57 - .../tools/profiler-film/package-lock.json | 1129 +++++++++++++++++ src/website/tools/profiler-film/package.json | 9 + src/website/tools/profiler-film/studio.mjs | 226 ---- .../tools/profiler-film/walkthrough.mjs | 265 ---- src/website/video-figure.js | 24 +- 76 files changed, 2972 insertions(+), 626 deletions(-) create mode 100644 src/website/src/assets/screenshots/profiler/profiler-fsharp-graph.webp create mode 100644 src/website/src/assets/screenshots/profiler/profiler-fsharp-heatmap.webp create mode 100644 src/website/src/assets/screenshots/profiler/profiler-fsharp-leaks.webp create mode 100644 src/website/src/assets/screenshots/profiler/profiler-fsharp-results.webp create mode 100644 src/website/src/assets/videos/profiler-fsharp-poster.webp create mode 100644 src/website/src/assets/videos/profiler-fsharp.en.vtt create mode 100644 src/website/src/assets/videos/profiler-fsharp.ja.vtt create mode 100644 src/website/src/assets/videos/profiler-fsharp.mp4 create mode 100644 src/website/src/assets/videos/profiler-fsharp.webm create mode 100644 src/website/src/assets/videos/profiler-fsharp.zh.vtt create mode 100644 src/website/src/docs/fsharp-profiling.md create mode 100644 src/website/src/ja/docs/fsharp-profiling.md create mode 100644 src/website/src/zh/docs/fsharp-profiling.md create mode 100644 src/website/tools/profiler-film/README.md rename src/website/tools/profiler-film/{ => demos}/fs-storefront/Catalog.fs (100%) rename src/website/tools/profiler-film/{ => demos}/fs-storefront/Checkout.fs (100%) rename src/website/tools/profiler-film/{ => demos}/fs-storefront/Directory.Build.props (100%) rename src/website/tools/profiler-film/{ => demos}/fs-storefront/Domain.fs (100%) rename src/website/tools/profiler-film/{ => demos}/fs-storefront/FsStorefront.fsproj (100%) rename src/website/tools/profiler-film/{ => demos}/fs-storefront/FsStorefront.slnx (100%) rename src/website/tools/profiler-film/{ => demos}/fs-storefront/Program.fs (100%) rename src/website/tools/profiler-film/{ => demos}/fs-storefront/Sessions.fs (100%) rename src/website/tools/profiler-film/{ => demos}/storefront/Catalog.cs (100%) rename src/website/tools/profiler-film/{ => demos}/storefront/CheckoutPipeline.cs (100%) rename src/website/tools/profiler-film/{ => demos}/storefront/Directory.Build.props (100%) rename src/website/tools/profiler-film/{ => demos}/storefront/Program.cs (100%) rename src/website/tools/profiler-film/{ => demos}/storefront/SessionCache.cs (100%) rename src/website/tools/profiler-film/{ => demos}/storefront/Storefront.csproj (100%) rename src/website/tools/profiler-film/{ => demos}/storefront/Storefront.slnx (100%) create mode 100644 src/website/tools/profiler-film/film.mjs create mode 100644 src/website/tools/profiler-film/films/fsharp.mjs create mode 100644 src/website/tools/profiler-film/films/fsharp.narration.json create mode 100644 src/website/tools/profiler-film/films/leak.mjs rename src/website/tools/profiler-film/{ => films/profiler}/capture.mjs (75%) rename src/website/tools/profiler-film/{ => films/profiler}/film.html (94%) rename src/website/tools/profiler-film/{ => films/profiler}/film.js (99%) rename src/website/tools/profiler-film/{ => films/profiler}/flamegraph.mjs (100%) rename src/website/tools/profiler-film/{ => films/profiler}/render.mjs (95%) create mode 100644 src/website/tools/profiler-film/kit/acts/cpu.mjs create mode 100644 src/website/tools/profiler-film/kit/acts/memory.mjs create mode 100644 src/website/tools/profiler-film/kit/camera.mjs rename src/website/tools/profiler-film/{vscode.mjs => kit/editor.mjs} (99%) rename src/website/tools/profiler-film/{ => kit}/encode.mjs (66%) rename src/website/tools/profiler-film/{ => kit}/overlay.mjs (98%) create mode 100644 src/website/tools/profiler-film/kit/scenes.mjs rename src/website/tools/profiler-film/{ => kit}/screencast.mjs (100%) create mode 100644 src/website/tools/profiler-film/kit/set.mjs create mode 100644 src/website/tools/profiler-film/kit/sound/score.mjs create mode 100644 src/website/tools/profiler-film/kit/sound/soundtrack.mjs create mode 100644 src/website/tools/profiler-film/kit/sound/voice.mjs create mode 100644 src/website/tools/profiler-film/kit/workbench.mjs delete mode 100644 src/website/tools/profiler-film/leak.mjs create mode 100644 src/website/tools/profiler-film/package-lock.json create mode 100644 src/website/tools/profiler-film/package.json delete mode 100644 src/website/tools/profiler-film/studio.mjs delete mode 100644 src/website/tools/profiler-film/walkthrough.mjs diff --git a/Makefile b/Makefile index 78588bf1..809dac3f 100644 --- a/Makefile +++ b/Makefile @@ -40,7 +40,7 @@ # sidecars ONCE and emitting that platform's VSIX and its standalone server # archive ([DIST-ARCHIVE]); VERSION is optional and defaults to 0.0.0. Then # `_print-publish-commands`. Also private: `_screenshots`, `_website-build`, -# `_website-dev`, `_test-website` and `_uninstall-dotnet-10`. +# `_website-dev`, `_test-website`, `_film` and `_uninstall-dotnet-10`. # ── OS detection ────────────────────────────────────────────────── # All recipes assume a POSIX shell. On Windows we use Git Bash (bundled with @@ -145,7 +145,7 @@ KOVER_PERCENT = $(if $(COVERAGE_TOOLS_PREBUILT),$(DOTNET) target/coverage-tools/ .PHONY: build ci test lint fmt clean audit setup _build-profiler-ui \ reinstall-vsix install-vsix uninstall-vsix install-dotnet-10 -.PHONY: _screenshots _website-build _website-dev _uninstall-dotnet-10 \ +.PHONY: _screenshots _website-build _website-dev _film _uninstall-dotnet-10 \ _package-vsix-linux-x64 _package-vsix-linux-arm64 \ _package-vsix-darwin-arm64 _package-vsix-darwin-x64 \ _package-vsix-win32-x64 _package-vsix-win32-arm64 \ @@ -805,6 +805,16 @@ _website-dev: @echo "==> Starting website development server..." npm run dev --prefix src/website +# [WEB-DESIGN-MEDIA] Shoot and publish one website film from the packaged VSIX: +# `make _film FILM=fsharp`, or `PUBLISH=1` to publish the last shoot again. +# The films live in src/website/tools/profiler-film/films; its README says how. +_film: + @test -n "$(FILM)" || { echo "usage: make _film FILM= [PUBLISH=1]" >&2; exit 1; } + $(if $(PUBLISH),,$(MAKE) _build-vsix VSIX_PREBUILT=1) + @# Install once: npm ci would wipe the voice model kokoro-js caches in node_modules. + test -d src/website/tools/profiler-film/node_modules || npm ci --prefix src/website/tools/profiler-film + node src/website/tools/profiler-film/film.mjs $(FILM) $(if $(PUBLISH),--publish,) + _test-website: @echo "==> Running website Playwright tests..." npm ci --prefix src/website diff --git a/docs/specs/DESIGN-SYSTEM.md b/docs/specs/DESIGN-SYSTEM.md index 16ee0826..37697c05 100644 --- a/docs/specs/DESIGN-SYSTEM.md +++ b/docs/specs/DESIGN-SYSTEM.md @@ -126,17 +126,18 @@ Page-level grids and marketing card styles must not leak into `.prose`. Do not r Screenshots and video of a SharpLsp feature show the real extension, never a mock-up. -- **Capture.** The profiler media is shot by `src/website/tools/profiler-film/`. `capture.mjs` drives an isolated VS Code (its own user-data and extensions directories, never the user's editor) running the packaged VSIX against a real .NET app, and records stills and real-time screencasts. `render.mjs` turns them into the site's assets. +- **Capture.** The profiler media is shot by `src/website/tools/profiler-film/` (its README says how). Every shot runs on `kit/set.mjs`'s `onSet`: an isolated VS Code (its own user-data and extensions directories, never the user's editor) running the packaged VSIX against a real .NET demo app from `demos/`. A walkthrough is one file in `films/`, a list of scenes that `film.mjs` shoots as real-time clips and publishes; `kit/acts/` tells the CPU and memory stories, so each film supplies only its words, its demo and its fix. `films/leak.mjs` films the C# Storefront demo and `films/fsharp.mjs` its F# twin, FsStorefront, so every F# claim the docs make about the profiler is shown on F# code. The composited one-minute film is `films/profiler/`: `capture.mjs` records stills and screencasts, and `render.mjs` turns them into the site's assets. - **Honesty.** Footage plays in real time unless the frame says otherwise: every fast-forwarded stretch carries an on-screen speed marker, and captions name time-lapses. Motion graphics (title cards, camera moves, highlights, the cursor) frame the footage but never alter what the editor shows. - **Privacy.** No frame may show a user name, home directory, hostname or terminal. The demo app runs from a neutral temporary path and the terminal is never opened. - **Formats.** - Films ship as H.264 MP4 and VP9 WebM with a WebP poster. - Stills ship as WebP, 2400px wide, with `width` and `height` set. - - Marketing pages may autoplay a film only when it is muted and looping; docs pages use controls. + - Marketing pages may autoplay a film only when it is muted and looping; docs pages use controls and are never muted. - Every autoplaying video pauses and shows its controls under `prefers-reduced-motion: reduce`. - Every video ships a WebP poster, the films and the recordings alike. +- **Narration.** A walkthrough may be narrated (`kit/sound/`). Its reviewed script (`films/.narration.json`) is spoken by Kokoro-82M through `kokoro-js` on the capturing machine, never by a speech service or the operating system's voices, whose licences forbid publishing. Each line starts just after its scene's cut, and a scene holds its last frame until its line ends. The music is an original score synthesised from code (`kit/sound/score.mjs`), so no film carries licensed or sampled music, and it ducks under the voice. A narrated film ships AAC in its MP4 and Opus in its WebM, plays unmuted with controls, and carries WebVTT captions in every site language (`..vtt`, the page's own language shown by default), listed under `captions` in the catalogue. - **One catalogue.** Every video on the site is declared once in `src/website/src/_data/videos.json`: encodings, size, length, upload date, and its label and caption in each language. Pages place a video with the `{% video "" %}` shortcode (`src/website/video-figure.js`) and never repeat `