[brycebaril]

The N|Solid MCP server's list_behaviors now marks two kinds of behavior, and ns-analyze-behaviors explains both.

• notInScan: the package was seen running programs or compiling code, but its published code imports none of the Node.js modules that do it. N|Solid marks it only when the scan read all of that version's code and found no computed import and no eval. So neither explains the mark, and the skill says what can:

  • the app or another package hands it a function from that module;
  • a loader the scan doesn't follow;
  • code on disk that isn't the published package's.

  If the version's checks show risk/uses-dynamic-require or risk/uses-eval anyway, the skill treats the scan as stale or the checks as another version's, and confirms which before using them. The package version also fails observed-not-in-scan in app_packages.

• compiled: the calls came from code the package compiled from a string (new Function, eval). That's the shape of an injected payload. The skill asks whether the package compiles code by design: a template engine, a schema validator or an expression evaluator does, and such packages rarely run programs or reach the network from that code. With notInScan or versionChange as well, it treats the behavior as a likely compromise.

Each mark weighs a step more. No tool or argument changes.

🤖 Generated with Claude Code

[ns-control-tower]

Walkthrough

Docs-only change to skills/ns-analyze-behaviors/SKILL.md (final diff vs nsolid-console-7, merge base 29aad46: 1 file, +5 lines, 4 commits; no code, tool, or argument changes). It documents two marks list_behaviors can put on a behavior:

• Lines 15, 24–25 — notInScan (commits 1–2): the package was seen running programs or compiling code its published code never imports; N|Solid marks it only when the scan read all of that version's code and found no computed import and no eval, so neither of those explains it; it weighs a step more, and the package version fails observed-not-in-scan in app_packages. Investigation guidance lists what can explain it (a function handed from the module, a loader the scan doesn't follow, code on disk that isn't the published package's) and treats risk/uses-dynamic-require / risk/uses-eval as non-explanations — confirm the exact version and scan freshness instead.
• Lines 16, 26 — compiled (commits 3–4): the calls came from code compiled from a string (new Function, eval) — the shape of an injected payload; it does not name the compiling package: the calls count for the nearest package below it on the stack, the one that called it, and another package may have compiled it and handed it over (a callback, an option); it weighs a step more too. Judging guidance: check whether that package, or one handing it functions, compiles code by design (a template engine, a schema validator or an expression evaluator) and whether the calls fit what it compiles; with notInScan or versionChange as well, treat it as a likely compromise only while the calls stay unexplained or other evidence backs it.

Commit 2e113e2c reworks lines 16, 24 and 26 (3 insertions, 3 deletions vs 596d50c) to address the three findings from the latest human recheck at 596d50c.

Assessment

• ✔ Resolved at 2e113e2c — P2 on compiled attribution: line 16 no longer implies the named package compiled the code — the calls count for the nearest package below it on the stack (the one that called it), and another package may have compiled it and handed it over.
• ✔ Resolved at 2e113e2c — P2 on the compromise conclusion: line 26 now conditions "likely compromise" on the calls staying unexplained or other evidence backing it, with the template-engine example inline.
• ✔ Resolved at 2e113e2c — P3 on the observed modules: line 24 uses app_packages only for the failing observed-not-in-scan check and points to the behavior's modules in list_behaviors for the observed modules.
• The earlier P2 on notInScan gating remains fixed — lines 15 and 25 are untouched by 2e113e2c.
• No new findings at 2e113e2c: the reworked prose is internally consistent with the rest of the skill, and notInScan / compiled / observed-not-in-scan / risk/uses-* still appear only in this file.
• Repo hygiene checks re-run at 2e113e2c in the review clone: node scripts/sync-plugin-assets.mjs --check → "plugin:sync up to date."; node scripts/materialize-github-marketplace.mjs --check → "plugin:root:check OK — root manifests match bundle.json".
• CI: test run 42158523694 (.github/workflows/test.yml) succeeded for 2e113e2c. The workflow lints, builds and tests code and only asserts that skill files exist — it does not exercise skill prose, so no CI signal is relied on for this verdict.
• Assumption on record (as in the earlier reviews): app_packages, observed-not-in-scan, risk/uses-dynamic-require, risk/uses-eval and the compiled attribute belong to the external nsolid MCP server and appear nowhere in this repository; their semantics are taken as given from the server-side changes the commit messages describe.
• No PR Lens diagram: the diff is pure prose with no component interaction to draw (unchanged from the previous reviews).

Verdict: APPROVE — all three findings from the latest human recheck are verifiably fixed at 2e113e2c, the rework introduces no new findings, and the repository's hygiene checks and CI pass at this head.

[dobleuber]

[P2] Align the notInScan investigation with the backend's current gates

The current backend does not mark a scanned version notInScan when it contains a computed import or eval: seenNotInScan explicitly returns false for either scan.dynamicImport.count > 0 or scan.eval.count > 0, and its unit tests cover both exclusions.

This new guidance therefore sends the analyst looking for checks that prevent the mark from being emitted for that version. Please update this paragraph and the examples above to explain that the mark requires a complete scan with neither condition. If those checks appear alongside an existing mark, first verify the exact package version and scan freshness rather than presenting them as its normal explanation.

[brycebaril]

Right, thanks. Fixed in 1c68a46:

• Step 1 now says the mark needs a scan that read all of the version's code and found no computed import and no eval. So neither can explain it.
• This step lists what can: a function the app or another package hands it from the module, a loader the scan doesn't follow, or code on disk that isn't the published package's.
• If risk/uses-dynamic-require or risk/uses-eval do show up next to the mark, the skill treats it as a stale scan or another version. It confirms the exact version and when that version was scanned before using them.

[brycebaril]

@dobleuber, ready for another look when you have a moment. Your P2 is fixed in 1c68a46: the skill now says the notInScan mark needs a scan that read all of the version's code and found no computed import and no eval, so neither explains it. 596d50c adds the compiled mark, which the N|Solid MCP server's list_behaviors returns now. CI is green.

[dobleuber]

Rechecked at 596d50c. The earlier notInScan gating feedback is addressed. Three remaining findings are attached inline.

[dobleuber]

[P2] Qualify which package compiled the code

compiled establishes that the call came through string-compiled code, but it does not establish which package created that code. For example, package A can create a function with new Function and hand it to package B. If B invokes it and it calls a Node.js built-in, the profiler attributes the call to the nearest identifiable package on that stack, B. I reproduced that attribution with a synthetic stack and an ordinary-code control.

Saying B compiled the function would send the investigation to the wrong package. Please describe the flag as a call from string-compiled code, and qualify package attribution rather than treating it as proof of compiler provenance.

[brycebaril]

Right: a call from string-compiled code counts for the nearest package below it on the stack, the one that called it. So it shows who ran that code, not who compiled it. 2e113e2 says so, and that another package may have compiled the function and handed it over (a callback, an option).

[dobleuber]

[P2] Preserve the expected-use check before concluding likely compromise

A legitimate template-engine upgrade can intentionally start reading partials through node:fs from generated render functions. That can produce both compiled and versionChange, even when the calls fit the package's documented purpose.

The final sentence nevertheless instructs the analyst to treat that combination as a likely compromise, overriding the expected-use assessment earlier in the paragraph and the later distinction between unexplained and expected behavior. Please make this conclusion conditional on the calls remaining unexpected or unexplained, or on corroborating evidence.

[brycebaril]

Agreed. In 2e113e2 it's a likely compromise only while the calls stay unexplained or other evidence backs it, with your template-engine case as the example. The check also asks about a package that hands it functions, since that one may be the compiler.

[dobleuber]

[P3] Read the observed modules from list_behaviors

app_packages exposes the per-version failing check as group/name:severity (with its source), but its MCP response does not include the check title, details, or modules. Following this step therefore cannot retrieve the promised module evidence from that tool.

Please use app_packages to confirm the version's observed-not-in-scan check and point to list_behaviors.modules for the observed modules.

[brycebaril]

Right, app_packages gives the check only as group/name:severity. In 2e113e2 the failing check comes from app_packages and the modules from the behavior's modules in list_behaviors.

[dobleuber]

One remaining finding from the full-range re-review at 2e113e2 is attached inline.

[dobleuber]

[P2] Limit compiled to at least one observed call

The behavior-level compiled flag is true if any observed call came from string-compiled code; it does not apply to every entry in modules or functions. The aggregation retains all function names while combining the flag with a maximum/OR.

I reproduced a single package version with a generated node:fs.readFileSync call and an ordinary node:fs.writeFileSync call. Both functions belong to the same capability record, which has compiled: true. Saying "the calls came from code compiled from a string" can therefore make the analyst attribute both calls to generated code and explain away an ordinary call as expected template activity.

Please say that at least one observed call came from string-compiled code, and clarify that this flag alone does not identify which listed calls came through generated code.