From 7ef555f00fe146fec25ad14dd1ae799c5a0080b7 Mon Sep 17 00:00:00 2001 From: Bryce Baril Date: Thu, 22 Oct 2026 22:58:34 -0700 Subject: [PATCH 1/4] feat(nsolid): ns-analyze-behaviors explains a capability the package's code never imports list_behaviors now marks a behavior notInScan when the package's published code imports none of the modules it was seen calling. The skill says what that means, and how app_packages' failing checks (uses-dynamic-require, uses-eval) can explain it. Co-Authored-By: Claude Opus 5.5 --- skills/ns-analyze-behaviors/SKILL.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/skills/ns-analyze-behaviors/SKILL.md b/skills/ns-analyze-behaviors/SKILL.md index afe2a75..29f730c 100644 --- a/skills/ns-analyze-behaviors/SKILL.md +++ b/skills/ns-analyze-behaviors/SKILL.md @@ -12,6 +12,7 @@ The watch learns what each app's packages do from its CPU profiles, and reports - `list_behaviors` (open ones, most severe first), with `app` when the user names one, `kind` (`capability`, `package`, `regression`, `drift`) or `where`, e.g. `severity >= "high"`, `capability == "network"`, `versionChange`. - Each behavior names the app, the package and its versions, what it `does`, its severity, the modules and functions involved, when it was first and last seen, and how many profiles showed it against how many before did not. - `versionChange` means it arrived with a new version of the package while earlier versions never did it. That is how a compromised release of a dependency shows at run time, and it weighs a step more. +- `notInScan` means the package's published code imports none of the modules it was seen calling: it was seen running programs or compiling code that its source never imports, so something reached them another way (a module required by a computed name, code fetched and evaluated). It weighs a step more too, and the package version fails `observed-not-in-scan` in `app_packages`. ### 2. Judge Each One - **New capability** (`process`: runs programs; `code`: compiles code; `network`; `threads`; `fs`): is it expected of that package? A logging library that starts spawning processes, or a date library that starts using the network, is suspicious; a database driver using the network is not. @@ -19,6 +20,7 @@ The watch learns what each app's packages do from its CPU profiles, and reports - `compare_release_calls` for the app's release that brought it shows the call against the release before, and the package upgrades that came with it. - `find_package` with the package name shows which apps and processes run which versions of it. - `list_vulnerabilities` with `where: '"" in packages'` shows whether the version is a known malicious or vulnerable release (`MAL-…` reports are malicious packages). + - For a `notInScan` behavior, `app_packages` with the `app` lists the version's failing checks. `risk/uses-dynamic-require` (a module imported by a computed name) or `risk/uses-eval` (code compiled from a string) could explain how it reached the module. With neither, it got there by an unusual loader (such as `module.constructor._load`) or through code that isn't in the published package, fetched at run time: either is a strong sign. - **New package** (running at all for the first time): which dependency brought it in (`get_sbom` with the `app` and `package` gives the chain) and whether that was expected. - **Regression**: a package taking more of the CPU in a release; `compare_releases` shows the rest of that release. Hand it to `ns-analyze-release`. - **Drift**: dependency versions that changed with no new release (a rebuilt image of the same version, or a change on disk under a running process). Ask whether the build pins its lockfile; an unexpected change under a running process deserves an incident. From 1c68a46a26c89092881507faa999ca31332f4f07 Mon Sep 17 00:00:00 2001 From: Bryce Baril Date: Tue, 27 Oct 2026 14:58:08 -0700 Subject: [PATCH 2/4] fix(nsolid): ns-analyze-behaviors doesn't explain a notInScan mark by a computed import or eval N|Solid marks a behavior `notInScan` only when the scan read all of the version's code and found no computed import and no `eval`, so neither can explain it. The skill sent the analyst looking for those checks. It now lists what can: a function handed to the package from the module, a loader the scan doesn't follow, or code on disk that isn't the published package's. If those checks do show up next to the mark, the scan changed since the behavior was marked or they belong to another version, so the skill checks the exact version and when it was scanned first. Co-Authored-By: Claude Opus 5.5 --- skills/ns-analyze-behaviors/SKILL.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/skills/ns-analyze-behaviors/SKILL.md b/skills/ns-analyze-behaviors/SKILL.md index 29f730c..ddbcf0f 100644 --- a/skills/ns-analyze-behaviors/SKILL.md +++ b/skills/ns-analyze-behaviors/SKILL.md @@ -12,7 +12,7 @@ The watch learns what each app's packages do from its CPU profiles, and reports - `list_behaviors` (open ones, most severe first), with `app` when the user names one, `kind` (`capability`, `package`, `regression`, `drift`) or `where`, e.g. `severity >= "high"`, `capability == "network"`, `versionChange`. - Each behavior names the app, the package and its versions, what it `does`, its severity, the modules and functions involved, when it was first and last seen, and how many profiles showed it against how many before did not. - `versionChange` means it arrived with a new version of the package while earlier versions never did it. That is how a compromised release of a dependency shows at run time, and it weighs a step more. -- `notInScan` means the package's published code imports none of the modules it was seen calling: it was seen running programs or compiling code that its source never imports, so something reached them another way (a module required by a computed name, code fetched and evaluated). It weighs a step more too, and the package version fails `observed-not-in-scan` in `app_packages`. +- `notInScan` means the package was seen running programs or compiling code that its published code never imports. N|Solid marks it only when the scan read all of that version's code and found no computed import and no `eval`, so neither of those explains it. It weighs a step more too, and the package version fails `observed-not-in-scan` in `app_packages`. ### 2. Judge Each One - **New capability** (`process`: runs programs; `code`: compiles code; `network`; `threads`; `fs`): is it expected of that package? A logging library that starts spawning processes, or a date library that starts using the network, is suspicious; a database driver using the network is not. @@ -20,7 +20,8 @@ The watch learns what each app's packages do from its CPU profiles, and reports - `compare_release_calls` for the app's release that brought it shows the call against the release before, and the package upgrades that came with it. - `find_package` with the package name shows which apps and processes run which versions of it. - `list_vulnerabilities` with `where: '"" in packages'` shows whether the version is a known malicious or vulnerable release (`MAL-…` reports are malicious packages). - - For a `notInScan` behavior, `app_packages` with the `app` lists the version's failing checks. `risk/uses-dynamic-require` (a module imported by a computed name) or `risk/uses-eval` (code compiled from a string) could explain how it reached the module. With neither, it got there by an unusual loader (such as `module.constructor._load`) or through code that isn't in the published package, fetched at run time: either is a strong sign. + - For a `notInScan` behavior, `app_packages` with the `app` shows the version's `observed-not-in-scan` check and the modules it named. What can explain it: the app or another package hands it a function from that module (a callback or an option); a loader the scan doesn't follow (such as `module.constructor._load`); or code on disk that isn't the published package's, patched into the image or written at run time. The last is a strong sign, and so is a `drift` behavior on the same package. + - If that version's checks also include `risk/uses-dynamic-require` or `risk/uses-eval`, they don't explain the mark: the scan changed since the behavior was marked, or the checks are for another of its versions. Confirm the exact version and when it was scanned before drawing anything from them. - **New package** (running at all for the first time): which dependency brought it in (`get_sbom` with the `app` and `package` gives the chain) and whether that was expected. - **Regression**: a package taking more of the CPU in a release; `compare_releases` shows the rest of that release. Hand it to `ns-analyze-release`. - **Drift**: dependency versions that changed with no new release (a rebuilt image of the same version, or a change on disk under a running process). Ask whether the build pins its lockfile; an unexpected change under a running process deserves an incident. From 596d50ca871cfe60ab36a36df874d03f13166785 Mon Sep 17 00:00:00 2001 From: Bryce Baril Date: Tue, 3 Nov 2026 11:11:32 -0700 Subject: [PATCH 3/4] feat(nsolid): ns-analyze-behaviors explains calls made from code compiled from a string list_behaviors now marks a behavior `compiled` when its calls came from code the package compiled with `new Function` or `eval`, and weighs it a step more. The skill says what the mark means and how to judge it: a package that compiles code by design, such as a template engine, rarely runs programs or reaches the network from that code. Co-Authored-By: Claude Opus 5.5 --- skills/ns-analyze-behaviors/SKILL.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/skills/ns-analyze-behaviors/SKILL.md b/skills/ns-analyze-behaviors/SKILL.md index ddbcf0f..185248b 100644 --- a/skills/ns-analyze-behaviors/SKILL.md +++ b/skills/ns-analyze-behaviors/SKILL.md @@ -13,6 +13,7 @@ The watch learns what each app's packages do from its CPU profiles, and reports - Each behavior names the app, the package and its versions, what it `does`, its severity, the modules and functions involved, when it was first and last seen, and how many profiles showed it against how many before did not. - `versionChange` means it arrived with a new version of the package while earlier versions never did it. That is how a compromised release of a dependency shows at run time, and it weighs a step more. - `notInScan` means the package was seen running programs or compiling code that its published code never imports. N|Solid marks it only when the scan read all of that version's code and found no computed import and no `eval`, so neither of those explains it. It weighs a step more too, and the package version fails `observed-not-in-scan` in `app_packages`. +- `compiled` means the calls came from code the package compiled from a string (`new Function`, `eval`): the shape of an injected payload, which is how a remote code execution runs what it was handed. The compiling itself shows nowhere; the calls count for the package whose compiled code made them. It weighs a step more too. ### 2. Judge Each One - **New capability** (`process`: runs programs; `code`: compiles code; `network`; `threads`; `fs`): is it expected of that package? A logging library that starts spawning processes, or a date library that starts using the network, is suspicious; a database driver using the network is not. @@ -22,6 +23,7 @@ The watch learns what each app's packages do from its CPU profiles, and reports - `list_vulnerabilities` with `where: '"" in packages'` shows whether the version is a known malicious or vulnerable release (`MAL-…` reports are malicious packages). - For a `notInScan` behavior, `app_packages` with the `app` shows the version's `observed-not-in-scan` check and the modules it named. What can explain it: the app or another package hands it a function from that module (a callback or an option); a loader the scan doesn't follow (such as `module.constructor._load`); or code on disk that isn't the published package's, patched into the image or written at run time. The last is a strong sign, and so is a `drift` behavior on the same package. - If that version's checks also include `risk/uses-dynamic-require` or `risk/uses-eval`, they don't explain the mark: the scan changed since the behavior was marked, or the checks are for another of its versions. Confirm the exact version and when it was scanned before drawing anything from them. + - For a `compiled` behavior, ask whether the package compiles code by design (a template engine, a schema validator or an expression evaluator does) and whether the calls fit what it compiles. Such packages rarely run programs or reach the network from that code. With `notInScan` or `versionChange` as well, treat it as a likely compromise. - **New package** (running at all for the first time): which dependency brought it in (`get_sbom` with the `app` and `package` gives the chain) and whether that was expected. - **Regression**: a package taking more of the CPU in a release; `compare_releases` shows the rest of that release. Hand it to `ns-analyze-release`. - **Drift**: dependency versions that changed with no new release (a rebuilt image of the same version, or a change on disk under a running process). Ask whether the build pins its lockfile; an unexpected change under a running process deserves an incident. From 2e113e2c0abd0d84b4bad02cd58ce932dcbf00fd Mon Sep 17 00:00:00 2001 From: Bryce Baril Date: Thu, 5 Nov 2026 20:44:36 -0700 Subject: [PATCH 4/4] fix(nsolid): ns-analyze-behaviors doesn't name a compiled call's compiler A call from code compiled from a string counts for the nearest package below it on the stack, the one that called it. Another package may have compiled that function and handed it over, so the skill no longer says the named package compiled it. A compiled behavior that also has notInScan or versionChange is a likely compromise only while its calls stay unexplained or other evidence backs it: a template engine's new version may read partials through node:fs by design. app_packages names the failing observed-not-in-scan check but not the modules; those are the behavior's modules in list_behaviors. Co-Authored-By: Claude Opus 5.5 --- skills/ns-analyze-behaviors/SKILL.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/skills/ns-analyze-behaviors/SKILL.md b/skills/ns-analyze-behaviors/SKILL.md index 185248b..ba2d88b 100644 --- a/skills/ns-analyze-behaviors/SKILL.md +++ b/skills/ns-analyze-behaviors/SKILL.md @@ -13,7 +13,7 @@ The watch learns what each app's packages do from its CPU profiles, and reports - Each behavior names the app, the package and its versions, what it `does`, its severity, the modules and functions involved, when it was first and last seen, and how many profiles showed it against how many before did not. - `versionChange` means it arrived with a new version of the package while earlier versions never did it. That is how a compromised release of a dependency shows at run time, and it weighs a step more. - `notInScan` means the package was seen running programs or compiling code that its published code never imports. N|Solid marks it only when the scan read all of that version's code and found no computed import and no `eval`, so neither of those explains it. It weighs a step more too, and the package version fails `observed-not-in-scan` in `app_packages`. -- `compiled` means the calls came from code the package compiled from a string (`new Function`, `eval`): the shape of an injected payload, which is how a remote code execution runs what it was handed. The compiling itself shows nowhere; the calls count for the package whose compiled code made them. It weighs a step more too. +- `compiled` means the calls came from code compiled from a string (`new Function`, `eval`): the shape of an injected payload, which is how a remote code execution runs what it was handed. It doesn't say which package compiled that code: the calls count for the nearest package below it on the stack, the one that called it, and another package may have compiled it and handed it over (a callback, an option). It weighs a step more too. ### 2. Judge Each One - **New capability** (`process`: runs programs; `code`: compiles code; `network`; `threads`; `fs`): is it expected of that package? A logging library that starts spawning processes, or a date library that starts using the network, is suspicious; a database driver using the network is not. @@ -21,9 +21,9 @@ The watch learns what each app's packages do from its CPU profiles, and reports - `compare_release_calls` for the app's release that brought it shows the call against the release before, and the package upgrades that came with it. - `find_package` with the package name shows which apps and processes run which versions of it. - `list_vulnerabilities` with `where: '"" in packages'` shows whether the version is a known malicious or vulnerable release (`MAL-…` reports are malicious packages). - - For a `notInScan` behavior, `app_packages` with the `app` shows the version's `observed-not-in-scan` check and the modules it named. What can explain it: the app or another package hands it a function from that module (a callback or an option); a loader the scan doesn't follow (such as `module.constructor._load`); or code on disk that isn't the published package's, patched into the image or written at run time. The last is a strong sign, and so is a `drift` behavior on the same package. + - For a `notInScan` behavior, `app_packages` with the `app` shows the version failing `observed-not-in-scan`, and the behavior's `modules` in `list_behaviors` are the ones it was seen using. What can explain it: the app or another package hands it a function from that module (a callback or an option); a loader the scan doesn't follow (such as `module.constructor._load`); or code on disk that isn't the published package's, patched into the image or written at run time. The last is a strong sign, and so is a `drift` behavior on the same package. - If that version's checks also include `risk/uses-dynamic-require` or `risk/uses-eval`, they don't explain the mark: the scan changed since the behavior was marked, or the checks are for another of its versions. Confirm the exact version and when it was scanned before drawing anything from them. - - For a `compiled` behavior, ask whether the package compiles code by design (a template engine, a schema validator or an expression evaluator does) and whether the calls fit what it compiles. Such packages rarely run programs or reach the network from that code. With `notInScan` or `versionChange` as well, treat it as a likely compromise. + - For a `compiled` behavior, ask whether that package, or one handing it functions, compiles code by design (a template engine, a schema validator or an expression evaluator does) and whether the calls fit what it compiles. Such packages rarely run programs or reach the network from that code. With `notInScan` or `versionChange` as well, treat it as a likely compromise only while the calls stay unexplained or other evidence backs it: a template engine's new version may read its partials through `node:fs` by design. - **New package** (running at all for the first time): which dependency brought it in (`get_sbom` with the `app` and `package` gives the chain) and whether that was expected. - **Regression**: a package taking more of the CPU in a release; `compare_releases` shows the rest of that release. Hand it to `ns-analyze-release`. - **Drift**: dependency versions that changed with no new release (a rebuilt image of the same version, or a change on disk under a running process). Ask whether the build pins its lockfile; an unexpected change under a running process deserves an incident.