diff --git a/.claude/skills/warlock/SKILL.md b/.claude/skills/warlock/SKILL.md index bcbfc0b..5849bad 100644 --- a/.claude/skills/warlock/SKILL.md +++ b/.claude/skills/warlock/SKILL.md @@ -11,9 +11,8 @@ The Warlock is a security-critical YARA-based content scanner for PostHog's agen ## When to use this skill - Editing any file under `/warlock` -- Porting rules into the Warlock from wizard's legacy `src/lib/yara-scanner.ts` - Proposing or reviewing a new rule, category, or severity -- Modifying the Warlock's public API, `scan()` signature, or integration-facing docs ([INTEGRATING.md](../../../INTEGRATING.md)) +- Modifying the Warlock's public API, `scan()` signature, or the integration-facing sections of [README.md](../../../README.md) - Reviewing a Warlock PR ## Non-negotiables @@ -67,7 +66,6 @@ Every PR description must state (a) the problem this change addresses in one to ## Further reading -- [README.md](../../../README.md) – charter, audience, design decisions, API stability, public API reference +- [README.md](../../../README.md) – charter, audience, design decisions, API stability, public API reference; update the API reference when changing the public API - [CONTRIBUTING.md](../../../CONTRIBUTING.md) – contribution process, rule-writing guide, category-addition policy -- [INTEGRATING.md](../../../INTEGRATING.md) – guide for engineers integrating the Warlock into a consumer application; update this when changing the public API - [.github/pull_request_template.md](../../../.github/pull_request_template.md) – PR template used for every Warlock PR diff --git a/README.md b/README.md index 4f839e2..b5005c6 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ the Warlock is the content-scanning engine behind PostHog's agentic tooling. You **Who this is for:** -- PostHog's [wizard](https://github.com/PostHog/wizard) – the Warlock replaces the hand-rolled JS-regex scanner at `src/lib/yara-scanner.ts` with real YARA rules. +- PostHog's [wizard](https://github.com/PostHog/wizard) – the Warlock replaced the wizard's hand-rolled JS-regex scanner with real YARA rules ([wizard#804](https://github.com/PostHog/wizard/pull/804)); the wizard wires it into its agent hooks via `src/lib/yara-hooks.ts`. - The context mill and other PostHog context infrastructure. - Future PostHog-internal consumers that need to scan untrusted content flowing through AI agents. @@ -95,13 +95,13 @@ When an implementation choice forces a trade-off between "ugly code in the Warlo - **Why:** the Warlock has one maintainer. Consumers have many call-sites. Paying the complexity tax once is way cheaper than paying it over and over. - **Examples:** The CommonJS / ESM bridge (below) and metadata normalization (below). -### CommonJS package – ESM-only yara-x bridged via dynamic `import()` +### ESM package – yara-x loaded lazily via dynamic `import()` -the Warlock ships as a CommonJS package. +the Warlock ships as an ESM package (`"type": "module"`; it migrated from CommonJS in v0.2.x). -- **Why:** CJS maximizes compatibility with consumers that haven't migrated to ESM. CJS can be imported from both CJS and ESM contexts – the reverse is awkward. Keeping the Warlock CJS concentrates the bridging code in a single file (`src/scanner/engine.ts`), so consumers don't have to deal with it themselves. We can add a dual CJS/ESM build later if it's needed. -- **Cost:** yara-x is ESM-only, so the Warlock uses a dynamic `await import('@virustotal/yara-x')` inside `loadRules()`. This is why `scan()` is async – the dynamic import returns a Promise. -- **When to revisit:** If future consumers are ESM-native, or Node itself deprecates CJS, this trade-off might flip. +- **Why lazy:** yara-x's WASM engine is expensive to initialize and its rules are compiled on first use. Loading it with a dynamic `await import('@virustotal/yara-x')` inside `loadRules()` keeps `import '@posthog/warlock'` itself cheap and synchronous – consumers pay the engine cost on the first `scan()`, not at import time. This is why `scan()` is async. +- **Cost:** ESM-only means CommonJS consumers can't `require()` the Warlock – they need a dynamic `import()` of their own. Both current consumers (the wizard, the context mill) are fine with that. +- **When to revisit:** If a CJS-only consumer appears, a dual build could be added. ### Why `yara-x` over classic YARA @@ -250,7 +250,8 @@ type Category = | 'destructive_operations' | 'supply_chain' | 'posthog_pii' - | 'posthog_hardcoded_key'; + | 'posthog_hardcoded_key' + | 'hardcoded_secret'; type Severity = 'critical' | 'high' | 'medium' | 'low'; diff --git a/src/scanner/__tests__/rules.test.ts b/src/scanner/__tests__/rules.test.ts index 19c7a1b..5a95e46 100644 --- a/src/scanner/__tests__/rules.test.ts +++ b/src/scanner/__tests__/rules.test.ts @@ -11,9 +11,12 @@ * - "metadata" (required-field contract) * * Category sections in use: - * ┌─ PostHog API (posthog_pii, posthog_hardcoded_key) - * └─ (prompt_injection, exfiltration, destructive_operations, - * supply_chain – coming in future PRs) + * └─ PostHog API (posthog_pii, posthog_hardcoded_key) + * + * Every other category (prompt_injection, exfiltration, + * destructive_operations, supply_chain, hardcoded_secret) has since moved + * to one file per rule under `__tests__/rules/` – add new rule tests + * there, not here. */ import { describe, it, expect } from 'vitest';