Skip to content

Replace #63 stopgap in posthog_feature_attack (tighten TP shape) + consumers must not block trusted first-party skills untriaged #65

Description

@sarahxsanders

Summary

Two separate problems surfaced when prompt_injection_posthog_feature_attack blocked a first-party skill install. #63 (shipped in 0.2.4) was a stopgap aimed at one phrasing; it does not resolve either root cause. This issue tracks both.

Problem 1 — the rule over-fires on non-instruction framing (rule-level)

The rule matches a verb + feature substring (disable autocapture) anywhere, regardless of the surrounding grammar. That catches informational and even anti-injection prose. Two real first-party shapes trip it:

  1. Permissive-modal config docs — "You can disable autocapture entirely by setting autocapture to false." (fix(rules): don't flag config-doc opt-out framing in posthog_feature_attack #63's $benign_config_doc guard suppresses this one.)
  2. Negated / prohibitive guidance — "Do not disable autocapture, do not disable session recording." This is guidance to keep the feature on, but the rule matches the disable autocapture substring. fix(rules): don't flag config-doc opt-out framing in posthog_feature_attack #63 does not cover this — it still fires.

#63's guard is also a whole-buffer negative match (any of ($attack_*) and not $benign_config_doc): because a bare string in the condition is true if it matches anywhere, any content containing the benign phrase suppresses the entire rule. A real injection can append a benign-shaped clause to evade. It also pushes consumer-context judgment into the engine.

Proposed direction (rule)

Replace the negative guard with a tightened true-positive shape: require actual instruction framing so neither permissive-modal ("you can disable X") nor negated ("do not disable X") prose matches, while bare imperatives ("Disable X.") still do.

Constraint: yara-x's regex engine (Rust-based) is not expected to support lookbehind/lookahead — confirm during implementation. If so, "match disable X unless preceded by you can / do not" can't be expressed directly; use an imperative/clause-boundary lead-in instead.

Problem 2 — a consumer can hard-block trusted first-party skills (the bigger half)

Warlock only detects; consumers decide what a match means. A consumer's skill-install scan currently terminates on a match without the false-positive triage pass that other scan paths use, so any benign match on trusted first-party skill content blocks the install. Tightening the rule (Problem 1) reduces the noise but does not fix this — the next benign match on any skill re-triggers it. The guarantee that first-party skills don't get blocked lives in the consumer wiring, not the rule.

The consumer-side fix (ensure the install path triages, and/or treat genuinely first-party skills by provenance) is tracked and PR'd in the consumer repo. Design constraint carried there: a trusted-source bypass must key off genuine first-party provenance, not a loose path pattern, or a real injection planted at that path would go unscanned.

Acceptance criteria

Rule (this repo):

Consumer (tracked in the consumer repo, noted here for context):

  • A benign match on trusted first-party skill content does not hard-block/terminate the install without triage.
  • Any trusted-source bypass keys off first-party provenance, not a loose path pattern.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions