Skip to content

P6 false positive on HTML Output Rules documentation heading #512

Description

@chrisknvidia

P6 reports a harmless HTML report-formatting heading as a HIGH-severity Direct Prompt Extraction finding.

Reproduced on main at 69dcdfb74487d361ba4c811d088cfdea2ff3a9dc (2.11.2) with this SKILL.md:

---
name: report-example
description: Generate a local HTML report from user-provided data.
---

# Report Example

## HTML Output Rules (Offline-Safe)

- Include figure titles, axis labels, and legends.
- Keep the generated HTML usable without network access.

Run:

skillspector scan ./report-example --no-llm --format json --output report.json

The report contains P6 at SKILL.md:8, matched text Output Rules, severity HIGH, confidence 0.85. The scan completes successfully; this finding contributes a risk score of 21 (CAUTION).

The heading describes the generated report's formatting requirements. It does not request disclosure of the assistant's system prompt or hidden instructions. P6 currently allows both your and system to be absent, so it interprets the noun phrase as an imperative. The existing exception for exactly ## Output Rules (Both Modes) does not cover this heading.

Expected behavior:

  • Recognize unambiguous report-formatting headings without a P6 finding.
  • Preserve actual prompt/rule extraction instructions, including instructions in headings, code blocks, and adjacent text.
  • Preserve finding locations, accounting, and JSON/Markdown/SARIF output correctness.
  • Add paired benign and malicious regressions and verify the complete CLI scan, not only the regex.

Related: #37 covers documentation false positives broadly; #453 addresses a different nominal print rules phrase. This issue tracks the report-formatting heading case specifically.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions