Add opt-in structured XML exemption for Europass CV PDFs - #658
Conversation
C2paManifestVerifier did two jobs: resolve every embedded stream the way a PDF reader would, and judge whether the bytes are a C2PA manifest. The structural walk becomes EmbeddedFileVerifier and the JUMBF logic moves unchanged into C2paPayloadValidator, behind a new EmbeddedPayloadValidatorInterface. Every embedded stream must now be accepted by at least one supplied validator. The walk also reads each file specification's /F and /UF name, taken from the innermost dictionary around the /EF, so validators can match on it. Names are optional, since C2PA signers often write neither. A name that is present but unreadable, duplicated, non-ASCII, or that differs between /F and /UF rejects the file, and so does one stream reached under two different names. No behaviour change for C2PA: the existing harnesses pass unmodified apart from class names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Europass CV builder attaches a machine-readable copy of every CV it exports as an embedded XML file, so the PDF scanner rejected all Europass CVs. The real export has no /Subtype, /AFRelationship or /AF, so only the payload bytes can identify it. StructuredXmlPayloadValidator accepts an embedded file only when its file specification name is an exact key in FILE_UPLOAD_PDF_STRUCTURED_XML_ALLOWLIST (attachment.xml, Europass-XML-Attachment.xml) and the payload: - is at most 5 MB, - is valid UTF-8 with no control bytes, - contains no DOCTYPE or ENTITY, - parses completely with XMLReader under LIBXML_NONET, - has the allowlisted root element and namespace. Opt-in through the new fileSecurityPdfAllowStructuredXml filter, which is off by default, compared with === true and requires qpdf, exactly as fileSecurityPdfAllowC2pa does. The diagnostics tab gains an xmlreader row. tests/security/verify-structured-xml.php is a dependency-free harness. Its fixtures are synthetic PDFs shaped like a real Europass export, so no real CV is committed. It covers: - name tricks, - spoofed roots and namespaces, - XXE and DTDs, - appended payloads, - executables under an allowlisted name, - mixed attachments, - the size cap and cost bounds, - the scanner's decision with and without the filter and qpdf. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- EmbeddedFileVerifier::fileSpecName() now returns null (not false) for an unreadable /F or /UF, so a C2PA-only site no longer rejects a valid manifest just because its file name is non-ASCII, empty, or the /F and /UF values disagree. Only the XML validator needs a name, and it already rejects null. - Extract PDF literal/hex string reading (skipLiteral, readAsciiText) into a new PdfStrings helper, and the /EF owner-dictionary lookup into innermostOpenDictionary(), to keep EmbeddedFileVerifier focused on structure. - Config::FILE_UPLOAD_PDF_STRUCTURED_XML_ALLOWLIST entries use named root/namespace keys instead of positional tuples. - Add regression checks in verify-c2pa.php for the name-handling fix (absent, ASCII, non-ASCII UTF-16, mismatched, and empty names). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- EmbeddedFileVerifier: replace the by-reference innermostOpenDictionary() walk with owningDictionaryOffsets(), a pure single-pass lookup over all /EF offsets in a region. - Document that name pinning across two file specifications applies whichever payload validators are enabled, and add C2PA coverage for a stream reached under one name twice vs. two different names. - FileSecurityDiagnostics: mark the xmlreader extension row optional, matching the qpdf row. - Config: collapse the C2PA/structured-XML exemption docblock to one paragraph pointing at EmbeddedFileVerifier and its validators, rather than repeating their per-class docs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Move the shared "structure only, not a safety proof" warning to the EmbeddedPayloadValidatorInterface docblock instead of repeating it in both C2paPayloadValidator and StructuredXmlPayloadValidator, and shorten several comments in EmbeddedFileVerifier and StructuredXmlPayloadValidator that restated what the adjacent code already shows. No behavioural change; all three security test harnesses still pass (57/22/56 checks). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
goranalkovic-infinum
left a comment
There was a problem hiding this comment.
The Europass exception looks well scoped, and keeping it off by default is a good choice. I’d request two changes before merging:
-
Fix repeated scanning of nested dictionaries. The new filename lookup repeatedly scans the same content when PDF dictionaries are nested. With a roughly 1 MB PDF containing 100 nested dictionaries, verification increased from 0.09s to 8.48s after qpdf processing. This affects the existing C2PA exception too and could let repeated uploads tie up server workers. Please collect filenames in one pass or cap the processing work, and add a regression test.
-
Run the security tests in CI.
composer testdoesn’t execute the three security harnesses, so GitHub can stay green even if attachment validation breaks. Please run them in CI with generated fixtures, qpdf, and the required PHP extensions.
piqusy
left a comment
There was a problem hiding this comment.
Reproduced both open findings locally (PHP 8.4.25, qpdf 12.4.1, xmlreader loaded). All three harnesses pass on this branch. Requesting changes on the same two points, with measurements.
1. Nested dictionaries re-scan the region per level (perf regression)
collectEmbeddedFileReferences() in src/Validation/FileSecurity/EmbeddedFileVerifier.php caches owner names by $ownerStart, so a body where every nesting level owns an /EF re-runs readDictionary() plus fileSpecName() (a mask() byte loop and two preg_match_all passes) over the remaining region for each level. The "read each owner once" comment only holds when owner dictionaries are small. The inline-filespecs case in the harness stays linear; this shape does not. It also affects C2PA-only opt-ins, because name reading happens regardless of which validators are enabled.
Measured on the same 0.95 MB body with 100 nested dictionaries and one embedded stream, identical verdict on both sides:
| code | time |
|---|---|
C2paManifestVerifier (release/10.6.0) |
0.08s |
EmbeddedFileVerifier (this branch) |
7.41s |
That is ~93x. Fix direction: record each << extent during the existing bracket walk and derive the name in one pass, or cap cumulative name-scan work and fail closed to name = null. The second keeps semantics intact: the C2PA validator ignores names, and the XML validator then rejects, which is the same treatment a readable-but-unverified name already gets. Please also add a timed nested-dictionary case to tests/security/verify-structured-xml.php; the cost section currently covers inline filespecs and many references to one stream, not nesting.
2. Security harnesses run nowhere in CI
.github/workflows/ci.yml runs composer lint and node lint only. composer test excludes tests/, and the pre-commit hook skips tests-only PHP commits (noted in the PR description). These harnesses are the only executable tests for this code, and the Playwright E2E never reaches the PDF verdicts. Please add a CI job that runs the three harnesses directly:
php tests/security/verify-c2pa.php
php tests/security/verify-structured-xml.php
php tests/security/verify-pdf-scanner.php
They need no vendor directory, they need dom, xmlreader and qpdf, and the qpdf-dependent checks skip cleanly when qpdf is absent, so a plain setup-php runner works.
Verified clean on this branch
Name pinning (one stream reached under two names rejects), /F + /UF disagree, duplicate and non-ASCII rejection, UTF-16BE hex names, XXE / DOCTYPE / SettingContent-ms / wrong-namespace rejection, DTD search on raw bytes before libxml, LIBXML_NONET with no LIBXML_NOENT, and the existing /RF, /ObjStm, /Filter and endstream checks. array_any is fine given "php": ">=8.4" in composer.json.
Each nesting level that owns an /EF re-read the dictionaries inside it, so a ~1 MB body with 100 nested Filespecs took 7.5s to verify. Name reads are now capped at twice the region; owners past the cap go unnamed, which the structured XML validator rejects and C2PA ignores. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
composer test never executes tests/security, so attachment validation could break with CI green. The new job installs qpdf, generates the fixtures, runs all three harnesses and fails on any SKIP, so a missing qpdf or fixture cannot pass silently. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An owner left unnamed by the name budget also fails the one-stream, one-name rule when the same stream is reached from a named owner, so the cap can reject a C2PA-only upload too, not just a structured XML one. PdfStrings has a single caller, so its header no longer claims several scanners share it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
piqusy
left a comment
There was a problem hiding this comment.
Both review findings are addressed and I verified the fixes locally (PHP 8.4.25, qpdf 12.4.1, xmlreader loaded), so approving.
1. Name-reading cap (afb61859). Measured on the same 0.95 MB body with 100 nested dictionaries and one embedded stream that reproduced the original regression:
| code | time | verdict |
|---|---|---|
| release/10.6.0 | 0.08s | accepted |
| f144724 | 7.41s | accepted |
| fa56125 | 0.27s | rejected, fail-closed |
Cost is linear again. The new 100 nested Filespecs around 1 MB case in tests/security/verify-structured-xml.php passes in 0.27s against a 1.0s budget and pins the fail-closed verdict, so a future regression fails the harness. I also checked the edge where an exhausted budget makes the read limit drop below the owner start: readDictionary() returns null immediately, the capped path records an unnamed owner, and the $limit === strlen($region) guard still rejects a genuinely unreadable uncapped owner. One behavior change worth restating for the record: a stream reached from both a named owner and a capped owner now fails the one-stream, one-name rule, including for C2PA-only opt-ins. Fail-closed and deliberate, fine by me.
2. CI coverage (250ea3dd). The security job installs the right extensions and qpdf, generates fixtures through the existing tests/security/generate-test-files.sh, and fails on both harness failures (bash -e plus pipefail) and skips (the grep '^SKIP' guard). Verified the YAML parses and all three harnesses pass on this head.
Two non-blocking nits are filed in #659; happy either way, they can ride in a follow-up PR if you agree they are worth it.
Description
This adds an opt-in exemption so the PDF scanner stops rejecting Europass CVs, whose builder attaches a machine-readable XML copy of every CV it exports.
Why
attachment.xml, a<Candidate>document in thehttp://www.europass.eu/1.0namespace./Subtype,/AFRelationshipor/AF, so only the payload bytes can identify it./EmbeddedFilewas only exempt for C2PA manifests, so every Europass CV came back asvalidationFilePdfUnsafe.Embedded-file verifier refactor
C2paManifestVerifiertoEmbeddedFileVerifier, which keeps only the structural walk.C2paPayloadValidator.EmbeddedPayloadValidatorInterface(maxBytes(),accepts($payload, $name))./ObjStm,/RF, duplicate objects,/Filter, theendstreamextent check and the backstop.File specification names
/Fand/UFfrom the innermost dictionary that contains the/EF, with a single linear pass over the brackets./Fand/UF.attachment.xml, saved aspayload.exe".Structured XML exemption
StructuredXmlPayloadValidator, which checks in this order:attachment.xml,Europass-XML-Attachment.xml)<!DOCTYPEor<!ENTITYin the raw bytesXMLReaderparse withLIBXML_NONETConfig::FILE_UPLOAD_PDF_STRUCTURED_XML_ALLOWLISTandFILE_UPLOAD_PDF_STRUCTURED_XML_MAX_BYTES.fileSecurityPdfAllowStructuredXmlfilter, registered inFilters.php.trueand needs qpdf, the same asfileSecurityPdfAllowC2pa.xmlreaderrow to the File security diagnostics tab.Deliberately not exempted
Original.odt), which are ZIPs that can carry macros.attachfile/embedfile.Accepted residual risk
/JS,/Launch,/RichMediaand the other blocked keys still apply to the whole PDF.Tests
verify-c2pa.phpandverify-pdf-scanner.phpchanged only in class names, and every existing case passes.tests/security/verify-structured-xml.phpharness has 59 checks, and its synthetic fixtures mean no real CV is committed.1, and without qpdf.composer test,bun run lintJsandbun run lintStyleall pass.Changelog (for the release)
fileSecurityPdfAllowStructuredXmlfilter. When enabled, the PDF scanner accepts Europass CVs, whose only dangerous key is the embedded machine-readable CV XML. The file name must be on an allowlist and the payload must parse as the expected Europass document with no DTD. Disabled by default and requires qpdf, likefileSecurityPdfAllowC2pa.Known follow-ups
SkillsPassportformat against a real 2013–2020 export.tests/PHP files, because PHPStan excludestests/.🤖 Generated with Claude Code