Add closed shadow DOM capture via CDP

[aryanku-dev]

Summary

• Add expose_closed_shadow_roots() using CDP to discover and expose closed shadow roots
• Add _walk_nodes() helper for CDP DOM tree traversal
• Add 6 unit tests covering tree walking, non-Chromium skip, CDP errors

Ported from percy/percy-playwright#609

Test plan

• Unit tests pass
• Verify no regression on pages without shadow DOM

🤖 Generated with Claude Code

[github-actions]

This PR is stale because it has been open for more than 14 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[pranavz28]

Cross-SDK review across the 6 ports of the iframe + closed-shadow-DOM rollout. This PR has the narrowest scope (closed-shadow-DOM only — no CORS iframe work), which is fine. Two HIGH findings inline + one MEDIUM around responsive-reload re-exposure.

Verified-pass items

• Hardcoded JS in Runtime.callFunctionOn — no user-controlled string interpolation, low security risk.
• 7 unit tests covering tree walking, non-Chromium skip, CDP errors, detach-error suppression.
• No module-level shared state; concurrent snapshots on different pages are isolated by WeakMap key.
• cache.py defensive cleanup (drive-by) is safe.

Verdict

Request changes. Two HIGH issues to align with selenium-dotnet #436's closed-shadow port (same-origin iframe walker + DOM.disable pairing). One MEDIUM around responsive-reload re-exposure (selenium-ruby #32 got this right; Python should mirror).

[pranavz28]

HIGH — Walker skips ALL iframes; selenium-dotnet #436 had the same initial bug and was fixed on CE re-review.

if "contentDocument" in node:
    return

Same-origin iframes share the parent JS realm and the window.__percyClosedShadowRoots WeakMap that PercyDOM.serialize reads. Closed shadow roots inside same-origin iframes SHOULD be captured. Cross-origin iframes should still be skipped (different JS realm; resolveNode objectIds wouldn't belong to our execution context anyway).

selenium-dotnet's fix (commit fix(snapshot): recurse closed shadow walk into same-origin iframes):

• Same origin (GetOrigin(documentURL) == pageOrigin) → recurse INTO contentDocument.
• Cross origin → skip.
• Missing documentURL → defensive skip.

Suggested Python port:

if "contentDocument" in node:
    cd = node["contentDocument"]
    document_url = cd.get("documentURL")
    if not document_url:
        return
    if _same_origin(document_url, page_url):
        _walk_nodes(cd, closed_pairs)
    return

_same_origin should use urllib.parse.urlparse to compare scheme + host + port. Add tests for all three branches (same-origin captured, cross-origin skipped, missing URL skipped) — mirroring the dotnet test suite added in that PR.

[pranavz28]

HIGH — DOM.enable is called but DOM.disable is never invoked.

selenium-dotnet #436 and selenium-ruby #32 both pair enable/disable explicitly in their try/finally and gate disable on whether enable succeeded (so a failing enable doesn't emit a spurious disable).

cdp_session.detach() in the finally block does mostly release the domain in practice, but the explicit pairing is the documented pattern and matches every sibling SDK that uses CDP.

Suggested fix:

dom_enabled = False
try:
    cdp_session.send("DOM.enable")
    dom_enabled = True
    # ... existing walk + resolve + register ...
finally:
    if dom_enabled:
        try:
            cdp_session.send("DOM.disable")
        except Exception:
            pass
    try:
        cdp_session.detach()
    except Exception:
        pass

Also add a test mirroring dotnet's DOM.disable lifecycle suite: DOM.disable is sent after a successful run, sent after a mid-walk failure, NOT sent when DOM.enable itself raised.

[pranavz28]

MEDIUM — expose_closed_shadow_roots is called once but the WeakMap is wiped on responsive reloads.

Responsive captures reload the page at each width and re-evaluate percy_dom_script. The window.__percyClosedShadowRoots WeakMap is gone after page.reload(), so closed shadow roots will be missing in responsive DOMs at widths beyond the first.

selenium-ruby #32 explicitly re-primes after refresh in its responsive snapshot loop (lib/percy.rb:549). Search every page.evaluate(percy_dom_script) and page.reload() site in this file and precede each with expose_closed_shadow_roots(page).

Add a unit test that drives the responsive capture path through change_window_dimension_and_wait and asserts the WeakMap is re-populated at the second width.

[github-actions]

This PR is stale because it has been open for more than 14 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[aryanku-dev]

🤖 stack:pr-review — Automated review (BrowserStack AI harness)

Summary: Adds closed-shadow-DOM capture via CDP for Chromium (expose_closed_shadow_roots + _walk_nodes/_get_origin/_same_origin), recursing into same-origin iframe content documents while skipping cross-origin; re-primes the __percyClosedShadowRoots WeakMap after responsive reloads; hardens Cache.cleanup_cache against missing keys / concurrent mutation.

Priority	Category	Check	Status
High	Security	No hardcoded secrets	Pass
High	Security	Input validation	Pass — _get_origin parses defensively, returns "" on bad URLs; origin compared scheme+netloc
High	Security	AuthZ / IDOR / SQLi	N/A
High	Correctness	Logic & edge cases	Pass — same-origin recursion matches dotnet canonical; cross/unknown-origin skipped; empty closed_pairs early-return
High	Correctness	Explicit error handling	Pass — failures debug-logged; # pragma: no cover only on best-effort DOM.disable/detach cleanup
High	Correctness	No race conditions	Pass — cleanup_cache iterates a list(...) copy + .get() — fixes dict-mutation race
Medium	Testing	New code / error paths tested	Pass — TestClosedShadowDOM (12) + cache orphan + responsive re-prime; DOM.disable covered in all 3 branches
Medium	Testing	Existing tests pass	Pass — 95 tests OK, 100% branch coverage
Medium	Performance	No unbounded fetch	Pass — DOM walked once; CDP bounded by closed-root count
Medium	Quality	Patterns / focused	Pass — faithful port of percy-selenium-dotnet canonical
Low	Quality	Names / comments / deps	Pass — stdlib urllib.parse only

Findings: no Fail. Non-blocking note: dotnet wraps each (host, shadow) pair in its own try/except; the Python version relies on the outer try (first failing pair stops the loop) — still safe/logged/non-fatal and matches the canonical JS, left as-is to keep the diff minimal.

Verification: pylint 10.00/10; full unittest suite 95 OK; coverage 100%. The three original reviewer threads (same-origin iframe recursion, DOM.disable pairing, WeakMap re-prime) all hold.

Commits: the fixes are at e7e8626 (already on PER-7292); nothing further needed this pass.

Overall: ✅ Pass — Approve. No human-decision blockers.

_{Posted by the BrowserStack stack:pr-review harness.}