Add an axe baseline for known third-party and upstream findings by cwickham · Pull Request #2208 · quarto-dev/quarto-web

cwickham · 2026-09-25T15:27:44Z

Summary

This PR adds _axe-baseline.json, the baseline file for quarto call axe (quarto-dev/quarto-cli#14815). A scan compares its findings with the baseline. It reports only the findings that are not in the baseline as new.

The baseline has 118 signatures. Each one is a finding that quarto-web cannot fix in its own content, or a false positive:

third-party: the markup comes from a library, for example Observable Plot or leaflet.
upstream: the markup comes from Quarto, and a quarto-cli issue tracks the fix.
false-positive: axe reports a defect that a user does not see.

Each entry has a note that gives the cause, the owner, and the condition to remove the entry. Each note is complete by itself, so the context stays if another entry is removed.

Most entries apply to all pages. An entry is limited to specific pages when its signature can also match a real defect in quarto-web content. Then a new defect of that type on a different page still shows as new.

Entries

Upstream: Quarto

Signature	Count	Issue	WCAG 2.2
`scrollable-region-focusable :: #cb` (all pages)	1 signature	quarto-dev/quarto-cli#14378: scrollable code blocks have no `tabindex`. Fixed by quarto-dev/quarto-cli#14816	2.1.1 Keyboard
Other `scrollable-region-focusable` code blocks and cell outputs (only the pages that have them)	36 signatures, 64 instances	quarto-dev/quarto-cli#14378. Fixed by quarto-dev/quarto-cli#14816	2.1.1 Keyboard
`aria-allowed-attr` on collapsed callout headers (all pages)	3 signatures, 13 instances	quarto-dev/quarto-cli#4934: collapsed callouts use a `<div aria-expanded>`, not the disclosure pattern	4.1.2 Name, Role, Value
`image-alt` on `{{< placeholder >}}` images (`docs/authoring/placeholder.html` only)	2 signatures, 2 instances	quarto-dev/quarto-cli#14769: the shortcode makes an `<img>` with no `alt`	1.1.1 Non-text Content
`aria-required-parent :: #ref-xie` (`docs/authoring/article-layout.html` only)	1 instance	quarto-dev/quarto-cli#14755: margin citations keep `role="listitem"`, but Quarto removes their `role="list"` parent	1.3.1 Info and Relationships
`frame-title` on `code-preview` iframes (`docs/presentations/revealjs/index.html`, `advanced.html`, and `docs/authoring/code-annotation.html` only)	18 signatures, 35 instances	quarto-dev/quarto-cli#14770: `code-preview` makes an `<iframe>` with no `title`. Fixed by quarto-dev/quarto-cli#14933	4.1.2 Name, Role, Value
`meta-viewport :: meta` (all pages)	1 signature	quarto-dev/quarto-cli#14952: the revealjs template sets `maximum-scale=1.0, user-scalable=no` on every deck	1.4.4 Resize Text
`color-contrast` on faded lines in revealjs line highlighting (`docs/presentations/revealjs/examples/line-highlighting-1.html` and `line-highlighting-2.html` only)	3 signatures, 28 instances	quarto-dev/quarto-cli#14959: `code-line-numbers` fades the other lines to `opacity: 0.4`. No opacity value meets 4.5:1 and still shows the highlight	1.4.3 Contrast (Minimum)
`aria-prohibited-attr` on revealjs code line numbers (`docs/presentations/revealjs/demo/index.html`, `demo/mini/auto-animate-code.html`, `examples/code-echo.html`, and `examples/line-highlighting-1.html` to `line-highlighting-4.html` only)	2 signatures, 87 instances	quarto-dev/quarto-cli#14961: revealjs removes the `href` from line-number links, so their `aria-label` is on an `<a>` that is not a link	4.1.2 Name, Role, Value
`scrollable-region-focusable` on revealjs slides, the slide menu, and a code block (`body`, `#slide`, `#scrollable-slide`, `.active-menu-panel`, `.latex`; 26 revealjs decks only)	5 signatures, 30 instances	quarto-dev/quarto-cli#14817: revealjs slides get `overflow-y: auto`, and the arrow keys move between slides instead of scrolling, so keyboard users cannot reach overflowing content	2.1.1 Keyboard
`color-contrast :: #2a76dd on #ffffff` (`docs/presentations/revealjs/demo/index.html` only)	1 instance	quarto-dev/quarto-cli#14962: the default revealjs `$link-color` is 4.44:1 on white, which fails for small text such as the footer	1.4.3 Contrast (Minimum)
`color-contrast :: #2a76dd on #bbbbbb` (`docs/presentations/revealjs/examples/tabset.html` only)	1 instance	quarto-dev/quarto-cli#14963: below 30em, the selected revealjs tab has a `#bbbbbb` fill, so its text is 2.31:1	1.4.3 Contrast (Minimum)

The #cb entry came from the baseline of the old axe harness on feat/axe-a11y-harness. It was the only one of 27 old entries with a finding that still occurs and a note that is still true.

All 37 quarto-dev/quarto-cli#14378 entries clear when quarto-dev/quarto-cli#14816 ships. A scan of the 144 affected pages, rendered with quarto-cli main and then with #14816 merged on top, removed these 37 signatures and added no scrollable findings. The 36 new entries are limited to their pages because signatures such as scrollable-region-focusable :: pre also match revealjs slides, which #14816 does not change. Of the 10 scrollable signatures that #14816 does not clear, the 5 revealjs signatures are in the baseline under quarto-dev/quarto-cli#14817, and the Arquero notebook table is under third-party. The visually hidden code copies are not in the baseline.

The frame-title entries clear when quarto-dev/quarto-cli#14933 ships, because #2220 adds a code-preview-title to every preview. They are limited to the pages that have them because signatures such as frame-title :: iframe also match untitled video embeds, which quarto-web can fix.

The line-highlighting entries are limited to the two example decks because signatures such as color-contrast :: #bbbbbb on #ffffff can also match gray text in quarto-web content. These entries do not have a removal condition yet. quarto-dev/quarto-cli#14959 asks for documentation or a different highlight method, not a fix to the fade.

The two line-number signatures are one defect. axe adds #cb to the selector when a page has more than one code block, or copies of a block for stepped highlighting. The entries are limited to the seven decks because #cb > a[aria-label="*"] can also match line numbers on format: html pages, and a[aria-label="*"] can match any labeled link. They clear when quarto-dev/quarto-cli#14961 is fixed.

The quarto-dev/quarto-cli#14817 entries are limited to their decks because signatures such as scrollable-region-focusable :: body can also match a page in quarto-web content. body is 25 of the 30 instances: at 320x568, reveal.js switches to scroll view, and the whole page scrolls. The barrier is real, because a short viewport also brings back the slide scrollbar. The .latex code block did not reproduce in a local render, so this entry can disappear from a scan. The entries clear when quarto-dev/quarto-cli#14817 is fixed.

The two revealjs link-colour entries are limited to their decks because #2a76dd is also the default link colour of other revealjs content that quarto-web can override. They clear when quarto-dev/quarto-cli#14962 and quarto-dev/quarto-cli#14963 are fixed. axe flags both only at 320x568. At wider viewports reveal.js scales the slide, and axe reports the contrast as incomplete.

The image-alt signatures are limited to the placeholder page because p > .img-fluid also matches content images on other pages that need alt text.

Third-party: Observable Plot (20 signatures, 54 instances, all pages)

Rule: aria-prohibited-attr, 4.1.2 Name, Role, Value.

Plot puts an aria-label on the <g> of every mark and axis, with no role. There are 20 signatures because the same label occurs under different selector paths. The signatures are in two groups:

10 axis-tick and grid signatures (30 instances). Plot 0.6.14 removed these (fix: hide axis ticks from aria since they are just adding noise observablehq/plot#2018). Quarto bundles Plot 0.6.11. A scan of the same pages with Plot 0.6.17 removed all 10 signatures and added no findings. Update the bundled Observable Plot and Inputs libraries quarto-cli#14934 asks Quarto to update Plot. When Quarto ships a newer Plot, remove these entries.
10 data-mark signatures (24 instances): rect, frame, line, rule, tick. These still occur in Plot 0.6.17. Plot labels marks on purpose (ariaLabel, ariaDescription, ariaHidden observablehq/plot#710, [a11y] Accessibility issue with plot rect observablehq/plot#944). The open request for a role is SVGGElements without text content are not meaningful to user agents observablehq/plot#1760.

Third-party: Observable Inputs (6 signatures, 60 instances on 11 pages)

Rule: label, 4.1.2 Name, Role, Value.

Inputs.range sliders (3 signatures, 12 instances). Every call in quarto-web gives a label. But Inputs connects the <label for> to the number box only (range.js L57). The slider has no accessible name.
Inputs.table selection checkboxes (3 signatures, 48 instances). The row and header checkboxes have no label, and no option adds one (table.js L100-L101). Inputs 0.11.0 added select: false, which removes the checkboxes. But then the scroll container of the table has no focusable content, and axe reports scrollable-region-focusable (2.1.1 Keyboard).

Scans with Inputs 0.11.0 and 0.12.0 gave the same 60 instances. No issue in observablehq/inputs reports these defects.

The two .oi-3a86ea-input signatures are specific to Inputs, so they apply to all pages. When Quarto updates Inputs (quarto-dev/quarto-cli#14934), the class changes to .inputs-3a86ea-input and these two entries need the new selector. The other four signatures can match any unlabeled input, so they apply only to the pages that have them.

Third-party: leaflet (8 signatures)

aria-allowed-role and presentation-role-conflict on .leaflet-marker-icon (9 instances). leaflet.js gives each marker <img> role="button" and tabindex="0", but also alt="". These two rules are axe best practices, not WCAG failures. Three of the signatures are specific to the three maps on docs/authoring/article-layout.html.
color-contrast on the attribution control (2 signatures, 4 instances), 1.4.3 Contrast (Minimum). leaflet.css has no dark theme.
link-in-text-block :: a on the ESDIS link in the attribution of the map tiles (2 instances), 1.4.1 Use of Color. The link is #0078a8 on the #343a40 page text, 2.32:1. This entry applies only to docs/interactive/index.html and docs/interactive/widgets/jupyter.html. The signature drops the href, so it can also match any other plain link that has low contrast.

No leaflet issue matches these findings exactly.

Third-party: dygraphs (1 signature, 15 instances, `docs/interactive/widgets/htmlwidgets.html` only)

Rule: color-contrast :: #000000 on #151515, 1.4.3 Contrast (Minimum).

dygraph.js sets the color of the axis labels in an inline style, so CSS cannot change it. dygraph.js 2.0 moved the color into a stylesheet rule. But the R dygraphs package still bundles dygraph.js 1.1.1 (rstudio/dygraphs#241).

This entry applies to htmlwidgets.html only. The same signature also occurs on docs/authoring/penguins-preview.html, where it is a Quarto defect in the notebook cell labels.

Third-party: the `@uwdata/arquero` Observable notebook (1 signature, 2 instances, `docs/interactive/ojs/examples/arquero.html` only)

Rule: scrollable-region-focusable :: #ojs-cell > .observablehq > div, 2.1.1 Keyboard.

The example imports Arquero from the @uwdata/arquero Observable notebook, not from npm. The notebook adds a .view() method to tables. This method puts the table in a <div style="max-height: 270px; overflow: auto"> that has no tabindex, so a keyboard user cannot scroll it. The notebook is the correct place for the fix. The notebook is not in a repo, so no upstream issue is filed.

Third-party: vega-embed (1 signature, 4 instances on 4 pages)

Rule: summary-name :: summary, 4.1.2 Name, Role, Value.

vega-embed adds an actions menu to every Altair and Vega chart. The menu is a <details> element. Its <summary> contains only an icon SVG, so axe finds no name. The label ("Click to view actions") is a title on the <details>. VoiceOver reads this label, so no upstream issue is filed.

This entry applies only to the four pages with Altair charts: docs/authoring/notebook-embed.html, penguins-preview.html, penguins.html, and docs/presentations/revealjs/examples/executable-code-figure-size.html. The signature can also match any other <summary> that has no text.

Third-party: knitr and Arquero tables (6 signatures)

empty-table-header on knitr::kable() output (5 signatures, 7 instances). This rule is an axe best practice. kable() leaves the corner cell above the row names as an empty <th> on purpose. feature request: more accessibility/navigation tags in kable_html yihui/knitr#1747, merged as kable(): accessibility scope attributes in HTML tables (#1747) yihui/knitr#2500, added scope to the other header cells but kept the corner cell empty.
color-contrast :: #f8f8f8 on #ffffff in the Arquero table viewer (1 signature, 22 instances), 1.4.3 Contrast (Minimum). Arquero sets background: #fff on the sticky header in an inline style. In dark mode, the header text gets the light text color of the theme.

Four other empty-table-header signatures are not in the baseline. They come from tables written in quarto-web content, so quarto-web can fix them.

False positive: the closed revealjs slide menu (1 signature, 1 instance, `docs/presentations/revealjs/examples/executable-code-figure-size.html` only)

Rule: color-contrast :: #aaaaaa on #ffffff, 1.4.3 Contrast (Minimum).

When the slide menu is closed, it is moved 300px to the left, and its overflow is hidden. This slide has no heading, so the menu uses the text of the slide as the title. The title is long, and about 12px of it shows at the left edge of the slide. axe calculates the contrast against the white slide. When the menu is open, the title is #aaaaaa on #333333, 5.44:1.

This entry applies to this deck only, because #aaaaaa on #ffffff can also match gray text in quarto-web content. A heading on the slide would give the menu a short title, and the entry could then be removed.

Re-checked all 27 entries from the old harness baseline (origin/feat/axe-a11y-harness:_tools/axe/baseline.json) against the 2026-09-15 quarto-web 1.11.4 scan. Only scrollable-region-focusable :: #cb still has both a live signature and a note that still holds (quarto-cli#14378, still open). Three entries were fixed upstream (#14615, #14376) and no longer appear. The remaining 23 carried empty notes with no recorded reasoning, so they're held open for a fresh re-triage (accessibility repo strand ax-xmy5f5qm) rather than carried forward blind.

dygraph.js sets the axis-label color with an inline style, so no CSS can override it. dygraph.js >= 2.0 moves it into a stylesheet rule, but the R dygraphs package still bundles 1.1.1 (rstudio/dygraphs#241). Scoped to htmlwidgets.html: the same signature on penguins-preview.html is a real Quarto defect that must stay visible.

Plot puts an aria-label with no role on the <g> of every mark and axis. 20 signatures in two groups: - 10 axis-tick and grid signatures are fixed in Plot 0.6.14 (observablehq/plot#2018). Quarto bundles Plot 0.6.11, so these wait on quarto-cli#14934. Prune them when Quarto updates Plot. - 10 data-mark signatures still occur in Plot 0.6.17. Plot labels marks on purpose (observablehq/plot#944); the open request for a role is observablehq/plot#1760.

…s third-party Inputs.range puts its <label for> on the number box only, so the slider has no accessible name even when `label` is set. Inputs.table row and header checkboxes have no label, and no option adds one. Inputs 0.12.0 has the same markup, and no upstream issue reports either defect. The two .oi-3a86ea-input signatures are site-wide; the selector becomes .inputs-3a86ea-input when Quarto updates Inputs (quarto-cli#14934). The four generic signatures are scoped to their pages, so a new unlabeled input elsewhere still shows as new.

quarto-cli#14755: Quarto relocates bibliography entries to the margin but drops the role="list" wrapper they need, leaving role="listitem" orphaned. 1 instance on docs/authoring/article-layout.html.

quarto-cli#4934: collapsed callouts use a plain <div aria-expanded> instead of the ARIA disclosure pattern (button + aria-controls), open since 2023. 3 signatures / 13 instances across brand, callouts, front-matter, the three manuscript editor pages, github-pages, and the get-started editor pages.

quarto-cli#14769: {{< placeholder >}} builds its <img> from pandoc.Image with an always-empty caption, so no alt attribute reaches the output. 2 instances on docs/authoring/placeholder.html, scoped to that page -- the broader p > .img-fluid signature also covers real content images on other pages that still need alt text of their own.

leaflet.js sets role="button" and tabindex="0" on marker icons that also carry alt="" (implicit presentation role), and its attribution control's CSS isn't dark-mode aware. 7 signatures / 9 marker instances + 2 contrast findings (2 instances each) across docs/interactive/index, docs/interactive/widgets/{htmlwidgets,jupyter}, the revealjs demo, and the three leaflet maps on docs/authoring/article-layout. No upstream issue — this is leaflet's own markup and CSS, not Quarto's.

knitr::kable() leaves the row-name corner cell as an empty <th> by design, not oversight: yihui/knitr#1747, merged as PR #2500 on 2026-09-19, added scope="col"/scope="row" to kable's other header cells but explicitly kept the corner cell blank and unscoped. 5 signatures / 7 instances across notebook-embed, penguins(-preview), three kable() examples on article-layout, and the revealjs demo's Data tab. Arquero's table viewer hardcodes background: #fff via inline JS style, so its sticky header text fails contrast in dark mode (1 signature, 22 instances, reused on both the arquero example and libraries pages). Each of the 5 kable entries repeats the full note, so the context survives if any one entry is pruned.

github-actions · 2026-09-25T15:35:37Z

📝 Preview Deployment

🔍 Full site preview: https://deploy-preview-2208.quarto.org

The 35 code-preview iframes have no title (quarto-cli#14770). They clear when quarto-cli#14933 ships, because fix/code-preview-titles already sets code-preview-title on every preview.

github-actions · 2026-09-25T21:33:52Z

📝 Preview Deployment

🔍 Full site preview: https://deploy-preview-2208.quarto.org

…upstream

github-actions · 2026-09-25T21:40:37Z

📝 Preview Deployment

🔍 Full site preview: https://deploy-preview-2208.quarto.org

github-actions · 2026-09-28T22:15:48Z

📝 Preview Deployment

🔍 Full site preview: https://deploy-preview-2208.quarto.org

github-actions · 2026-09-29T16:15:07Z

📝 Preview Deployment

🔍 Full site preview: https://deploy-preview-2208.quarto.org

github-actions · 2026-09-29T17:04:35Z

📝 Preview Deployment

🔍 Full site preview: https://deploy-preview-2208.quarto.org

github-actions · 2026-09-29T17:37:58Z

📝 Preview Deployment

🔍 Full site preview: https://deploy-preview-2208.quarto.org

…hird-party

github-actions · 2026-09-29T18:09:48Z

📝 Preview Deployment

🔍 Full site preview: https://deploy-preview-2208.quarto.org

…#14817

…-cli#14962 and #14963

github-actions · 2026-09-29T20:38:06Z

📝 Preview Deployment

🔍 Full site preview: https://deploy-preview-2208.quarto.org

…itive

github-actions · 2026-09-29T21:04:47Z

📝 Preview Deployment

🔍 Full site preview: https://deploy-preview-2208.quarto.org

cwickham added 9 commits September 25, 2026 08:27

Accept the margin-citation aria-required-parent finding as upstream

200c89e

quarto-cli#14755: Quarto relocates bibliography entries to the margin but drops the role="list" wrapper they need, leaving role="listitem" orphaned. 1 instance on docs/authoring/article-layout.html.

cwickham added 2 commits September 25, 2026 14:22

Accept the code-preview frame-title findings as upstream

e9c7bf2

The 35 code-preview iframes have no title (quarto-cli#14770). They clear when quarto-cli#14933 ships, because fix/code-preview-titles already sets code-preview-title on every preview.

Accept the revealjs meta-viewport finding as upstream

8a9af6b

Accept the scrollable-region findings that quarto-cli#14816 fixes as …

968a973

…upstream

Accept the revealjs line-highlight fade findings as upstream

0126704

Accept the revealjs line-number anchor findings as upstream

3a7295a

Accept the vega-embed actions-menu summary-name findings as third-party

89201f8

Accept the Leaflet ESDIS attribution link findings as third-party

cd51137

Accept the Arquero notebook table-view scrollable-region finding as t…

c30387d

…hird-party

cwickham added 2 commits September 29, 2026 11:14

Accept the revealjs scrollable-region findings as upstream quarto-cli…

9faf1c9

…#14817

Accept the revealjs link and tab contrast findings as upstream quarto…

61e8d7a

…-cli#14962 and #14963

Accept the revealjs closed slide-menu contrast finding as a false pos…

96200d1

…itive

Navigation Menu

Add an axe baseline for known third-party and upstream findings - #2208

Add an axe baseline for known third-party and upstream findings#2208
cwickham wants to merge 20 commits into
mainfrom
axe-baseline-seed

cwickham commented Sep 25, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 25, 2026

Uh oh!

github-actions Bot commented Sep 25, 2026

Uh oh!

github-actions Bot commented Sep 25, 2026

Uh oh!

github-actions Bot commented Sep 28, 2026

Uh oh!

github-actions Bot commented Sep 29, 2026

Uh oh!

github-actions Bot commented Sep 29, 2026

Uh oh!

github-actions Bot commented Sep 29, 2026

Uh oh!

github-actions Bot commented Sep 29, 2026

Uh oh!

github-actions Bot commented Sep 29, 2026

Uh oh!

github-actions Bot commented Sep 29, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

cwickham commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Entries

Upstream: Quarto

Third-party: Observable Plot (20 signatures, 54 instances, all pages)

Third-party: Observable Inputs (6 signatures, 60 instances on 11 pages)

Third-party: leaflet (8 signatures)

Third-party: dygraphs (1 signature, 15 instances, docs/interactive/widgets/htmlwidgets.html only)

Third-party: the @uwdata/arquero Observable notebook (1 signature, 2 instances, docs/interactive/ojs/examples/arquero.html only)

Third-party: vega-embed (1 signature, 4 instances on 4 pages)

Third-party: knitr and Arquero tables (6 signatures)

False positive: the closed revealjs slide menu (1 signature, 1 instance, docs/presentations/revealjs/examples/executable-code-figure-size.html only)

Uh oh!

github-actions Bot commented Sep 25, 2026

📝 Preview Deployment

Uh oh!

github-actions Bot commented Sep 25, 2026

📝 Preview Deployment

Uh oh!

github-actions Bot commented Sep 25, 2026

📝 Preview Deployment

Uh oh!

github-actions Bot commented Sep 28, 2026

📝 Preview Deployment

Uh oh!

github-actions Bot commented Sep 29, 2026

📝 Preview Deployment

Uh oh!

github-actions Bot commented Sep 29, 2026

📝 Preview Deployment

Uh oh!

github-actions Bot commented Sep 29, 2026

📝 Preview Deployment

Uh oh!

github-actions Bot commented Sep 29, 2026

📝 Preview Deployment

Uh oh!

github-actions Bot commented Sep 29, 2026

📝 Preview Deployment

Uh oh!

github-actions Bot commented Sep 29, 2026

📝 Preview Deployment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

cwickham commented Sep 25, 2026 •

edited

Loading

Third-party: dygraphs (1 signature, 15 instances, `docs/interactive/widgets/htmlwidgets.html` only)

Third-party: the `@uwdata/arquero` Observable notebook (1 signature, 2 instances, `docs/interactive/ojs/examples/arquero.html` only)

False positive: the closed revealjs slide menu (1 signature, 1 instance, `docs/presentations/revealjs/examples/executable-code-figure-size.html` only)