Repository navigation
feat: add render_embedding_atlas() for Embedding Atlas explorers - #294
Merged
Merged
Conversation
Renders embedding_atlas.widget.EmbeddingAtlasWidget. The explorer hard-codes a 650px container height, so shinywidgets.css fits it to explicit output/widget heights (filling layouts already work). embedding-atlas requires Python 3.11+, so it is added to the test/dev deps with a python_version marker.
embedding_atlas.widget never imports torch/sentence-transformers/umap-learn (only embedding_atlas.projection and the CLI do), so override them away via [tool.uv] to avoid CUDA wheels in CI. Also document the CSS selectors, drop unneeded type: ignore, and make the Playwright tests less brittle.
The explorer answers its queries by running SQL that the browser sends over the widget's comm, so in a deployed app any visitor can run arbitrary SQL in the server process. Against the example app as it was, a few lines in the browser console read /etc/hosts and wrote a file to the server's disk. The docstring now explains the exposure and the fix: build the widget on its own connection, then `SET enable_external_access = false` and `SET lock_configuration = true` (doing it at connect() time breaks the widget's data loading). The example and the test app use that recipe, and a new Playwright test sends the explorer's SQL messages from the browser to check that file reads, file writes, and undoing the lockdown are refused while the explorer's own queries still work.
…aims The docstring said trait changes on `.widget` update the explorer in place, but the explorer ignores them (it only listens to `_props`, and re-applying those has no visible effect), and setting `_predicate` from Python silently desyncs Python's selection() from what the browser shows. It now points to the pattern that does work, reactive_read(.widget, "_predicate") plus .selection(), and says changes need a re-render. It also says the embedding view needs WebGPU's shader-f16 feature, not just WebGPU (plain headless Chromium has an adapter without it, and the map stays blank), and that the explorer needs about 600px of height outside filling layouts, where the 400px default leaves the map ~45px tall. The example's table no longer shows the synthetic __row_index__ column.
# Conflicts: # shinywidgets/_render_widget.py # tests/unit/test_render_widget.py # uv.lock
litellm is only imported lazily by embedding_atlas's projection/CLI code paths (never by embedding_atlas.widget), but its fastuuid dependency has no cp315 wheels and its PyO3 version can't build on Python 3.15. Exclude it via uv override-dependencies, same as torch/sentence-transformers.
…subagents/pr294-embedding-atlas-dd9f1a9-32c2-s0-t0
Resolve conflicts from merging origin/main (per-renderer extras, AGENTS.md consolidation, jscatter/ipydatagrid renderers): - CHANGELOG.md: keep both sides' entries; drop install commands from the embedding_atlas entry to match main's style - _render_widget.py: union of render_embedding_atlas and render_jscatter - test_render_widget.py: union of embedding_atlas and ipydatagrid tests - CLAUDE.md: accept deletion (superseded by AGENTS.md) Add embedding_atlas extra (embedding-atlas>=0.26, Python 3.11+), point the decorator docstring and README extras table at 'pip install shinywidgets[embedding_atlas]', and switch the example's requirements.txt to shinywidgets[embedding_atlas].
…ngAtlasWidget; exclude intrinsic heights from embedding-atlas CSS overrides
The explorer runs its queries by sending SQL from the browser to the widget's DuckDB connection, so an unrestricted connection lets any visitor run arbitrary SQL (including reads/writes of server files) with the server process's permissions. render_embedding_atlas now locks the connection down before display (SET enable_external_access = false; SET lock_configuration = true) unless the app author passes allow_unrestricted_connection=True or already locked the connection's configuration themselves (their settings are left untouched). Addresses Copilot review on #294.
The app no longer constructs a connection itself (the renderer locks down the widget's default connection), and duckdb comes with embedding-atlas anyway.
render_pygwalker (#289) reads the same two settings (via a duckdb_settings() helper) to enforce its lockdown, just with the opposite policy: it runs browser SQL on DuckDB's process-wide default connection, which shinywidgets can't lock down on the app's behalf, so it fails closed and requires the app to do it. Adopt its duckdb_settings() here and add a generic restrict_duckdb_connection() next to it, so the SET idiom lives in one place once both PRs land.
cpsievert
added a commit
that referenced
this pull request
Oct 27, 2026
Stacks this PR on render-embedding-atlas so both DuckDB-backed renderers share the lockdown mechanics: duckdb_settings() now lives once (next to restrict_duckdb_connection(), which render_embedding_atlas uses) and this branch's duplicate is dropped. require_duckdb_lockdown() keeps its fail-closed policy for PyGWalker's process-wide default connection. Conflicts: CHANGELOG.md (keep both entries) and the TYPE_CHECKING block in _render_widget.py (keep both widget imports).
cpsievert
marked this pull request as ready for review
October 27, 2026 21:10
Only conflict: the runtime alias chain in _render_widget.py's non-TYPE_CHECKING block -- keep both EmbeddingAtlasWidget and K3dPlot.
test_k3d_rerender_closes_old_models clicks #rerender 3x waiting only on the #render_count text output, then screenshots #fill_plot canvas. The widget re-render trails the text update (old canvas detached, new one attached and painted asynchronously), so Playwright could resolve the locator to a canvas that a still in-flight re-render detached mid-screenshot, failing with 'Element is not attached to the DOM'. count_red_pixels() now treats a transient screenshot error as 'not drawn yet' so wait_for_red_pixels() keeps polling until the final canvas is stable and painted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #278.
Apple's Embedding Atlas explorer can now be used in a Shiny app with
@render_embedding_atlas: return anEmbeddingAtlasWidgetand the interactive embedding/table/chart explorer fills its card, tracks window resizes, and cleans up on re-render. Because the widget syncs the filter a user builds in the browser (clicking chart bars, selecting points, SQL predicates),reactive_read()lets other outputs react to the selected rows (seeexamples/embedding_atlas).Notes
EmbeddingAtlasWidget(df, x=, y=, text=, ...)needs column arguments a bare DataFrame can't supply, so there is no coercion; the docstring documents the constructor.output_widget(height=...)/widget.layout.height. A small CSS rule inshinywidgets.cssfits it to explicit heights (320px -> 320, layout 360px -> 360, measured). Filling layouts needed no change (900x700: output 818x411 == explorer 818x411; 700x520: 618x231 == 618x231; grows past 650px when there is room).python_version >= '3.11', and the unit testsimportorskipit. The test environment skips torch, sentence-transformers and umap-learn (via[tool.uv] override-dependencies, uv-only; published metadata is unaffected), which the widget doesn't import -- onlyembedding_atlas.projectionand the CLI do. This keeps CUDA wheels out of CI._predicatetrait (the only selection state the widget syncs;selection()applies it). Upstream is pre-1.0, so this may need adjusting if the trait changes.Verification
Format, pyright (3.10 and 3.12), unit tests (185 passed), and Playwright (render_embedding_atlas, explicit heights, plus matplotlib fill, maplibre, plotly and altair re-render) in the torch-free environment.
🤖 Generated with Claude Code