diff --git a/AGENTS.md b/AGENTS.md index 275c2ab4..34f2f96c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -230,7 +230,6 @@ How most packages work, so the bullets below don't repeat it: - `vp dlx skills add onmax/comark-ui` lists them beside comark-ui and installs the ones picked. - Each installs alone, so it carries the syntax of the blocks it uses in `references/blocks.md` and links nothing outside its folder. Its `references/example.md` is a finished document. - Each has a page in `docs/pages/skills/` that leads with that example rendered. -- `skills/data-explorer` writes `report.json` instead of Comark: a shared template in `assets/` draws it as an interactive page, and `assets/page.html` is the one page: `scripts/build.mjs` checks `report.json`, fills the page's slots, inlines Vega when comark-ui is installed, and screenshots every tab, and an agent without Node fills the slots by hand. The page loads anything missing from jsDelivr, so `app.js` and `style.css` must keep working from `comark-ui/dist/skills/data-explorer/assets/` on npm. Its page links a live example on Drop instead of rendering one. ### Docs site diff --git a/docs/pages/skills/data-explorer.md b/docs/pages/skills/data-explorer.md deleted file mode 100644 index ad0d1b32..00000000 --- a/docs/pages/skills/data-explorer.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -title: Data explorer -description: Your agent explores a dataset and turns what it finds into an interactive page of findings, graphs, matrices and tables, from one report.json. -icon: layout-dashboard -keywords: data, data science, dashboard, analytics, report.json, filters, charts, tables, heatmap, vega-lite -tagline: "Hand your agent a dataset and get back what's in it: the findings, and the graphs and tables behind them, on a page you click through, filter and share." ---- - -::skill{name="data-explorer"} -:: - -[Open the example](https://drop.vitehub.dev/d/qmjw7dwepwfa/who-builds-nuxt-dashboard): an agent built it with this skill from the public [Nuxters](https://github.com/nuxt/nuxters) data, in one run. - -## What it does - -The agent starts from the data, not the page. It writes down the questions you want answered, profiles every field, and computes every number in a prep script, so a rerun on new data rebuilds the page. Then it writes `report.json`: the data rows, the filters, and the blocks on each tab. A shared template draws it, so the agent writes no HTML, CSS or page script. Every page gets the same parts: - -- a **top bar** with the tabs and a light and dark theme -- a **strip of numbers** that stays on every tab, and the **rows behind them** at the bottom -- **flat sections** of charts, segmented bar lists, heatmap matrices and tables, two to a row -- a **filter sidebar** with search, the period, and one list per entity, where every name keeps one color across the page and a click filters every block - -The build rejects text past its limits and names the fix, so the page stays numbers first. It then screenshots every tab in light and dark, and the first on a phone, and the agent reads each one before it hands the page back. - -The result is one HTML file. It opens from disk or any static host, and you can publish it to [Drop](https://drop.vitehub.dev/?utm_source=comark-ui&utm_medium=skills) to share it privately, with a new version on each rebuild and comments on the numbers. - -## When to use it - -Ask with the data in hand, such as "What does this export say about who contributes to our repos?" or "Make this CSV explorable by country and month." To be sure, say "Use the data-explorer skill." - -- You explore data and filter it yourself: this skill. -- You choose between options or approve a plan: [Report](/skills/report). -- Findings about something that already exists: [Review document](/skills/review-document). - -## What your agent needs - -Node with `playwright` installed, for the build that checks the page against its limits and screenshots it. With `comark-ui` installed too, the page carries Vega and opens offline. Without Node, as in a chat app, the agent fills the skill's page template itself, and the page loads whatever it couldn't fill, such as Vega, from jsDelivr when it opens. - -## It's working if - -- The first tab opens with two or three numbers that answer what you asked, each with one sentence. -- Every tab mixes block types: a table, a matrix or a bar list next to its charts. -- A name has the same color in the sidebar, the tables and the charts, and clicking it filters the page. -- Definitions and caveats sit on the Method tab, not under every chart. -- The agent hands back the screenshots it read and what the data couldn't answer. diff --git a/docs/pages/skills/index.md b/docs/pages/skills/index.md index 828cec9e..f36ae24c 100644 --- a/docs/pages/skills/index.md +++ b/docs/pages/skills/index.md @@ -14,7 +14,7 @@ tagline: Each skill teaches your agent one kind of document, what goes in it and - A visible change needs proof on a pull request: [Before/after evidence](/skills/before-after-evidence). - A review is done, and someone has to decide what to do: [Review document](/skills/review-document). - Someone has to choose between options or approve a plan, from your research: [Report](/skills/report). -- Someone hands over data and wants to see what's in it, with graphs and filters: [Data explorer](/skills/data-explorer). +- Someone hands over data and wants a dashboard with graphs and filters: Drop's [data-explorer skill](https://drop.vitehub.dev/.well-known/skills/data-explorer/SKILL.md?utm_source=comark-ui&utm_medium=skills), since it writes an HTML page rather than Comark. - Someone asked how a system, a flow, or a change works: [Visual explanation](/skills/visual-explanation). - Someone needs to learn one concept and prove it: [Teach lesson](/skills/teach-lesson). - Your agent needs a block's syntax, not a whole document: the [comark-ui skill](/guide/agents). diff --git a/docs/pages/skills/report.md b/docs/pages/skills/report.md index 9b028ea0..08008506 100644 --- a/docs/pages/skills/report.md +++ b/docs/pages/skills/report.md @@ -29,7 +29,7 @@ Choose by what you have: - Options to compare, or a plan to approve: this skill. - Findings about code, a design, or a system that already exists: [Review document](/skills/review-document). - A question about how something works: [Visual explanation](/skills/visual-explanation). -- Data you explore and filter yourself: [Data explorer](/skills/data-explorer). +- Data you explore and filter yourself: Drop's [data-explorer skill](https://drop.vitehub.dev/.well-known/skills/data-explorer/SKILL.md?utm_source=comark-ui&utm_medium=skills). ## It's working if diff --git a/skills/comark-ui/SKILL.md b/skills/comark-ui/SKILL.md index 3339e654..87a224b7 100644 --- a/skills/comark-ui/SKILL.md +++ b/skills/comark-ui/SKILL.md @@ -53,7 +53,7 @@ Problems land in `meta.issues` after `parseMarkdown`, one sentence each, and a h ## Shape the document -This skill teaches the blocks. Five sibling skills teach a whole document, what goes in it and in what order: `before-after-evidence`, `review-document`, `report`, `visual-explanation`, and `teach-lesson`. A sixth, `data-explorer`, builds an interactive data page from a `report.json` instead of Comark. When the user asks for one of those and it isn't installed, suggest `vp dlx skills add onmax/comark-ui` and picking it from the list. +This skill teaches the blocks. Five sibling skills teach a whole document, what goes in it and in what order: `before-after-evidence`, `review-document`, `report`, `visual-explanation`, and `teach-lesson`. When the user asks for one of those and it isn't installed, suggest `vp dlx skills add onmax/comark-ui` and picking it from the list. ## Pick the blocks diff --git a/skills/data-explorer/SKILL.md b/skills/data-explorer/SKILL.md deleted file mode 100644 index 63776557..00000000 --- a/skills/data-explorer/SKILL.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -name: data-explorer -description: Turns a dataset into an interactive explorer page, a dashboard of findings, graphs, matrices and tables behind tabs and filters. Use when someone hands over data, such as a CSV, a JSON export or an API, and wants to see what's in it, or asks for a dashboard of it. -license: MIT ---- - -# Data explorer - -A data explorer is one `report.json`: the data rows, the filters, and the blocks on each tab. This skill's template draws it as an _instrument_, the way an analytics tool looks: a top bar with the tabs, a strip of numbers that stays on every tab, flat sections of charts and tables, the rows behind the numbers at the bottom, and a filter sidebar on the right where every name keeps one color across the page. You decide **what to show**; the template owns **how it looks**. You write no HTML, CSS or page script. - -Words are the scarcest thing on the page. A reader scans numbers, colors and bars; prose goes in `method`. The build rejects text past its limit. - -When the reader has to choose between options or approve a plan, write a `report` instead: a page with the answer first. - -The block reference is [references/blocks.md](references/blocks.md); read it before step 3. - -## Steps - -1. **Questions.** From the request, write the 3 to 6 questions the reader wants answered. Each becomes a tab. Name each one's _shape_: change over time, share of a whole, ranking, two numbers per item, spread, two categories crossed, or a lookup of one item. - Done when every question has a shape and none overlaps another. - -2. **Profile.** Write a short script that prints, for each source file, its row count, every field with its type, range and share of nulls, and a few sample rows. Read the source's README for what each field means and how it was counted. - Done when you can name the fields and the join keys each question needs, and the caveats the README states. - -3. **Entities and filters.** Pick the one or two _entities_ the reader thinks in (customers, repositories, countries, types of work) and the period or mode they switch between. Each entity becomes a `list` filter: it gets a color that every block reuses, a row in the sidebar with its count, and a click target in every table and bar list. The period becomes a `tabs` filter. Give every dataset that should follow a filter that filter's column. - Done when each entity and period is a filter and you know which datasets carry its column. - -4. **Prep.** Write `prep.py` (or `prep.mjs`) next to the report. It reads the raw files and writes `report.json` with aggregated rows: one dataset per block shape, plus one dataset of evidence rows (the people, tickets or items behind the numbers). Every number on the page is computed here, so a rerun on new data rebuilds the report. Fold any long tail into one `Other` row. - Done when the script runs from scratch and prints the headline numbers you'll quote. - -5. **Blocks.** Put 4 to 6 numbers in `top` (a `stats` block) and the evidence rows in `bottom` (a `table` with `"style": "rows"`); both stay on every tab. On each tab, pick 2 to 6 blocks by the question's shape, using the table in [references/blocks.md](references/blocks.md), and mix them: a tab of only line charts reads as a chart dump, a tab with a table, a matrix and a bar list reads as an instrument. Open the first tab with a `facts` block: 2 or 3 numbers that answer the request, each with one sentence. - Done when every block answers its tab's question and no two blocks show the same numbers. - -6. **Words, last.** Write a short product-like `title` and a `subtitle` (the source and the date range), a `title` per block that names what is measured, and a `note` only where a block would mislead without it or says how to use it ("Click a row to filter"). Definitions, caveats and how you counted go in `method`. - Done when no block has a sentence the numbers already show. - -7. **Build and look.** Build the page as [Build it](#build-it) says, and fix every error the build prints; each one says how. Then look at it like a reader, in **every** screenshot (light, dark and phone), or without Node in your host's preview if it has one: labels collide or are cut, a block is empty or flat, colors disagree between blocks, a tab without a table or matrix, a fact the blocks don't support. Change the blocks or the prep, not the template. - Done when the build exits 0 and you have opened every screenshot of the last build, or, without Node, when `report.json` keeps every limit and you have looked at the page. - -8. **Hand back** the path to `report.html`, the screenshot folder if you built with `--shots`, and three lines: what the report answers, the most surprising finding, and anything the data couldn't answer. The page is one file that opens from disk or any static host. When the reader wants to share it, publish it to [Drop](https://drop.vitehub.dev/?utm_source=comark-ui&utm_medium=skills): the link is private, a rebuild on new data becomes a new version at the same URL, and readers can comment on the numbers. - -## Build it - -The page is [assets/page.html](assets/page.html) with `report.json`, the template's [assets/style.css](assets/style.css) and [assets/app.js](assets/app.js) in its slots. Fill it the first way your environment allows: - -1. **With Node**, run the build. It checks `report.json` against every limit and names the fix, fills the page, and screenshots every tab. Copy this skill's `scripts/` and `assets/` folders next to the report, keeping them side by side, then: - - ```sh - node scripts/build.mjs report.json report.html --shots shots - ``` - - `--shots` needs `playwright` installed where you run it; without the flag, the build needs nothing installed. The build finds both from the folder you run it in, so install them in a scratch folder with its own `package.json` (`pnpm add --ignore-workspace comark-ui playwright`) and run the build from there, with paths to `scripts/build.mjs` and the report. With `comark-ui` there too, the page carries Vega inline and opens offline. On Linux, when Playwright has no browser of its own version, the build uses the newest Chromium in `~/.cache/ms-playwright`. - -2. **Without Node**, such as in a chat app's sandbox, fill [assets/page.html](assets/page.html) yourself, with any language: `report.json` goes inside ` - - - - - - diff --git a/skills/data-explorer/assets/style.css b/skills/data-explorer/assets/style.css deleted file mode 100644 index 0433a43e..00000000 --- a/skills/data-explorer/assets/style.css +++ /dev/null @@ -1,884 +0,0 @@ -:root { - --rp-bg: #ffffff; - --rp-fg: #0b0b0d; - --rp-text: #2a2a30; - --rp-muted: #6e6e78; - --rp-border: #ebebef; - --rp-grid: #f0f0f3; - --rp-soft: #f4f4f6; - --rp-hover: #f7f7f9; - --rp-accent: #4d97de; - --rp-up: #2f8f57; - --rp-down: #cf3f4e; - --rp-side: 280px; - --rp-topbar: 52px; - color-scheme: light; -} -:root[data-theme="dark"] { - --rp-bg: #0b0b0c; - --rp-fg: #ececef; - --rp-text: #d4d4da; - --rp-muted: #8b8b94; - --rp-border: #222226; - --rp-grid: #19191c; - --rp-soft: #161619; - --rp-hover: #131316; - --rp-up: #4fbf85; - --rp-down: #ec7280; - color-scheme: dark; -} -* { - box-sizing: border-box; -} -html { - background: var(--rp-bg); -} -body { - margin: 0; - color: var(--rp-text); - background: var(--rp-bg); - font: - 13.5px/1.5 "DM Sans", - ui-sans-serif, - system-ui, - -apple-system, - "Segoe UI", - Roboto, - sans-serif; - -webkit-font-smoothing: antialiased; - font-variant-numeric: tabular-nums; -} -a { - color: inherit; - text-decoration-color: color-mix(in srgb, currentColor 35%, transparent); - text-underline-offset: 3px; -} -code { - font: - 0.88em ui-monospace, - SFMono-Regular, - Menlo, - monospace; - background: var(--rp-soft); - padding: 0.1em 0.35em; - border-radius: 4px; -} -button { - font: inherit; - color: inherit; -} -.rp-muted { - color: var(--rp-muted); - font-weight: 400; -} -.rp-dot { - display: inline-block; - flex: none; - width: 7px; - height: 7px; - border-radius: 50%; - border: 1px solid transparent; -} - -/* top bar */ -.rp-top { - position: sticky; - top: 0; - z-index: 10; - display: flex; - align-items: stretch; - gap: 24px; - height: var(--rp-topbar); - padding: 0 20px; - background: color-mix(in srgb, var(--rp-bg) 92%, transparent); - backdrop-filter: blur(8px); - border-bottom: 1px solid var(--rp-border); -} -.rp-brand { - display: flex; - align-items: center; - gap: 10px; - padding-right: 24px; - border-right: 1px solid var(--rp-border); - text-decoration: none; - white-space: nowrap; -} -.rp-brand b { - color: var(--rp-fg); - font-size: 15px; - font-weight: 600; - letter-spacing: -0.01em; -} -.rp-brand span { - color: var(--rp-muted); - font-size: 12.5px; -} -.rp-tabs { - display: flex; - gap: 2px; - overflow-x: auto; - scrollbar-width: none; -} -.rp-tabs a { - position: relative; - display: flex; - align-items: center; - padding: 0 10px; - color: var(--rp-muted); - text-decoration: none; - white-space: nowrap; -} -.rp-tabs a:hover { - color: var(--rp-fg); -} -.rp-tabs a[aria-current] { - color: var(--rp-fg); -} -.rp-tabs a[aria-current]::after { - content: ""; - position: absolute; - left: 10px; - right: 10px; - bottom: -1px; - height: 2px; - background: var(--rp-accent); -} -.rp-top-right { - display: flex; - align-items: center; - gap: 18px; - margin-left: auto; - color: var(--rp-muted); - font-size: 12.5px; -} -.rp-check { - display: flex; - align-items: center; - gap: 7px; - white-space: nowrap; - cursor: pointer; -} -.rp-side-toggles { - display: none; - flex-wrap: wrap; - gap: 6px 14px; - margin-bottom: 14px; -} -.rp-check input { - margin: 0; - accent-color: var(--rp-fg); -} -.rp-seg { - display: inline-flex; - padding: 2px; - border-radius: 8px; - background: var(--rp-soft); -} -.rp-seg button { - display: grid; - place-items: center; - border: 0; - background: none; - min-width: 26px; - height: 24px; - padding: 0 8px; - border-radius: 6px; - cursor: pointer; - color: var(--rp-muted); - font-size: 12px; -} -.rp-seg.is-theme button { - padding: 0; -} -.rp-seg button:hover { - color: var(--rp-fg); -} -.rp-seg button[aria-pressed="true"] { - background: var(--rp-bg); - color: var(--rp-fg); - box-shadow: - 0 1px 2px rgb(0 0 0 / 0.08), - 0 0 0 1px rgb(0 0 0 / 0.04); -} - -/* layout */ -.rp-layout { - display: grid; - grid-template-columns: minmax(0, 1fr) var(--rp-side); -} -.rp-layout.is-full { - grid-template-columns: minmax(0, 1fr); -} -.rp-main { - min-width: 0; - padding: 16px 28px 0; - border-right: 1px solid var(--rp-border); -} -.rp-layout.is-full .rp-main { - border-right: 0; - max-width: 1100px; -} -.rp-grid { - display: grid; - grid-template-columns: repeat(2, minmax(0, 1fr)); - column-gap: 40px; -} -.rp-block { - min-width: 0; - padding: 20px 0 22px; - border-top: 1px solid var(--rp-border); -} -.rp-block.is-wide, -.rp-summary, -.rp-method { - grid-column: 1 / -1; -} -.rp-block.is-stats, -.rp-block.is-facts { - border-top: 0; - padding: 8px 0 12px; -} -.rp-block.is-stats + .rp-block.is-facts { - padding-top: 0; -} -.rp-block-head { - display: flex; - align-items: flex-start; - justify-content: space-between; - gap: 12px; - margin-bottom: 12px; -} -.rp-block-head h3 { - margin: 0; - color: var(--rp-fg); - font-size: 14px; - font-weight: 600; - letter-spacing: -0.005em; -} -.rp-block-head p { - margin: 2px 0 0; - color: var(--rp-muted); - font-size: 12.5px; - max-width: 80ch; - text-wrap: pretty; -} -.rp-summary { - margin: 4px 0 16px; - color: var(--rp-fg); - font-size: 15px; - line-height: 1.45; - font-weight: 500; - max-width: 80ch; - text-wrap: pretty; -} -.rp-chart { - min-height: 100px; -} -.rp-chart-svg svg { - display: block; - max-width: 100%; - overflow: visible; -} -.rp-empty { - color: var(--rp-muted); - font-size: 13px; - margin: 16px 0; -} - -/* sidebar */ -.rp-side { - position: sticky; - top: var(--rp-topbar); - align-self: start; - max-height: calc(100dvh - var(--rp-topbar)); - overflow-y: auto; - padding: 16px 20px 32px; - scrollbar-width: thin; -} -.rp-side-title { - color: var(--rp-fg); - font-size: 15px; - font-weight: 600; - margin-bottom: 12px; -} -.rp-side-search { - display: flex; - align-items: center; - gap: 8px; - padding: 0 2px 8px; - border-bottom: 1px solid var(--rp-border); - color: var(--rp-muted); -} -.rp-side-search input { - flex: 1; - min-width: 0; - border: 0; - outline: 0; - background: none; - font: inherit; - color: var(--rp-fg); -} -.rp-side-search:focus-within { - border-color: var(--rp-fg); -} -.rp-side-tabs { - display: flex; - flex-wrap: wrap; - gap: 4px 14px; - margin: 12px 0 4px; -} -.rp-side-tabs button { - border: 0; - background: none; - padding: 2px 0; - cursor: pointer; - color: var(--rp-muted); - border-bottom: 1px solid transparent; -} -.rp-side-tabs button:hover { - color: var(--rp-fg); -} -.rp-side-tabs button[aria-pressed="true"] { - color: var(--rp-fg); - border-bottom-color: var(--rp-fg); -} -.rp-side-group { - margin-top: 20px; -} -.rp-side-select { - display: flex; - align-items: center; - justify-content: space-between; - gap: 10px; - margin: 12px 0 4px; - color: var(--rp-fg); - font-weight: 600; -} -.rp-side-select select { - font: inherit; - font-weight: 400; - color: var(--rp-fg); - background: var(--rp-soft); - border: 0; - border-radius: 6px; - padding: 4px 8px; -} -.rp-side-head { - display: flex; - justify-content: space-between; - align-items: baseline; - margin-bottom: 4px; - color: var(--rp-fg); - font-weight: 600; -} -.rp-side-head button { - border: 0; - background: none; - color: var(--rp-muted); - font-size: 12px; - font-weight: 400; - cursor: pointer; - padding: 0; -} -.rp-side-group ul { - list-style: none; - margin: 0; - padding: 0; -} -.rp-side-group li button { - display: flex; - align-items: center; - gap: 9px; - width: 100%; - border: 0; - background: none; - padding: 3px 6px; - margin: 0 -6px; - width: calc(100% + 12px); - border-radius: 5px; - text-align: left; - cursor: pointer; - color: var(--rp-text); -} -.rp-side-group li button:hover { - background: var(--rp-hover); -} -.rp-side-group li button.is-on { - background: var(--rp-soft); - color: var(--rp-fg); - font-weight: 500; -} -.rp-side-group li button.is-zero { - color: var(--rp-muted); -} -.rp-side-group li button span:nth-child(2) { - flex: 1; - overflow: hidden; - text-overflow: ellipsis; - white-space: nowrap; -} -.rp-count { - color: var(--rp-muted); - font-size: 12px; -} - -/* stats and facts */ -.rp-stats { - display: grid; - grid-template-columns: repeat(auto-fit, minmax(150px, 1fr)); - border: 1px solid var(--rp-border); - border-radius: 8px; -} -.rp-stat { - padding: 12px 16px; - border-left: 1px solid var(--rp-border); -} -.rp-stat:first-child { - border-left: 0; -} -.rp-stat-value { - color: var(--rp-fg); - font-size: 22px; - line-height: 1.15; - font-weight: 500; - letter-spacing: -0.02em; -} -.rp-stat-value small, -.rp-fact-value small { - font-size: 0.6em; - font-weight: 400; - color: var(--rp-muted); - letter-spacing: 0; - margin-left: 1px; -} -.rp-stat-label { - margin-top: 2px; - font-size: 12.5px; -} -.rp-stat-foot { - display: flex; - gap: 6px; - color: var(--rp-muted); - font-size: 12px; -} -.rp-delta.is-up { - color: var(--rp-up); -} -.rp-delta.is-down { - color: var(--rp-down); -} -.rp-facts { - display: grid; - grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); - gap: 10px; -} -.rp-fact { - padding: 14px 16px; - border-radius: 8px; - background: var(--rp-soft); -} -.rp-fact-value { - color: var(--rp-fg); - font-size: 22px; - line-height: 1.2; - font-weight: 500; - letter-spacing: -0.02em; -} -.rp-fact p { - margin: 4px 0 0; - font-size: 12.5px; - text-wrap: pretty; -} - -/* legend and html bars */ -.rp-legend { - display: flex; - flex-wrap: wrap; - gap: 4px 14px; - margin: -4px 0 10px; - font-size: 12px; - color: var(--rp-muted); -} -.rp-legend span { - display: inline-flex; - align-items: center; - gap: 6px; -} -.rp-legend i { - width: 9px; - height: 9px; - border-radius: 2px; -} -.rp-bars { - display: grid; - gap: 1px; -} -.rp-bars-row { - display: grid; - grid-template-columns: minmax(90px, 32%) minmax(0, 1fr); - align-items: center; - gap: 10px; - min-height: 21px; - font-size: 12.5px; - border-radius: 4px; -} -.rp-bars-row.is-click { - cursor: pointer; -} -.rp-bars-row.is-click:hover, -.rp-bars-row.is-on { - background: var(--rp-hover); -} -.rp-bars-label { - display: flex; - align-items: center; - gap: 8px; - min-width: 0; -} -.rp-bars-label span { - overflow: hidden; - text-overflow: ellipsis; - white-space: nowrap; -} -.rp-bars-track { - display: flex; - align-items: center; - gap: 6px; - min-width: 0; -} -.rp-bars-fill { - display: flex; - height: 12px; - min-width: 2px; -} -.rp-bars-fill span { - flex: none; - min-width: 0; -} -.rp-stack span { - flex: none; -} -.rp-bars-fill span:first-child { - border-radius: 2px 0 0 2px; -} -.rp-bars-fill span:last-child { - border-radius: 0 2px 2px 0; -} -.rp-bars-end { - flex: none; - color: var(--rp-muted); - font-size: 11.5px; - white-space: nowrap; -} - -/* matrix */ -.rp-matrix { - border-collapse: separate; - border-spacing: 2px; - font-size: 11.5px; -} -.rp-matrix thead th { - height: 34px; - max-width: 96px; - vertical-align: bottom; - font-weight: 400; - color: var(--rp-muted); - line-height: 1.25; - padding: 0 4px 4px; -} -.rp-matrix.is-rotated thead th { - white-space: nowrap; - padding: 0 0 4px; -} -.rp-matrix.is-rotated thead th { - height: 96px; -} -.rp-matrix.is-rotated thead th span { - display: inline-block; - transform: translate(14px, -6px) rotate(-40deg); - transform-origin: left bottom; - width: 24px; -} -.rp-matrix tbody th { - text-align: left; - font-weight: 400; - padding: 0 12px 0 0; - white-space: nowrap; - max-width: 180px; - overflow: hidden; - text-overflow: ellipsis; -} -.rp-matrix tbody th.is-click { - cursor: pointer; -} -.rp-matrix td { - min-width: 46px; - height: 22px; - padding: 0 6px; - text-align: center; - background: var(--rp-soft); - border-radius: 2px; - color: var(--rp-text); -} -.rp-entity { - display: inline-flex; - align-items: center; - gap: 7px; - max-width: 100%; - white-space: nowrap; -} - -/* tables */ -.rp-scroll { - overflow-x: auto; -} -.rp-table table { - width: 100%; - border-collapse: collapse; - font-size: 12.5px; -} -.rp-table th { - text-align: left; - font-weight: 400; - color: var(--rp-muted); - font-size: 12px; - padding: 0 10px 6px; - border-bottom: 1px solid var(--rp-border); - white-space: nowrap; -} -.rp-table th button { - border: 0; - background: none; - padding: 0; - cursor: pointer; - color: inherit; -} -.rp-table th button:hover { - color: var(--rp-fg); -} -.rp-table td { - padding: 5px 10px; - border-bottom: 1px solid var(--rp-border); - white-space: nowrap; -} -.rp-table.is-rows td { - padding: 5px 10px; -} -.rp-table tr:last-child td { - border-bottom: 0; -} -.rp-table tbody tr:hover td { - background: var(--rp-hover); -} -.rp-table tr.is-click { - cursor: pointer; -} -.rp-table tr.is-on td { - background: var(--rp-soft); -} -.rp-table th:first-child, -.rp-table td:first-child { - padding-left: 0; -} -.rp-table th:last-child, -.rp-table td:last-child { - padding-right: 0; -} -.rp-num { - text-align: right !important; -} -.rp-rank { - width: 1%; - color: var(--rp-muted); -} -.rp-trunc { - max-width: 0; - width: 50%; - overflow: hidden; - text-overflow: ellipsis; -} -.rp-user a { - display: inline-flex; - align-items: center; - gap: 7px; - text-decoration: none; -} -.rp-user img { - border-radius: 50%; - background: var(--rp-soft); -} -.rp-barcell { - min-width: 140px; -} -.rp-barcell { - display: flex; - align-items: center; - justify-content: flex-end; - gap: 8px; -} -.rp-inbar { - flex: 1; - display: flex; - justify-content: flex-start; - height: 8px; -} -.rp-inbar span { - height: 100%; - border-radius: 2px; - background: var(--rp-accent); - opacity: 0.85; -} -.rp-stack-cell { - width: 1%; -} -.rp-stack { - display: flex; - width: 64px; - height: 9px; - border-radius: 2px; - overflow: hidden; -} -.rp-chips { - display: inline-flex; - gap: 4px; -} -.rp-chip { - padding: 1px 7px; - border: 1px solid var(--rp-border); - border-radius: 999px; - font-size: 11.5px; -} -.rp-chip b { - font-weight: 400; - color: var(--rp-muted); -} -.rp-heat { - background: color-mix(in srgb, var(--rp-accent) calc(var(--t) * 55%), transparent); -} -.rp-search { - display: block; - width: 100%; - margin: 0 0 8px; - font: inherit; - color: var(--rp-fg); - background: none; - border: 0; - border-bottom: 1px solid var(--rp-border); - padding: 4px 0; - outline: 0; -} -.rp-more { - display: block; - margin: 6px 0 0; - border: 0; - background: none; - padding: 4px 0; - color: var(--rp-muted); - font-size: 12.5px; - cursor: pointer; -} -.rp-more:hover { - color: var(--rp-fg); -} - -.rp-method { - max-width: 68ch; - padding: 16px 0 32px; - font-size: 14px; -} -.rp-method h3 { - color: var(--rp-fg); - font-size: 15px; - margin: 24px 0 6px; -} -.rp-foot { - padding: 16px 0 28px; - color: var(--rp-muted); - font-size: 12px; - border-top: 1px solid var(--rp-border); -} -.rp-tip { - position: fixed; - z-index: 50; - pointer-events: none; - min-width: 140px; - max-width: 300px; - padding: 7px 10px; - border-radius: 7px; - background: var(--rp-bg); - border: 1px solid var(--rp-border); - box-shadow: - 0 1px 2px rgb(0 0 0 / 0.06), - 0 8px 24px rgb(0 0 0 / 0.1); - font-size: 12px; -} -.rp-tip b { - display: block; - margin-bottom: 3px; - color: var(--rp-fg); - font-weight: 600; -} -.rp-tip div { - display: flex; - justify-content: space-between; - gap: 16px; - color: var(--rp-muted); -} -.rp-tip div span:last-child { - color: var(--rp-fg); -} - -@media (max-width: 1000px) { - .rp-layout { - grid-template-columns: minmax(0, 1fr); - } - .rp-main { - border-right: 0; - } - .rp-side { - position: static; - max-height: none; - order: -1; - border-bottom: 1px solid var(--rp-border); - } - .rp-side-group ul { - display: flex; - flex-wrap: wrap; - gap: 2px 12px; - } - .rp-side-group li button { - width: auto; - margin: 0; - } - .rp-brand span, - .rp-top .rp-check { - display: none; - } - .rp-side-toggles { - display: flex; - } -} -@media (max-width: 760px) { - .rp-grid { - grid-template-columns: minmax(0, 1fr); - } - .rp-top { - gap: 12px; - padding: 0 12px; - } - .rp-brand { - padding-right: 12px; - } - .rp-main { - padding: 12px 16px 0; - } -} -@media (prefers-reduced-motion: no-preference) { - .rp-seg button, - .rp-side-group li button, - .rp-tabs a { - transition: - color 120ms, - background-color 120ms; - } -} diff --git a/skills/data-explorer/references/blocks.md b/skills/data-explorer/references/blocks.md deleted file mode 100644 index a497b11d..00000000 --- a/skills/data-explorer/references/blocks.md +++ /dev/null @@ -1,121 +0,0 @@ -# report.json reference - -```json -{ - "title": "shoplens", - "subtitle": "Till exports · Jan 2024 to Jun 2026", - "intro": "What sells, where, and whether customers come back. Data from the till exports.", - "source": { "label": "Till exports", "url": "https://example.com/exports" }, - "updated": "2026-07-01", - "search": "Search titles", - "filters": [ - { "field": "period", "type": "tabs", "options": ["12 months", "30 days", "All time"] }, - { "field": "store", "label": "Stores", "count": "sales" }, - { "field": "genre", "label": "Genres", "count": "sales" }, - { "field": "gift", "type": "toggle", "label": "Include gift cards" } - ], - "data": { - "totals": [{ "period": "12 months", "store": "North", "revenue": 41800, "orders": 2210 }], - "byStore": [{ "period": "12 months", "store": "North", "fiction": 1840, "nonfiction": 960, "end": "2,800 · 66% fiction" }], - "sales": [{ "period": "12 months", "date": "2026-06-28", "store": "North", "genre": "Fiction", "title": "The Long Way", "copies": 3, "gift": false }] - }, - "top": [{ "type": "stats", "items": [{ "label": "revenue", "value": { "data": "totals", "field": "revenue" }, "format": "money", "hint": "in this period" }] }], - "views": [ - { - "id": "overview", - "label": "Overview", - "blocks": [ - { "type": "facts", "items": [{ "value": "66%", "text": "of copies sold are fiction, up from 58% last year." }] }, - { "type": "bars", "title": "Copies by store", "data": "byStore", "label": "store", "segments": ["fiction", "nonfiction"], "names": { "fiction": "Fiction", "nonfiction": "Non-fiction" }, "end": "end" } - ] - } - ], - "bottom": [{ "type": "table", "style": "rows", "title": "Sales", "data": "sales", "columns": [{ "field": "date", "format": "date" }, { "field": "store", "format": "entity" }, { "field": "genre", "format": "entity" }, { "field": "title", "truncate": true }, { "field": "copies" }], "sort": "date", "limit": 20, "more": 50 }], - "method": "Markdown: paragraphs, `- ` lists, **bold**, `code`, [links](https://…)." -} -``` - -## Limits - -| Text | Limit | -| --- | --- | -| `title` | 40 characters | -| `subtitle` | 60 characters | -| `intro` | 280 characters, two sentences (shown on the Method tab) | -| tab `label` | 22 characters | -| tab `summary` (optional) | 140 characters, one sentence | -| block `title` | 60 characters | -| block `note` | 120 characters, one sentence | -| fact `text` | 110 characters, one sentence | -| `method` | 1,600 characters | -| stat `label` | 28 characters | -| stat `hint` | 32 characters | -| tabs 7, blocks per tab 8, series per chart 12, table columns 9, `top` 2 blocks, `bottom` 1 block | -| `stats` 6 items, `facts` 4 items | -| `bars` 30 rows and bar charts 40 bars, unless `limit` keeps fewer; matrix 24 columns, or swap `x` and `y` | -| data 3 MB in all | - -A view `id` is kebab-case, and `method` is taken by the Method tab. A key the reference doesn't list is an error. Notes, fact `text`, tab `summary`, `intro` and `method` take `**bold**`, `` `code` `` and links; titles and labels print as written. - -## Which block for which question - -| Question shape | Block | -| --- | --- | -| Headline numbers | `stats` in `top` | -| The answer to the request | `facts`, first tab | -| A number over time | `chart` `line`; `bar` for one series with 15 periods or fewer | -| Share of a whole over time | `chart` `stacked` with `"stack": "percent"`, or `area` | -| Ranking of names, or a whole split per name | `bars` (one `value`, or `segments` that add up) | -| Many numbers per name | `table` with `entity`, `bar`, `stack`, `chips` and `percent` columns | -| Two categories crossed (customer × type, cohort × year) | `matrix` | -| Two numbers per item | `chart` `scatter`, `logX`/`logY` when values span 100× or more | -| Spread of one number | `chart` `histogram`, `logToggle` when it's skewed | -| The rows behind the numbers | `table` with `"style": "rows"` in `bottom` | - -## Layout - -Blocks sit in two columns. Charts and `bars` take half a row, so put related ones next to each other; tables, matrices, `stats` and `facts` take the full row. `"wide": true` or `"half": true` overrides that. Sections have no boxes: a hairline separates them. - -## Colors - -Every value of a `list` filter gets one color, used by the sidebar, `entity` columns, `bars` labels, matrix rows or columns, and chart series on that field. So name a chart's `series`, a bar list's `label` or a matrix's `x`/`y` after the filter's field when they show that entity. For anything else, `tones` sets colors in order: `good`, `fair`, `warn`, `bad`, `muted`, `accent`, a palette index, or a `#hex`. Use the semantic tones for scales that run from good to bad (answered → only the team knows). - -## Blocks - -Every block takes `title` and `note` (a `note` needs a `title`), and every block but `stats` and `facts` takes `wide` and `half`. - -**stats** (in `top`): `items` of `{ label, value, format, unit, delta, deltaFormat, good, hint }`. `value` and `delta` are a number, or `{ "data", "field", "agg", "where" }` computed on the filtered rows; `agg` is `sum` (default), `count` (counts the rows when there's no `field`), `mean`, `min`, `max`, `first`, `last`. A trailing unit (`%`, `K`, `h`) prints small. `format: "percent"` takes a 0–1 value. `delta` prints signed, green when up unless `"good": "down"`, as a percent of a 0–1 change unless `deltaFormat` says otherwise (`number`, `integer`, `compact`, `money`). - -**facts**: 2 to 4 `items` of `{ value, text }`. `value` is a short string ("53%", "10 of 19", "1.4 h") or a number with `format`; `text` is one sentence that says what it means. - -**bars**: horizontal bars in HTML, one row per `label`. Either `value` (one number) or `segments` (fields that add up to the bar, with `names` and `tones`, and a legend). `end` is a field printed after the bar, like `"55 · 47% today"`, built in the prep script; without it the total prints. `percent: true` makes every bar full width. Sorted by total unless `"sort": "data"`; `limit` keeps the top N. Rows click to filter when `label` is a list filter's field. - -**matrix**: `x`, `y`, `value`. Cells tint by their share of the largest value (`"scale": "row"` compares within a row). `tint`: `single` (default), `columns` (each column in its entity color), `rows`. Long column names rotate. Rows and columns keep the data's order, or list `columns`. - -**chart**: `chart` is one of: - -- `line`, `area`: `x` (a year, an ISO date, or a category), `y`, optional `series`; or wide rows with `"y": ["fix", "feat"]` and `names`. Lines are named at their end; `endValues: true` adds the last value. `emphasis` draws one series (a total, "All") in the text color and thicker. `points: true` marks every value. `area` stacks with a series; `"stack": "percent"` on an `area` makes shares. -- `bar`, `stacked`, `grouped`: `x` the category, `y` the value, optional `series`. `horizontal: true` sorts by value; `limit` keeps the top N; `"stack": "percent"` turns stacked bars (`stacked`, or `bar` with a `series`) into shares. -- `scatter`: `x`, `y`, optional `size`, `series`, `label` (names the top `labelTop` points, moved apart so they don't overlap). `logX`/`logY` keep points at 0 on the axis. A click on a point filters by its `label` or `series` when that field is a list filter's, and otherwise, when the report has `search`, searches for its `label`, which narrows every table to that row. -- `histogram`: `x` only; `bins` (default 30). With `logToggle` on log, values at or below 0 are left out. - -Options for any chart: `format` (`number`, `integer`, `compact`, `percent` for 0–1 values, `money` in US dollars), `height` (default 240), `highlight` (one series in color, the rest gray), `tones`, `logToggle`, `xTitle`, `yTitle`. - -**table**: `data`, `columns` of `{ field, label, format, number }`, `sort` (a field, descending), `limit` (rows shown, default 12), `more` (rows each "Show more" adds), `search: true`, `rank: false`. `"style": "rows"` is the dense evidence list: no header, no rank, its title shows the matching count, and the sidebar search filters it. Column `format`: - -- `entity`: a colored dot and the name, for a list filter's field. Rows click to filter. -- `bar`: inline bar and number; `heat`: tinted cell; `delta`: signed and colored, a 0–1 change printed as a percent; `number`/`integer`/`compact`/`percent`/`money`. -- `stack`: a small bar split by `fields` (with `names`, `tones`), for a mix per row. Give it any `field` name as its key. -- `chips`: an array of strings or `[label, count]` pairs, as pills, like the two things a customer most needs. -- `text` (`truncate: true` cuts it to the cell), `date`, `user` (GitHub avatar), `country` (flag from an ISO code), `link`. `suffix` prints another field after a number, small. - -**vega** (escape hatch): `{ "type": "vega", "title", "spec": { …Vega-Lite…, "data": { "name": "" } } }`. - -## Filters - -- `tabs`: periods or modes in the sidebar, one always on; more than 5 options become a dropdown, so list every year if the reader needs them. Give every dataset that should follow it the field, with one set of rows per option. -- `list` (default for more than 5 options): the entity list in the sidebar, each name with its color and a count of rows in `count` (a dataset name; `countField` sums a field instead). Click to filter every dataset with that field; click again to clear. `colors` maps a value to a tone (`{ "Bug": "bad" }`) when its meaning has a color. -- `toggle`: a checkbox in the top bar. Rows whose field is true are hidden until it's checked, like "Include thanks and scheduling". -- `search` (report level, the placeholder text): filters every `table`. - -Datasets without a filter's field ignore it. diff --git a/skills/data-explorer/scripts/build.mjs b/skills/data-explorer/scripts/build.mjs deleted file mode 100644 index 5c52f996..00000000 --- a/skills/data-explorer/scripts/build.mjs +++ /dev/null @@ -1,181 +0,0 @@ -// node build.mjs report.json report.html [--shots dir] -// Checks report.json, then writes one self-contained HTML page. With --shots, it opens the page and saves a -// screenshot of every view in light and dark, plus the first view on a phone, and fails on any chart that broke. -// It needs nothing installed. Where comark-ui is, it inlines Vega so the page works offline; --shots needs playwright. -import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from "node:fs"; -import { createRequire } from "node:module"; -import { homedir } from "node:os"; -import { dirname, join, resolve } from "node:path"; -import { fileURLToPath, pathToFileURL } from "node:url"; -import { check } from "./check.mjs"; - -const here = dirname(fileURLToPath(import.meta.url)); -const assets = join(here, "../assets"); -const [input, output, ...rest] = process.argv.slice(2); -if (!input || !output) { - console.error("Usage: node build.mjs report.json report.html [--shots dir]"); - process.exit(2); -} -const shots = rest[0] === "--shots" ? resolve(rest[1] ?? "shots") : null; - -let report; -try { - report = JSON.parse(readFileSync(input, "utf8")); -} catch (error) { - console.error(`${input} isn't valid JSON: ${error.message}`); - process.exit(1); -} -const result = check(report); -for (const w of result.warnings) console.log(`warning ${w}`); -for (const e of result.errors) console.log(`error ${e}`); -if (result.errors.length) { - console.log(`\n${result.errors.length} errors. Fix them and build again.`); - process.exit(1); -} - -// pnpm keeps vega inside comark-charts, so look in the current folder first, then follow comark-ui to it. Without -// either, the page loads Vega from jsDelivr when it opens. -const local = createRequire(join(process.cwd(), "noop.js")); -function packageDir(name) { - const tries = [local]; - try { - const ui = createRequire(local.resolve("comark-ui/package.json")); - tries.push(createRequire(ui.resolve("comark-charts/package.json"))); - } catch {} - for (const require of tries) { - try { - for (let dir = dirname(require.resolve(name)); dir !== dirname(dir); dir = dirname(dir)) { - const manifest = join(dir, "package.json"); - if (existsSync(manifest) && JSON.parse(readFileSync(manifest, "utf8")).name === name) - return dir; - } - } catch {} - } - return null; -} -const script = (code) => ``; -const vegaDirs = [packageDir("vega"), packageDir("vega-lite")]; -const vega = vegaDirs.every(Boolean) - ? [ - script(readFileSync(join(vegaDirs[0], "build/vega.min.js"), "utf8")), - script(readFileSync(join(vegaDirs[1], "build/vega-lite.min.js"), "utf8")), - ].join("\n") - : null; -if (!vega) - console.log( - "note Vega isn't installed here, so the page loads it from jsDelivr when it opens. Run the build where comark-ui is installed for a page that works offline.", - ); - -// The same page an agent without Node fills by hand: put each part in its slot. -const esc = (s) => - String(s).replace( - /[&<>"]/g, - (c) => ({ "&": "&", "<": "<", ">": ">", '"': """ })[c], - ); -// Each slot is an element of the page whose contents are replaced, keeping its tags. -const fill = (id, text) => [new RegExp(`(<(\\w+) id="${id}"[^>]*>)[\\s\\S]*?()`), text]; -const slots = [ - [/(<(title)>)[\s\S]*?(<\/title>)/, esc(report.title)], - fill("rp-style", `\n${readFileSync(join(assets, "style.css"), "utf8")}`), - fill("report-data", JSON.stringify(report).replace(/", vega ?? ""], -]; -let html = readFileSync(join(assets, "page.html"), "utf8"); -for (const [slot, text] of slots) { - if (!html.match(slot)) throw new Error(`assets/page.html lost its ${slot} slot.`); - html = html.replace(slot, (...m) => (typeof slot === "string" ? text : `${m[1]}${text}${m[3]}`)); -} -mkdirSync(dirname(resolve(output)), { recursive: true }); -writeFileSync(output, html); -console.log( - `Wrote ${output} (${(html.length / 1024).toFixed(0)} kB, ${result.rows} rows, ${report.views.length} views${result.escapes ? `, ${result.escapes} raw Vega-Lite blocks` : ""}).`, -); - -if (shots) { - mkdirSync(shots, { recursive: true }); - let path; - try { - path = local.resolve("playwright"); - } catch { - console.log( - "error --shots needs playwright: run the build where it's installed (pnpm add playwright).", - ); - process.exit(1); - } - const playwright = await import(pathToFileURL(path).href); - const { chromium } = playwright.chromium ? playwright : playwright.default; - // Playwright's own browser may be missing; fall back to the newest one in its cache. - const cache = join(homedir(), ".cache/ms-playwright"); - const fallback = existsSync(cache) - ? readdirSync(cache) - .filter((d) => /^chromium-\d+$/.test(d)) - .sort((a, b) => Number(b.slice(9)) - Number(a.slice(9))) - .map((d) => join(cache, d, "chrome-linux64/chrome")) - .find((path) => existsSync(path)) - : undefined; - let browser; - try { - browser = await chromium.launch(); - } catch { - browser = await chromium.launch({ executablePath: fallback }); - } - const problems = []; - const url = pathToFileURL(resolve(output)).href; - for (const theme of ["light", "dark"]) { - const page = await browser.newPage({ - viewport: { width: 1440, height: 900 }, - deviceScaleFactor: 1, - }); - page.on("pageerror", (e) => problems.push(`page error: ${e.message}`)); - page.on( - "console", - (m) => - m.type() === "error" && - !/Failed to load resource/.test(m.text()) && - problems.push(`console: ${m.text()}`), - ); - await page.goto(`${url}?theme=${theme}`); - await page.evaluate(() => document.fonts.ready); - const ids = await page.evaluate(() => window.__report?.views ?? []); - for (const id of ids) { - await page.evaluate((v) => window.__report.go(v), id); - await page - .waitForFunction((v) => document.body.dataset.ready === v, id, { timeout: 20000 }) - .catch(() => problems.push(`view ${id}: charts didn't finish drawing in 20 s`)); - await page.waitForTimeout(150); - if (theme === "light") { - const broken = await page.$$eval(".rp-chart[data-state=error]", (els) => - els.map((el) => `${el.getAttribute("aria-label")}: ${el.dataset.error}`), - ); - for (const b of broken) problems.push(`view ${id}: chart "${b}"`); - const clipped = await page.$$eval(".rp-block", (cards) => - cards - .filter((c) => c.scrollWidth > c.clientWidth + 2 && !c.querySelector(".rp-scroll")) - .map((c) => c.querySelector("h3")?.textContent ?? "a block"), - ); - for (const c of clipped) problems.push(`view ${id}: "${c}" is wider than its card`); - } - await page.screenshot({ path: join(shots, `${id}-${theme}.png`), fullPage: true }); - } - await page.close(); - } - const phone = await browser.newPage({ - viewport: { width: 390, height: 844 }, - deviceScaleFactor: 2, - }); - await phone.goto(`${url}?theme=light`); - await phone - .waitForFunction(() => document.body.dataset.ready, null, { timeout: 20000 }) - .catch(() => {}); - await phone.screenshot({ path: join(shots, "phone.png"), fullPage: true }); - await browser.close(); - for (const p of new Set(problems)) console.log(`error ${p}`); - console.log( - `Screenshots in ${shots}: one per view and theme, plus phone.png. Open every one before you call the report done.`, - ); - if (problems.length) process.exit(1); -} diff --git a/skills/data-explorer/scripts/check.mjs b/skills/data-explorer/scripts/check.mjs deleted file mode 100644 index 0c4068e6..00000000 --- a/skills/data-explorer/scripts/check.mjs +++ /dev/null @@ -1,477 +0,0 @@ -// Checks a report.json before it's built. Every issue names where it is and how to fix it. - -const LIMITS = { - title: 40, - subtitle: 60, - intro: 280, - label: 22, - summary: 140, - blockTitle: 60, - note: 120, - method: 1600, - statLabel: 28, - hint: 32, - fact: 110, -}; -const KEYS = { - report: [ - "title", - "subtitle", - "intro", - "source", - "updated", - "filters", - "search", - "top", - "views", - "bottom", - "method", - "data", - ], - filter: ["field", "label", "type", "default", "options", "count", "countField", "colors"], - view: ["id", "label", "summary", "blocks"], - stats: ["type", "items", "id", "title", "note"], - stat: ["label", "value", "format", "unit", "delta", "deltaFormat", "good", "hint"], - facts: ["type", "items", "id", "title", "note"], - fact: ["value", "format", "text"], - chart: [ - "type", - "chart", - "title", - "note", - "data", - "x", - "y", - "series", - "names", - "labels", - "value", - "size", - "label", - "labelTop", - "format", - "height", - "wide", - "half", - "horizontal", - "stack", - "limit", - "sort", - "highlight", - "emphasis", - "tones", - "endValues", - "points", - "yMax", - "logX", - "logY", - "logToggle", - "xTitle", - "yTitle", - "xType", - "xFormat", - "bins", - "curve", - "tint", - "scale", - "columns", - "id", - ], - bars: [ - "type", - "title", - "note", - "data", - "label", - "value", - "segments", - "names", - "tones", - "end", - "format", - "percent", - "sort", - "limit", - "wide", - "half", - "id", - ], - matrix: [ - "type", - "title", - "note", - "data", - "x", - "y", - "value", - "columns", - "tint", - "tones", - "scale", - "format", - "wide", - "half", - "id", - ], - table: [ - "type", - "title", - "note", - "data", - "columns", - "style", - "header", - "sort", - "limit", - "more", - "search", - "rank", - "wide", - "half", - "id", - ], - column: [ - "field", - "label", - "format", - "number", - "fields", - "names", - "tones", - "colorBy", - "truncate", - "suffix", - ], - vega: ["type", "title", "note", "spec", "height", "wide", "half", "id"], -}; -const TYPES = ["stats", "facts", "chart", "bars", "matrix", "table", "vega"]; -const CHARTS = ["line", "area", "bar", "stacked", "grouped", "scatter", "heatmap", "histogram"]; -const FORMATS = ["number", "integer", "compact", "percent", "money"]; -const COLUMN_FORMATS = [ - "text", - "entity", - "user", - "country", - "link", - "date", - "bar", - "heat", - "stack", - "chips", - "delta", - ...FORMATS, -]; -const TONE_NAMES = ["good", "fair", "warn", "bad", "muted", "accent"]; -const MAX_SERIES = 12; - -const sentences = (s) => (String(s).match(/[.!?](\s|$)/g) ?? []).length; - -export function check(report) { - const errors = []; - const warnings = []; - const err = (where, msg) => errors.push(`${where}: ${msg}`); - const warn = (where, msg) => warnings.push(`${where}: ${msg}`); - const keys = (obj, kind, where) => { - for (const k of Object.keys(obj ?? {})) - if (!KEYS[kind].includes(k)) - err( - where, - `\`${k}\` isn't a ${kind} option. The options are ${KEYS[kind].map((x) => `\`${x}\``).join(", ")}.`, - ); - }; - const text = (value, limit, where, what, one) => { - if (value == null) return; - if (typeof value !== "string") return err(where, `${what} must be text.`); - if (value.length > limit) - err( - where, - `${what} is ${value.length} characters; the limit is ${limit}. Keep the one thing the reader needs and move the rest to \`method\`.`, - ); - if (one && sentences(value) > 1) - err( - where, - `${what} has ${sentences(value)} sentences; write one. Put the rest in \`method\` or let a block show it.`, - ); - }; - const tones = (list, where) => { - for (const t of list ?? []) - if (typeof t === "string" && !t.startsWith("#") && !TONE_NAMES.includes(t)) - err( - where, - `tone \`${t}\` isn't one of ${TONE_NAMES.join(", ")}, a palette index, or a #hex color.`, - ); - }; - - keys(report, "report", "report"); - if (!report.title) - err( - "report", - "add a `title`: the product-like name in the top bar, like `shoplens` or `Who builds Nuxt`.", - ); - text(report.title, LIMITS.title, "report", "`title`"); - text(report.subtitle, LIMITS.subtitle, "report", "`subtitle`"); - text(report.intro, LIMITS.intro, "report", "`intro`"); - if (report.intro && sentences(report.intro) > 2) - err( - "report", - "`intro` has more than two sentences; say what the report answers and where the data comes from, nothing else.", - ); - text(report.method, LIMITS.method, "report", "`method`"); - if (report.source && (!report.source.label || !report.source.url)) - err("report.source", "give both `label` and `url`."); - - const data = report.data ?? {}; - if (!Object.keys(data).length) - err("report", "add `data`: one array of rows per dataset, keyed by name."); - let total = 0; - for (const [name, rows] of Object.entries(data)) { - if ( - !Array.isArray(rows) || - rows.some((r) => typeof r !== "object" || r === null || Array.isArray(r)) - ) - err(`data.${name}`, "must be an array of row objects."); - else total += rows.length; - } - const size = JSON.stringify(data).length; - if (size > 3_000_000) - err( - "data", - `is ${(size / 1e6).toFixed(1)} MB; aggregate it in the prep script so the page stays under 3 MB.`, - ); - else if (total > 20000) - warn( - "data", - `${total} rows in all; aggregate in the prep script unless a table needs every row.`, - ); - const fieldsOf = (name) => { - const set = new Set(); - for (const r of data[name] ?? []) for (const k of Object.keys(r)) set.add(k); - return set; - }; - const needData = (name, where) => { - if (!name) return err(where, "add `data`: the name of a dataset.") ?? false; - if (!(name in data)) - return ( - err( - where, - `there's no dataset \`${name}\`. The datasets are ${Object.keys(data) - .map((x) => `\`${x}\``) - .join(", ")}.`, - ) ?? false - ); - return true; - }; - const needField = (name, f, where, what) => { - if (f == null) return; - const fields = fieldsOf(name); - for (const one of Array.isArray(f) ? f : [f]) - if (!fields.has(one)) - err( - where, - `${what} \`${one}\` isn't a field of \`${name}\`. Its fields are ${[...fields].map((x) => `\`${x}\``).join(", ")}.`, - ); - }; - - const listFields = new Set(); - for (const [i, f] of (report.filters ?? []).entries()) { - const where = `filters[${i}]`; - keys(f, "filter", where); - if (f.type && !["tabs", "list", "toggle"].includes(f.type)) - err( - where, - "`type` must be `tabs` (a few periods or modes), `list` (names with a color and a count) or `toggle` (a checkbox in the top bar).", - ); - if (!f.field) err(where, "add the `field` it filters on."); - else if (!Object.values(data).some((rows) => rows.some?.((r) => f.field in r))) - err(where, `no dataset has a \`${f.field}\` field, so this filter would do nothing.`); - if (f.count) needData(f.count, where); - if ((f.type ?? (f.options && f.options.length <= 5 ? "tabs" : "list")) === "list") - listFields.add(f.field); - } - - let escapes = 0; - const checkBlock = (b, at) => { - const kind = b.type; - if (!TYPES.includes(kind)) - return err(at, `\`type\` must be one of ${TYPES.map((t) => `\`${t}\``).join(", ")}.`); - keys(b, kind, at); - text(b.title, LIMITS.blockTitle, at, "`title`"); - text(b.note, LIMITS.note, at, "`note`", true); - if (b.note && !b.title) - err(at, "a `note` shows under the block's `title`; add a `title` or drop the `note`."); - if (kind === "stats" || kind === "facts") { - const max = kind === "stats" ? 6 : 4; - if (!b.items?.length) err(at, "add `items`."); - if (b.items?.length > max) - err(at, `has ${b.items.length} items; keep the ${max} or fewer that matter.`); - for (const [si, s] of (b.items ?? []).entries()) { - const w = `${at} > items[${si}]`; - keys(s, kind === "stats" ? "stat" : "fact", w); - if (kind === "stats") { - text(s.label, LIMITS.statLabel, w, "`label`"); - text(s.hint, LIMITS.hint, w, "`hint`"); - } else { - if (!s.text) err(w, "add `text`: one sentence that says what the number means."); - text(s.text, LIMITS.fact, w, "`text`", true); - } - for (const slot of ["value", "delta"]) { - const v = s[slot]; - if (v && typeof v === "object" && needData(v.data, `${w}.${slot}`)) { - if (v.field == null && v.agg !== "count") - err(`${w}.${slot}`, 'add `field`, or `"agg": "count"` to count the rows.'); - needField(v.data, v.field, `${w}.${slot}`, "`field`"); - } - } - if (s.format && !FORMATS.includes(s.format)) - err(w, `\`format\` must be one of ${FORMATS.join(", ")}.`); - } - return; - } - if (kind === "vega") { - escapes++; - if (!b.spec || typeof b.spec !== "object") - err(at, 'add `spec`: a Vega-Lite spec whose `data` is `{ "name": "" }`.'); - else if (b.spec.data?.name) needData(b.spec.data.name, at); - warn(at, "is a raw Vega-Lite block. Use another block when one fits."); - return; - } - if (!b.title && kind !== "table") err(at, "add a `title` that says what is measured."); - if (!needData(b.data, at)) return; - const rows = data[b.data] ?? []; - tones(b.tones, at); - if (b.format && !FORMATS.includes(b.format)) - err(at, `\`format\` must be one of ${FORMATS.join(", ")}.`); - if (kind === "bars") { - if (!b.label) err(at, "add `label`: the field that names each row."); - if (!b.value && !b.segments) - err(at, "add `value` (one number per row) or `segments` (fields that add up to the bar)."); - needField(b.data, b.label, at, "`label`"); - needField(b.data, b.value, at, "`value`"); - needField(b.data, b.segments, at, "`segments`"); - needField(b.data, b.end, at, "`end`"); - if (rows.length > 30 && !b.limit) - err(at, `has ${rows.length} rows; set \`limit\` (at most 30) or use a table.`); - } - if (kind === "matrix" || (kind === "chart" && b.chart === "heatmap")) { - needField(b.data, b.x, at, "`x`"); - needField(b.data, b.y, at, "`y`"); - needField(b.data, b.value, at, "`value`"); - if (b.tint && !["columns", "rows", "single"].includes(b.tint)) - err( - at, - "`tint` must be `columns` (each column in its value's color), `rows`, or `single`.", - ); - const cols = new Set(rows.map((r) => r[b.x])).size; - if (cols > 24) err(at, `has ${cols} columns; keep at most 24, or swap \`x\` and \`y\`.`); - if (kind === "matrix") return; - } - if (kind === "chart") { - if (!CHARTS.includes(b.chart)) - return err(at, `\`chart\` must be one of ${CHARTS.join(", ")}.`); - if (b.chart === "heatmap") return; - needField(b.data, b.x, at, "`x`"); - if (b.chart !== "histogram") needField(b.data, b.y, at, "`y`"); - needField(b.data, b.series, at, "`series`"); - needField(b.data, b.size, at, "`size`"); - needField(b.data, b.label, at, "`label`"); - if (!b.x) err(at, "add `x`."); - if (!b.y && b.chart !== "histogram") err(at, "add `y`."); - const series = Array.isArray(b.y) - ? b.y.length - : b.series - ? new Set(rows.map((r) => r[b.series])).size - : 1; - if (series > MAX_SERIES) - err( - at, - `has ${series} series; show at most ${MAX_SERIES}. Keep the largest and sum the rest into one "Other" series in the prep script, or use a matrix.`, - ); - if (["bar", "stacked", "grouped"].includes(b.chart) && !b.limit) { - const cats = new Set(rows.map((r) => r[b.x])).size; - if (cats > 40) - err(at, `has ${cats} bars; set \`limit\` to the top 10–25, or use a line or histogram.`); - } - if (b.stack != null && ["line", "scatter", "histogram"].includes(b.chart)) - err(at, "`stack` applies to area and bar charts; draw shares as an `area` chart."); - if (b.horizontal && !["bar", "stacked", "grouped"].includes(b.chart)) - err( - at, - "`horizontal` only applies to bar, stacked and grouped charts; a `bars` block is often better.", - ); - } - if (kind === "table") { - if (!b.columns?.length) - err(at, "add `columns`: `{ field, label, format }` for each column to show, at most 9."); - if (b.columns?.length > 9) err(at, `has ${b.columns.length} columns; keep at most 9.`); - if (b.style && b.style !== "rows") - err( - at, - '`style` can only be `"rows"`: a dense list with no header, for the evidence behind the numbers.', - ); - for (const [ci, c] of (b.columns ?? []).entries()) { - const w = `${at} > columns[${ci}]`; - keys(c, "column", w); - if (c.format === "stack") { - if (!c.fields?.length) - err( - w, - "a `stack` column needs `fields`: the numeric fields that make up the little bar.", - ); - needField(b.data, c.fields, w, "`fields`"); - tones(c.tones, w); - } else needField(b.data, c.field, w, "`field`"); - if (c.format && !COLUMN_FORMATS.includes(c.format)) - err(w, `\`format\` must be one of ${COLUMN_FORMATS.join(", ")}.`); - if (c.format === "entity" && !listFields.has(c.field)) - warn( - w, - `\`entity\` shows a colored dot only for a field with a list filter; \`${c.field}\` has none.`, - ); - if (c.suffix) needField(b.data, c.suffix, w, "`suffix`"); - } - } - }; - - for (const [i, b] of (report.top ?? []).entries()) checkBlock(b, `top[${i}]`); - for (const [i, b] of (report.bottom ?? []).entries()) checkBlock(b, `bottom[${i}]`); - if ((report.top ?? []).length > 2) - err("report", "`top` holds at most 2 blocks (the numbers that stay on every tab)."); - if ((report.bottom ?? []).length > 1) - err( - "report", - "`bottom` holds at most 1 block (the list of evidence rows that stays on every tab).", - ); - - const views = report.views ?? []; - if (!views.length) - err("report", "add `views`: the tabs of the report, each with `id`, `label` and `blocks`."); - if (views.length > 7) - err("report", `has ${views.length} views; keep at most 7 tabs and merge the rest.`); - const ids = new Set(); - for (const [vi, view] of views.entries()) { - const where = `views[${vi}] "${view.label ?? view.id}"`; - keys(view, "view", where); - if (!view.id || !/^[a-z0-9-]+$/.test(view.id)) - err(where, "give it an `id` in kebab-case, like `growth`."); - if (ids.has(view.id)) err(where, `the id \`${view.id}\` is used twice.`); - if (view.id === "method" && (report.method || report.intro)) - err( - where, - "the id `method` belongs to the Method tab that `method` and `intro` add; rename this view.", - ); - ids.add(view.id); - if (!view.label) err(where, "add a `label` for its tab."); - text(view.label, LIMITS.label, where, "`label`"); - text(view.summary, LIMITS.summary, where, "`summary`", true); - const blocks = view.blocks ?? []; - if (!blocks.length) err(where, "add `blocks`."); - if (blocks.length > 8) - err( - where, - `has ${blocks.length} blocks; keep at most 8 per view, or split it into two views.`, - ); - for (const [bi, b] of blocks.entries()) - checkBlock(b, `${where} > blocks[${bi}]${b.title ? ` "${b.title}"` : ""}`); - } - return { errors, warnings, escapes, rows: total, size }; -} diff --git a/skills/report/SKILL.md b/skills/report/SKILL.md index 4def3e14..9431c4c5 100644 --- a/skills/report/SKILL.md +++ b/skills/report/SKILL.md @@ -8,7 +8,7 @@ license: MIT The reader scans a report before reading it. They want the answer, the decisions they have to make, and the numbers behind both, and they check the rest only when they doubt something. So the page has a fixed shape: the answer on top, one picture per section, short table cells, and detail folded underneath. -For findings about something that already exists, such as a code review or an audit, use `review-document`. For how something works, use `visual-explanation`. For data the reader explores with filters, use `data-explorer`. This skill shapes a report you already have the material for; gathering the material is the research itself. +For findings about something that already exists, such as a code review or an audit, use `review-document`. For how something works, use `visual-explanation`. For a dashboard the reader explores with filters, use Drop's `data-explorer` skill. This skill shapes a report you already have the material for; gathering the material is the research itself. ## 1. Write the answer and the decisions diff --git a/test/data-explorer.test.ts b/test/data-explorer.test.ts deleted file mode 100644 index e2c99533..00000000 --- a/test/data-explorer.test.ts +++ /dev/null @@ -1,56 +0,0 @@ -// The data-explorer skill's build fills assets/page.html, the same page an agent without Node fills by hand. This runs -// it on the example in references/blocks.md, as an agent would, and reads the page it writes. -import { execFileSync } from "node:child_process"; -import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { fileURLToPath } from "node:url"; -import { afterAll, describe, expect, it } from "vite-plus/test"; - -const skill = fileURLToPath(new URL("../skills/data-explorer/", import.meta.url)); -const reference = readFileSync(join(skill, "references/blocks.md"), "utf8"); -const example = JSON.parse(/```json\n([\s\S]*?)\n```/.exec(reference)![1]!); -const dir = mkdtempSync(join(tmpdir(), "data-explorer-")); -afterAll(() => rmSync(dir, { recursive: true, force: true })); - -function build(report: unknown) { - const input = join(dir, "report.json"); - const output = join(dir, "report.html"); - writeFileSync(input, JSON.stringify(report)); - try { - const log = execFileSync("node", [join(skill, "scripts/build.mjs"), input, output], { - cwd: dir, - encoding: "utf8", - }); - return { ok: true, log, html: readFileSync(output, "utf8") }; - } catch (error) { - return { ok: false, log: String((error as { stdout: string }).stdout), html: "" }; - } -} - -describe("the data-explorer build", () => { - it("fills every slot of the page with the example", () => { - const { ok, html } = build(example); - expect(ok).toBe(true); - expect(html).toContain(`${example.title}`); - const data = /