feat(cli): Add shiny docs command for component and controller documentation lookup with autocomplete

[karangattu]

Overview

Adds the shiny docs CLI command and the shiny-docs Agent Skill to inspect signatures, types, parameters, and docstrings directly from the terminal for Shiny functions, classes, Playwright controllers, and methods.

---

Key Capabilities & Features

1. UI Components & Functions (Implied shiny. prefix)

You don't need to type shiny.ui.value_box—typing ui.value_box is enough because shiny. is implied.

$ shiny docs ui.value_box

Output:

def value_box(title: TagChild, value: TagChild, *args: TagChild, showcase: TagChild | None = None, showcase_layout: ShowcaseLayout | None = None, full_screen: bool = False, theme: str | ValueBoxTheme | None = None, height: str | None = None, max_height: str | None = None, fill: bool = True, class_: str | None = None, id: str | None = None, **kwargs: TagAttrValue) -> Tag:
Value box

An opinionated (:func:`~shiny.ui.card`-powered) box, designed for
displaying a `value` and `title`. Optionally, a `showcase` can provide context
for what the `value` represents (for example, it could hold an icon, or even a
:func:`~shiny.ui.output_plot`).
...

2. Playwright Controllers & Methods

Inspect test controller classes along with all their available methods, or inspect a single method directly.

# Inspect a controller class and all its methods
$ shiny docs playwright.controller.Accordion

# Inspect a specific method
$ shiny docs playwright.controller.Accordion.expect_height

Output:

def expect_height(self, height: str | int, *, timeout: float | None = None) -> None:
Expects the accordion to have the specified height.

Parameters
----------
height : str | int
    The expected height of the accordion.
timeout : float | None
    The maximum time to wait for the expectation to be met.

3. Multiple Lookups with Error-First Output

When requesting multiple items in a single command, any errors are printed first before rendering documentation for the valid items.

$ shiny docs ui.card non_existent_symbol ui.value_box

Output:

Error: Could not find documentation for 'non_existent_symbol'.

def card(*args: TagChild, full_screen: bool = False, height: str | None = None, max_height: str | None = None, min_height: str | None = None, fill: bool = True, class_: str | None = None, id: str | None = None, **kwargs: TagAttrValue) -> Tag:
A Bootstrap card component

...

---

def value_box(title: TagChild, value: TagChild, *args: TagChild, ...) -> Tag:
Value box

...

4. Typo & "Did You Mean" Fuzzy Matching Suggestions

If you make a typo or omit a module path, shiny docs uses fuzzy matching to suggest close matches.

# Typo in function name
$ shiny docs ui.value_bx
# Error: Could not find documentation for 'ui.value_bx'. Did you mean 'ui.value_box'?

# Bare name without module path
$ shiny docs Accordion
# Error: Could not find documentation for 'Accordion'. Did you mean one of: 'playwright.controller.Accordion', 'ui.accordion', 'express.ui.accordion'?

# Typo in method name
$ shiny docs playwright.controller.Accordion.expect_hight
# Error: Could not find documentation for 'playwright.controller.Accordion.expect_hight'. Did you mean 'playwright.controller.Accordion.expect_height'?

5. Explicit Paths Required (No Bare Magic Lookups)

Bare names like Accordion or value_box without module paths are not resolved implicitly. This prevents confusion between UI components (e.g. ui.accordion) and test controllers (e.g. playwright.controller.Accordion).

$ shiny docs Accordion
# Error: Could not find documentation for 'Accordion'. Did you mean one of: 'playwright.controller.Accordion', 'ui.accordion', 'express.ui.accordion'?

6. Machine-Readable JSON Mode (--json)

Pass --json to output structured JSON data containing AST-extracted signatures, parameter lists with type annotations and defaults, return types, and docstrings.

$ shiny docs --json ui.card

Output:

[
  {
    "name": "shiny.ui.card",
    "signature": "def card(*args: TagChild, full_screen: bool = False, ...):",
    "type": "function",
    "return_type": "Tag",
    "parameters": [
      {
        "name": "*args",
        "type": "TagChild",
        "default": null,
        "kind": "VAR_POSITIONAL"
      },
      {
        "name": "full_screen",
        "type": "bool",
        "default": "False",
        "kind": "POSITIONAL_OR_KEYWORD"
      }
    ],
    "docstring": "A Bootstrap card component\n\n..."
  }
]

7. Shell Tab Completion & Autocomplete (--complete)

Supports interactive shell tab completion (bash, zsh, fish) and programmatic query completion with the --complete flag.

$ shiny docs --complete playwright.controller.Accordion.expect

Output:

playwright.controller.Accordion.expect_class
playwright.controller.Accordion.expect_height
playwright.controller.Accordion.expect_open
playwright.controller.Accordion.expect_width

8. AST-Based Signature Extraction

Signatures are parsed directly from source files using Python's ast module rather than reconstructed runtime representations. This guarantees accurate parameter annotations, positional-only/keyword-only markers, default values, and return types as written in the source.

9. Bundled shiny-docs Agent Skill

Includes a bundled Agent Skill in shiny/.agents/skills/shiny-docs/SKILL.md (and .claude/skills/shiny-docs/SKILL.md) so coding agents automatically know how to query shiny docs when writing Shiny apps or tests.

---

Multi-Model Evaluation: Control vs. Treatment

We tested the impact of the shiny-docs skill across Gemini 3.7 Flash, GPT-5.6 Luna, and GLM-5.3-Flash comparing a Control group (without docs lookup) and a Treatment group (with shiny docs reference).

The Benchmark Task Given to All Models

Create a complete Shiny for Python application (`app.py`) and Playwright test suite (`test_app.py`).

Requirements:
1. UI Layout (`app.py`):
   - Use `ui.page_sidebar` with `ui.sidebar`.
   - In the main area, arrange 3 KPI `ui.value_box` components inside `ui.layout_columns`.
   - Each `ui.value_box` must specify:
     * `showcase` (e.g., an icon or plot)
     * `showcase_layout` set to "left center"
     * `theme` using presets like "teal", "primary", "bg-gradient-blue-purple"
     * `full_screen=True`
   - Include a collapsible `ui.accordion` with id="analytics_acc" containing two `ui.accordion_panel` components.
   - Include an interactive table using `@render.data_frame` returning a `render.DataGrid` with `selection_mode="rows"`.

2. Reactivity (`app.py`):
   - Use `@reactive.calc` for filtered metrics.

3. Automated Playwright Test Suite (`test_app.py`):
   - Use `shiny.playwright.controller.ValueBox` with `.expect_title()` and `.expect_value()`.
   - Use `shiny.playwright.controller.Accordion` with `.expect_open()` and `.set()`.

---

Benchmark Results: Gemini 3.7 Flash & GPT-5.6 Luna

1. Functional Area Differences

Functional Area	Control (Without shiny docs)	Treatment (With shiny docs)
Value Boxes (ui.value_box)	❌ Hallucinates icon=..., color=... (from R Shiny), causing TypeError at startup	✅ Uses exact showcase=ui.span(...) and theme="teal" / theme="bg-gradient-blue-purple"
Data Tables (render.data_frame)	❌ Attempts render.data_frame(df, grid=True), causing TypeError	✅ Returns render.DataGrid(df, selection_mode="rows")
ValueBox Test Assertions	❌ Hallucinates .get_value(), causing AttributeError in test suites	✅ Uses .expect_value() and .expect_title()
Accordion Test Assertions	❌ Hallucinates .should_be_open() or .is_open(), causing AttributeError	✅ Uses .expect_open(["panel"]) and .set(["panel"])
Reactivity	Filters inline inside renderers	Structured @reactive.calc pipeline

2. Live Performance, Latency, and Cost

Model	Provider	Condition	First-Turn Pass	Input Tokens	Output Tokens	Latency	Actual Cost (USD)
Gemini 3.7 Flash	Google	Control	❌ Failed (API Errors)	153	636	13.68s	$0.000270
Gemini 3.7 Flash	Google	Treatment	✅ Passed (0 Errors)	240	466	20.20s	$0.000210
GPT-5.6 Luna	OpenAI	Control	❌ Failed (API Errors)	291	2,500	32.36s	$0.018009
GPT-5.6 Luna	OpenAI	Treatment	✅ Passed (0 Errors)	497	2,500	26.92s	$0.018370

Benchmark Results: GLM-5.3-Flash

1. Live Performance and Cost

Condition	First-Turn Result	Input Tokens	Output Tokens	Latency	Cost (USD)
Control (without shiny docs)	❌ Failed (used output tokens on speculative guessing)	308	4,000	91.79s	$0.00102
Treatment (with shiny docs)	✅ Passed (0 Errors)	510	4,000	87.72s	$0.00104

2. Internal Chain-of-Thought (Reasoning Analysis)

GLM-5.3-Flash outputs internal reasoning steps in reasoning_content:

• Control (Speculation Loop): Without documentation, the model generated over 24,000 characters in reasoning trying to remember method names and guessing source file layouts ("Let me check the shiny docs page... I believe it exists with methods expect_title, expect_value... class ValueBox(UiWithVisibility)..."), exhausting its token budget before completing the code.
• Treatment (Direct Signature Grounding): With shiny docs, the model immediately validated the signatures (ui.value_box(showcase=..., theme=...), Accordion.expect_open(), render.DataGrid(selection_mode="rows")) and produced working code without token exhaustion.

3. Takeaway

Providing documentation references stops reasoning models from wasting token budgets and time trying to guess missing API parameters.