Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,30 @@ product surface; all optional dependencies degrade gracefully).
- **Assistant frontend component** — chat interface with draft confirmation cards for all semantic layer entities
- **Assistant tests** — 33 unit tests for agent normalizers and service branching

### Added (Phase 2 - Durable analytics artifacts)
- **Saved queries** (migration `005`) — named, owned, re-runnable NL question + pinned SQL with
typed parameters (`{{date_from}}`, `{{region}}`); versioned, clone/fork. Connection-scoped,
mirroring the semantic-layer ownership model. Save directly from a query result.
- **Result cache + snapshots** — runs are persisted to a Postgres `result_snapshots` table that
doubles as a cache keyed by `sha256(final_sql + params + connection_id)`; cache-first re-runs
within `RESULT_CACHE_TTL_SECONDS` (default 300) with a manual-refresh override, so dashboards
don't re-hit the warehouse on every load.
- **Charts** — a persisted chart config per saved query (table/line/bar/area/pie/scatter),
rendered with Recharts.
- **Export** — client-side CSV/JSON of any result, plus a backend CSV/JSON/XLSX export endpoint
for saved queries (`openpyxl` via the optional `export` extra).
- **Type-safe parameter rendering** — `saved_query_service.render_sql` substitutes `{{param}}`
placeholders with validated, escaped SQL literals (defense-in-depth on top of the read-only
SQL safety blocklist).
- **Dashboards** (migration `006`) — workspace-scoped, shareable dashboards composed of tiles in a
draggable/resizable grid (react-grid-layout); each tile renders a saved query as a chart or
table with optional per-tile auto-refresh.
- **Dashboard-level filters** — a dashboard defines named filters whose values flow into every
tile's run; they reuse the saved-query parameter system, so a tile consumes only the filters its
SQL references.
- New optional dependency extra: `export` (`openpyxl`). Frontend adds `recharts` and
`react-grid-layout`.

## [1.0.0] - 2026-06-04

First stable release: natural-language-to-SQL with a semantic metadata layer.
Expand Down
16 changes: 14 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Project Overview

QueryWise — a text-to-SQL application with a semantic metadata layer. Users ask natural language questions, an LLM generates SQL using business context, executes against their database, and returns human-readable answers. The conversational Assistant enables editing the semantic layer in plain language.
QueryWise — a text-to-SQL application with a semantic metadata layer. Users ask natural language questions, an LLM generates SQL using business context, executes against their database, and returns human-readable answers. The conversational Assistant enables editing the semantic layer in plain language. Answers become durable, shareable artifacts: saved queries (pinned SQL + typed params), charts, and workspace dashboards.

## Tech Stack

Expand Down Expand Up @@ -94,7 +94,7 @@ frontend/src/
├── api/ # Axios API clients (one per resource)
├── components/layout/ # AppShell with sidebar navigation
├── hooks/ # React Query hooks
├── pages/ # Route pages (Query, Connections, Glossary, Metrics, Dictionary, Knowledge, History)
├── pages/ # Route pages (Query, SavedQueries, Dashboards, Connections, Glossary, Metrics, Dictionary, Knowledge, History)
└── types/ # TypeScript interfaces matching backend schemas
```

Expand Down Expand Up @@ -246,3 +246,15 @@ Real users, teams, roles, and ownership. Single-tenant per deployment; isolation
- **AuthZ in services** (per the existing convention): `connection_service` scopes by org+workspace and enforces role; metadata endpoints authorize through the connection (the cascade root) via `app/api/v1/deps.py` (`require_connection_read/write`, `require_column_read/write`). Non-request entry points — startup auto-setup, the MCP server, the seed script via `DISABLE_AUTH` — act under `identity_service.system_context()` (admin in the default workspace).
- **Endpoints:** `/auth/*` (login, register, magic-link request/verify, logout, me, providers), `/teams` + `/teams/{id}/members` (admin-managed), `/api-keys` (per-user, plaintext shown once).
- **Heads-up:** once auth is enforced, the current (pre-auth) frontend gets 401s — run with `DISABLE_AUTH=true` until the Phase 1 frontend (login + auth context + workspace switcher) lands.

## Durable analytics artifacts (Phase 2)

One-shot answers become saved, owned, re-runnable, shareable objects. Two milestones; migrations `005` (artifacts) and `006` (dashboards).

- **Models** (`app/db/models/`): `SavedQuery` (pinned SQL + typed `params` + `version`/`status`), `Chart` (viz config per saved query), `ResultSnapshot` (result persistence that doubles as the cache), `Dashboard` + `DashboardTile`.
- **Scoping:** saved queries / charts / snapshots are **connection-scoped** (carry `organization_id` + `connection_id`, authorize through the connection via `require_connection_read/write`), matching the semantic-layer convention. `Dashboard` is the first **workspace-scoped** artifact (`workspace_id`); its endpoints use `get_org_context` + `ctx.require_role(...)` directly (like `teams.py`), and `dashboard_service._assert_access` mirrors `connection_service._assert_access`.
- **Re-runs & cache** (`app/services/saved_query_service.py`): `render_sql` substitutes `{{param}}` placeholders with **type-safe, escaped SQL literals** (defense-in-depth atop the read-only blocklist), then `run_saved_query` is cache-first — a `ResultSnapshot` keyed by `sha256(final_sql + params + connection_id)` within `RESULT_CACHE_TTL_SECONDS` (default 300), with a `refresh` override. Execution reuses `query_service.execute_raw_sql`.
- **Dashboards** (`app/services/dashboard_service.py`): tiles run via `run_saved_query` (so they inherit connection auth + the cache). Dashboard-level **filters reuse the param system** — filter values are passed as supplied params and a tile only consumes the `{{name}}`s its SQL references. `_finalize` refreshes server-side `onupdate` timestamps after UPDATEs to avoid async lazy-load errors during response serialization.
- **Export:** client-side CSV/JSON in the frontend; backend CSV/JSON/XLSX for saved queries (XLSX needs the optional `export` extra → `openpyxl`).
- **Endpoints:** `/connections/{id}/saved-queries` (+ `/run`, `/clone`, `/export`, `/charts`), `/dashboards` (+ `/tiles`, `/layout`, `/tiles/{id}/run`).
- **Frontend:** Recharts (`components/charts/ChartView.tsx`) for viz; `react-grid-layout` for the dashboard grid; shared typed `components/common/ParamInputs.tsx` for params/filters. Charts are managed inside the saved-query view (no separate Charts page). Note: the frontend container's anonymous `node_modules` volume means new deps (recharts, react-grid-layout) need `docker compose exec frontend npm install` or an image rebuild.
62 changes: 54 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,9 @@ A full-stack application that translates natural language questions into SQL que
- **Schema introspection** — auto-discovers tables, columns, types, relationships from target databases
- **Conversational Assistant** — chat panel for NL queries and semantic layer editing (glossary terms, metrics, dictionary entries, knowledge)
- **Identity, teams, and ownership** — real users, roles (viewer/editor), teams, and workspace-based ownership
- **Saved queries** — name and pin a question + SQL with typed parameters (`{{region}}`); re-run, version, clone, and export (CSV/JSON/XLSX)
- **Charts & result caching** — visualize a saved query (line/bar/area/pie/scatter via Recharts); results are snapshotted to a Postgres cache so re-runs don't re-hit the warehouse
- **Dashboards** — compose saved queries into a shareable, draggable tile grid with dashboard-level filters that flow into every tile's SQL
- **Production hardening** — rate limiting, async job queue, OpenTelemetry tracing, structured logging, health probes


Expand Down Expand Up @@ -448,7 +451,12 @@ querywise/
│ │ │ ├── dictionary.py # DictionaryEntry (value mappings)
│ │ │ ├── knowledge.py # KnowledgeDocument + KnowledgeChunk (with embedding vector)
│ │ │ ├── sample_query.py # SampleQuery (with embedding vector)
│ │ │ └── query_history.py# QueryExecution (full audit log)
│ │ │ ├── query_history.py# QueryExecution (full audit log)
│ │ │ ├── saved_query.py # SavedQuery (pinned SQL + typed params)
│ │ │ ├── chart.py # Chart (viz config per saved query)
│ │ │ ├── result_snapshot.py # ResultSnapshot (result persistence + cache)
│ │ │ ├── dashboard.py # Dashboard (workspace-scoped, with filters)
│ │ │ └── dashboard_tile.py # DashboardTile (grid position + refresh)
│ │ ├── api/v1/
│ │ │ ├── router.py # Aggregates all endpoint routers
│ │ │ ├── endpoints/
Expand All @@ -461,7 +469,9 @@ querywise/
│ │ │ │ ├── sample_queries.py
│ │ │ │ ├── knowledge.py # Knowledge document CRUD + URL fetch
│ │ │ │ ├── query.py # POST /query (full pipeline), POST /query/sql-only
│ │ │ │ └── query_history.py# History list + favorite toggle
│ │ │ │ ├── query_history.py# History list + favorite toggle
│ │ │ │ ├── saved_queries.py# Saved query CRUD + run/clone/export + charts
│ │ │ │ └── dashboards.py # Dashboard + tile CRUD, layout, tile run
│ │ │ └── schemas/ # Pydantic request/response models
│ │ ├── services/
│ │ │ ├── query_service.py # Full pipeline orchestrator
Expand Down Expand Up @@ -519,24 +529,37 @@ querywise/
├── main.tsx # MantineProvider + QueryClient + Router
├── App.tsx # Route definitions
├── api/
│ ├── client.ts # Axios instance
│ ├── client.ts # Axios instance (session cookie + workspace header)
│ ├── connectionApi.ts # Connection endpoints
│ ├── queryApi.ts # Query + history endpoints
│ ├── glossaryApi.ts # Glossary + metrics + dictionary endpoints
│ └── knowledgeApi.ts # Knowledge document CRUD + URL fetch
│ ├── knowledgeApi.ts # Knowledge document CRUD + URL fetch
│ ├── savedQueriesApi.ts # Saved query CRUD + run/clone/export + charts
│ └── dashboardsApi.ts # Dashboard + tile CRUD, layout, tile run
├── components/
│ └── layout/
│ └── AppLayout.tsx # Mantine AppShell with sidebar nav
│ ├── layout/
│ │ └── AppLayout.tsx # Mantine AppShell with sidebar nav
│ ├── charts/
│ │ └── ChartView.tsx # Recharts renderer (line/bar/area/pie/scatter)
│ ├── common/
│ │ └── ParamInputs.tsx # Typed param/filter inputs (shared)
│ ├── savedQueries/ # Run drawer + form modal
│ └── dashboards/ # Grid, tile card, filters bar, modals
├── hooks/
│ └── useConnections.ts # React Query hooks for connections
│ ├── useConnections.ts # React Query hooks for connections
│ ├── useSavedQueries.ts # Saved query + chart hooks
│ └── useDashboards.ts # Dashboard + tile hooks
├── pages/
│ ├── QueryPage.tsx # NL input → SQL preview → results table
│ ├── ConnectionsPage.tsx # Add/edit/delete/test/introspect connections
│ ├── GlossaryPage.tsx # Business glossary term management
│ ├── MetricsPage.tsx # Metric definition management
│ ├── DictionaryPage.tsx # Column value mapping management
│ ├── KnowledgePage.tsx # Knowledge document import/manage (text + URL fetch)
│ └── HistoryPage.tsx # Query execution history + favorites
│ ├── HistoryPage.tsx # Query execution history + favorites
│ ├── SavedQueriesPage.tsx# Saved queries: run, chart, export
│ ├── DashboardsPage.tsx # Dashboard list
│ └── DashboardDetailPage.tsx # Dashboard grid + filters
└── types/
└── api.ts # TypeScript interfaces
```
Expand Down Expand Up @@ -708,6 +731,29 @@ All endpoints are under `/api/v1`.
| `POST` | `/query` | Execute NL query (full pipeline) |
| `POST` | `/query/sql-only` | Generate SQL without executing |

### Saved Queries

| Method | Path | Description |
|--------|------|-------------|
| `GET/POST` | `/connections/{id}/saved-queries` | List/create saved queries |
| `GET/PUT/DELETE` | `/connections/{id}/saved-queries/{sq_id}` | Get/update/delete saved query |
| `POST` | `/connections/{id}/saved-queries/{sq_id}/run` | Run (cache-first; `refresh` to bypass) |
| `POST` | `/connections/{id}/saved-queries/{sq_id}/clone` | Clone a saved query |
| `GET` | `/connections/{id}/saved-queries/{sq_id}/export` | Export results (`format=csv\|json\|xlsx`) |
| `GET/POST` | `/connections/{id}/saved-queries/{sq_id}/charts` | List/create charts |
| `PUT/DELETE` | `/connections/{id}/saved-queries/{sq_id}/charts/{chart_id}` | Update/delete chart |

### Dashboards

| Method | Path | Description |
|--------|------|-------------|
| `GET/POST` | `/dashboards` | List/create dashboards (workspace-scoped) |
| `GET/PUT/DELETE` | `/dashboards/{id}` | Get/update/delete dashboard |
| `POST` | `/dashboards/{id}/tiles` | Add a tile |
| `PUT/DELETE` | `/dashboards/{id}/tiles/{tile_id}` | Update/delete a tile |
| `PUT` | `/dashboards/{id}/layout` | Bulk-save tile positions |
| `POST` | `/dashboards/{id}/tiles/{tile_id}/run` | Run a tile with dashboard filters |

### History

| Method | Path | Description |
Expand Down
98 changes: 98 additions & 0 deletions backend/alembic/versions/006_dashboards.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
"""Dashboards and tiles (Phase 2 — Milestone 2)

Revision ID: 006
Revises: 005
Create Date: 2026-06-08

Adds workspace-scoped dashboards and their tiles. A dashboard composes saved
queries (from any connection in the workspace) into a draggable grid; tiles
render a saved query as a chart or table, and dashboard-level filters flow into
each tile's run via the saved-query parameter system.
"""

from collections.abc import Sequence

import sqlalchemy as sa
from sqlalchemy.dialects.postgresql import JSONB, UUID

from alembic import op

# revision identifiers, used by Alembic.
revision: str = "006"
down_revision: str = "005"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None


def upgrade() -> None:
op.create_table(
"dashboards",
sa.Column("id", UUID(as_uuid=True), primary_key=True),
sa.Column(
"organization_id",
UUID(as_uuid=True),
sa.ForeignKey("organizations.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
"workspace_id",
UUID(as_uuid=True),
sa.ForeignKey("teams.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
"owner_id",
UUID(as_uuid=True),
sa.ForeignKey("users.id", ondelete="SET NULL"),
nullable=True,
),
sa.Column("name", sa.String(255), nullable=False),
sa.Column("description", sa.Text, nullable=True),
sa.Column("filters", JSONB, server_default=sa.text("'[]'::jsonb")),
sa.Column("is_public", sa.Boolean, nullable=False, server_default=sa.text("false")),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
)
op.create_index("ix_dashboards_workspace_id", "dashboards", ["workspace_id"])

op.create_table(
"dashboard_tiles",
sa.Column("id", UUID(as_uuid=True), primary_key=True),
sa.Column(
"organization_id",
UUID(as_uuid=True),
sa.ForeignKey("organizations.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
"dashboard_id",
UUID(as_uuid=True),
sa.ForeignKey("dashboards.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
"saved_query_id",
UUID(as_uuid=True),
sa.ForeignKey("saved_queries.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
"chart_id",
UUID(as_uuid=True),
sa.ForeignKey("charts.id", ondelete="SET NULL"),
nullable=True,
),
sa.Column("title", sa.String(255), nullable=True),
sa.Column("position", JSONB, server_default=sa.text("'{}'::jsonb")),
sa.Column("refresh_interval", sa.Integer, nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
)
op.create_index("ix_dashboard_tiles_dashboard_id", "dashboard_tiles", ["dashboard_id"])


def downgrade() -> None:
op.drop_index("ix_dashboard_tiles_dashboard_id", table_name="dashboard_tiles")
op.drop_table("dashboard_tiles")
op.drop_index("ix_dashboards_workspace_id", table_name="dashboards")
op.drop_table("dashboards")
Loading
Loading