Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions agents/connect-external-mcp-servers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,8 @@ MCP servers are configured per project and can be attached to one or more agents
- **Client ID** and **Client secret** (optional) — only needed if the MCP provider doesn't support Dynamic Client Registration (DCR). To get them, go to the MCP provider and register a new OAuth client, provide the callback URL below as the redirect URI, and copy the client ID and client secret the provider gives you back into Lightdash. Leave both blank if the provider supports DCR — Lightdash will register a client automatically.

**Callback URL:** `https://<instance-name>.lightdash.cloud/api/v1/aiAgents/mcp/oauth/callback`

If the provider's consent screen shows something like "Access to this domain is restricted" or "Your admin has blocked this domain" for that URL, the block is on the provider's side — nothing in Lightdash can grant it. Ask that provider's organization admin to allow the callback URL for the OAuth app, then click **Connect** again.
- **Allow shared OAuth credential** — turn this on if you want a manager to connect a single shared account that everyone can fall back to. Leave it off to require each user to connect their own account.
- **None** — no credentials required.
</Step>
Expand Down
2 changes: 1 addition & 1 deletion agents/data-access.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Three controls decide what an agent can reach on behalf of the person asking, an

The first two apply to every answer the agent builds from the semantic layer, which is its default mode. The third sits outside the semantic layer: raw SQL is not filtered by user attributes, and only users who could already open the SQL Runner can turn it on. Agent [instructions](/agents/set-up-agents#instructions) shape how the agent reasons and answers; they can't widen what it reaches. For a worked setup over explores that some users can only partly see, read [Agent access to restricted data](/agents/agent-access-to-restricted-data).

The agent and its model never connect to your warehouse themselves. The model asks for a query through a tool, and Lightdash compiles and runs it over the project's warehouse connection, the same query path Explore and the SQL Runner use. The model only sees the results Lightdash returns, and only when [data access](#data-access-modes) is enabled.
The agent and its model never connect to your warehouse themselves. The model asks for a query through a tool, and Lightdash compiles and runs it over the project's warehouse connection, the same query path Explore and the SQL Runner use. The model only sees the results Lightdash returns, and only when [data access](#data-access-modes) is enabled. On a project that requires personal warehouse credentials, that connection is the signed-in person's own saved credentials — the same ones their charts and Explore queries use. See [When are personal warehouse credentials used?](/personal-settings/personal-warehouse-connections#when-are-personal-warehouse-credentials-used).

On Snowflake, BigQuery, Databricks, Athena, Postgres, Redshift, Trino and ClickHouse, [agent identity](/agents/agent-identity) also lets the warehouse decide what agent queries can read. With agent identity on, Lightdash also checks who can read each saved result. See [Who can read a saved result](/agents/agent-identity#who-can-read-a-saved-result).

Expand Down
2 changes: 1 addition & 1 deletion agents/deep-research.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Because the run executes on the server, you can close the tab or leave the threa
Use the Ask AI composer on the homepage, start a new agent thread, or open an existing thread that you own. Deep research is unavailable in read-only threads, such as another user's thread, a thread started in Slack, or a thread continued from [AI analysis](/data-apps/ai-analysis) of a data app or dashboard. You also can't start a run while [impersonating](/support/support-access#impersonation-by-your-admins) another user.
</Step>
<Step title="Enable Deep research">
Select **+** in the composer, then select **Deep research**. A checkmark shows the mode is active, and the send button starts research instead of sending a chat message. CSV and document attachments are unavailable while the mode is on. When you type a longer investigative question, the **+** button can pulse to suggest this mode; you can still continue in regular chat.
Select **+** in the composer, then select **Deep research**. A checkmark shows the mode is active, and the send button starts research instead of sending a chat message. [CSV and document attachments](/agents/use-ai-agents#attach-a-file) are unavailable while the mode is on. When you type a longer investigative question, the **+** button can pulse to suggest this mode; you can still continue in regular chat.
</Step>
<Step title="Describe the outcome you need">
Include the decision or question, relevant time period, important segments, and any definitions or constraints the agent should preserve.
Expand Down
31 changes: 29 additions & 2 deletions agents/use-ai-agents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,33 @@ The options you see depend on your permissions and your organization's settings:
- **SQL Runner** turns on [SQL mode](#sql-mode) for the conversation. It appears if you can use the SQL Runner.
- **Deep research** turns your question into a [deep research](/agents/deep-research) run, a long-running investigation that ends in a report. It appears when deep research is enabled and your role can start runs.

### Attach a file

Select **+** in the message input and choose **Attach a document**, or drag a file straight into the chat box — the composer shows a **Drop files to analyze them** overlay while you drag. On self-hosted deployments, dragging and dropping a file needs Lightdash v2.425.0 or later; attaching through the **+** menu works on any version.

The document picker accepts `.md`, `.markdown`, `.txt`, `.json`, `.yml`, `.yaml`, `.sql`, `.log`, `.xml`, `.html`, `.toml`, `.csv` and `.tsv`, and the server accepts any UTF-8 text file regardless of extension. Excel files (`.xlsx`) can't be attached — export the sheet to CSV first.

Each file is limited to 1 MB. A prompt inlines only the first 32 KB of its attached files, combined, into the model's context; the agent reads anything beyond that straight from the attached files.

<Note>
When [External sources](/explore/external-sources) is available for your project, a CSV or TSV file you drop or pick imports as a queryable table instead of attaching as a text document. Otherwise it attaches as a plain text document, like any other file.
</Note>

Attachments aren't available in [Deep research](/agents/deep-research#start-a-run) mode.

### Attach a file

The same **+** menu can attach a file for the agent to use:

- **Attach a CSV** uploads the file as a queryable [external source](/explore/external-sources), so the agent can run queries against it alongside your warehouse data. It only appears when external sources are enabled for the project and you can manage external sources (Editor role or above) — see [External sources](/explore/external-sources) for setup and size limits.
- **Attach a document** uploads a plain-text file for the agent to read inline as reference material; it isn't queryable the way an external source is. The picker accepts `.md`, `.markdown`, `.txt`, `.json`, `.yml`, `.yaml`, `.sql`, `.log`, `.xml`, `.html` and `.toml`, but the server accepts any UTF-8 text file regardless of its extension. Each file is capped at 1024 KB; a larger file is rejected with an error naming the limit.

Dragging a file onto the message input works the same way, without opening the menu: a dropped `.csv` or `.tsv` file becomes a queryable external source when **Attach a CSV** is available, and otherwise — including when that option doesn't appear — attaches as a document, the same as any non-CSV file.

<Note>
Excel files (`.xlsx`) can't be attached through either route. Export the sheet to CSV, or to a plain-text format, first.
</Note>

### Model and reasoning

When more than one model is available, the model picker beside the send button shows which model answers. Click it to choose another model. For models that support reasoning, the same menu has a **Reasoning** setting with two levels, **Default** and **High**, and the picker shows **High** beside the model name when it is on.
Expand Down Expand Up @@ -105,7 +132,7 @@ Slack-triggered prompts can only use SQL mode when your organization has the **\
/>
</Frame>

<FeatureBadge label="Beta" /> Some questions need a step after the query: joining two explores, comparing results against a CSV you attach, ranking or bucketing rows, or fitting a forecast. For these, the agent runs one or more queries, then post-processes their results with DuckDB SQL or Python before it answers. It is off by default. [Reach out to us](mailto:support@lightdash.com) to turn it on for your organization.
<FeatureBadge label="Beta" /> Some questions need a step after the query: joining two explores, comparing results against an [attached CSV](#attach-a-file), ranking or bucketing rows, or fitting a forecast. For these, the agent runs one or more queries, then post-processes their results with DuckDB SQL or Python before it answers. It is off by default. [Reach out to us](mailto:support@lightdash.com) to turn it on for your organization.

The semantic layer runs first (or the SQL query, if approved in [SQL mode](#sql-mode)). DuckDB and Python only ever see the rows those queries returned, so metric definitions, joins, [data access](/agents/data-access) rules and SQL approvals apply exactly as they do for any other query. Neither DuckDB nor Python connects to your warehouse.

Expand All @@ -130,7 +157,7 @@ The semantic layer runs first (or the SQL query, if approved in [SQL mode](#sql-
| **Warehouse SQL** | A read-only `SELECT` or `WITH` statement, with [approval](#how-approvals-work) | Your warehouse, in [SQL mode](#sql-mode) |
| **DuckDB** | DuckDB SQL over the results of other steps | A sandbox inside Lightdash |
| **Python** | pandas code over the results of other steps | A sandbox inside Lightdash |
| **External data** | DuckDB SQL over a CSV attached to the thread | A sandbox inside Lightdash |
| **External data** | DuckDB SQL over a [CSV attached to the thread](#attach-a-file) | A sandbox inside Lightdash |

A question that one metric query can answer still runs as a single query. The agent only adds DuckDB or Python steps when the question needs them.

Expand Down
13 changes: 13 additions & 0 deletions explore/dashboards/filter.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,19 @@ Locked filters take precedence over every override mechanism:

The override-stripping behaviour runs regardless of whether the feature flag is on, so dashboards with locked filters stay safe even if the flag is later turned off.

### Locked filters on hidden fields

A filter chip can also show locked without anyone locking it. This happens when the field a saved filter uses is later set to `hidden: true` ([dimensions](/semantic-layer/dimensions#dimension-configuration), [metrics](/semantic-layer/metrics#metric-configuration)) and no tile on the dashboard offers that field in its filter picker any more. Lightdash can't show a picker for a field nothing exposes, so it locks the chip instead, with the same closed-padlock icon and a tooltip reading "This filter applies to your charts. You cannot change it here."

The filter still applies to every tile that used it — hiding the field doesn't remove the filter, it only stops anyone changing its value from the dashboard. Viewers can't edit or remove it, even though no one set it as a deliberate lock.

To replace it with a filter on the new field:

1. Open the dashboard and click **Edit dashboard**.
2. Hover over the locked chip and click its **x** to remove it.
3. Click **Add filter** and pick the field that replaced the hidden one.
4. Click **Save**.

## Group filters into sections

<div className="feature-badges"><FeatureBadge label="Experimental" /></div>
Expand Down
11 changes: 10 additions & 1 deletion explore/table-calculations/functions/row-functions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@ Row functions let you reference values from other rows without writing raw SQL w
Row functions use your query's current sort order to determine row positions. Changing the sort order of your results will change the output of these functions.
</Note>

Row functions operate across your entire results table, with no equivalent to SQL's `PARTITION BY`. If a dimension in your rows splits the data into categories (for example a region or product column), `offset`, `index`, `offset_list`, `lookup`, and `row` don't reset at each category — an `offset(${orders.total_revenue}, -1)` can return a value from a different category's row. If you need an offset or lag that resets per group, either:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this shouldnt be a callout


- Use the [Formula table calculation](/explore/table-calculations/formulas)'s window functions (for example `LAG`), which accept an inline `PARTITION BY` clause, or
- Pivot the dimension into columns and use [pivot functions](/explore/table-calculations/functions/pivot-functions) like `pivot_offset`, which partition by row automatically.

## row

Returns the 1-based row number of the current row.
Expand Down Expand Up @@ -51,6 +56,10 @@ offset(column, rowOffset)
| `column` | column reference | The column to get the value from |
| `rowOffset` | integer | Number of rows to offset. Negative = previous rows, positive = next rows, 0 = current row |

`offset` counts rows across your whole results table, in sort order — it doesn't restart at each value of a dimension. This makes it safe when your results have one row per step (for example a single time dimension), but unreliable once another dimension adds more than one row per step: with three categories alongside a weekly time dimension, `offset(x, -52)` moves back about 17 weeks, not 52, because it also steps across the other categories' rows.

To offset within a group, use a [formula table calculation with `PARTITION BY`](/explore/table-calculations/formulas#window-clauses-partition-by-and-order-by), which restarts `LAG`/`LEAD` for each value of a dimension. For a prior-year or prior-period comparison on a chart with extra dimensions, use [period comparison](/explore/period-over-period), which compares each group against its own prior period.

**Example**

Get the previous row's revenue to calculate period-over-period changes:
Expand All @@ -70,7 +79,7 @@ For positive offsets (next rows):
LEAD(${orders.total_revenue}, 1) OVER (ORDER BY ...)
```

For an offset of 0, the column value is returned directly with no window function.
For an offset of 0, the column value is returned directly with no window function. None of these include a `PARTITION BY`, so the offset always spans the whole results table.
</Accordion>

---
Expand Down
6 changes: 5 additions & 1 deletion personal-settings/personal-warehouse-connections.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Personal warehouse credentials allow you to connect to your data warehouse using

## When are personal warehouse credentials used?

Your personal warehouse credentials are used for every query you run in a project, including charts, dashboards, Explore, and the SQL runner, when:
Your personal warehouse credentials are used for every query you run in a project, including charts, dashboards, Explore, the SQL runner, and AI agents (Ask AI), when:

- The project is configured to require user credentials for data access, or
- The project uses a shared connection that accepts personal credentials. Databricks and Trino projects always do. BigQuery projects do when **Use personal credentials when available** is turned on under **Advanced configuration options** in the project's BigQuery connection settings.
Expand All @@ -24,6 +24,10 @@ On a project with [multiple warehouse connections](/workspace-admin/multiple-war

When a project is created with [organization credentials](/workspace-admin/organization-credentials) but also requires user credentials, your personal credentials override the organization credential for your own queries.

<Note>
If your personal sign-in has expired or fails, Ask AI doesn't show the reconnect prompt that charts, dashboards and Explore show. It replies with text only, for example that the warehouse connection has expired, and that wording can tell you to ask your data team instead of pointing you at your own settings. Reconnect under **Settings → My warehouse connections**, then ask your question again.
</Note>

## Setting up personal warehouse credentials

To configure your personal warehouse credentials:
Expand Down
4 changes: 4 additions & 0 deletions self-host/upgrading.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ The check tells you whether the new version can replace the old one with a rolli

Each release publishes a `release-safety.json` file, on its [GitHub release](https://github.com/lightdash/lightdash/releases) and in its Docker image. It records whether the release can run alongside the version before it, the migrations it runs, and any version you must upgrade through first. The CLI reads a public index of these files, so the check needs no Lightdash login or access to your instance.

<Tip>
Ask your AI coding agent to do this for you with the `upgrade-preflight` [agent skill](/workflow/install-agent-skills#review-what-changed-between-releases), which runs `lightdash upgrade-check` and `migrate preflight` and walks you through the result.
</Tip>

1. Install or update the [Lightdash CLI](/workflow/cli/install) to version `1.126.0` or later.
2. Run `lightdash upgrade-check` with the version you run now and the version you want:

Expand Down
2 changes: 1 addition & 1 deletion semantic-layer/dimensions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ The table below shows all the dimension properties you can customize:
| [description](#description) | No | string | Description of the dimension in Lightdash. You can use this to override the description you have for the dimension in dbt. |
| sql | No | string | Custom SQL applied to the column used to define the dimension. |
| [time_intervals](#time-intervals) | No | 'default' or OFF or an array[] containing elements of [date](#date-options), [numeric](#numeric-options), [string](#string-options) options, or [custom granularity](#using-custom-granularities) names | 'default' (or not setting the time_intervals property) will be converted into ['DAY', 'WEEK', 'MONTH', 'QUARTER', 'YEAR'] for dates and ['RAW', 'DAY', 'WEEK', 'MONTH', 'QUARTER', 'YEAR'] for timestamps; if you want no time intervals set 'OFF'. You can also include custom granularity names defined in `lightdash.config.yml`. |
| hidden | No | boolean | If set to true, the dimension is hidden from Lightdash. By default, this is set to false if you don't include this property. Hidden dimensions are also excluded from drilldowns (View underlying data). |
| hidden | No | boolean | If set to true, the dimension is hidden from Lightdash. By default, this is set to false if you don't include this property. Hidden dimensions are also excluded from drilldowns (View underlying data). Hiding a dimension that a dashboard filter already uses doesn't remove the filter — it still applies, but shows as a [locked filter chip](/explore/dashboards/filter#locked-filters-on-hidden-fields) until an editor replaces it. |
| [compact](#compact) | No | string | This option will compact the number value (e.g. 1,500 to 1.50K). Currently supports one of the following: ['auto', 'thousands', 'millions', 'billions', 'trillions', 'kilobytes', 'megabytes', 'gigabytes', 'terabytes', 'petabytes', 'kibibytes', 'mebibytes', 'gibibytes', 'tebibytes', 'pebibytes']. Use `auto` to dynamically pick K/M/B/T based on each value's magnitude. |
| [format](#format) | No | string | This option will format the output value on the results table and CSV export. Supports spreadsheet-style formatting (e.g. #,##0.00). Use [this website](https://customformats.com) to help build your custom format. |
| [separator](#separator) | No | string | This option controls the grouping and decimal characters used when rendering numbers (e.g. `1.234.567,50` for European locales). One of: `default`, `commaPeriod`, `spacePeriod`, `periodComma`, `noSeparatorPeriod`, `apostrophePeriod`. |
Expand Down
Loading
Loading