From 2734e7ab4ccfa993bfbb2e6b194bb3749884e6c0 Mon Sep 17 00:00:00 2001 From: Lightdash Bot <131011178+lightdash-bot@users.noreply.github.com> Date: Fri, 30 Oct 2026 01:51:12 +0000 Subject: [PATCH 1/5] Document how to troubleshoot the self-hosted usage analytics nightly refresh --- self-host/enterprise-features/usage-analytics.mdx | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/self-host/enterprise-features/usage-analytics.mdx b/self-host/enterprise-features/usage-analytics.mdx index 2e261c6b..d6420d19 100644 --- a/self-host/enterprise-features/usage-analytics.mdx +++ b/self-host/enterprise-features/usage-analytics.mdx @@ -39,6 +39,18 @@ Generate activity in the enabled organization, then allow the [scheduled data re An organization administrator can then [create the analytics project](/workspace-admin/usage-analytics-project#create-your-analytics-project) and run a chart for a closed day with known activity. Check that the user, content and date match that activity. To update installed dashboards after an upgrade, follow [Update models and dashboards](/workspace-admin/usage-analytics-project#update-models-and-dashboards). +### Troubleshoot the nightly refresh + +The scheduler runs usage processing once a night, at 00:30 UTC. If that run fails, it does not retry the same day; the next opportunity is the following night's run. + +Search the scheduler logs for the paired lines `Usage processing stage started` and `Usage processing stage finished`, logged once for each of the `compaction`, `dimensions` and `users` stages. The finished line carries an `outcome` of `success`, `partial` or `failed`. A `partial` or `failed` outcome on `compaction` or `dimensions` means snapshots used for names and content inventory are stale or incomplete. + +The `users` stage builds the user-activity data that the **People** dashboards read; backfill does not build it. A dashboard built on another Explore can show data while **People** stays empty, until a nightly run completes that stage successfully. Look for `User activity refresh: {...}` to confirm the stage ran, and `User activity failed for org_id=.../stream=.../dt=...` for a partition that failed and kept its previous output. + +Each run only processes the last 7 closed UTC days and compacts at most 5,000 partitions; anything past that limit is picked up by the next night's run, not the same night. + +If the run does not populate data at all, check the license and the scheduler's object storage access before anything else: usage processing needs both the applied license and the usage storage from [Configure the deployment](#configure-the-deployment) to reach the scheduler. An S3 access error in the scheduler logs usually means the scheduler's credentials or bucket permissions are wrong rather than the license. Fix the cause and data appears once the next nightly run completes; this job has no manual trigger. + On releases that support it, [historical backfill](/workspace-admin/usage-analytics-project#historical-backfill) can import supported records still retained in the Lightdash database, including records from before capture was enabled. The linked guide explains included data, exclusions and how to run it for an existing project. Usage files can contain user identities and content metadata. Restrict bucket access and choose a retention policy that matches your organization's requirements. Disabling the feature stops future eligible capture and processing; it does not delete previously stored files. From 91ed2d33553bc5e9adf89def9f1582e05d8bf5fa Mon Sep 17 00:00:00 2001 From: Lightdash Bot <131011178+lightdash-bot@users.noreply.github.com> Date: Fri, 30 Oct 2026 01:55:58 +0000 Subject: [PATCH 2/5] Note that there's no admin view of personal warehouse credentials --- personal-settings/personal-warehouse-connections.mdx | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/personal-settings/personal-warehouse-connections.mdx b/personal-settings/personal-warehouse-connections.mdx index 9dcc764b..45b6db4a 100644 --- a/personal-settings/personal-warehouse-connections.mdx +++ b/personal-settings/personal-warehouse-connections.mdx @@ -7,6 +7,10 @@ doc-type: reference Personal warehouse credentials allow you to connect to your data warehouse using your own individual credentials instead of shared project credentials. This provides better security and audit trails for data access. + + There's no admin view or API that lists which users have personal warehouse credentials configured for a project. Each person's credentials are visible and manageable only by that person, under their own **Settings → My warehouse connections**. + + ## When are personal warehouse credentials used? Your personal warehouse credentials are used for every query you run in a project, including charts, dashboards, Explore, and the SQL runner, when: From 50327ecc30d3b9bb939646d83626d0aecfb74332 Mon Sep 17 00:00:00 2001 From: Lightdash Bot <131011178+lightdash-bot@users.noreply.github.com> Date: Fri, 30 Oct 2026 01:56:39 +0000 Subject: [PATCH 3/5] Document that service accounts can't hold or use personal warehouse credentials --- workspace-admin/service-accounts.mdx | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/workspace-admin/service-accounts.mdx b/workspace-admin/service-accounts.mdx index f2345c12..69f11f30 100644 --- a/workspace-admin/service-accounts.mdx +++ b/workspace-admin/service-accounts.mdx @@ -140,6 +140,12 @@ In the service account list, project-scoped service accounts show an **N project The projects and roles assigned to a project-scoped service account can be edited in place — see [Editing a service account](#editing-a-service-account). The token is preserved across edits, so integrations using it keep working. +## Limitations + +A service account is an authentication principal for the API and CLI, not a login, and it can't hold warehouse credentials. It can't run queries against a project that has **Require users to provide their own credentials** turned on, or against a connection — such as Databricks or Trino — that always needs [personal warehouse credentials](/personal-settings/personal-warehouse-connections). Those queries fail with the same missing-credentials error a real user gets before adding their own. + +For a project set up this way, use a shared or service-account-based warehouse connection instead, or create a Lightdash user with their own personal credentials. A service account doesn't substitute for either. + ## Using service accounts ### With Lightdash CLI From aa9a18027453474928b34ce7cca013a0f368bf6a Mon Sep 17 00:00:00 2001 From: Lightdash Bot <131011178+lightdash-bot@users.noreply.github.com> Date: Fri, 30 Oct 2026 08:51:31 +0000 Subject: [PATCH 4/5] Document API-based service account provisioning --- workspace-admin/service-accounts.mdx | 39 +++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/workspace-admin/service-accounts.mdx b/workspace-admin/service-accounts.mdx index 69f11f30..64e294a8 100644 --- a/workspace-admin/service-accounts.mdx +++ b/workspace-admin/service-accounts.mdx @@ -167,6 +167,39 @@ curl -H "Authorization: ldsvc_abc123xyz..." \ For full API documentation, see the [API Reference](/api-reference/v1/introduction). +## Automate account provisioning + +Automate creating, listing, updating, rotating, and deleting service accounts by calling the `/api/v1/service-accounts` endpoints headlessly — useful on a self-hosted instance, or when you manage service accounts across many Lightdash instances from a deployment pipeline. + +Call [`POST /api/v1/service-accounts`](https://docs.lightdash.com/api-reference/service-accounts/create-service-account) with an existing credential that already has organization administration permission: + +- An admin personal access token — `Authorization: ApiKey ` +- An admin service account token — `Authorization: Bearer ` + +```bash +curl -X POST "$LIGHTDASH_URL/api/v1/service-accounts" \ + -H "Authorization: ApiKey $ADMIN_PAT" \ + -H "Content-Type: application/json" \ + -d '{ + "description": "reporting-bot", + "expiresAt": "2026-01-01T00:00:00.000Z", + "scopes": ["system:member"], + "projectAccess": [{"projectUuid": "'"$PROJECT_UUID"'", "role": "viewer"}] + }' +``` + +The request body takes the same permission shapes as the UI form: + +- **Project-scoped** — `scopes: ["system:member"]` plus a `projectAccess` array of `{ projectUuid, role }` grants. +- **Organization-wide** — a `system:*` scope such as `scopes: ["system:viewer"]`, with no `projectAccess`. +- **Custom role** — `roleUuid` instead of `scopes`, for a [custom organization or project role](/workspace-admin/custom-roles). + + +The token is only returned in the response to create and rotate calls — store both `results.uuid` and `results.token` immediately, since there's no way to retrieve the token afterwards. Creating a service account isn't an upsert: there's no idempotency key, so don't automatically retry a `POST` that timed out. Check the account list first and only create a replacement if none exists. + + +Use the same admin credential with `GET /api/v1/service-accounts`, `PATCH /api/v1/service-accounts/{tokenUuid}`, `PATCH /api/v1/service-accounts/{tokenUuid}/rotate`, and `DELETE /api/v1/service-accounts/{tokenUuid}` to list, update, rotate, or revoke accounts. Managing service accounts requires `manage:Organization` permission, so a scoped service account token — one created with `system:member`, `system:viewer`, or a non-admin custom role — can't create or manage other service accounts. + ## Example Use cases ### CI/CD automation @@ -175,7 +208,11 @@ Use service accounts to automatically deploy dbt changes to Lightdash in your CI ### Instance initialization -When setting up a new self-hosted instance, configure `LD_SETUP_SERVICE_ACCOUNT_TOKEN` to initialize with a service account. See the [environment variables guide](/self-host/customize-deployment/environment-variables#initialize-instance-enterprise) for details. +When setting up a new self-hosted instance, configure `LD_SETUP_SERVICE_ACCOUNT_TOKEN` to create a one-time, org-admin-scoped service account during bootstrap. See the [environment variables guide](/self-host/customize-deployment/environment-variables#initialize-instance-enterprise) for details. + + +Remove `LD_SETUP_SERVICE_ACCOUNT_TOKEN` and the other `LD_SETUP_*` settings from the instance's configuration once setup finishes. On every restart, startup setup reconciliation deletes existing org-admin-scoped service accounts — including custom-role accounts with no `scopes` set — before recreating the bootstrap one, so leaving it configured can delete accounts you provisioned afterwards through the API. + ### Automated reporting From c4bbd410f48c31f5a0deca78857e73ae65b24c94 Mon Sep 17 00:00:00 2001 From: Lightdash Bot <131011178+lightdash-bot@users.noreply.github.com> Date: Fri, 30 Oct 2026 09:47:12 +0000 Subject: [PATCH 5/5] Document that the Helm chart's automatic scale-down needs helm upgrade, not GitOps syncs --- self-host/upgrading.mdx | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/self-host/upgrading.mdx b/self-host/upgrading.mdx index c7828a1a..9a8f979d 100644 --- a/self-host/upgrading.mdx +++ b/self-host/upgrading.mdx @@ -95,10 +95,12 @@ migrationJob: ``` With chart `2.16.284` or later, this combination makes the chart remove the backend HorizontalPodAutoscaler (HPA), scale the backend and worker Deployments to zero, and wait for their pods to stop before running migrations. The chart stops the upgrade if shutdown or migration fails. + +This automatic scale-down only runs under `helm upgrade`, which the chart can detect as an upgrade. A GitOps tool that renders the chart with `helm template` and applies the output itself — ArgoCD, or Flux in template mode — never triggers it: the migration Job still runs as a hook against the old pods, but the scale-down is silently skipped. If you apply this chart through ArgoCD or another GitOps tool, [scale workloads down by hand](#scale-application-workloads-manually-when-required) before syncing whenever the check reports `UNSAFE`. - -Only for chart `2.16.283` and earlier, or a custom deployment, with an `UNSAFE` verdict and the migration Job enabled. Old pods must not run against the new schema, so remove the backend HPA, scale to zero and wait until no Lightdash pods remain: + +Required with an `UNSAFE` verdict and the migration Job enabled, whenever the automatic scale-down above doesn't run: chart `2.16.283` and earlier, a custom deployment, or the chart applied through ArgoCD or another GitOps tool — do this before the sync. Old pods must not run against the new schema, so remove the backend HPA, scale to zero and wait until no Lightdash pods remain: ```bash kubectl delete hpa -l app.kubernetes.io/instance=lightdash