Document that the Helm chart's automatic scale-down needs helm upgrade, not GitOps syncs (+4 more)

personal-settings/personal-warehouse-connections.mdx

@@ -7,6 +7,10 @@ doc-type: reference
 
 Personal warehouse credentials allow you to connect to your data warehouse using your own individual credentials instead of shared project credentials. This provides better security and audit trails for data access.
 
+<Note>
+    There's no admin view or API that lists which users have personal warehouse credentials configured for a project. Each person's credentials are visible and manageable only by that person, under their own **Settings → My warehouse connections**.
+</Note>
+
 ## When are personal warehouse credentials used?
 
 Your personal warehouse credentials are used for every query you run in a project, including charts, dashboards, Explore, and the SQL runner, when:

self-host/enterprise-features/usage-analytics.mdx

@@ -39,6 +39,18 @@ Generate activity in the enabled organization, then allow the [scheduled data re
 
 An organization administrator can then [create the analytics project](/workspace-admin/usage-analytics-project#create-your-analytics-project) and run a chart for a closed day with known activity. Check that the user, content and date match that activity. To update installed dashboards after an upgrade, follow [Update models and dashboards](/workspace-admin/usage-analytics-project#update-models-and-dashboards).
 
+### Troubleshoot the nightly refresh
+
+The scheduler runs usage processing once a night, at 00:30 UTC. If that run fails, it does not retry the same day; the next opportunity is the following night's run.
+
+Search the scheduler logs for the paired lines `Usage processing stage started` and `Usage processing stage finished`, logged once for each of the `compaction`, `dimensions` and `users` stages. The finished line carries an `outcome` of `success`, `partial` or `failed`. A `partial` or `failed` outcome on `compaction` or `dimensions` means snapshots used for names and content inventory are stale or incomplete.
+
+The `users` stage builds the user-activity data that the **People** dashboards read; backfill does not build it. A dashboard built on another Explore can show data while **People** stays empty, until a nightly run completes that stage successfully. Look for `User activity refresh: {...}` to confirm the stage ran, and `User activity failed for org_id=.../stream=.../dt=...` for a partition that failed and kept its previous output.
+
+Each run only processes the last 7 closed UTC days and compacts at most 5,000 partitions; anything past that limit is picked up by the next night's run, not the same night.
+
+If the run does not populate data at all, check the license and the scheduler's object storage access before anything else: usage processing needs both the applied license and the usage storage from [Configure the deployment](#configure-the-deployment) to reach the scheduler. An S3 access error in the scheduler logs usually means the scheduler's credentials or bucket permissions are wrong rather than the license. Fix the cause and data appears once the next nightly run completes; this job has no manual trigger.
+
 On releases that support it, [historical backfill](/workspace-admin/usage-analytics-project#historical-backfill) can import supported records still retained in the Lightdash database, including records from before capture was enabled. The linked guide explains included data, exclusions and how to run it for an existing project.
 
 Usage files can contain user identities and content metadata. Restrict bucket access and choose a retention policy that matches your organization's requirements. Disabling the feature stops future eligible capture and processing; it does not delete previously stored files.

self-host/upgrading.mdx

@@ -95,10 +95,12 @@ migrationJob:
 ```
 
 With chart `2.16.284` or later, this combination makes the chart remove the backend HorizontalPodAutoscaler (HPA), scale the backend and worker Deployments to zero, and wait for their pods to stop before running migrations. The chart stops the upgrade if shutdown or migration fails.
+
+This automatic scale-down only runs under `helm upgrade`, which the chart can detect as an upgrade. A GitOps tool that renders the chart with `helm template` and applies the output itself — ArgoCD, or Flux in template mode — never triggers it: the migration Job still runs as a hook against the old pods, but the scale-down is silently skipped. If you apply this chart through ArgoCD or another GitOps tool, [scale workloads down by hand](#scale-application-workloads-manually-when-required) before syncing whenever the check reports `UNSAFE`.
 </Step>
 
-<Step title="Optional: scale workloads down by hand" id="scale-application-workloads-manually-when-required">
-Only for chart `2.16.283` and earlier, or a custom deployment, with an `UNSAFE` verdict and the migration Job enabled. Old pods must not run against the new schema, so remove the backend HPA, scale to zero and wait until no Lightdash pods remain:
+<Step title="Scale workloads down by hand, when required" id="scale-application-workloads-manually-when-required">
+Required with an `UNSAFE` verdict and the migration Job enabled, whenever the automatic scale-down above doesn't run: chart `2.16.283` and earlier, a custom deployment, or the chart applied through ArgoCD or another GitOps tool — do this before the sync. Old pods must not run against the new schema, so remove the backend HPA, scale to zero and wait until no Lightdash pods remain:
 
 ```bash
 kubectl delete hpa -l app.kubernetes.io/instance=lightdash

workspace-admin/service-accounts.mdx

@@ -140,6 +140,12 @@ In the service account list, project-scoped service accounts show an **N project
 
 The projects and roles assigned to a project-scoped service account can be edited in place — see [Editing a service account](#editing-a-service-account). The token is preserved across edits, so integrations using it keep working.
 
+## Limitations
+
+A service account is an authentication principal for the API and CLI, not a login, and it can't hold warehouse credentials. It can't run queries against a project that has **Require users to provide their own credentials** turned on, or against a connection — such as Databricks or Trino — that always needs [personal warehouse credentials](/personal-settings/personal-warehouse-connections). Those queries fail with the same missing-credentials error a real user gets before adding their own.
+
+For a project set up this way, use a shared or service-account-based warehouse connection instead, or create a Lightdash user with their own personal credentials. A service account doesn't substitute for either.
+
 ## Using service accounts
 
 ### With Lightdash CLI
@@ -161,6 +167,39 @@ curl -H "Authorization: ldsvc_abc123xyz..." \
 
 For full API documentation, see the [API Reference](/api-reference/v1/introduction).
 
+## Automate account provisioning
+
+Automate creating, listing, updating, rotating, and deleting service accounts by calling the `/api/v1/service-accounts` endpoints headlessly — useful on a self-hosted instance, or when you manage service accounts across many Lightdash instances from a deployment pipeline.
+
+Call [`POST /api/v1/service-accounts`](https://docs.lightdash.com/api-reference/service-accounts/create-service-account) with an existing credential that already has organization administration permission:
+
+- An admin personal access token — `Authorization: ApiKey <PAT>`
+- An admin service account token — `Authorization: Bearer <token>`
+
+```bash
+curl -X POST "$LIGHTDASH_URL/api/v1/service-accounts" \
+  -H "Authorization: ApiKey $ADMIN_PAT" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "description": "reporting-bot",
+    "expiresAt": "2026-01-01T00:00:00.000Z",
+    "scopes": ["system:member"],
+    "projectAccess": [{"projectUuid": "'"$PROJECT_UUID"'", "role": "viewer"}]
+  }'
+```
+
+The request body takes the same permission shapes as the UI form:
+
+- **Project-scoped** — `scopes: ["system:member"]` plus a `projectAccess` array of `{ projectUuid, role }` grants.
+- **Organization-wide** — a `system:*` scope such as `scopes: ["system:viewer"]`, with no `projectAccess`.
+- **Custom role** — `roleUuid` instead of `scopes`, for a [custom organization or project role](/workspace-admin/custom-roles).
+
+<Warning>
+The token is only returned in the response to create and rotate calls — store both `results.uuid` and `results.token` immediately, since there's no way to retrieve the token afterwards. Creating a service account isn't an upsert: there's no idempotency key, so don't automatically retry a `POST` that timed out. Check the account list first and only create a replacement if none exists.
+</Warning>
+
+Use the same admin credential with `GET /api/v1/service-accounts`, `PATCH /api/v1/service-accounts/{tokenUuid}`, `PATCH /api/v1/service-accounts/{tokenUuid}/rotate`, and `DELETE /api/v1/service-accounts/{tokenUuid}` to list, update, rotate, or revoke accounts. Managing service accounts requires `manage:Organization` permission, so a scoped service account token — one created with `system:member`, `system:viewer`, or a non-admin custom role — can't create or manage other service accounts.
+
 ## Example Use cases
 
 ### CI/CD automation
@@ -169,7 +208,11 @@ Use service accounts to automatically deploy dbt changes to Lightdash in your CI
 
 ### Instance initialization
 
-When setting up a new self-hosted instance, configure `LD_SETUP_SERVICE_ACCOUNT_TOKEN` to initialize with a service account. See the [environment variables guide](/self-host/customize-deployment/environment-variables#initialize-instance-enterprise) for details.
+When setting up a new self-hosted instance, configure `LD_SETUP_SERVICE_ACCOUNT_TOKEN` to create a one-time, org-admin-scoped service account during bootstrap. See the [environment variables guide](/self-host/customize-deployment/environment-variables#initialize-instance-enterprise) for details.
+
+<Warning>
+Remove `LD_SETUP_SERVICE_ACCOUNT_TOKEN` and the other `LD_SETUP_*` settings from the instance's configuration once setup finishes. On every restart, startup setup reconciliation deletes existing org-admin-scoped service accounts — including custom-role accounts with no `scopes` set — before recreating the bootstrap one, so leaving it configured can delete accounts you provisioned afterwards through the API.
+</Warning>
 
 ### Automated reporting