diff --git a/fabric_resources/CompatibilityMatrix.md b/fabric_resources/CompatibilityMatrix.md index ec91d34..39389ed 100644 --- a/fabric_resources/CompatibilityMatrix.md +++ b/fabric_resources/CompatibilityMatrix.md @@ -46,7 +46,7 @@ Storage tier is inferred from each `-OutputPath*` value's form (see §4a). All f | DeviceCode | ✅ | ✅ | ✅ | ✅ | | AppRegistration | ✅ | ✅ | ✅ | ✅ | | ManagedIdentity | ✅ | ✅ | ✅ | ✅ | -| Agent365 enrichment | Interactive / DeviceCode / AppRegistration only — **ManagedIdentity is rejected** (Graph requires a delegated context for the underlying endpoints). | +| Agent365 enrichment | Supported under **all** auth modes. **Delegated** (Interactive / DeviceCode / Credential / Silent) requires the signed-in user to hold the **AI Administrator** or **Global Administrator** directory role. **App-only** (`AppRegistration` certificate/secret **or** `ManagedIdentity`) uses the application permissions `CopilotPackages.Read.All` + `Application.Read.All` (admin-consented) and performs no interactive sign-in, so it is valid on headless hosts (containers / ACA Jobs). App-only Agent 365 is **opt-in / pre-GA** — validate before relying on it in production; a missing app-role, unlicensed tenant, or absent program enrollment surfaces as a runtime **403**, not an up-front auth-mode rejection. | > **`-Deidentify` is auth- and tier-agnostic.** Optional output anonymization works under every auth mode and on all four destination tiers (it is applied on the host before any write/upload) and requires no additional Graph scope. See §4b for its effect on merge keys. @@ -87,9 +87,11 @@ Three trailing columns are appended at merge time to **any appended file**: | `Latest_Append_Date` | `YYYY-MM-DD` | Latest run timestamp that touched the file. Same value on every row; updated each append. | | `In_Latest_Append` | `TRUE` / `FALSE` | Whether the row appeared in the latest run's audit window or membership snapshot. `FALSE` for retained-but-departed rows. | -The CopilotInteraction rollup Fact CSV additionally carries two trailing identity columns (always present, not just at append time) so per-run integer surrogates remain stable across appends: **`Message_Id_Raw`** (raw audit `Messages[].Id` GUID) and **`ThreadId_Raw`** (raw `CopilotEventData.ThreadId` GUID). Merge keys: `Message_Id` for the rollup Fact CSV, `RecordId` for the non-rollup raw audit CSV, `PersonId_Normalized` for the EntraUsers CSV, `AgentId` for the Agent 365 CSV. +The CopilotInteraction rollup Fact CSV additionally carries stable identity columns (always present, not just at append time) so per-run integer surrogates remain stable across appends: the two trailing raw keys **`Message_Id_Raw`** (raw audit `Messages[].Id` GUID) and **`ThreadId_Raw`** (raw `CopilotEventData.ThreadId` GUID), plus a stable normalized user-identity column — **`User_Id_Normalized`** on the AIO fact (additive; parity with the **`Audit_UserId_Normalized`** column the AIBV fact already carries). Merge keys: the rollup Fact CSV dedups on a **composite key** — the profile grain (every grain column, with per-run integer surrogates replaced by their stable forms: `UserKey` → `User_Id_Normalized` on AIO / `Audit_UserId_Normalized` on AIBV, and `ThreadId` → `ThreadId_Raw`) **plus `Message_Id_Raw`** — because one message can fan out to several rows (one per distinct grain, e.g. per accessed resource) that must each reconcile independently rather than collapse to one. The non-rollup raw audit CSV keys on `RecordId`, the EntraUsers CSV on `PersonId_Normalized`, and the Agent 365 CSV on `AgentId`. -**Deidentification (`-Deidentify`) and merge keys.** Under `-Deidentify`, identity values are replaced with irreversible deterministic tokens before any write. The documented merge keys remain usable: `PersonId_Normalized` is tokenized but deterministic (so EntraUsers/Users joins and distinct counts are preserved across runs), while `Message_Id_Raw` / `ThreadId_Raw` are message/thread GUIDs that are **not** tokenized. A run's deidentify state is detected from the target's identity columns; appending a `-Deidentify` run into a non-deidentified target (or the reverse) is hard-rejected at pre-flight to prevent token/raw identity mixing. `-Deidentify` is persisted in the checkpoint and restored on `-Resume`. +**Rollup Fact CSV one-time re-baseline (parallel to the Users schema-compatibility probe below).** When an existing rollup Fact CSV `-AppendFile` target predates these keys — i.e. it is missing `Message_Id_Raw` or any composite-key column (for example an older AIO file written before `User_Id_Normalized` existed) — PAX does **not** silently overwrite or mis-merge it. It leaves the old target untouched, writes the current run to a new timestamped file, and prints re-baseline guidance. The practical effect is a one-time re-baseline with no data lost. The 0-row overwrite refusal and union-shrink guards remain in force alongside it. + +**Deidentification (`-Deidentify`) and merge keys.** Under `-Deidentify`, identity values are replaced with irreversible deterministic tokens before any write. The documented merge keys remain usable: `PersonId_Normalized` and the fact composite key's user component (`User_Id_Normalized` / `Audit_UserId_Normalized`) are tokenized but deterministic (so EntraUsers/Users joins, distinct counts, and cross-run fact reconciliation are preserved across runs), while `Message_Id_Raw` / `ThreadId_Raw` are message/thread GUIDs that are **not** tokenized. A run's deidentify state is detected from the target's identity columns; appending a `-Deidentify` run into a non-deidentified target (or the reverse) is hard-rejected at pre-flight to prevent token/raw identity mixing. `-Deidentify` is persisted in the checkpoint and restored on `-Resume`. **Org / manager-hierarchy columns and Delta schema evolution.** AIO / AIBV rollups add org/manager-hierarchy columns to the Users output (`OrgLevel`, `Manager_UserKey`, `TopOfChain_UserKey`, `HierarchyPath`, `IsManager`, `DirectReports`, `TotalReports`, and `Level0`…`Level14` name/key pairs; the M365 dashboard has none). On Fabric, the Users append runs a schema-compatibility probe that **tolerates added columns but rejects missing ones**: appending a current (hierarchy-bearing) run into a Delta table created by an older PAX simply adds the new columns, but appending older output that lacks the hierarchy columns into a table that already has them fails the compatibility check. Keep a given Fabric Users table on a single PAX version line, or recreate it, to avoid mixed-schema appends. The hierarchy needs no new Graph scope (manager data comes from the existing `$expand=manager` under `User.Read.All`). @@ -134,7 +136,7 @@ major exceeds 2. Operators should never edit checkpoints by hand. ## 8. Known incompatibilities -- `-Auth ManagedIdentity` with `-IncludeAgent365Info` / `-OnlyAgent365Info` — rejected at startup. +- `-Auth ManagedIdentity` with `-IncludeAgent365Info` / `-OnlyAgent365Info` **without** the Agent 365 application permissions granted — the app-only Agent 365 phase returns a runtime **403** until the identity holds `CopilotPackages.Read.All` + `Application.Read.All` (grant via `Grant-PAXPermissions.ps1 -IncludeAgent365`; opt-in / pre-GA). It is **not** rejected at startup. - Two simultaneous PAX runs against the same `OutputPath` — guarded by an exclusive checkpoint lock; operators should still set ACA `parallelism = 1`. - Remote (URL) `OutputPath` for checkpoint storage — checkpoints MUST be on the local diff --git a/fabric_resources/Deploy/Deploy-PAXAcaJob.ps1 b/fabric_resources/Deploy/Deploy-PAXAcaJob.ps1 index b5ecae6..ad88cd8 100644 --- a/fabric_resources/Deploy/Deploy-PAXAcaJob.ps1 +++ b/fabric_resources/Deploy/Deploy-PAXAcaJob.ps1 @@ -21,9 +21,11 @@ GroupMember.Read.All (always); AuditLogsQuery-Exchange.Read.All, AuditLogsQuery-OneDrive.Read.All, AuditLogsQuery-SharePoint.Read.All (only when PAX is invoked with -IncludeM365Usage). - Note: Agent 365 scopes (CopilotPackages.Read.All, Application.Read.All) - and the legacy AuditLog.Read.All Entra-audit scope are NOT used by PAX - and are intentionally not granted. + Agent 365 scopes (CopilotPackages.Read.All, Application.Read.All) are + granted only when Grant-PAXPermissions.ps1 is run with -IncludeAgent365 + (opt-in / pre-GA app-only Agent 365 enrichment); they are not granted by + default. The legacy AuditLog.Read.All Entra-audit scope is not used by PAX + and is never granted. * Sites.ReadWrite.All + Files.ReadWrite.All (SharePoint mode), OR Contributor / Storage Blob Data Contributor on the Fabric workspace (Fabric mode) diff --git a/fabric_resources/Deploy/README.md b/fabric_resources/Deploy/README.md index 383b522..44cb06f 100644 --- a/fabric_resources/Deploy/README.md +++ b/fabric_resources/Deploy/README.md @@ -1,8 +1,8 @@ # PAX Purview Audit Log Processor — ACA Job Deployment (Container path) > [!IMPORTANT] -> **Microsoft Agent 365 enrichment is fully supported.** -> The `-IncludeAgent365Info`, `-OnlyAgent365Info`, `-OutputPathAgent365Info`, and `-AppendAgent365Info` switches require interactive (delegated) sign-in by an AI Administrator or Global Administrator; ManagedIdentity is not supported for this stream. +> **Microsoft Agent 365 enrichment is supported on this container path via app-only auth (opt-in / pre-GA).** +> The `-IncludeAgent365Info`, `-OnlyAgent365Info`, `-OutputPathAgent365Info`, and `-AppendAgent365Info` switches run under `-Auth ManagedIdentity` (app-only) once the managed identity holds the application permissions `CopilotPackages.Read.All` + `Application.Read.All` (admin-consented) — no interactive sign-in is required. Grant them by running `../Prereqs/Grant-PAXPermissions.ps1` with `-IncludeAgent365` (not granted by default). This app-only path is **pre-GA** — validate it in your tenant before relying on it in production. > **Two ways to use PAX with Fabric.** This README covers the **container path** — a scheduled, unattended run hosted on Azure Container Apps Jobs using a managed identity. If you want to run PAX directly from a laptop, on-prem server, or Azure VM and still write to Fabric/OneLake, see [`../LocalRun/README.md`](../LocalRun/README.md) (no container build, no ACR, no ACA — just PowerShell + an Entra identity that has Fabric workspace access). The top-level [`../README.md`](../README.md) compares the two paths side-by-side. @@ -149,6 +149,8 @@ Rules enforced at parameter validation: every supplied URL must resolve to the s -CronExpression '0 6 * * *' ``` +> **This example uses `-IncludeAgent365Info` under `-Auth ManagedIdentity` (app-only).** It requires the managed identity to hold `CopilotPackages.Read.All` + `Application.Read.All` — grant them with `../Prereqs/Grant-PAXPermissions.ps1 -IncludeAgent365`. App-only Agent 365 is opt-in / pre-GA; validate it in your tenant before relying on it in production. (Omit the Agent 365 switches, or the `-IncludeAgent365` grant, if you don't need this stream.) + ### Provenance columns on appended files Every appended file (rollup Fact CSV under `-AppendFile`, raw audit CSV under non-rollup `-AppendFile`, EntraUsers CSV under `-AppendUserInfo`) gains three trailing columns at merge time: `Date_Added`, `Latest_Append_Date`, `In_Latest_Append`. Rows that departed from the current run's audit window are retained in the union with `In_Latest_Append = FALSE` so historical fact-table joins continue to resolve. The CopilotInteraction rollup Fact CSV additionally carries two stable identity columns (`Message_Id_Raw`, `ThreadId_Raw`) so per-run integer surrogates remain stable across appends. Under `-AppendUserInfo`, the raw Entra membership snapshot the audit phase writes is kept **pristine** — the union lands at the `-AppendUserInfo` target, the raw file keeps its natural timestamped name (or gets a `_raw` suffix on the rare path-and-leaf collision). @@ -325,7 +327,7 @@ Or open the file share in Azure Storage Explorer / Azure portal. The failed cont ## Notes -- **Agent 365 enrichment is NOT supported under `-Auth ManagedIdentity`.** PAX rejects `-IncludeAgent365Info` and `-OnlyAgent365Info` up-front when `-Auth ManagedIdentity` is in effect. The Microsoft Graph Agent Package Management API requires the AI Administrator or Global Administrator directory role, which can only be held by a signed-in user; a managed identity has no user principal and cannot satisfy the requirement, even with admin-consented application permissions. If you need Agent 365 enrichment, run PAX interactively (`-Auth WebLogin` or `DeviceCode`), or with `-Auth AppRegistration` (PAX will interactively top up a delegated context for the Agent 365 phase only). For the unattended ACA-Job pattern this README covers, simply omit those switches. +- **Agent 365 enrichment under `-Auth ManagedIdentity` (app-only, opt-in / pre-GA).** PAX runs `-IncludeAgent365Info` / `-OnlyAgent365Info` app-only under `-Auth ManagedIdentity` (and `-Auth AppRegistration`) with no interactive sign-in, provided the identity holds the application permissions `CopilotPackages.Read.All` + `Application.Read.All` (admin-consented). Grant them with `../Prereqs/Grant-PAXPermissions.ps1 -IncludeAgent365` — they are not granted by default. A missing app-role, unlicensed tenant, or absent program enrollment surfaces as a runtime 403 (the rest of the run completes normally). This app-only path is **pre-GA** — validate it before relying on it in production. (Delegated Agent 365 instead relies on the signed-in user's AI Administrator / Global Administrator directory role, e.g. on the local-run path.) - **Fabric `-OutputPath` shapes the script accepts:** - `https://.onelake.dfs.fabric.microsoft.com//.Lakehouse` — main Delta tables go under the Lakehouse's default `Tables/` area. - `…/.Lakehouse/Tables` — explicit non-Schemas form. diff --git a/fabric_resources/LocalRun/README.md b/fabric_resources/LocalRun/README.md index 05dd040..223ed8b 100644 --- a/fabric_resources/LocalRun/README.md +++ b/fabric_resources/LocalRun/README.md @@ -1,6 +1,6 @@ # PAX + Fabric — Local / Direct run (Path A) -Run PAX directly from a Windows laptop, on-prem server, or Azure VM and write outputs to a Fabric Lakehouse. **No container build, no ACR, no ACA, and no `Grant-PAXPermissions.ps1` are required.** This is the lower-setup path; pick it when you do not need a fully unattended Azure-hosted scheduled run, or when you need Microsoft Agent 365 enrichment (which is unsupported under managed identity — see the container path README). +Run PAX directly from a Windows laptop, on-prem server, or Azure VM and write outputs to a Fabric Lakehouse. **No container build, no ACR, no ACA, and no `Grant-PAXPermissions.ps1` are required.** This is the lower-setup path; pick it when you do not need a fully unattended Azure-hosted scheduled run, or when you want the simplest route to **delegated** Microsoft Agent 365 enrichment. (Agent 365 also runs **app-only** under managed identity on the container path — opt-in / pre-GA; see the container path README.) For the side-by-side comparison of this path vs. the container path, see [`../README.md`](../README.md). @@ -58,7 +58,9 @@ When you intend to pass `-IncludeM365Usage`: - `AuditLogsQuery-OneDrive.Read.All` - `AuditLogsQuery-SharePoint.Read.All` -If you want Microsoft Agent 365 enrichment (`-IncludeAgent365Info` / `-OnlyAgent365Info`), the **signed-in user** also needs the **AI Administrator** or **Global Administrator** directory role. This requirement is a server-side check on the Agent 365 endpoint and cannot be satisfied by application permissions alone. (Use `-Auth WebLogin`, `DeviceCode`, `Credential`, `Silent`, or `AppRegistration`. Managed identity is rejected up-front for Agent 365.) +> **`-UserInfoFile` can reduce the required scopes.** If you supply the user/organization directory from your own CSV via `-UserInfoFile` and every row includes a license value, PAX skips the live `/users` pull and the `/subscribedSkus` license lookup — so `User.Read.All` and `Organization.Read.All` may not be needed for that run. A single blank license value re-triggers the online license lookup (`User.Read.All` + `Organization.Read.All`). + +If you want Microsoft Agent 365 enrichment (`-IncludeAgent365Info` / `-OnlyAgent365Info`) under **delegated** auth (`-Auth WebLogin` / `DeviceCode` / `Credential` / `Silent`), the **signed-in user** needs the **AI Administrator** or **Global Administrator** directory role — a server-side check on the Agent 365 endpoint. Under **app-only** auth (`-Auth AppRegistration` certificate/secret, or `-Auth ManagedIdentity`) no directory role or interactive sign-in is required; instead grant the application permissions **`CopilotPackages.Read.All`** + **`Application.Read.All`** (admin-consented). App-only Agent 365 is **opt-in / pre-GA** — validate it in your tenant before relying on it in production. --- @@ -171,7 +173,7 @@ If you want unattended runs but prefer not to manage a cert or secret, host PAX -Rollup ``` -This is functionally identical to the container path but skips the ACR + ACA layer. **Remember: Agent 365 enrichment is rejected up-front under `-Auth ManagedIdentity`.** +This is functionally identical to the container path but skips the ACR + ACA layer. **Note: Agent 365 enrichment runs app-only under `-Auth ManagedIdentity` when the identity holds `CopilotPackages.Read.All` + `Application.Read.All` (opt-in / pre-GA — see the Microsoft Graph permissions section above).** --- @@ -262,7 +264,7 @@ Every appended file gains three trailing columns at merge time: | `Latest_Append_Date` | `YYYY-MM-DD` | Latest run that touched the file (same on every row). | | `In_Latest_Append` | `TRUE` / `FALSE` | Whether the row appeared in this run's audit window / membership snapshot. `FALSE` for rows retained from prior runs that no longer surface today. | -The CopilotInteraction rollup Fact CSV additionally carries two stable identity columns: **`Message_Id_Raw`** and **`ThreadId_Raw`**. These keep the per-run integer surrogates (`Message_Id`, `ThreadId`) stable across appends so multi-month threads remain a single thread in downstream models. +The CopilotInteraction rollup Fact CSV additionally carries stable identity columns: the raw keys **`Message_Id_Raw`** and **`ThreadId_Raw`**, plus a normalized user column (**`User_Id_Normalized`** on the AIO fact; **`Audit_UserId_Normalized`** on AIBV). These keep the per-run integer surrogates (`Message_Id`, `ThreadId`, `UserKey`) stable across appends — the cross-run append merge dedups on the full grain plus `Message_Id_Raw`, so multi-month threads remain a single thread and fan-out rows (many per message) reconcile independently in downstream models. ### Pristine raw EntraUsers under `-AppendUserInfo` diff --git a/fabric_resources/Prereqs/Grant-PAXPermissions.ps1 b/fabric_resources/Prereqs/Grant-PAXPermissions.ps1 index 2e41722..31b030b 100644 --- a/fabric_resources/Prereqs/Grant-PAXPermissions.ps1 +++ b/fabric_resources/Prereqs/Grant-PAXPermissions.ps1 @@ -20,17 +20,22 @@ - AuditLogsQuery-Exchange.Read.All (-IncludeM365Usage only) - AuditLogsQuery-OneDrive.Read.All (-IncludeM365Usage only) - AuditLogsQuery-SharePoint.Read.All (-IncludeM365Usage only) + - CopilotPackages.Read.All (-IncludeAgent365 only — opt-in / pre-GA) + - Application.Read.All (-IncludeAgent365 only — opt-in / pre-GA) - Sites.ReadWrite.All, Files.ReadWrite.All (SharePoint mode only) + OPT-IN — Microsoft Agent 365 (app-only, pre-GA): + PAX supports Agent 365 enrichment (-IncludeAgent365Info / -OnlyAgent365Info) + under -Auth ManagedIdentity (and -Auth AppRegistration) app-only, using the + application permissions CopilotPackages.Read.All + Application.Read.All with NO + interactive sign-in. These are NOT granted by default (least-privilege). Pass + -IncludeAgent365 to grant + admin-consent them. This path is pre-GA — validate + in your tenant before relying on it in production. (Delegated Agent 365 instead + relies on the signed-in user's AI Admin / Global Admin directory role.) + NOT granted (intentionally): - AuditLog.Read.All — different endpoint (Entra audit activities), not used by PAX. - - CopilotPackages.Read.All — Agent 365 enrichment requires a user-bound role - (AI Admin / Global Admin) that a managed identity - cannot hold. PAX rejects -IncludeAgent365Info / - -OnlyAgent365Info under -Auth ManagedIdentity. - - Application.Read.All — only consumed by the Agent 365 path, which is - unsupported under -Auth ManagedIdentity. 4. (SharePoint mode) Adds Sites.ReadWrite.All and Files.ReadWrite.All to the scope list above so PAX can write outputs to the destination SharePoint @@ -70,6 +75,13 @@ AuditLogsQuery-SharePoint.Read.All). Omit when you only need the unified AuditLogsQuery.Read.All umbrella scope. +.PARAMETER IncludeAgent365 + Opt-in / pre-GA. Grant the two Microsoft Agent 365 application permissions + (CopilotPackages.Read.All + Application.Read.All) so the managed identity can run + PAX's -IncludeAgent365Info / -OnlyAgent365Info app-only (no interactive sign-in). + Omit unless you intend to run Agent 365 enrichment on the container path. This path + is pre-GA — validate in your tenant before relying on it in production. + .EXAMPLE # SharePoint mode ./Grant-PAXPermissions.ps1 ` @@ -87,6 +99,16 @@ -Mode Fabric ` -FabricWorkspaceResourceId '/subscriptions/.../workspaces/PAX-Workspace' ` -IncludeM365Usage + +.EXAMPLE + # Fabric mode, opt-in app-only Microsoft Agent 365 enrichment (pre-GA) + ./Grant-PAXPermissions.ps1 ` + -SubscriptionId 'xxx' -ResourceGroup 'rg-pax' ` + -ManagedIdentityName 'uai-pax' -Location 'eastus' ` + -AcrResourceId '/subscriptions/.../registries/paxacr' ` + -Mode Fabric ` + -FabricWorkspaceResourceId '/subscriptions/.../workspaces/PAX-Workspace' ` + -IncludeAgent365 #> [CmdletBinding()] param( @@ -97,7 +119,8 @@ param( [Parameter(Mandatory)] [string] $AcrResourceId, [Parameter(Mandatory)] [ValidateSet('SharePoint','Fabric')] [string] $Mode, [Parameter()] [string] $FabricWorkspaceResourceId, - [Parameter()] [switch] $IncludeM365Usage + [Parameter()] [switch] $IncludeM365Usage, + [Parameter()] [switch] $IncludeAgent365 ) $ErrorActionPreference = 'Stop' @@ -217,6 +240,17 @@ if ($IncludeM365Usage) { 'AuditLogsQuery-SharePoint.Read.All' ) } +if ($IncludeAgent365) { + # Opt-in / pre-GA: application permissions for app-only Microsoft Agent 365 enrichment + # (-IncludeAgent365Info / -OnlyAgent365Info) under -Auth ManagedIdentity. No interactive + # sign-in is performed; a missing app-role, unlicensed tenant, or absent program + # enrollment surfaces as a runtime 403 in PAX. Validate before relying on it in production. + Write-Host " Agent 365 (opt-in / pre-GA): granting CopilotPackages.Read.All + Application.Read.All." -ForegroundColor Yellow + $requiredScopes += @( + 'CopilotPackages.Read.All', + 'Application.Read.All' + ) +} if ($Mode -eq 'SharePoint') { $requiredScopes += @('Sites.ReadWrite.All','Files.ReadWrite.All') } diff --git a/fabric_resources/Prereqs/README.md b/fabric_resources/Prereqs/README.md index 67a9b40..1d93721 100644 --- a/fabric_resources/Prereqs/README.md +++ b/fabric_resources/Prereqs/README.md @@ -1,8 +1,8 @@ # PAX Purview Audit Log Processor — Prerequisites (Container path) > [!IMPORTANT] -> **Microsoft Agent 365 enrichment is fully supported.** -> The `-IncludeAgent365Info`, `-OnlyAgent365Info`, `-OutputPathAgent365Info`, and `-AppendAgent365Info` switches require interactive (delegated) sign-in by an AI Administrator or Global Administrator; ManagedIdentity is not supported for this stream. +> **Microsoft Agent 365 enrichment is supported on the container path via app-only auth (opt-in / pre-GA).** +> The `-IncludeAgent365Info`, `-OnlyAgent365Info`, `-OutputPathAgent365Info`, and `-AppendAgent365Info` switches run under `-Auth ManagedIdentity` (app-only) once the managed identity holds the application permissions `CopilotPackages.Read.All` + `Application.Read.All` (admin-consented) — no interactive sign-in is required. Those permissions are **not** granted by default; pass **`-IncludeAgent365`** to `Grant-PAXPermissions.ps1` to add them. This app-only path is **pre-GA** — validate it in your tenant before relying on it in production. (Delegated sign-in by an AI Administrator or Global Administrator also works, e.g. on the local-run path.) One-time setup before deploying the ACA Job. **Only required when you intend to run PAX as a containerized scheduled job on Azure Container Apps.** If you are running PAX directly from a host (laptop, on-prem server, Azure VM) and writing to Fabric/OneLake, see [`../LocalRun/README.md`](../LocalRun/README.md) instead — the prerequisites there are smaller and `Grant-PAXPermissions.ps1` is not required. @@ -23,16 +23,19 @@ One-time setup before deploying the ACA Job. **Only required when you intend to - `AuditLogsQuery-OneDrive.Read.All` - `AuditLogsQuery-SharePoint.Read.All` + With `-IncludeAgent365` (opt-in / pre-GA — enables app-only Microsoft Agent 365 enrichment): + - `CopilotPackages.Read.All` + - `Application.Read.All` + `-Mode SharePoint` only: - `Sites.ReadWrite.All`, `Files.ReadWrite.All` 4. **(Fabric mode only)** Grants `Storage Blob Data Contributor` on the Fabric workspace's OneLake (Azure RBAC). -### Permissions intentionally NOT granted +### Permissions not granted by default -- `AuditLog.Read.All` — this is the Entra audit-activities permission, a different endpoint that PAX does not call. Earlier versions of this script granted it by mistake. -- `CopilotPackages.Read.All` — Microsoft Agent 365 enrichment requires the AI Administrator or Global Administrator directory role, which can only be held by a signed-in user. A managed identity has no user principal and cannot satisfy this requirement. The PAX script rejects `-IncludeAgent365Info` / `-OnlyAgent365Info` under `-Auth ManagedIdentity` up-front. If you need Agent 365 enrichment, run PAX interactively (`-Auth WebLogin` or `DeviceCode`) or with `-Auth AppRegistration` (the script will interactively top up a delegated context for the Agent 365 phase only). -- `Application.Read.All` — only consumed by the Agent 365 path, which is unsupported under `-Auth ManagedIdentity`. +- `AuditLog.Read.All` — this is the Entra audit-activities permission, a different endpoint that PAX does not call. Earlier versions of this script granted it by mistake. **Never** granted. +- `CopilotPackages.Read.All` and `Application.Read.All` — the Microsoft Agent 365 application permissions. PAX **does** support Agent 365 under app-only auth (`-Auth ManagedIdentity` or `-Auth AppRegistration`) using these permissions, with no interactive sign-in. They are **opt-in** — not granted by default, to keep the identity least-privileged. Pass **`-IncludeAgent365`** to this script to grant + admin-consent them. This app-only Agent 365 path is **pre-GA**; validate it in your tenant before relying on it in production. (Delegated Agent 365 instead relies on the signed-in user's AI Administrator / Global Administrator directory role and needs no application permission.) ## Operator pre-reqs @@ -83,6 +86,22 @@ One-time setup before deploying the ACA Job. **Only required when you intend to -IncludeM365Usage ``` +### Fabric destination, also enabling app-only Agent 365 (opt-in / pre-GA) + +```powershell +./Grant-PAXPermissions.ps1 ` + -SubscriptionId '00000000-0000-0000-0000-000000000000' ` + -ResourceGroup 'rg-pax' ` + -ManagedIdentityName 'uai-pax' ` + -Location 'eastus' ` + -AcrResourceId '/subscriptions/.../registries/paxacr' ` + -Mode Fabric ` + -FabricWorkspaceResourceId '/subscriptions/.../workspaces/PAX-Workspace' ` + -IncludeAgent365 +``` + +> **Agent 365 is pre-GA.** `-IncludeAgent365` grants `CopilotPackages.Read.All` + `Application.Read.All` so the managed identity can run `-IncludeAgent365Info` / `-OnlyAgent365Info` app-only (no interactive sign-in). Validate in your tenant before relying on it in production. + > **Fabric note:** Azure RBAC alone is sometimes insufficient for OneLake DFS write access. After running the script, also add the managed identity as a **Contributor** on the Fabric workspace via the Fabric portal (Workspace settings → Manage access). ## Output diff --git a/fabric_resources/README.md b/fabric_resources/README.md index 5b26a67..f64fb50 100644 --- a/fabric_resources/README.md +++ b/fabric_resources/README.md @@ -8,15 +8,15 @@ PAX writes Delta tables and files to a Microsoft Fabric Lakehouse whenever an `- | **Setup complexity** | Low — no Azure resources required beyond the Fabric workspace itself | High — ACR, ACA environment, managed identity, image build | | **What invokes the script** | You (or Task Scheduler / cron on the host) | ACA Job runtime, on a cron expression | | **Auth modes that fit** | `-Auth WebLogin`, `-Auth DeviceCode`, `-Auth AppRegistration` (cert or secret), `-Auth Credential`, `-Auth Silent` | `-Auth ManagedIdentity` (recommended) or `-Auth AppRegistration` | -| **Agent 365 enrichment (`-IncludeAgent365Info` / `-OnlyAgent365Info`) supported?** | Yes — under any delegated auth mode | No (under `-Auth ManagedIdentity`). Yes if you use `-Auth AppRegistration` (PAX top-ups a delegated context for the agent phase only). | +| **Agent 365 enrichment (`-IncludeAgent365Info` / `-OnlyAgent365Info`) supported?** | Yes — delegated (the signed-in user needs the **AI Administrator** or **Global Administrator** directory role) | Yes — via **app-only** auth (`-Auth ManagedIdentity` or `-Auth AppRegistration`) using the application permissions `CopilotPackages.Read.All` + `Application.Read.All`. **Opt-in / pre-GA** — enable with `Grant-PAXPermissions.ps1 -IncludeAgent365` and validate in your tenant before relying on it in production. | | **Secret rotation** | Your responsibility (if using `AppRegistration` with a secret) | None when using managed identity | | **Setup scripts in this folder** | None required. See [`LocalRun/README.md`](LocalRun/README.md) for the step-by-step. | [`Prereqs/Grant-PAXPermissions.ps1`](Prereqs/Grant-PAXPermissions.ps1) and [`Deploy/Deploy-PAXAcaJob.ps1`](Deploy/Deploy-PAXAcaJob.ps1). See [`Prereqs/README.md`](Prereqs/README.md) and [`Deploy/README.md`](Deploy/README.md). | | **Image** | Not needed | Built from [`Dockerfile/PAX.Dockerfile`](Dockerfile/PAX.Dockerfile), pushed to ACR | ## Which one should I pick? -- **Pick Path A** if you want the fastest possible time-to-first-write, you are okay with the run being tied to whatever host you put it on, and / or you need Agent 365 enrichment. -- **Pick Path B** if you want a fully unattended scheduled run hosted entirely inside Azure, with no long-lived secrets, and you do not need Agent 365 enrichment. +- **Pick Path A** if you want the fastest possible time-to-first-write, you are okay with the run being tied to whatever host you put it on, and / or you want the simplest route to **delegated** Agent 365 enrichment. +- **Pick Path B** if you want a fully unattended scheduled run hosted entirely inside Azure, with no long-lived secrets. Agent 365 enrichment is also available on this path via **app-only** auth (opt-in / pre-GA — see `Prereqs/Grant-PAXPermissions.ps1 -IncludeAgent365`). Both paths write to the **same Fabric workspace and lakehouse** in the same way — the script does not detect whether it is containerized. The only difference is where the script runs and how its identity is established. @@ -50,7 +50,15 @@ With `-IncludeM365Usage`: - `AuditLogsQuery-OneDrive.Read.All` - `AuditLogsQuery-SharePoint.Read.All` -(The container path's `Prereqs/Grant-PAXPermissions.ps1` provisions all of these for you. The local path defers to whatever auth mode you pick — see `LocalRun/README.md`.) +With Agent 365 enrichment (`-IncludeAgent365Info` / `-OnlyAgent365Info`) under **app-only** auth: +- `CopilotPackages.Read.All` +- `Application.Read.All` + + **Opt-in / pre-GA.** On the container path, grant these with `Grant-PAXPermissions.ps1 -IncludeAgent365`; they are not granted by default. Under **delegated** auth the signed-in user instead needs the **AI Administrator** or **Global Administrator** directory role (no extra application permission). + +> **`-UserInfoFile` can reduce the required Graph scopes.** When you supply the user/organization directory from your own CSV via `-UserInfoFile` and every row carries a license value, PAX skips the live `/users` pull and the `/subscribedSkus` license lookup — so `User.Read.All` and `Organization.Read.All` may not be needed for that run. A single blank license value re-triggers the online license lookup (which needs `User.Read.All` + `Organization.Read.All`). + +(The container path's `Prereqs/Grant-PAXPermissions.ps1` provisions the always-on and `-IncludeM365Usage` scopes for you, plus the Agent 365 scopes when you pass `-IncludeAgent365`. The local path defers to whatever auth mode you pick — see `LocalRun/README.md`.) ## Destination model