-
Notifications
You must be signed in to change notification settings - Fork 238
Feature: Run FinOps hubs on your own hardware #2187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
MSBrett
wants to merge
82
commits into
dev
Choose a base branch
from
features/ftklocal
base: dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
82 commits
Select commit
Hold shift + click to select a range
2c7e32c
feat(finops-hub-local): two-DB ftklocal foundation (T-5000.1 + T-5000.2)
a01cebe
Merge T-5000.1+T-5000.2: ftklocal two-DB foundation into features/ftkβ¦
f6108e8
fix(finops-hub-local): make kql defaults to Hub; split KQL_DB/PARITY_DB
5bbf5c3
Merge T-5000.2 fix: make kql defaults to Hub (KQL_DB/PARITY_DB split)
0fa2a96
docs(finops-hub-local): export-config + manual staging contract (T-50β¦
98ebb14
Merge T-5000.4: export-config + staging contract into features/ftklocal
a4f6106
feat(finops-hub-local): local settings.json + config-container decisiβ¦
01e5e42
Merge T-5000.10: settings.json + config-container decision into featuβ¦
1f90c55
feat(finops-hub-local): add localized ADX dashboard artifact (T-5000.5)
f554584
merge(T-5000.5): integrate localized ADX dashboard into features/ftklβ¦
a5342c3
merge(features/ftklocal): sync .8 worktree to resolve notes/ references
bb422ad
feat(finops-hub-local): proactive backfill chunking + fail-fast doublβ¦
abf9d0d
merge(features/ftklocal): sync .3 worktree (brings in .5 dashboard)
8be8346
chore(finops-hub-local): genericize EXPORT_DIR example path (avoid /Uβ¦
e3b13ee
feat(agent-skills): add finops-hub-local skill + fix ftk.ps1 catalog β¦
e264089
fix(finops-hub-local): guard git fallback in ftk.ps1 Get-FtkRepo
2d8a2ff
merge(T-5000.8): integrate finops-hub-local agent skill + ftk.ps1 catβ¦
1614eb1
Merge branch 'features/ftklocal' into features/ftklocal-t5000.3
20fd3d3
refactor(finops-hub-local): gate chunked backfill on row count alone β¦
a9471b6
merge(T-5000.3): integrate ingest reliability (row-gated chunking, doβ¦
f574559
docs(finops-hub-local): README + adapter/parity notes; drop stale croβ¦
e474163
merge(T-5000.7): integrate finops-hub-local docs + ftk.ps1 cross-db sβ¦
7a615f7
docs(finops-hub-local): measured performance + limits (T-5000.6)
2fe9be7
merge(T-5000.6): integrate measured performance + limits doc
5ec214c
docs(finops-hub-local): correct ADX dashboard connectivity (UAT-measuβ¦
801981e
docs(finops-hub-local): Windows-primary + macOS install/deploy guidesβ¦
69912ba
style(finops-hub-local): apply Prettier formatting to docs
91899bd
chore(finops-hub-local): pin .md to LF in .gitattributes
c4a3908
docs(finops-hub-local): add 'build by hand' runbook
41abdd4
refactor(finops-hub-local): load each schema bundle in one call
ed1f4ca
fix(finops-hub-local): correct the by-hand ingest steps (update policβ¦
a961d73
docs(finops-hub-local): align README by-hand summary with the update-β¦
421b9a7
fix(finops-hub-local): add .build.config to keep runtime data out of β¦
5278e2f
Revert "refactor(finops-hub-local): load each schema bundle in one call"
e016dc3
fix(finops-hub-local): fix single-element-array unwrap in ingest backβ¦
4ef6457
docs(hubs): add Run hubs locally guide; remove finops-hub-local template
bce7f65
docs: clarify local hub prerequisites
MSBrett 96bd183
docs: improve local hub ingestion guidance
MSBrett 7aee803
docs: broaden local hub positioning
MSBrett 7e951a8
docs: remove unused local ingest helper
MSBrett a32f12b
docs: update mslearn dates
MSBrett 3fc373f
docs: add ftklocal agent skill
MSBrett 195869d
feat(finops-hub): build-generated local open-data loader, Initialize-β¦
MSBrett 66b66be
Merge remote-tracking branch 'origin/dev' into features/ftklocal
MSBrett b3c40a2
feat(hubs): add ftk-local-dashboard canvas extension
a133f66
feat(dashboard): add KPI interpretive annotations (T-5001.1)
13b389c
feat(dashboard): add triage framing to Overview tab (T-5001.4)
435bc00
fix(dashboard): correct Overspend tab target and null guards in triagβ¦
7f627ca
feat(dashboard): elevate source line to diagnostic rail (T-5001.2)
807c02c
feat(dashboard): add open KQL escape hatch per chart (T-5001.5)
4ebc90e
fix(dashboard): wire Run button to chart re-render and add HTTP errorβ¦
4e23bce
feat(dashboard): introduce visual hierarchy between primary and referβ¦
db3531f
fix(dashboard): fix KPI ordering, grid span, and border stacking (T-5β¦
64dddf7
fix(dashboard): fix reference KPI shadow cascade and remove dead fontβ¦
2fa1846
fix(dashboard): reset row count on error state and fix orphaned separβ¦
db01317
feat(dashboard): add /api/kql endpoint, refactor error handling (T-50β¦
bce4ee4
fix(dashboard): fix tainted format string in logError (CodeQL js/tainβ¦
b60441d
feat(dashboard): add CUS, MACC, top-txns, AI-by-app from INDEX.md catβ¦
c5a23e1
feat(canvas): add interactive click-to-filter to dashboard
c215872
chore: add PRODUCT.md and ignore impeccable cache
b1b6654
feat(dashboard): impeccable polish β P1-P3 design fixes
6f1366c
feat(dashboard): harden β abort races, error boundary, a11y, null guards
ef634f4
feat(dashboard): P1+P2 UX polish β KPI tooltips, triage callouts, ariβ¦
bfd2d81
fix(docs): mount kustodata volume in docker run so databases survive β¦
c12b8a5
harden(powershell): validate params, thread TimeoutSec, wrap release β¦
4ddb10c
fix(dashboard): correct KQL metric semantics and finish cross-tab layβ¦
6f9a5fb
Merge remote-tracking branch 'origin/dev' into features/ftklocal
MSBrett 549a6d0
chore: update ms.date in docs-mslearn files
MSBrett a91c072
test: fix Pester 6 Discovery-phase and mock-assertion bugs
MSBrett 98aa69c
feat(ftk-local-dashboard): make Kusto emulator port/database configurβ¦
MSBrett 9104089
feat(ftklocal-dashboard): add experimental Monaco KQL and ADX Web UI β¦
MSBrett 0e6177b
fix(ftklocal-dashboard): correct ADX iframe status copy per iframe emβ¦
MSBrett c0cbdc4
fix(ftklocal-dashboard): give tool tabs real vertical space, cut clutter
MSBrett 8397775
fix(ftk-local-dashboard): remove ADX tab, fix KQL query editor
MSBrett 7041399
fix(ftk-local-dashboard): survive host restarts without losing state
MSBrett 0376bac
chore: update ms.date in docs-mslearn files
MSBrett 3e77651
harden(powershell): guard blank error payload in Invoke-FinOpsHubLocaβ¦
MSBrett 9149de7
Merge remote-tracking branch 'origin/dev' into features/ftklocal
MSBrett b7fd743
chore: update ms.date in docs-mslearn files
MSBrett b4d5eb2
Address review feedback: remove undisclosed dashboard extension, fix β¦
MSBrett 3b6d1b3
chore: Update ms.date in docs-mslearn files
MSBrett 91d73e5
Merge remote-tracking branch 'origin/dev' into features/ftklocal
MSBrett File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
80 changes: 80 additions & 0 deletions
80
docs-mslearn/toolkit/powershell/hubs/initialize-finopshublocal.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| --- | ||
| title: Initialize-FinOpsHubLocal command | ||
| description: Set up a local FinOps hub in a running Kusto emulator using the Initialize-FinOpsHubLocal command in the FinOpsToolkit module. | ||
| author: MSBrett | ||
| ms.author: brettwil | ||
| ms.date: 07/31/2026 | ||
| ms.topic: reference | ||
| ms.service: finops | ||
| ms.subservice: finops-toolkit | ||
| ms.reviewer: brettwil | ||
| #customer intent: As a FinOps user, I want to set up a local FinOps hub with one command so I can explore my cost data without deploying Azure resources. | ||
| --- | ||
|
|
||
| # Initialize-FinOpsHubLocal command | ||
|
|
||
| The **Initialize-FinOpsHubLocal** command sets up a local FinOps hub in a running [Kusto emulator](/azure/data-explorer/kusto-emulator-overview). It creates the `Ingestion` and `Hub` databases, then downloads and applies the released FinOps hub setup scripts and open data load script, so the local hub uses the same KQL, transforms, and open data as a deployed hub. | ||
|
|
||
| The command doesn't install Docker or manage the emulator container, and it doesn't ingest cost data. Start the emulator first, then run this command against its endpoint. For the full walkthrough, see [Run FinOps hubs locally](../../hubs/run-hubs-locally.md). | ||
|
|
||
| <br> | ||
|
|
||
| ## Syntax | ||
|
|
||
| ```powershell | ||
| Initialize-FinOpsHubLocal ` | ||
| [-ClusterUri <string>] ` | ||
| [-ReleaseUri <string>] ` | ||
| [-RawRetentionInDays <int>] ` | ||
| [-OpenDataPath <string>] ` | ||
| [-SkipOpenData] ` | ||
| [-Destination <string>] ` | ||
| [-WhatIf] | ||
| ``` | ||
|
|
||
| <br> | ||
|
|
||
| ## Parameters | ||
|
|
||
| | Name | Description | | ||
| | --------------------- | --------------------------------------------------------------------------------------------------------------- | | ||
| | `βClusterUri` | Optional. Base URI of the running Kusto emulator. Default = `http://localhost:8082`. | | ||
| | `βReleaseUri` | Optional. Base URI to download the setup scripts and open data load script from. Default = the latest FinOps toolkit GitHub release. Point this at a local file server or a specific release to run offline or pin a version. | | ||
| | `βRawRetentionInDays` | Optional. Number of days to keep raw data in the `Ingestion` database. Default = `90`. | | ||
| | `βOpenDataPath` | Optional. Path, as seen by the emulator, to the folder that holds the open data CSV files. Default = `/data/export/open-data`. | | ||
| | `βSkipOpenData` | Optional. Skips loading the open data reference tables. Default = `false`. | | ||
| | `βDestination` | Optional. Local folder used to download the setup scripts. Default = temp folder. | | ||
| | `βWhatIf` | Optional. Shows what would happen if the command runs without actually running it. | | ||
|
|
||
| <br> | ||
|
|
||
| ## Examples | ||
|
|
||
| ### Set up a local hub | ||
|
|
||
| ```powershell | ||
| Initialize-FinOpsHubLocal | ||
| ``` | ||
|
|
||
| Sets up a local FinOps hub in the emulator at `http://localhost:8082` using the latest release. | ||
|
|
||
| ### Set up the schema only | ||
|
|
||
| ```powershell | ||
| Initialize-FinOpsHubLocal ` | ||
| -RawRetentionInDays 30 ` | ||
| -SkipOpenData | ||
| ``` | ||
|
|
||
| Creates the databases and applies the schema with 30-day raw retention, and skips loading open data. | ||
|
|
||
| <br> | ||
|
|
||
| ## Related content | ||
|
|
||
| Related solutions: | ||
|
|
||
| - [Run FinOps hubs locally](../../hubs/run-hubs-locally.md) | ||
| - [FinOps hubs](../../hubs/finops-hubs-overview.md) | ||
|
|
||
| <br> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| # Copyright (c) Microsoft Corporation. | ||
| # Licensed under the MIT License. | ||
|
|
||
| <# | ||
| .SYNOPSIS | ||
| Sends a single command or query to a local FinOps hub (Kusto emulator) HTTP endpoint. | ||
|
|
||
| .DESCRIPTION | ||
| Posts a management command or query to a running Kusto emulator over its REST API. Used by Initialize-FinOpsHubLocal to set up a local FinOps hub. This command does not manage the container; the emulator must already be running. | ||
|
|
||
| .PARAMETER ClusterUri | ||
| Required. Base URI of the running Kusto emulator (for example, http://localhost:8082). | ||
|
|
||
| .PARAMETER Database | ||
| Required. Name of the database to run the command against. | ||
|
|
||
| .PARAMETER Command | ||
| Required. The management command (starting with '.') or query to run. | ||
|
|
||
| .PARAMETER Endpoint | ||
| Optional. The REST endpoint to use: 'mgmt' for management commands (default) or 'query' for queries. | ||
|
|
||
| .PARAMETER TimeoutSec | ||
| Optional. Maximum number of seconds to wait for a response. Default = 0 (wait indefinitely). | ||
| #> | ||
| function Invoke-FinOpsHubLocalCommand | ||
| { | ||
| [Diagnostics.CodeAnalysis.SuppressMessage("PSAvoidUsingEmptyCatchBlock", "", Justification="Not all failures have a JSON error body (for example, a connection failure); ignore parse errors and fall through to rethrow the original exception.")] | ||
| [CmdletBinding()] | ||
| param | ||
| ( | ||
| [Parameter(Mandatory)] | ||
| [ValidateNotNullOrEmpty()] | ||
| [string] | ||
| $ClusterUri, | ||
|
|
||
| [Parameter(Mandatory)] | ||
| [ValidateNotNullOrEmpty()] | ||
| [string] | ||
| $Database, | ||
|
|
||
| [Parameter(Mandatory)] | ||
| [ValidateNotNullOrEmpty()] | ||
| [string] | ||
| $Command, | ||
|
|
||
| [Parameter()] | ||
| [ValidateSet('mgmt', 'query')] | ||
| [string] | ||
| $Endpoint = 'mgmt', | ||
|
|
||
| [Parameter()] | ||
| [ValidateRange(0, [int]::MaxValue)] | ||
| [int] | ||
| $TimeoutSec = 0 | ||
| ) | ||
|
|
||
| $uri = '{0}/v1/rest/{1}' -f $ClusterUri.TrimEnd('/'), $Endpoint | ||
| $body = @{ db = $Database; csl = $Command } | ConvertTo-Json -Compress | ||
|
|
||
| try | ||
| { | ||
| return Invoke-RestMethod -Uri $uri -Method 'Post' -ContentType 'application/json' -Body $body -TimeoutSec $TimeoutSec -ErrorAction 'Stop' | ||
| } | ||
| catch | ||
| { | ||
| # Kusto returns a structured JSON error body. Surface its message/code when present; | ||
| # otherwise rethrow the original exception (for example, a connection failure) as-is. | ||
| $content = $null | ||
| if ($null -ne $_.ErrorDetails -and -not [string]::IsNullOrWhiteSpace($_.ErrorDetails.Message)) | ||
| { | ||
| try | ||
| { | ||
| $content = $_.ErrorDetails.Message | ConvertFrom-Json -Depth 10 | ||
| } | ||
| catch {} | ||
| } | ||
|
|
||
| if ($null -ne $content -and $content.PSObject.Properties['error']) | ||
| { | ||
| $errorBody = $content.error | ||
| throw ($script:LocalizedData.Common_ErrorResponse -f $errorBody.message, $errorBody.code) | ||
| } | ||
|
|
||
| throw | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.