Skip to content

HyperDX Dashboard Provisioner#190

Open
ZeynelKoca wants to merge 1 commit intoClickHouse:mainfrom
ZeynelKoca:inject-json-dashboards
Open

HyperDX Dashboard Provisioner#190
ZeynelKoca wants to merge 1 commit intoClickHouse:mainfrom
ZeynelKoca:inject-json-dashboards

Conversation

@ZeynelKoca
Copy link
Copy Markdown

@ZeynelKoca ZeynelKoca commented Mar 13, 2026

Summary

  • Add dashboard provisioning via k8s-sidecar that discovers ConfigMaps labeled hyperdx.io/dashboard: "true" across all namespaces
  • k8s-sidecar writes dashboard JSON to a shared volume, HyperDX upserts them natively
  • Dashboards matched by name for idempotency (update existing, create new, never delete)
  • Two ways to provide dashboards: inline in values or via labeled ConfigMaps from any namespace

Design decisions

  • k8s-sidecar + native HyperDX provisioner: Sidecar handles ConfigMap discovery (same pattern as Grafana). HyperDX handles MongoDB upsert natively
  • ClusterRole: Needed for cross-namespace ConfigMap discovery.

Dependencies

@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Mar 13, 2026

🦋 Changeset detected

Latest commit: a437052

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 13, 2026

CLA assistant check
All committers have signed the CLA.

@ZeynelKoca
Copy link
Copy Markdown
Author

Implements feature request #170

@ZeynelKoca ZeynelKoca force-pushed the inject-json-dashboards branch from 0243f23 to 752d6a5 Compare March 13, 2026 12:36
@heiba
Copy link
Copy Markdown

heiba commented Mar 22, 2026

@dhable Can we please look at this PR and see how we can progress it ?

@ZeynelKoca
Copy link
Copy Markdown
Author

As per PR description, I redesigned the initial implementation to handle upsert behavior natively through hyperdx (same pattern as Grafana). This PR is dependent on hyperdxio/hyperdx#1962

@ZeynelKoca ZeynelKoca force-pushed the inject-json-dashboards branch 3 times, most recently from 04bbee1 to 5714065 Compare April 3, 2026 16:02
@dhable dhable self-requested a review April 6, 2026 18:34
Copy link
Copy Markdown
Collaborator

@dhable dhable left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for following up the feature change with a helm change as well. Great to keep these two up-to-date.

I think the need to provision RBAC roles and scan for tagged config maps is more than our typical users would expect from an application install. Using the default ConfigMap functionality to mount the dashboard data as a file would avoid the need for an additional service account, RBAC, etc.

dashboards:
enabled: false
# Image for the k8s-sidecar that watches for labeled ConfigMaps
sidecarImage: "kiwigrid/k8s-sidecar:2.5.0"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the motivation for using a sidecar to store the ConfigMap content as a file vs using the built in ability of ConfigMaps to be used as a file? I am slightly concerned about expanding 3rd party dependencies and increasing the security attack surface area of a deployment.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use case is: the team managing ClickStack deploys and manages the observability platform independently from application teams. Each application team manages their own HyperDX dashboards as ConfigMaps in their own namespace, deployed via their own Helm charts- GitOps style

Without the sidecar, the ClickStack team would need to track every application team's dashboards and redeploy ClickStack every time a dashboard is added or changed. The sidecar dynamically discovers labeled ConfigMaps across namespaces (hence the RBAC)

This pattern is directly taken from Grafana's Helm chart, which uses the same kiwigrid/k8s-sidecar image for the same purpose (see https://github.com/grafana-community/helm-charts/blob/main/charts/grafana/values.yaml)

imagePullSecrets:
{{- toYaml .Values.global.imagePullSecrets | nindent 8 }}
{{- end }}
{{- if .Values.hyperdx.dashboards.enabled }}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We merged in a SA change recently and if that field is set and dashboards are enabled, the resulting template ends up with two serviceAccountName entries. We need any change to be compatible with user defined SA's in order to work within AWS hosted clusters effectively.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I rebased onto main and changed SA handling to support user defined SAs

k8s-sidecar watches ConfigMaps labeled "hyperdx.io/dashboard: true"
across all namespaces and writes dashboard JSON to a shared volume.
HyperDX reads and upserts them natively via file-based provisioner.

Requires hyperdxio/hyperdx#1962
@ZeynelKoca ZeynelKoca force-pushed the inject-json-dashboards branch from bd006b9 to a437052 Compare April 7, 2026 18:56
@ZeynelKoca
Copy link
Copy Markdown
Author

Thank you for following up the feature change with a helm change as well. Great to keep these two up-to-date.

I think the need to provision RBAC roles and scan for tagged config maps is more than our typical users would expect from an application install. Using the default ConfigMap functionality to mount the dashboard data as a file would avoid the need for an additional service account, RBAC, etc.

No worries, the dashboard provisioning would be really nice for us as we're planning to go into production with clickstack soon and implementing missing features is faster than creating an issue 😄

Ready for another review!

@ZeynelKoca ZeynelKoca requested a review from dhable April 7, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants