Skip to content

feat(monitoring): set up initial GMP metrics & minimal test dashboard - #5964

Open
jess-lowe wants to merge 11 commits into
google:masterfrom
jess-lowe:feat/dashboard
Open

feat(monitoring): set up initial GMP metrics & minimal test dashboard#5964
jess-lowe wants to merge 11 commits into
google:masterfrom
jess-lowe:feat/dashboard

Conversation

@jess-lowe

Copy link
Copy Markdown
Contributor

This sets up the basic Prometheus scraping and Grafana infrastructure, instrumenting the worker with a single test metric (osv_worker_tasks_processed_total) to confirm end-to-end telemetry on oss-vdb-test.

  • Prometheus Metrics (go/internal/metrics):
    • Added osv_worker_tasks_processed_total counter with status label (success, error, skipped).
    • Added HTTP server on :9090 serving /metrics and /healthz.
  • Worker Instrumentation (go/cmd/worker/, go/internal/worker/):
    • Worker daemon starts the metrics HTTP server on startup.
    • Task processing pipeline increments task count on completion.
  • GKE & Grafana (deployment/clouddeploy/.../environments/oss-vdb-test/monitoring/):
    • Added a PodMonitoring CRD for GMP to scrape workers every 15s.
    • Deployed stateless Grafana.
    • Provisioned a minimal test dashboard showing total tasks processed and task processing rate over time.
    • Configured GKE Ingress, ManagedCertificate, and IAP BackendConfig (scoped strictly to oss-vdb-test).
  • Terraform (deployment/terraform/modules/osv_pipeline/):
    • Declared the osv-monitoring-sa GCP service account, monitoring/logging viewer roles, and Workload Identity binding for default/grafana-sa.

Comment thread go/internal/worker/subscriber.go Outdated
Comment on lines +30 to +43
- name: GF_SECURITY_ALLOW_EMBEDDING
value: "true"
- name: GF_AUTH_ANONYMOUS_ENABLED
value: "false"
- name: GF_AUTH_BASIC_ENABLED
value: "false"
- name: GF_AUTH_PROXY_ENABLED
value: "true"
- name: GF_AUTH_PROXY_HEADER_NAME
value: "X-Goog-Authenticated-User-Email"
- name: GF_AUTH_PROXY_HEADER_PROPERTY
value: "email"
- name: GF_AUTH_PROXY_AUTO_SIGN_UP
value: "true"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to need more work to get working I believe.

Apparently the Auth proxy header doesn't work with google. Try JWTs.

You'll also need to look into how oauth and cloud IAM is setup and how it can be used to authenticate grafana.

Also, we probably want to set: GF_USERS_AUTO_ASSIGN_ORG_ROLE to be admin, instead of the default viewer. We'll need to figure out persistent accounts later. Right now for testing allow anyone with the google cloud perms to have write access.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have done the oauth and iam service account. 👍🏻

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants