feat: add bounded game performance and relay monitoring by Baylem · Pull Request #838 · GameplanePanel/Gameplane

Baylem · 2026-11-05T03:54:51Z

Operators need to distinguish missing measurements, game degradation, resource pressure and relay failures. This adds cached numeric game-performance measurements and relay metrics with Prometheus configuration, alerts and a workload dashboard.

Modules can declare tick/frame rate or tick duration, units and an optional degradation threshold. One bounded source feeds both dashboard routes and Prometheus; scrapes issue no RCON commands. Unknown, invalid and stale readings omit value series, while genuine zero remains visible. Source RCON has an aggregate response limit and absolute exchange deadline; NuclearOption applies its deadline before request writes as well.

Tunnel metrics use the same sanitized health evidence as readiness. Playit’s pinned client still reports unknown live connectivity. Infrastructure alerts require the documented kubelet/cAdvisor and kube-state-metrics collectors; no external game module is opted in by this PR.

The chart validates scrape-policy configuration and supports externally managed policies. The Prometheus example aligns monitor labels, namespace discovery and ingress permissions. A Cilium CI fixture checks actual agent/tunnel scraping, ingress revocation/restoration and denial from an unapproved namespace. Promtool fixtures cover alert firing/recovery and duplicate operator readings.

Supersedes #771. Prior review discussion remains on that PR.

Native stack, bottom to top: #837 → #838.

Closes #725.

Validation: Linux operator, agent and tunnel builds, controller regression-source and E2E source compilation passed. Generated CRDs and chart bundles are synchronized. Runtime tests, rule checks and cluster scraping run in CI under the repository’s compile-only local policy.

Current validation: all 108 main CI jobs passed on 51b76792 (CI run). Portable-path and visual-regression workflows also passed.

Signed-off-by: Baylem <zelathis@gmail.com>

coderabbitai · 2026-11-05T03:54:58Z

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 38 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Source mode: Auto
Repository: GameplanePanel/Gameplane
Branch: stack/pr-771
File: .coderabbit.yaml
Commit: 51b7679
Layers: Repository YAML
Review profile: CHILL
Plan: Advanced
Run ID: 8ddff12a-8385-45d0-9600-9a905c4e2fd5

Commits

Reviewing files that changed from the base of the PR and between d35927f and 51b7679.

Files selected for processing: 70

.github/workflows/ci.yaml
agent/cmd/main.go
agent/internal/caps/caps.go
agent/internal/rcon/nuclearoption.go
agent/internal/rcon/nuclearoption_deadline_test.go
agent/internal/rcon/nuclearoption_test.go
agent/internal/rcon/rcon.go
agent/internal/rcon/rcon_bounds_test.go
agent/internal/status/performance.go
agent/internal/status/performance_test.go
agent/internal/status/status.go
agent/specs.md
charts/gameplane/charts/gameplane-crds/crds/gameplane.local_backups.yaml
charts/gameplane/charts/gameplane-crds/crds/gameplane.local_backupschedules.yaml
charts/gameplane/charts/gameplane-crds/crds/gameplane.local_clusters.yaml
charts/gameplane/charts/gameplane-crds/crds/gameplane.local_gameservers.yaml
charts/gameplane/charts/gameplane-crds/crds/gameplane.local_gametemplates.yaml
charts/gameplane/charts/gameplane-crds/crds/gameplane.local_modules.yaml
charts/gameplane/charts/gameplane-crds/crds/gameplane.local_modulesources.yaml
charts/gameplane/charts/gameplane-crds/crds/gameplane.local_networkcaptures.yaml
charts/gameplane/charts/gameplane-crds/crds/gameplane.local_restores.yaml
charts/gameplane/crd-manifests/gameplane.local_backups.yaml
charts/gameplane/crd-manifests/gameplane.local_backupschedules.yaml
charts/gameplane/crd-manifests/gameplane.local_clusters.yaml
charts/gameplane/crd-manifests/gameplane.local_gameservers.yaml
charts/gameplane/crd-manifests/gameplane.local_gametemplates.yaml
charts/gameplane/crd-manifests/gameplane.local_modules.yaml
charts/gameplane/crd-manifests/gameplane.local_modulesources.yaml
charts/gameplane/crd-manifests/gameplane.local_networkcaptures.yaml
charts/gameplane/crd-manifests/gameplane.local_restores.yaml
charts/gameplane/dashboards/gameplane-workloads.json
charts/gameplane/templates/NOTES.txt
charts/gameplane/templates/grafana-dashboard.yaml
charts/gameplane/templates/monitoring-rules.yaml
charts/gameplane/templates/monitoring-validation.yaml
charts/gameplane/templates/networkpolicies.yaml
charts/gameplane/templates/servicemonitors.yaml
charts/gameplane/tests/monitoring-rules.sh
charts/gameplane/tests/monitoring-rules.test.yaml
charts/gameplane/values.yaml
charts/tests/check_gateway.py
charts/tests/check_standalone.py
deploy/kind/monitoring.sh
docs/examples/monitoring/gameplane-values.yaml
docs/examples/monitoring/prometheus.yaml
docs/monitoring.md
docs/tunnels.md
hack/ci_scope.py
hack/test_ci_scope.py
operator/api/v1alpha1/gametemplate_types.go
operator/api/v1alpha1/zz_generated.deepcopy.go
operator/config/crd/gameplane.local_gametemplates.yaml
operator/internal/controller/backup_forget_envtest_test.go
operator/internal/controller/gameserver_relay_health_test.go
operator/internal/controller/gameserver_tunnel.go
operator/internal/controller/gametemplate_performance_envtest_test.go
operator/specs.md
test/e2e/buckets.sh
test/e2e/ci_selection_e2e_test.go
test/e2e/monitoring_e2e_test.go
test/e2e/specs.md
test/e2e/suite-components.json
tunnel/main.go
tunnel/relay_health.go
tunnel/relay_metrics.go
tunnel/relay_metrics_test.go
tunnel/specs.md
tunnel/tailscale.go
tunnel/tailscale_test.go
web/src/types.ts

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

github-actions · 2026-11-05T06:43:34Z

CI Report - `51b7679`

run 42397774462 - attempt 1

0 failed - 33 passed - 0 skipped - 109m10s elapsed

Test selection: full (selection-rule change) - per-suite table

Coverage

Module	Measured	Gate	Margin
web	L95.95 S94.45 B89.33 F91.95	L92 S92 B82 F76	+2.45

Changed

Area	Files	+/-
agent/	11	+691/-99
charts/	30	+752/-22
tunnel/	7	+266/-30
docs/	4	+243/-0
test/	5	+232/-2
operator/	8	+164/-21
.github/	1	+24/-8
deploy/	1	+25/-0
hack/	2	+12/-2
web/	1	+5/-0

Slowest jobs

Job	Duration
e2e telemetry / arm64 (kind)	23m51s
build e2e images	14m17s
e2e restricted-workloads / arm64 (kind)	14m16s
e2e web live / amd64 (kind)	13m54s
e2e capture overhead (kind)	12m30s

e2e buckets run

api-agent, api-auth, api-mods, api-rbac, api-roles, bot-fast, monitoring, multicluster, operator, optional-components, postgres-availability, ratelimit, restricted-workloads, telemetry, upgrade

Signed-off-by: Baylem <zelathis@gmail.com>

Baylem added 27 commits October 21, 2026 12:10

merge: combine relay and enforced-network monitoring prerequisites

2e109f9

Signed-off-by: Baylem <zelathis@gmail.com>

feat: expose bounded game performance and relay monitoring

94d56a4

Signed-off-by: Baylem <zelathis@gmail.com>

fix(agent): bound NuclearOption request writes

0c5fdf5

Signed-off-by: Baylem <zelathis@gmail.com>

fix: align monitoring validation and relay regression fixtures

6cd3f48

Signed-off-by: Baylem <zelathis@gmail.com>

fix: include safe operator upgrades in monitoring installs

6b9681e

Signed-off-by: Baylem <zelathis@gmail.com>

fix(e2e): read metrics probes from completed pod logs

55ddd2d

Signed-off-by: Baylem <zelathis@gmail.com>

fix(e2e): distinguish policy verdicts from probe diagnostics

7b794f2

Signed-off-by: Baylem <zelathis@gmail.com>

test(e2e): drive event streams and retry monitoring downloads

6923113

Signed-off-by: Baylem <zelathis@gmail.com>

fix(ci): integrate upstream selection and current Tailscale SDK

a072caf

Signed-off-by: Baylem <zelathis@gmail.com>

chore: synchronize CRDs with workspace generator

d31f279

Signed-off-by: Baylem <zelathis@gmail.com>

ci: use Go 1.27 compatible golangci-lint

3248d53

Signed-off-by: Baylem <zelathis@gmail.com>

test: synchronize closed RCON response fixtures

dcc7161

Signed-off-by: Baylem <zelathis@gmail.com>

test: clarify closed response deadline synchronization

59a0f44

Signed-off-by: Baylem <zelathis@gmail.com>

ci: align lint tooling with Go 1.27 export data

9522fff

Signed-off-by: Baylem <zelathis@gmail.com>

docs: classify lint tool dependency versions

9197f6c

Signed-off-by: Baylem <zelathis@gmail.com>

fix(operator): use explicit waits for expected reconcile retries

71cd384

Signed-off-by: Baylem <zelathis@gmail.com>

fix(agent): confine mod deletion to an open directory root

5042ab1

Signed-off-by: Baylem <zelathis@gmail.com>

fix: align tailscale condition documentation and coverage

d1ded7f

Signed-off-by: Baylem <zelathis@gmail.com>

fix: preserve known relay outages and reviewed prerequisites

cfe8c19

Signed-off-by: Baylem <zelathis@gmail.com>

chore: merge upstream namespace controls into monitoring

18b4999

Signed-off-by: Baylem <zelathis@gmail.com>

chore: reconcile monitoring branch with upstream

4137e81

Signed-off-by: Baylem <zelathis@gmail.com>

test: exercise production relay observation and scoped wipe inventory

cb5e73b

Signed-off-by: Baylem <zelathis@gmail.com>

chore: reconcile relay monitoring with upstream forwarding

8a59abb

Signed-off-by: Baylem <zelathis@gmail.com>

Merge upstream FRP dependency security fix

a80d3c2

Signed-off-by: Baylem <zelathis@gmail.com>

chore: stack monitoring on current network isolation

42e047f

Signed-off-by: Baylem <zelathis@gmail.com>

test(operator): seed finished backups before forget manager startup

4701667

Signed-off-by: Baylem <zelathis@gmail.com>

merge: update network isolation base for observability

843d3b0

Signed-off-by: Baylem <zelathis@gmail.com>

Baylem added area: agent in-pod sidecar area: chart Helm chart and deployment labels Nov 5, 2026 — with ChatGPT Codex Connector

Baylem added this pull request to stack #842 November 5, 2026 03:57

This was referenced Nov 5, 2026

feat: add bounded game performance and relay monitoring #771

Closed

feat(network): add opt-in sentinel isolation and strict policy profile #837

Merged

test: exercise panel database and encryption-key recovery #839

Merged

Baylem marked this pull request as ready for review November 6, 2026 06:56

Baylem requested a review from ValgulNecron as a code owner November 6, 2026 06:56

Baylem added 8 commits November 6, 2026 02:14

Merge updated network isolation parent into observability

ea8de2b

Signed-off-by: Baylem <zelathis@gmail.com>

Merge network isolation workspace checksum into observability

e564740

Signed-off-by: Baylem <zelathis@gmail.com>

merge: carry shipped x/net security update into observability

38332c5

Signed-off-by: Baylem <zelathis@gmail.com>

merge: carry sentinel exposure cleanup and read regression fixes

6aa7651

Signed-off-by: Baylem <zelathis@gmail.com>

merge: carry source checksum normalization into observability

272cb71

Signed-off-by: Baylem <zelathis@gmail.com>

merge: carry workspace graph metadata into observability

9ac545e

Signed-off-by: Baylem <zelathis@gmail.com>

merge: integrate upstream and isolation changes into monitoring

27bf78d

Signed-off-by: Baylem <zelathis@gmail.com>

chore: merge settled inventory fixture into monitoring stack

51b7679

Signed-off-by: Baylem <zelathis@gmail.com>

ValgulNecron approved these changes Nov 7, 2026

View reviewed changes

ValgulNecron merged commit 77fb50b into master Nov 7, 2026
113 checks passed

ValgulNecron mentioned this pull request Nov 10, 2026

chore(release): v0.4.0 version bumps, status wording and CHANGELOG section #865

Merged

ValgulNecron deleted the stack/pr-771 branch November 11, 2026 00:03

This was referenced Nov 11, 2026

feat(tunnel): measure relay saturation, packet loss and latency #879

Closed

feat(observability): discover tenant metrics and alert on missing game signals #880

Open

Navigation Menu

Repository navigation

feat: add bounded game performance and relay monitoring - #838

feat: add bounded game performance and relay monitoring#838
ValgulNecron merged 35 commits into
stack/pr-763from
stack/pr-771

Baylem commented Nov 5, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Nov 5, 2026 •

edited

Loading

Review limit reached

Uh oh!

github-actions Bot commented Nov 5, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Baylem commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

github-actions Bot commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI Report - 51b7679

Coverage

Changed

Slowest jobs

e2e buckets run

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Baylem commented Nov 5, 2026 •

edited

Loading

coderabbitai Bot commented Nov 5, 2026 •

edited

Loading

github-actions Bot commented Nov 5, 2026 •

edited

Loading

CI Report - `51b7679`