Skip to content

fix(metrics): serve the process collector's memory and CPU metrics - #750

Merged
MegaRedHand merged 1 commit into
mainfrom
fix/metrics-process-collector
Oct 30, 2026
Merged

MegaRedHand merged 1 commit into
mainfrom
fix/metrics-process-collector

Conversation

@MegaRedHand

Copy link
Copy Markdown
Collaborator

The beacon dashboard's "Resident memory" panel reads process_resident_memory_bytes{job="beacon"}, which no ethlambda build exported. The prometheus crate only registers its process collector with the default registry when its process feature is on. Only the embedded-ethrex follower had it, because ethrex turns the feature on and features unify. Every other node's memory panel was blank, eth-2, eth-3 and eth-4 included.

What changed

  • Cargo.toml: prometheus = { version = "0.14", features = ["process"] }. On Linux the default registry, which /metrics gathers from (gather_default_metrics), now carries process_resident_memory_bytes, process_virtual_memory_bytes, process_cpu_seconds_total, process_open_fds, process_max_fds, process_threads and process_start_time_seconds, read from /proc/self at each scrape. Other platforms are unchanged.
  • Cargo.lock: adds procfs/procfs-core 0.17.0, plus the older rustix 0.38 and linux-raw-sys 0.4 they depend on. make cooldown-check passes.
  • docs/metrics.md: the process metrics, under Node Info Metrics.
  • A Linux-only unit test that the default registry serves process_resident_memory_bytes, so dropping the feature fails CI instead of blanking the dashboard.

Tests

  • make lint clean, make cooldown-check passes (macOS; the new test is compiled out there).

The beacon dashboard's memory panel reads process_resident_memory_bytes, which
no ethlambda build exported: the prometheus crate only registers its process
collector with the default registry when its `process` feature is on. Only a
build that pulls ethrex's metrics in (the embedded follower) had it, through
feature unification, so every other node's memory panel was blank.

Turn the feature on for the workspace. On Linux the default registry, which the
metrics endpoint gathers from, now carries process_resident_memory_bytes and the
other process_* series; other platforms are unchanged.
@github-actions

Copy link
Copy Markdown

🤖 Kimi Code Review

I'll review this PR which enables Prometheus process metrics on Linux. Let me analyze each file.

Cargo.lock

No issues. The lockfile changes are mechanically correct consequences of enabling the process feature. The duplicate rustix versions (0.38.44 via procfs and 1.1.4 for direct uses) are an unfortunate but common occurrence in the Rust ecosystem when transitive dependencies haven't updated. This is not actionable in this PR.

Cargo.toml

Line 98: The feature gating is correct—prometheus = { version = "0.14", features = ["process"] } enables the process collector. Consider whether this should be default-features = false with explicit feature selection to avoid future surprise features, but that's pre-existing style.

crates/common/metrics/src/gather.rs

Line 30-31 (comment): The comment states the process collector "registers it with the default registry on Linux only." This is accurate for prometheus 0.14.

Line 32 (#[cfg(all(test, target_os = "linux"))]): The test module is correctly gated to Linux only.

Line 39-42 (test assertion): The test checks for process_resident_memory_bytes in the gathered output. This is a reasonable smoke test.

Potential issue — test fragility: Line 40 uses .unwrap() on gather_default_metrics(). If this test runs in an environment where the default registry has been poisoned by another test (e.g., duplicate metric registration), it will panic. However, given the test isolation in cargo test, this is acceptable.

Suggestion — stronger assertion: Consider also asserting that the metric value is non-empty or parses as a valid Prometheus exposition format, but the string containment check is sufficient for a smoke test.

docs/metrics.md

Lines 16-22: Documentation is accurate and appropriately caveats that these metrics are Linux-only and not part of the leanMetrics spec. Good practice for operational documentation.


Summary

Aspect Assessment
Correctness Correct
Security No concerns — /proc/self is a kernel-generated pseudofilesystem, not attacker-controlled
Performance Minimal — /proc reads at scrape time are cheap; this is standard Prometheus practice
Rust idioms Appropriate use of cfg gating
Memory safety N/A — no unsafe code

Verdict: Approve. The PR is correct, well-documented, and appropriately tested. The only minor note is that the duplicate rustix versions in Cargo.lock will slightly increase compile times and binary size, but this is a transitive dependency issue outside this PR's scope.


Automated review by Kimi (Moonshot AI) · kimi-k2.6 · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Claude Code Review

Review of PR 750: serve the process collector's memory and CPU metrics

The change looks correct and is well scoped. I found no blocking issues.

  • Fix (Cargo.toml): The prometheus crate registers its ProcessCollector with the default registry only when the process feature is on, and only on Linux. gather_default_metrics reads that registry, so enabling the feature is the right and minimal fix. Because the feature is set in [workspace.dependencies], every crate gets it. The old dependence on ethrex's feature unification is gone.
  • Lockfile: The new packages are procfs/procfs-core 0.17.0, plus rustix 0.38 and linux-raw-sys 0.4. They add a second rustix major version, which costs some compile time and binary size. procfs 0.17 pins 0.38, so you can't avoid it here. It's worth revisiting when procfs moves to rustix 1.x. The PR says make cooldown-check passes.
  • Test (gather.rs):
    • The Linux-only cfg matches the feature's behavior.
    • The test fails if the feature is dropped, and the {metrics} assertion message makes a failure easy to read.
    • Nit: process_resident_memory_bytes would also appear if another test registered a metric by that name, which is unlikely.
  • Docs (docs/metrics.md): The seven listed metric names match what the crate's collector exports, and the Linux-only caveat is stated. The new paragraph sits under the Node Info table, which is reasonable.
  • Cost: Each scrape reads /proc/self (stat, limits, fd directory). The process_open_fds read lists the fd directory, which is cheap at typical scrape intervals. A node holding many thousands of libp2p and RocksDB fds will take slightly longer per scrape. This is acceptable.

Approve.


Automated review by Claude (Anthropic) · sonnet · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Codex Code Review

Looks good overall. This PR is narrowly scoped, and I do not see consensus-layer, SSZ, fork-choice, attestation, or XMSS risks in the actual code changed.

  • No correctness bug stands out in crates/common/metrics/src/gather.rs:19. Enabling prometheus’s process feature is a reasonable way to restore Linux process metrics in the default registry, and the regression test in crates/common/metrics/src/gather.rs:37 directly guards the intended behavior.
  • The Linux-only test gating in crates/common/metrics/src/gather.rs:35 is appropriate; it avoids false failures on non-Linux targets where the collector is not exported.
  • The docs update in docs/metrics.md:16 matches the implementation and clearly scopes the behavior to Linux.

Minor note:

  • Cargo.toml:98 enables the process feature workspace-wide. That is fine here, but it does add procfs/libc on Linux to any crate pulling the workspace prometheus dependency. I don’t think that is a problem in this repo, just a small dependency-surface increase worth being aware of.

Validation note:

  • I attempted to run the new test, but the environment blocks rustup from creating files under /home/runner/.rustup, so I could not verify it locally.

Automated review by OpenAI Codex · gpt-5.4 · custom prompt

MegaRedHand added a commit that referenced this pull request Oct 30, 2026
)

The beacon dashboard's memory panel reads process_resident_memory_bytes, which
no ethlambda build exported: the prometheus crate only registers its process
collector with the default registry when its `process` feature is on. Only a
build that pulls ethrex's metrics in (the embedded follower) had it, through
feature unification, so every other node's memory panel was blank.

Turn the feature on for the workspace. On Linux the default registry, which the
metrics endpoint gathers from, now carries process_resident_memory_bytes and the
other process_* series; other platforms are unchanged.

Cherry-picked from fix/metrics-process-collector at a9d8119.
@MegaRedHand
MegaRedHand added this pull request to the merge queue Oct 30, 2026
Merged via the queue into main with commit 012dc4f Oct 30, 2026
18 checks passed
@MegaRedHand
MegaRedHand deleted the fix/metrics-process-collector branch October 30, 2026 23:25
MegaRedHand added a commit that referenced this pull request Nov 2, 2026
Brings in #750 (fix(metrics): serve the process collector's memory and
CPU metrics). nightly already carries it as squash 0a531fa, so the
merged tree is identical to nightly's; this only makes main an ancestor.
MegaRedHand added a commit that referenced this pull request Nov 2, 2026
Cargo.lock conflicted in the dependency lists of rustix 1.1.4 and
tempfile: this branch moved both to windows-sys 0.52.0, while main
(#750, prometheus's process feature) added rustix 0.38.44 and
linux-raw-sys 0.4.15, which forces the version-qualified names. The
resolution keeps both: windows-sys 0.52.0 with the qualified names.
`cargo metadata --locked` accepts the result unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants