[scheduler] Add end-to-end stress tests to scheduler #2425
Draft
DiegoTavares wants to merge 1 commit into
Draft
Conversation
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Booking + accounting stress suite for the Rust scheduler. cluster feed → job query → host matching → dispatch) against a realistic farm in two phases inside one process: 1. **drain** — farm capacity comfortably exceeds demand. Benchmarks booking throughput (frames/s over the active booking window) and requires ≥90% (`STRESS_DRAIN_TARGET`) of frames to dispatch. 2. **saturation** — demand vastly exceeds tight subscription bursts and per-job core caps, so the Redis Lua cap check becomes the binding constraint. Verifies enforcement (no booking above burst / job max-cores) and that rejections actually flowed through the accounting hot path. After each phase, an audit cross-checks every Redis `acct:*` hash against `SUM(proc)` in Postgres plus host/frame/stat invariants — with the recompute and limit-reseed loops pushed beyond the test horizon, agreement proves the dispatch hot path (Lua book + force-rollback) kept accounting exact on its own. Requirements - Postgres with migrations applied, from the repo root: `docker compose up -d flyway` - A Docker daemon (the suite starts a throwaway Redis container via testcontainers; all Redis state dies with the container)
6ba8d83 to
be86728
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Booking + accounting stress suite for the Rust scheduler.
cluster feed → job query → host matching → dispatch) against a realistic farm in
two phases inside one process:
throughput (frames/s over the active booking window) and requires ≥90%
(
STRESS_DRAIN_TARGET) of frames to dispatch.core caps, so the Redis Lua cap check becomes the binding constraint. Verifies
enforcement (no booking above burst / job max-cores) and that rejections
actually flowed through the accounting hot path.
After each phase, an audit cross-checks every Redis
acct:*hash againstSUM(proc)in Postgres plus host/frame/stat invariants — with the recompute andlimit-reseed loops pushed beyond the test horizon, agreement proves the dispatch
hot path (Lua book + force-rollback) kept accounting exact on its own.
Requirements
docker compose up -d flywayall Redis state dies with the container)