feat(stargate): add configurable Pulsar input TPS weights - #1002
Draft
barrygreengus wants to merge 2 commits into
Draft
feat(stargate): add configurable Pulsar input TPS weights#1002barrygreengus wants to merge 2 commits into
barrygreengus wants to merge 2 commits into
Conversation
Signed-off-by: Barry Greengus <bgreengus@nvidia.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
Contributor
🛡️ CodeQL Analysis🚨 Found 2 issue(s) Severity Breakdown:
📋 Top Issues🔗 View full details in Security tab 🕐 Last updated: 2026-08-19 15:47:50 UTC | Commit: f00fe01 |
Signed-off-by: Barry Greengus <bgreengus@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
Add an optional generation maximum input TPS measurement from Pylon through Stargate, and let Pulsar rendezvous routing select either the existing mean or that maximum as its capacity weight. Mean input TPS remains the default.
Additional Details
Pylon publishes the maximum already maintained by its generation-owned input TPS distribution. The value persists for that model generation and resets when the generation is replaced. There is no time window, bucket state, expiration path, or related runtime configuration.
Stargate sums the optional maximum only when every backend in the cluster supplies a valid positive finite value. Pulsar and Pulsar wait-and-widen accept last-mean-input-tps or max-input-tps for rendezvous weighting. Max mode does not silently fall back to mean when peak data is incomplete. Admission and work estimation continue using mean.
The ranking cache keys include the selected weight, so changing the unused statistic does not invalidate cached rankings. No dependencies or hash-version behavior changed.
For the Reviewer
Please focus on generation ownership and reset behavior, complete-only aggregation across mixed versions, and consistency between ranking cache invalidation and weight selection.
For QA
QA is not required beyond automated coverage. Verified with:
Issues
Closes #1001
Checklist