test(pyamber): cover the statistics and pause managers - #8049
Merged
Conversation
Contributor
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8049 +/- ##
=========================================
Coverage 93.41% 93.42%
Complexity 4696 4696
=========================================
Files 1179 1179
Lines 47713 47713
Branches 5314 5314
=========================================
+ Hits 44573 44576 +3
+ Misses 1698 1695 -3
Partials 1442 1442
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 713 | 0.435 | 14,600/16,394/16,394 us | 🟢 -30.5% / 🔴 +30.6% |
| ⚪ | bs=100 sw=10 sl=64 | 1,652 | 1.008 | 56,841/91,322/91,322 us | ⚪ within ±5% / 🟢 +42.7% |
| 🟢 | bs=1000 sw=10 sl=64 | 2,042 | 1.247 | 483,624/559,252/559,252 us | 🟢 -7.9% / 🟢 +72.2% |
Baseline details
Latest main b01b11f from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 713 tuples/sec | 707 tuples/sec | 893.99 tuples/sec | +0.8% | -20.2% |
| bs=10 sw=10 sl=64 | MB/s | 0.435 MB/s | 0.432 MB/s | 0.546 MB/s | +0.7% | -20.3% |
| bs=10 sw=10 sl=64 | p50 | 14,600 us | 12,830 us | 11,176 us | +13.8% | +30.6% |
| bs=10 sw=10 sl=64 | p95 | 16,394 us | 23,605 us | 14,009 us | -30.5% | +17.0% |
| bs=10 sw=10 sl=64 | p99 | 16,394 us | 23,605 us | 16,898 us | -30.5% | -3.0% |
| bs=100 sw=10 sl=64 | throughput | 1,652 tuples/sec | 1,588 tuples/sec | 1,157 tuples/sec | +4.0% | +42.7% |
| bs=100 sw=10 sl=64 | MB/s | 1.008 MB/s | 0.969 MB/s | 0.706 MB/s | +4.0% | +42.7% |
| bs=100 sw=10 sl=64 | p50 | 56,841 us | 58,354 us | 88,763 us | -2.6% | -36.0% |
| bs=100 sw=10 sl=64 | p95 | 91,322 us | 87,705 us | 95,096 us | +4.1% | -4.0% |
| bs=100 sw=10 sl=64 | p99 | 91,322 us | 87,705 us | 102,854 us | +4.1% | -11.2% |
| bs=1000 sw=10 sl=64 | throughput | 2,042 tuples/sec | 2,026 tuples/sec | 1,187 tuples/sec | +0.8% | +72.1% |
| bs=1000 sw=10 sl=64 | MB/s | 1.247 MB/s | 1.236 MB/s | 0.724 MB/s | +0.9% | +72.2% |
| bs=1000 sw=10 sl=64 | p50 | 483,624 us | 486,026 us | 874,968 us | -0.5% | -44.7% |
| bs=1000 sw=10 sl=64 | p95 | 559,252 us | 606,959 us | 917,815 us | -7.9% | -39.1% |
| bs=1000 sw=10 sl=64 | p99 | 559,252 us | 606,959 us | 949,868 us | -7.9% | -41.1% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,280.51,200,128000,713,0.435,14600.17,16394.40,16394.40
1,100,10,64,20,1210.62,2000,1280000,1652,1.008,56840.63,91322.18,91322.18
2,1000,10,64,20,9792.73,20000,12800000,2042,1.247,483624.46,559251.83,559251.83Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Xinyuan Lin <xinyual3@uci.edu>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
Two manager specs extended, 19 tests to 27.
statistics_manager.pypause_manager.py+7 fully-covered lines and +4 branch arms. Both files reach 100% on lines and branches.
statistics_manager.py's gap was94: 50% (1/2),95: MISS,101: 50% (1/2),102: MISS— the arms deciding whether an input or output statistics update is recorded at all.pause_manager.py's was the channel pause/resume paths, including bothchange_state=Falsevariants and the global-pause queue cycle.Two files were dropped from this bundle on measurement:
state_manager.py(1 line) andlinked_blocking_multi_queue.py, whose four partial lines did not survive scrutiny despite its 78 branch arms.core/util/customized_queue/inner.pysits in the same package and shows 2 partials plus 5 missed lines, but it is vendored third-party code and was refused rather than counted.Verification
Two reviewers returned eleven findings — the most of any bundle in this batch. The builder had reported
survivors: [], which the reviewers showed was materially wrong.46 mutations were run one at a time against the final tree; 44 die, and 15 of those die at exactly one test each. The two that survive are reported rather than dropped, both genuinely equivalent:
.add(channel_id)anddisable(channel_id)statements insidepause_input_channel— both statements are individually pinned by separate tests, so the order is unobservable.removewithdiscard—discardis by definitionremove if present.Measured from
amber/with the identical full-suite scope on both sides and--cov-branchpassed explicitly, since this branch is onmainandamber/pyproject.tomlhas no[tool.coverage.run]section (verified by reading it: only the threerufftables andpytest.ini_optionsexist). Per-linehitsandcondition-coveragewere read out ofcoverage.xmlby a parser script; summary percentages ignored. The before-state was produced by writing theHEADcontent of the two test files into the tree from a scratch extraction and restoring the new versions from a scratch snapshot afterwards, sha256-verified. Nogit restoreorgit checkoutwas run on any file.Full pyamber suite:
5 failed, 1167 passed, 7 errorsagainst main's1159 passed. Failure sets sorted and diffed — identical by identity, 12 entries.ruff checkandruff format --checkpass on CI's exact scope.No production file is touched;
git diff -- 'amber/src/main/*'is empty.Any related issues, documentation, discussions?
Closes #8045
How was this PR tested?
Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 5)