Skip to content

Add KumoTabular model size options to benchmarks - #983

Merged
akihironitta merged 2 commits into
mainfrom
feat/kumo-size-estimators
Sep 25, 2026
Merged

akihironitta merged 2 commits into
mainfrom
feat/kumo-size-estimators

Conversation

@cedric-lorenz

Copy link
Copy Markdown
Collaborator

No description provided.

@copy-pr-bot

copy-pr-bot Bot commented Sep 25, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@cedric-lorenz
cedric-lorenz force-pushed the feat/kumo-size-estimators branch 3 times, most recently from fdd1e7e to 468b9ff Compare September 25, 2026 20:39
@cedric-lorenz cedric-lorenz changed the title Set Kumo estimators by model size Use 16 Kumo estimators in benchmarks Sep 25, 2026
@cedric-lorenz
cedric-lorenz marked this pull request as ready for review September 25, 2026 21:22
@coderabbitai

coderabbitai Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: NVIDIA/structured-data-models/.coderabbit.yaml

Review profile: QUIET

Plan: Enterprise

Run ID: 8a54b17a-ee10-4504-98ac-0cb646125b98

📥 Commits

Reviewing files that changed from the base of the PR and between 468b9ff and 67e0653.

📒 Files selected for processing (2)
  • benchmark/tabular/scoringbench/models.py
  • benchmark/tabular/talent/models.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.


📝 Summary

Summary by CodeRabbit

  • Benchmark Updates
    • Tabular model benchmarks now use 16 estimators instead of 8, including the ScoringBench and Talent configurations. The default estimator count for tabular benchmarks has also increased to 16, keeping benchmark runs consistent across these setups. This changes the configuration used when running the benchmarks; it does not add or remove benchmark capabilities.

Walkthrough

The Kumo tabular model default and its ScoringBench and Talent benchmark configurations now use 16 estimators instead of 8.

Changes

Tabular estimator configuration

Layer / File(s) Summary
Update estimator count
benchmark/tabular/model.py, benchmark/tabular/scoringbench/models.py, benchmark/tabular/talent/models.py
The model default and both benchmark configurations increase the estimator count from 8 to 16.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~5 minutes

Merge Risk: ⚪ Minimal · up to 67e06

The estimator increase is supported by the model API and benchmark configurations. No concrete merge-blocking issue is established; performance at 16 remains unmeasured.

🚥 Pre-merge checks | ✅ 4 | ❓ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Description check ❓ Inconclusive No pull request description was provided. The changes are understandable from the title and diff summary, but the description check cannot confirm author-provided context. Add a brief description that states which benchmark configurations changed and that num_estimators increased from 8 to 16.
✅ Passed checks (4 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 3…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title is related to the changes because it identifies KumoTabular model sizing in benchmarks. It is broader than the actual change, which sets the estimator count to 16.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@cedric-lorenz
cedric-lorenz force-pushed the feat/kumo-size-estimators branch from 468b9ff to 67e0653 Compare September 25, 2026 21:29
@akihironitta akihironitta changed the title Use 16 Kumo estimators in benchmarks Add KumoTabular model size options to benchmarks Sep 25, 2026
@akihironitta
akihironitta merged commit 07db1cf into main Sep 25, 2026
3 of 4 checks passed
@akihironitta
akihironitta deleted the feat/kumo-size-estimators branch September 25, 2026 23:55
JingangQu added a commit that referenced this pull request Sep 26, 2026
[#983](#983) raised
every size to 16 estimators. This PR corrects it and sets 8 estimators
by default for small and medium `KumoTabular` in the
TabArena/BeyondArena, ScoringBench and TALENT benchmarks. Large keeps
16.

Signed-off-by: Jingang Qu <jqu@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants