Repository navigation
results summarizer is broken since 4.1.45 #481
Description
After compliance check adds scaling factors, the summarizer won't print out the summarized results since v4.1.44. Issue occurs after checking out branch at 4.1.45 commits or newer commits, but summarizer works as expected on the same results path when using the prior v4.1.44 commits as shown below.
This issue prevents any recent MLPerf Training results to display when checking submissions.
When running the mlperf logging verify script from v4.1.45 or later versions on the v6.0 results, you don't get the detailed summarized output, you only see the following:
logging]$ git checkout 782506c
Previous HEAD position was 77699df Increment version to 4.1.44
HEAD is now at 782506c Increment version to 4.1.45
logging]$ ./scripts/verify_for_v6.0_training.sh ../training_results_v6.0/Lambda
** Detailed log output is also at package_checker.log NOTICE: Applying scaling factor 1.0311762501465556 to dir ../training_results_v6.0/Lambda/results/gb300_n18_ngc25.09_nemo/gpt_oss_20b Strong Scaling Scores: INFO - Running repository checks. INFO - Running git-unfriendly file name checks. INFO - Running large file checks. INFO - ALL CHECKS PASSED.
When using the mlperf logging verify scripts from v4.1.44 on the v6.0 training results, you get this output for the Lamda results (which is expected):
logging]$ git checkout 77699df
Previous HEAD position was 782506c Increment version to 4.1.45
HEAD is now at 77699df Increment version to 4.1.44
logging]$ ./scripts/verify_for_v6.0_training.sh ../training_results_v6.0/Lambda
** Detailed log output is also at package_checker.log
NOTICE: Applying scaling factor 1.0311762501465556 to dir Lambda/results/gb300_n18_ngc25.09_nemo/gpt_oss_20b
Strong Scaling Scores:
division availability submitter system \
0 closed Available cloud Lambda Lambda-1-Click-Cluster_B200_n1
1 closed Available cloud Lambda Lambda_GB300_n18
number_of_nodes host_processor_model_name host_processors_count \
0 1 Intel(R) Xeon(R) Platinum 8570 2
1 18 Neoverse-V2 36
accelerator_model_name accelerators_count \
0 NVIDIA Blackwell GPU (B200-SXM-180GB) 8
1 NVIDIA Blackwell Ultra GPU (GB300) 72
framework notes llama31_8b dlrm_dcnv2 flux1 \
0 PyTorch NVIDIA Release 25.04 B200 1000W 85.263156 NaN NaN
1 PyTorch NVIDIA Release 25.09 11.585940 NaN NaN
llama2_70b_lora llama31_405b gpt_oss_20b deepseekv3_671b \
0 NaN NaN 96.457092 NaN
1 NaN NaN 18.359156 NaN
details_url \
0 https://github.com/mlcommons/training_results_v6.0/blob/main/Lambda/systems/b200_n1_ngc25.09_nemo.json
1 https://github.com/mlcommons/training_results_v6.0/blob/main/Lambda/systems/gb300_n18_ngc25.09_nemo.json
code_url
0 https://github.com/mlcommons/training_results_v6.0/blob/master/Lambda/benchmarks
1 https://github.com/mlcommons/training_results_v6.0/blob/master/Lambda/benchmarks
INFO - Running repository checks.
INFO - Running git-unfriendly file name checks.
INFO - Running large file checks.
INFO - ALL CHECKS PASSED.
Activity
leachkr commentedon Oct 8, 2026
@arav-agarwal2 @ShriyaRishab FYI.
ShriyaRishab commentedon Oct 8, 2026
@pgmpablo157321 can you please reproduce this and see if we can fix it easily this round?
leachkr commentedon Oct 8, 2026
I didn't explicitly state it, but this issue occurs for other submitter folders as well, so isn't unique to Lamda, or HPE's results.