Release v0.1 - #28
Open
ThrudPrimrose wants to merge 830 commits into
Open
Release v0.1#28ThrudPrimrose wants to merge 830 commits into
ThrudPrimrose wants to merge 830 commits into
Annotations
7 errors
|
Phase 3c -- mlscale-part2 references through the real ML rank driver (gloo)
Process completed with exit code 1.
|
|
Phase 3c -- mlscale-part2 references through the real ML rank driver (gloo)
tests/test_mlscale_part2_real_launch.py::test_a_submission_that_writes_zeros_grades_wrong[dist_moe_router]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10935] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_a_submission_that_writes_zeros_grades_wrong[dist_rmsnorm]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10946] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_a_submission_that_writes_zeros_grades_wrong[dist_split_kv_decode]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10957] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_a_submission_that_writes_zeros_grades_wrong[dist_sync_batchnorm]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10968] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_a_submission_that_writes_zeros_grades_wrong[dist_vocab_embedding]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10979] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
|
|
Phase 3c -- mlscale-part2 references through the real ML rank driver (gloo)
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_vocab_embedding-2]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10842] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_vocab_embedding-4]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '4', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10853] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_a_submission_that_writes_zeros_grades_wrong[dist_adamw_zero]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10880] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_a_submission_that_writes_zeros_grades_wrong[dist_all_to_all_transpose]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10891] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_a_submission_that_writes_zeros_grades_wrong[dist_causal_attention]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10902] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_a_submission_that_writes_zeros_grades_wrong[dist_contrastive_loss]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10913] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_a_submission_that_writes_zeros_grades_wrong[dist_conv2d_halo]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10924] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
|
|
Phase 3c -- mlscale-part2 references through the real ML rank driver (gloo)
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_split_kv_decode-1]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '1', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...g on\nexactly when Open MPI kills them.\n--------------------------------------------------------------------------\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_split_kv_decode-2]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10750] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_split_kv_decode-4]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '4', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10761] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_sync_batchnorm-1]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '1', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...g on\nexactly when Open MPI kills them.\n--------------------------------------------------------------------------\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_sync_batchnorm-2]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10796] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_sync_batchnorm-4]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '4', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10808] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_vocab_embedding-1]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '1', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...g on\nexactly when Open MPI kills them.\n--------------------------------------------------------------------------\n').returncode
|
|
Phase 3c -- mlscale-part2 references through the real ML rank driver (gloo)
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_conv2d_halo-4]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '4', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10624] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_moe_router-1]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '1', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...g on\nexactly when Open MPI kills them.\n--------------------------------------------------------------------------\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_moe_router-2]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10659] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_moe_router-4]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '4', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...] [[54573,0],0] ORTE_ERROR_LOG: Data unpack had inadequate space in file ../../../orte/util/show_help.c at line 513\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_rmsnorm-1]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '1', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...g on\nexactly when Open MPI kills them.\n--------------------------------------------------------------------------\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_rmsnorm-2]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10704] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_rmsnorm-4]: AssertionError: -------------------------------------------------------------------------- ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '4', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10715] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
|
|
Phase 3c -- mlscale-part2 references through the real ML rank driver (gloo)
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_causal_attention-2]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...g on\nexactly when Open MPI kills them.\n--------------------------------------------------------------------------\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_causal_attention-4]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '4', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10534] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_contrastive_loss-1]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '1', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...g on\nexactly when Open MPI kills them.\n--------------------------------------------------------------------------\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_contrastive_loss-2]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10568] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_contrastive_loss-4]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '4', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10579] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_conv2d_halo-1]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '1', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...g on\nexactly when Open MPI kills them.\n--------------------------------------------------------------------------\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_conv2d_halo-2]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10613] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
|
|
Phase 3c -- mlscale-part2 references through the real ML rank driver (gloo)
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_adamw_zero-1]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '1', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...g on\nexactly when Open MPI kills them.\n--------------------------------------------------------------------------\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_adamw_zero-2]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10432] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_adamw_zero-4]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '4', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10443] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_all_to_all_transpose-1]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '1', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...g on\nexactly when Open MPI kills them.\n--------------------------------------------------------------------------\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_all_to_all_transpose-2]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '2', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10477] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_all_to_all_transpose-4]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '4', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...mpi-abort\n[runnervm8df0l:10488] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages\n').returncode
tests/test_mlscale_part2_real_launch.py::test_the_kernels_own_reference_grades_correct_through_the_real_rank_driver[dist_causal_attention-1]: AssertionError: Traceback (most recent call last): ... + where 1 = CompletedProcess(args=['mpirun', '--oversubscribe', '-n', '1', '/home/runner/work/HPCAgent-Bench/HPCAgent-Bench/.venv/...g on\nexactly when Open MPI kills them.\n--------------------------------------------------------------------------\n').returncode
|
background
wait
wait-all
cancel
parallel
Loading