Skip to content

Can't run tests with array_api_strict 2.6 without a CUDA device #34456

Description

@betatim

Warning

This issue is not yet ready for a PR. If you are interested in contributing to scikit-learn, please have a look at our contributing guidelines, and in particular the sections for new contributors and the "Needs triage" label.

I was trying to run the array API tests using array_api_strict (version 2.6, newly released), as you might if you want to run the tests and don't have a MPS or CUDA device locally. I used pytest -k array_api_strict to do that. This works but also selects tests like "sklearn/metrics/tests/test_common.py::test_mixed_array_api_namespace_input_compliance[accuracy_score-array_api_strict to torch cpu]", which makes sense as I have torch installed. However the test fails with the following because my torch was compiled without CUDA support:

<snip>
sklearn/metrics/_classification.py:129: in _check_targets
    y_true, sample_weight = move_to(y_true, sample_weight, xp=xp, device=device)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sklearn/utils/_array_api.py:598: in move_to
    array_converted = xp.from_dlpack(array, device=device)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../miniconda/envs/sklearn-py3.13/lib/python3.13/site-packages/torch/utils/dlpack.py:169: in from_dlpack
    stream = torch.cuda.current_stream(f'cuda:{ext_device[1]}')
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../miniconda/envs/sklearn-py3.13/lib/python3.13/site-packages/torch/cuda/__init__.py:1133: in current_stream
    _lazy_init()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _lazy_init():
        global _initialized, _queued_calls
        if is_initialized() or hasattr(_tls, "is_initializing"):
            return
        with _initialization_lock:
            # We be double-checked locking, boys!  This is OK because
            # the above test was GIL protected anyway.  The inner test
            # is for when a thread blocked on some other thread which was
            # doing the initialization; when they get the lock, they will
            # find there is nothing left to do.
            if is_initialized():
                return
            # It is important to prevent other threads from entering _lazy_init
            # immediately, while we are still guaranteed to have the GIL, because some
            # of the C calls we make below will release the GIL
            if _is_in_bad_fork():
                raise RuntimeError(
                    "Cannot re-initialize CUDA in forked subprocess. To use CUDA with "
                    "multiprocessing, you must use the 'spawn' start method"
                )
            if not hasattr(torch._C, "_cuda_getDeviceCount"):
>               raise AssertionError("Torch not compiled with CUDA enabled")
E               AssertionError: Torch not compiled with CUDA enabled

../../miniconda/envs/sklearn-py3.13/lib/python3.13/site-packages/torch/cuda/__init__.py:417: AssertionError

This happens because the array_api_strict device device1 uses CUDA as the device ID (data-apis/array-api-strict#212).

I'm not sure yet what we should do about this. Revert data-apis/array-api-strict#212, live with this, tell people to uninstall torch, something else. With array-api-strict 2.5 I think this isn't a problem.

I thought I'd create the issue already so others don't run into the same problem/spend time debugging this.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions