Skip to content

Latest commit

 

History

History
80 lines (56 loc) · 4.81 KB

installation.md

File metadata and controls

80 lines (56 loc) · 4.81 KB

Installation Guide

English | 简体中文

HyperParallel provides two installation methods:

  • pip installation: Install a prebuilt hyper-parallel package and select runtime deep learning framework dependencies through extras.
  • Source build: Use ./build.sh to generate a wheel, with build options controlling whether native extensions are compiled.

Prefer pip install when you only need a published package. Build from source when you need to generate a wheel locally or customize native extension builds.

1. Install with pip

The extras for pip install control Python runtime dependencies only, not native extension compilation.

Command Installed dependencies Use case
pip install hyper-parallel Common dependencies only; no deep learning framework You manage framework versions yourself or only use framework-independent capabilities
pip install 'hyper-parallel[torch]' Common dependencies + torch==2.9.1 + torch-npu==2.9.1 Default PyTorch 2.9 backend
pip install 'hyper-parallel[torch26]' Common dependencies + torch==2.6.0 + torch-npu==2.6.0.post3 PyTorch 2.6 backend
pip install 'hyper-parallel[torch27]' Common dependencies + torch==2.7.1 + torch-npu==2.7.1 PyTorch 2.7 backend
pip install 'hyper-parallel[torch29]' Common dependencies + torch==2.9.1 + torch-npu==2.9.1 Explicit PyTorch 2.9 backend
pip install 'hyper-parallel[all]' Common dependencies + default PyTorch 2.9 Default PyTorch backend

In shells such as zsh, quote package names containing extras so that [] is not interpreted as a wildcard.

2. Build a Wheel from Source

The unified entry point invokes the build scripts for the selected components, then assembles the successful components into one wheel. The indexed Dataset C++ helper is always compiled. Multicore includes its private SHMEM dependency; there is no separate SHMEM switch.

Option Default Values Description
--multicore on on, off Build Torch Multicore and its private SHMEM.
--soc-list ascend910b,ascend910_93 SoC list Select Multicore kernel targets.
--strict off on, off Abort if a selected optional component fails.
--jobs nproc Positive integer Set native compilation parallelism.
--clean Disabled Flag without an argument Recreate work and installation directories for selected components.
./build.sh --help
./build.sh --multicore on --soc-list ascend910b,ascend910_93
./build.sh --strict on --jobs 24
# Install the exact wheel path printed by this build.
pip install /absolute/path/to/the-built-wheel.whl

General build requirements: Python 3.10–3.12, matching development headers, setuptools, wheel, pybind11, and a C++17 host toolchain. Each wheel targets one Python ABI and host architecture. Each native component defines its own SDK, framework, and device requirements. Runtime environments for source and prebuilt wheels must also satisfy the glibc, framework ABI, and SDK constraints recorded in the final ELF binaries.

Each unified build regenerates build/native/payload/hyper_parallel and ships only components that succeeded in that build. --strict on aborts when a selected component fails and is suitable for acceptance and release builds. The default permissive mode continues packaging after optional component failures, so generating a wheel does not prove that all optional components are available. python setup.py bdist_wheel does not trigger native compilation; without an explicit payload, it packages Python sources only. When Multicore is enabled, activate its CANN custom OPP before starting Python.

3. Activate the Multicore CANN Custom OPP

For source or editable development, use the payload generated by the unified build:

source /usr/local/Ascend/cann/set_env.sh
source build/native/payload/hyper_parallel/core/multicore/lib/set_env.bash
python your_program.py

After installing a wheel, use the locator script installed in the current Python environment:

source /usr/local/Ascend/cann/set_env.sh
source "$(command -v hyper_parallel_multicore_set_env.bash)"
python your_program.py

Run the activation script before starting the application Python process. It sets ASCEND_CUSTOM_OPP_PATH and the shared library paths required by the shipped Multicore vendor. A wheel installation does not depend on the source repository's build/ directory; source or editable development loads build artifacts from build/native/payload in the same checkout.

4. Verify the Installation

After installation, first verify that the core module imports successfully:

import hyper_parallel as hp
import importlib.metadata as metadata

print(metadata.version("hyper_parallel"))