Skip to content

Segfault in OpenVINOExecutionProvider with NPU on PVE kernel 6.17.9 (libopenvino.so.2541) #1043

Description

@eckmannmiles91

Description

onnxruntime-openvino 1.24.1 segfaults immediately when creating an InferenceSession with OpenVINOExecutionProvider on an Intel Arrow Lake system with NPU. The crash occurs in the bundled libopenvino.so.2541 (OpenVINO 2025.4.1) at a fixed offset during NPU plugin initialization.

CPUExecutionProvider works fine with the same model files and environment.

Environment

  • onnxruntime-openvino: 1.24.1 (latest on PyPI)
  • Bundled OpenVINO: 2025.4.1 (libopenvino.so.2541)
  • Python: 3.12
  • OS: Ubuntu 24.04.4 LTS (LXC container on Proxmox VE 9.1.5)
  • Kernel: 6.17.9-1-pve (Proxmox, host kernel)
  • CPU: Intel Core Ultra 7 265 (Arrow Lake)
  • NPU device: /dev/accel0 passed through to LXC
  • Kernel module: intel_vpu 1.0.0 (built into PVE kernel 6.17.9)
  • NPU userspace: intel-level-zero-npu 1.32.0, intel-fw-npu 1.32.0, intel-driver-compiler-npu 1.32.0
  • Also tested with: intel-level-zero-npu 1.28.0 — same segfault
  • Standalone openvino 2026.1.0: installed alongside ORT — no effect (ORT loads its bundled libs)

Reproduction

import onnxruntime as ort

# This segfaults:
sess = ort.InferenceSession(
    "encoder_model.onnx",
    providers=["OpenVINOExecutionProvider"]
)

# This works:
sess = ort.InferenceSession(
    "encoder_model.onnx",
    providers=["CPUExecutionProvider"]
)

The model is Moonshine Base encoder (~78MB ONNX). Any ONNX model triggers the crash — it happens during EP initialization, not model loading.

Crash Details

signal=SEGV (status=11)
segfault at 0 ip <offset 0xcdd56c> in libopenvino.so.2541
error 6 (write to NULL pointer in userspace)
  • Crash is 100% reproducible, always at the same offset in libopenvino.so.2541
  • dmesg on host confirms: python3[PID]: segfault at 0 ip ...cdd56c in libopenvino.so.2541

What We've Tried

  1. ✅ Upgraded NPU userspace from 1.28.0 → 1.32.0 (latest from intel/linux-npu-driver) — still segfaults
  2. ✅ Installed standalone openvino==2026.1.0 in the venv — ORT still loads its bundled libopenvino.so.2541, segfault persists
  3. ✅ Confirmed /dev/accel0 exists and intel_vpu module is loaded
  4. ldd shows no missing libraries
  5. ✅ CPUExecutionProvider works perfectly as a workaround

Expected Behavior

OpenVINOExecutionProvider should either:

  1. Successfully initialize and use the NPU, or
  2. Fail gracefully with an error message instead of segfaulting

Request

  • Is there a known incompatibility between ORT-OpenVINO 1.24.1's bundled OpenVINO 2025.4.1 and the intel_vpu driver in kernel 6.17.x?
  • Will the next onnxruntime-openvino release bundle a newer OpenVINO (2026.x) that may resolve this?
  • Is there a way to force ORT to use a system-installed OpenVINO instead of its bundled libopenvino.so.2541?

Workaround

Using CPUExecutionProvider — encoder inference is 88ms on CPU vs ~50ms on NPU. Acceptable but we'd prefer NPU when available.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions