Skip to content

[SYCL] Fix get mem error - #28227

Open
arthw wants to merge 5 commits into
ggml-org:masterfrom
arthw:fix_get_mem
Open

[SYCL] Fix get mem error#28227
arthw wants to merge 5 commits into
ggml-org:masterfrom
arthw:fix_get_mem

Conversation

@arthw

@arthw arthw commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Overview

Fixed the issue #28134 involved in #27968.

Additional information

  1. The PR [SYCL] Enhance to get the free memory of Intel GPU #27968 use level zero API to get free memory size to support more GPUs. But it can't work on WSL on windows, because level zero Sysman doesn't support it.
    The solution in this PR fallback to SYCL API when it's not supported.

  2. Add GGML_SYCL_DEV_DEBUG() for development debug.

Verified on Linux (Arc770,UHD Graphics 770 in 13700K), Windows (140T) and WSL (Ubuntu) on Windows.

Requirements

@arthw
arthw requested a review from a team as a code owner September 2, 2026 06:14
@github-actions github-actions Bot added documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language labels Sep 2, 2026
Comment thread docs/backend/SYCL.md Outdated
Co-authored-by: Titaniumtown <titaniumtown@proton.me>
@strtgbb

strtgbb commented Sep 2, 2026

Copy link
Copy Markdown

This does not compile for me. The base b10743 does compile.

Compiler log
- The C compiler identification is IntelLLVM 2026.0.0
-- The CXX compiler identification is IntelLLVM 2026.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/intel/oneapi/compiler/2026.0/bin/icx - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/intel/oneapi/compiler/2026.0/bin/icpx - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- llama.cpp version: 0.3.0-dev
CMAKE_BUILD_TYPE=Release
-- Found Git: /usr/bin/git (found version "2.55.0")
-- The ASM compiler identification is IntelLLVM
-- Found assembler: /opt/intel/oneapi/compiler/2026.0/bin/icx
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- ccache found, compilation results will be cached. Disable with GGML_CCACHE=OFF.
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- GGML_SYSTEM_ARCH: x86
-- Found OpenMP_C: -fiopenmp (found version "5.1")
-- Found OpenMP_CXX: -fiopenmp (found version "5.1")
-- Found OpenMP: TRUE (found version "5.1")
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu: -march=native 
-- Using RPC backend
--   RDMA transport enabled (auto-detected)
-- Including RPC backend
-- GGML_SYCL_TARGET=INTEL
-- Performing Test SUPPORTS_SYCL
-- Performing Test SUPPORTS_SYCL - Success
-- Using oneAPI Release SYCL compiler (icpx).
-- SYCL found
-- SYCL Compiler version: 20260000
-- SYCL_INCLUDE_DIR: /opt/intel/oneapi/compiler/2026.0/include
-- SYCL_LIBRARY=/opt/intel/oneapi/compiler/2026.0/lib/libsycl.so
-- Found IntelSYCL: /opt/intel/oneapi/compiler/2026.0/include (found version "202012")
-- GGML_SYCL_SUPPORT_LEVEL_ZERO_API ON
CMake Warning at ggml/src/ggml-sycl/CMakeLists.txt:121 (message):
  Level Zero loader or headers not found, Level Zero support disabled


-- Found oneDNN: /opt/intel/oneapi/dnnl/2026.0/lib/libdnnl.so.3.11
-- Skipping -ze-intel-greater-than-4GB-buffer-required for spir64_gen AOT
-- MKL_VERSION: 2026.0.0
-- MKL_ROOT: /opt/intel/oneapi/mkl/2026.0
-- MKL_ARCH: intel64
-- MKL_SYCL_LINK: None, set to ` dynamic` by default
-- MKL_LINK: None, set to ` dynamic` by default
-- MKL_SYCL_INTERFACE_FULL: None, set to ` intel_ilp64` by default
-- MKL_INTERFACE_FULL: None, set to ` intel_ilp64` by default
-- MKL_SYCL_THREADING: None, set to ` tbb_thread` by default
-- MKL_THREADING: None, set to ` intel_thread` by default
-- MKL_SYCL_MPI: None, set to ` intelmpi` by default
-- MKL_MPI: None, set to ` intelmpi` by default
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_scalapack_ilp64.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_cdft_core.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_intel_ilp64.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_intel_thread.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_core.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_blacs_intelmpi_ilp64.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_distributed_dft.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_blas.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_lapack.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_dft.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_sparse.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_data_fitting.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_rng.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_stats.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_sycl_vm.so
-- Found /opt/intel/oneapi/mkl/2026.0/lib/libmkl_tbb_thread.so
-- Found /opt/intel/oneapi/tbb/2023.0/lib/intel64/gcc4.8/libtbb.so.12
-- Found /opt/intel/oneapi/compiler/2026.0/lib/libiomp5.so
-- find GGML_SYCL_GRAPH
-- find GGML_SYCL_HOST_MEM_FALLBACK
-- GGML_SYCL_DEVICE_ARCH=bmg_g21 (AOT via spir64_gen)
-- Including SYCL backend
-- ggml version: 0.22.0
-- ggml commit:  d2f23593f
-- Found OpenSSL: /usr/lib/libcrypto.so (found version "3.6.4")
-- Performing Test OPENSSL_VERSION_SUPPORTED
-- Performing Test OPENSSL_VERSION_SUPPORTED - Success
-- OpenSSL found: 3.6.4
-- Generating embedded license file for target: llama-app
-- Configuring done (2.8s)
-- Generating done (0.3s)
-- Build files have been written to: ~/llama.cpp/build.sycl.b10743-5-gd2f23593f
12
[  0%] Building CXX object vendor/cpp-httplib/CMakeFiles/cpp-httplib.dir/httplib.cpp.o
[  0%] Building CXX object common/CMakeFiles/llama-common-base.dir/build-info.cpp.o
[  0%] Building CXX object tools/mtmd/CMakeFiles/llama-llava-cli.dir/deprecation-warning.cpp.o
[  0%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[  0%] Building CXX object vendor/hash/CMakeFiles/vendor-hash.dir/hash.cpp.o
[  1%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml.cpp.o
[  2%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[  2%] Building CXX object tools/ui/CMakeFiles/llama-ui-embed.dir/embed.cpp.o
[  2%] Building CXX object tools/mtmd/CMakeFiles/llama-qwen2vl-cli.dir/deprecation-warning.cpp.o
[  2%] Building CXX object tools/mtmd/CMakeFiles/llama-gemma3-cli.dir/deprecation-warning.cpp.o
[  2%] Building C object vendor/hash/CMakeFiles/vendor-hash.dir/xxhash/xxhash.c.o
[  2%] Building CXX object tools/mtmd/CMakeFiles/llama-minicpmv-cli.dir/deprecation-warning.cpp.o
[  2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[  2%] Linking CXX executable llama-ui-embed
[  2%] Linking CXX static library libllama-common-base.a
[  2%] Building CXX object vendor/hash/CMakeFiles/vendor-hash.dir/sha1/sha1.c.o
[  3%] Linking CXX executable ../../bin/llama-minicpmv-cli
[  3%] Linking CXX executable ../../bin/llama-gemma3-cli
[  3%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend-meta.cpp.o
[  4%] Linking CXX executable ../../bin/llama-qwen2vl-cli
[  4%] Linking CXX executable ../../bin/llama-llava-cli
[  4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[  4%] Building C object vendor/hash/CMakeFiles/vendor-hash.dir/sha256/sha256.c.o
[  4%] Linking CXX static library libcpp-httplib.a
[  4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[  5%] Linking CXX static library libvendor-hash.a
[  5%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[  6%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[  6%] Built target llama-common-base
[  6%] Built target cpp-httplib
[  6%] Linking CXX shared library ../../bin/libggml-base.so
[  6%] Built target vendor-hash
[  6%] Built target llama-gemma3-cli
[  6%] Built target llama-llava-cli
[  6%] Built target llama-qwen2vl-cli
[  6%] Built target llama-minicpmv-cli
[  6%] Built target llama-ui-embed
[  6%] Provisioning UI assets
-- UI: downloading from b10748: https://huggingface.co/buckets/ggml-org/llama-ui/resolve/b10748/dist.tar.gz
[  6%] Built target ggml-base
[  6%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/quants.c.o
[  6%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o
[  6%] Building CXX object ggml/src/ggml-rpc/CMakeFiles/ggml-rpc.dir/transport.cpp.o
[  6%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o
[  7%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/repack.cpp.o
[  7%] Building CXX object ggml/src/ggml-rpc/CMakeFiles/ggml-rpc.dir/ggml-rpc.cpp.o
[  7%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/iqp.cpp.o
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/hbm.cpp.o
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/traits.cpp.o
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o
[  8%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/ggml-sycl.cpp.o
[  8%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/add-id.cpp.o
[  8%] Linking CXX shared library ../../../bin/libggml-rpc.so
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/binary-ops.cpp.o
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/unary-ops.cpp.o
[  9%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/binbcast.cpp.o
[  9%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/col2im-1d.cpp.o
[  9%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/common.cpp.o
[  9%] Built target ggml-rpc
[ 10%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/vec.cpp.o
-- UI: download dist.tar.gz from b10748 failed: "HTTP response code said error"
-- UI: downloading from latest: https://huggingface.co/buckets/ggml-org/llama-ui/resolve/latest/dist.tar.gz
[ 10%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/concat.cpp.o
[ 10%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/conv.cpp.o
[ 10%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/conv2d-dw.cpp.o
[ 10%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ops.cpp.o
[ 10%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/llamafile/sgemm.cpp.o
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/conv2d-transpose.cpp.o
[ 11%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/arch/x86/quants.c.o
[ 11%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/arch/x86/repack.cpp.o
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/conv2d.cpp.o
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/conv3d.cpp.o
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/convert.cpp.o
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/count-equal.cpp.o
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/cpy.cpp.o
-- UI: archive verified and extracted
-- UI: HF download succeeded, stamp updated (latest)
-- UI: gzip compression applied (~/llama.cpp/build.sycl.b10743-5-gd2f23593f/tools/ui/dist/_gzip)
embed: write output file ~/llama.cpp/build.sycl.b10743-5-gd2f23593f/tools/ui/ui.h
embed: write output file ~/llama.cpp/build.sycl.b10743-5-gd2f23593f/tools/ui/ui.cpp
[ 11%] Built target llama-ui-assets
[ 11%] Building CXX object tools/ui/CMakeFiles/llama-ui.dir/ui.cpp.o
[ 11%] Linking CXX static library libllama-ui.a
[ 11%] Built target llama-ui
[ 11%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/cross_entropy_loss.cpp.o
[ 12%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/cumsum.cpp.o
[ 12%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/diag.cpp.o
[ 12%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/dmmv.cpp.o
[ 12%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/dsv4-hc.cpp.o
[ 12%] Building CXX object ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/element_wise.cpp.o
[ 12%] Linking CXX shared library ../../bin/libggml-cpu.so
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:314:8: error: unknown type name 'ze_result_t'; did you mean 'ur_result_t'?
  314 | static ze_result_t init_zes() {
      |        ^~~~~~~~~~~
      |        ur_result_t
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:956:3: note: 'ur_result_t' declared here
  956 | } ur_result_t;
      |   ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:315:5: error: unknown type name 'ze_result_t'; did you mean 'ur_result_t'?
  315 |     ze_result_t res = zesInit(0);
      |     ^~~~~~~~~~~
      |     ur_result_t
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:956:3: note: 'ur_result_t' declared here
  956 | } ur_result_t;
      |   ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:315:23: error: use of undeclared identifier 'zesInit'
  315 |     ze_result_t res = zesInit(0);
      |                       ^~~~~~~
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:316:16: error: use of undeclared identifier 'ZE_RESULT_SUCCESS'; did you mean 'UR_RESULT_SUCCESS'?
  316 |     if (res != ZE_RESULT_SUCCESS) {
      |                ^~~~~~~~~~~~~~~~~
      |                UR_RESULT_SUCCESS
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:787:3: note: 'UR_RESULT_SUCCESS' declared here
  787 |   UR_RESULT_SUCCESS = 0,
      |   ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:323:1: error: unknown type name 'ze_result_t'; did you mean 'ur_result_t'?
  323 | ze_result_t get_zes_init_res() {
      | ^~~~~~~~~~~
      | ur_result_t
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:956:3: note: 'ur_result_t' declared here
  956 | } ur_result_t;
      |   ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:324:12: error: unknown type name 'ze_result_t'; did you mean 'ur_result_t'?
  324 |     static ze_result_t zes_init_res = init_zes();
      |            ^~~~~~~~~~~
      |            ur_result_t
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:956:3: note: 'ur_result_t' declared here
  956 | } ur_result_t;
      |   ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:727:32: warning: unused parameter 'device_dst' [-Wunused-parameter]
  727 | static void dev2dev_memcpy(int device_dst, sycl::queue &q_dst, int device_src, sycl::queue &q_src, void *ptr_dst,
      |                                ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:727:68: warning: unused parameter 'device_src' [-Wunused-parameter]
  727 | static void dev2dev_memcpy(int device_dst, sycl::queue &q_dst, int device_src, sycl::queue &q_src, void *ptr_dst,
      |                                                                    ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:5651:35: error: use of undeclared identifier 'ZE_RESULT_SUCCESS'; did you mean 'UR_RESULT_SUCCESS'?
 5651 |             get_zes_init_res() == ZE_RESULT_SUCCESS ? (MemoryAPIType) g_ggml_sycl_get_mem_api : MemoryAPIType::MEMORY_API_TYPE_SYCL);
      |                                   ^~~~~~~~~~~~~~~~~
      |                                   UR_RESULT_SUCCESS
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:787:3: note: 'UR_RESULT_SUCCESS' declared here
  787 |   UR_RESULT_SUCCESS = 0,
      |   ^
~/llama.cpp/ggml/src/ggml-sycl/ggml-sycl.cpp:6078:31: error: use of undeclared identifier 'ZE_RESULT_SUCCESS'; did you mean 'UR_RESULT_SUCCESS'?
 6078 |         get_zes_init_res() == ZE_RESULT_SUCCESS ? (MemoryAPIType) g_ggml_sycl_get_mem_api : MemoryAPIType::MEMORY_API_TYPE_SYCL);
      |                               ^~~~~~~~~~~~~~~~~
      |                               UR_RESULT_SUCCESS
/opt/intel/oneapi/compiler/2026.0/include/unified-runtime/ur_api.h:787:3: note: 'UR_RESULT_SUCCESS' declared here
  787 |   UR_RESULT_SUCCESS = 0,
      |   ^
2 warnings and 8 errors generated.
make[2]: *** [ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/build.make:79: ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/ggml-sycl.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 12%] Built target ggml-cpu
make[1]: *** [CMakeFiles/Makefile2:2690: ggml/src/ggml-sycl/CMakeFiles/ggml-sycl.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

@arthw

arthw commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

@strtgbb

-- GGML_SYCL_SUPPORT_LEVEL_ZERO_API ON
CMake Warning at ggml/src/ggml-sycl/CMakeLists.txt:121 (message):
  Level Zero loader or headers not found, Level Zero support disabled

The root cause is you didn't install the level-zero dev package.
Because the GGML_SYCL_SUPPORT_LEVEL_ZERO_API ON is default.
It trigger the build error.

Two methods:
1.
Install level-zero-dev by sudo apt-get install -y libze-dev, refer to https://dgpu-docs.intel.com/installation-guides/installing-packages-from-the-intel-ppa.html.

Disable using level-zero by -DGGML_SYCL_SUPPORT_LEVEL_ZERO_API=OFF in build command.

Thank you!

@strtgbb

strtgbb commented Sep 2, 2026

Copy link
Copy Markdown

Thanks, I ran sudo pacman -S level-zero-headers and the build works now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants