[SYCL] Enhance to get the free memory of Intel GPU

[arthw]

Overview

The legacy code use SYCL API to get the free mem size.
It can support all GPUs to get free size. In such case, use the total to replace it.

This PR support both level zero and SYCL API to get the free memory.
level zero API can support more GPUs and cases.
It can be changed by env var: GGML_SYCL_GET_MEM_API

Additional information

SYCL API get free mem has limitation:

1. Not support all GPUs, like Arc770.
2. Depend on the env var: ZES_ENABLE_SYSMAN=1

There is risk to get the wrong free memory during running, it will bring some troubles.
Some issues would have relationship with it, like memory leak.

In this PR, use both level zero and SYCL API to get the free mem size.
It's controlled by env var: GGML_SYCL_GET_MEM_API.

It's verified on Arc770, B60, iGPU(UHD Graphics 770, Ultra 7 265K).
No performance impact.

Base:

0.45.008.589 I common_memory_breakdown_print: | memory breakdown [MiB]                     | total    free    self   model   context   compute    unaccounted |
0.45.008.589 I common_memory_breakdown_print: |   - SYCL0 (Intel(R) Arc(TM) A770 Graphics) | 15473 = 15473 + (5718 =  3577 +    2048 +      92) +       -5718 |
0.45.008.589 I common_memory_breakdown_print: |   - Host                                   |                    90 =    70 +       0 +      20                |

PR:

0.06.532.066 I common_memory_breakdown_print: | memory breakdown [MiB]                     | total    free    self   model   context   compute    unaccounted |
0.06.532.066 I common_memory_breakdown_print: |   - SYCL0 (Intel(R) Arc(TM) A770 Graphics) | 16288 = 10282 + (5718 =  3577 +    2048 +      92) +         287 |
0.06.532.067 I common_memory_breakdown_print: |   - Host                                   |                    90 =    70 +       0 +      20                |

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: NO

[HumerousGorgon]

Can confirm, this PR does remove the errors in llama.cpp being unable to query A770 free memory regions, even when using the Xe driver.
Thank you!

[NateHag]

Hey arthw, thanks for putting the PR together. I'm the one who posted the 4B test data and devcoredumps on #27595.

I want to validate this on a B70, since that card isn't in your test matrix. Same one from the original report: Battlemage G31, 32GB, 8086:E223, kernel 7.0.0-30, driver 26.27.39122.14 from the Intel PPA.

What's the cleanest way to exercise the change? Build this branch and compare the memory breakdown with --fit between GGML_SYCL_GET_MEM_API settings? Or is there a specific command you'd rather I run? Happy to post before and after numbers here.

I've also got a B60 going into a second box, so I can cover that card too once it's up.

[NateHag]

Validated this on an Arc Pro B70 (32GB Battlemage) with a 27B model holding about 23.4GB VRAM. All three paths plan the same context:

build	free mem reported	ctx planned
master (SYCL path)	9209 MiB	40960 -> 40448
this PR (Level Zero sysman, default)	9207 MiB	40960 -> 40448
this PR (GGML_SYCL_GET_MEM_API=1)	9209 MiB	40960 -> 40448

So both query paths are accurate on B70 with the current driver, and the change is a no-op here. That fills in a Battlemage datapoint for the matrix.

Separate observation, and I do not think it is caused by this PR: under sustained load (4 parallel streams of small prompts, 200 tokens each) both builds eventually hit a xe driver job timeout and engine reset, then llama-server dies with UR_RESULT_ERROR_DEVICE_LOST on FLASH_ATTN_EXT at ggml-sycl.cpp:5428. The crash reproduces identically on master and on this build, so it looks like a driver/SYCL issue independent of the memory query change. Happy to share dmesg and coredump details if that is useful for a separate issue.

[arthw]

@HumerousGorgon @NateHag
Thank you for your feedback!

When I check the issue: #27595, it hint me that the method to get the free mem size would impact it.
So, I create this PR.
It's great to help Arc770 user!

@NateHag
For B70, we will continue checking it.
By now, we think it has relationship with driver.
I will reply on #27595.

Thank you!