quantize: add IQ2_NL and IQ3_NL types (CPU + Metal + CUDA + Vulkan)

[EAddario]

Overview

K-quants and I-quants operate on 256-element super-blocks. A tensor can use them only if its row length (ncols) is an exact multiple of 256. When it isn't, the quantizer falls back to a different (usually sub-optimal) 32-element block type per tensor.

Mainline 32-block types stop at 4.50 bpw (Q4_0/IQ4_NL) creating a hard floor on every misaligned tensor and therefore on the whole file wherever misaligned tensors dominate because there is nothing better to substitute to.

This PR introduces two new CPU, Metal, CUDA & Vulkan only types IQ2_NL (2.50 bpw) and IQ3_NL (3.50 bpw) extending the existing non-linear family downward. Being 32-block types, they apply wherever ncols % 32 == 0.

Additional information

Test models and results including PPL, KLD and inference benchmarks available at HuggingFace.

PR for CPU backend: #27322
PR for CPU + Metal backend: #27324
PR for CPU + Metal + CUDA: #27325

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES. AI used to code review and to generate Metal, CUDA and Vulkan implementations.

[ggml-gh-bot]

Hi @EAddario, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 4 open PRs.

• Multiple backend changes in one PR: When adding support for a new model or feature, focus on CPU support only in the initial PR. Add support for other backends like CUDA in follow-up PRs. If you have a good reason to modify multiple backends in one PR, please explain it.

• Large PR: Large changes require prior discussion (e.g. an issue or RFC) and maintainers may not be able to review this PR as-is. Consider splitting it into smaller, focused PRs.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[arthw]

What's the purpose of this update in cpy.hpp

[arthw]

The update in ggml-sycl.cpp to skip the IQ2/3_NL is not enough.

Here is my patch, please check and use it.

diff --git a/ggml/src/ggml-sycl/ggml-sycl.cpp b/ggml/src/ggml-sycl/ggml-sycl.cpp
index b762ae446..b3830efa9 100644
--- a/ggml/src/ggml-sycl/ggml-sycl.cpp
+++ b/ggml/src/ggml-sycl/ggml-sycl.cpp
@@ -6081,7 +6081,9 @@ static bool do_ggml_backend_sycl_device_supports_op(ggml_backend_dev_t dev, cons
 
         case GGML_OP_SET_ROWS:
             {
-                if (op->type == GGML_TYPE_TQ2_0) {
+                if (op->type == GGML_TYPE_TQ2_0 ||
+                    op->type == GGML_TYPE_IQ2_NL ||
+                    op->type == GGML_TYPE_IQ3_NL) {
                     return false;
                 }
                 auto res = (op->src[0]->type == GGML_TYPE_F32 || op->src[0]->type == GGML_TYPE_F16 ||
@@ -6208,7 +6210,12 @@ static bool do_ggml_backend_sycl_device_supports_op(ggml_backend_dev_t dev, cons
                     }
                 }
 
-                if (src0_type == GGML_TYPE_TQ2_0 || src1_type == GGML_TYPE_TQ2_0) {
+                if (src0_type == GGML_TYPE_TQ2_0 ||
+                    src1_type == GGML_TYPE_TQ2_0 ||
+                    src0_type == GGML_TYPE_IQ2_NL ||
+                    src1_type == GGML_TYPE_IQ2_NL ||
+                    src0_type == GGML_TYPE_IQ3_NL ||
+                    src1_type == GGML_TYPE_IQ3_NL) {
                     return false;
                 }