quantize: add IQ2_NL and IQ3_NL types (CPU + Metal + CUDA)

[EAddario]

Overview

K-quants and I-quants operate on 256-element super-blocks. A tensor can use them only if its row length (ncols) is an exact multiple of 256. When it isn't, the quantizer falls back to a different (usually sub-optimal) 32-element block type per tensor.

Mainline 32-block types stop at 4.50 bpw (Q4_0/IQ4_NL) creating a hard floor on every misaligned tensor and therefore on the whole file wherever misaligned tensors dominate because there is nothing better to substitute to.

This PR introduces two new CPU, Metal & CUDA only types IQ2_NL (2.50 bpw) and IQ3_NL (3.50 bpw) extending the existing non-linear family downward. Being 32-block types, they apply wherever ncols % 32 == 0.

Additional information

Test models and results including PPL, KLD and inference benchmarks available at HuggingFace.

PR for CPU backend: #27322
PR for CPU + Metal backend: #27324
PR for CPU + Metal + CUDA + Vulkan backend: #27983

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES. AI used to code review and to generate Metal and CUDA implementations.

[ggml-gh-bot]

Hi @EAddario, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 3 open PRs.

• Multiple backend changes in one PR: When adding support for a new model or feature, focus on CPU support only in the initial PR. Add support for other backends like CUDA in follow-up PRs. If you have a good reason to modify multiple backends in one PR, please explain it.

• Large PR: Large changes require prior discussion (e.g. an issue or RFC) and maintainers may not be able to review this PR as-is. Consider splitting it into smaller, focused PRs.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.