Add Tile-Low-Rank Matrices in NextLA

[alecarraro]

Final GSoC deliverable: ext wraps the relevant vendor APIs, while the self-contained TLRmodule implements tile-low-rank storage, ARA compression, and TLR-GEMM on CPU and CUDA via KernelAbstractions.

Storage

Two packed, GEMM-friendly containers store exact per-tile ranks:

• CompressedFTLRMatrix: all tiles stored as low-rank U V' factors.
• TLRMatrix: dense diagonal with a CompressedFTLRMatrix off-diagonal.

ARA compression

Blocked Adaptive Randomized Approximation (Algorithm 2.3 of [1]) is batched across tiles. It uses mixed-precision Cholesky-QR, per-tile adaptive stopping, and rank-revealing truncation to maxrank/tol. Cholesky breakdown is treated as a rank signal, and tiles whose spectra exceed the requested rank are reported.

Example:

A_tlr = CompressedFTLRMatrix(A, tile_size; maxrank, tol)  #compress A and construct a full TRL matrix
A_tlr = TLRMatrix(A, b; maxrank, tol, diagonal=:dense)  #compress A and construct a TRL matrix
uncompress!(A_dense, A_tlr) #uncompress A_tlr

FTLRCompressionWorkspace supports reuse of compression staging.

TLR-GEMM

Both GEMM variants compute C ← α·op(A)·op(B) + β·C.

Dense accumulator: gemm!(C, A, B; ...) supports compressed×compressed, TLR×dense, and dense×TLR products using grouped/batched GEMM. Execution respects caller-specified workspace budgets and falls back tilewise when necessary. analyze_compressed_gemm provides reusable symbolic analysis and persistent grouped-GEMM descriptors.

Compressed accumulator: gemm(A, B; maxrank, rank_multiple, tol, ...) discovers output ranks with ARA. Fixed-width staging is compacted into packed storage on completion. A rolling-admission scheduler bounds workspace independently of grid size by retiring converged tiles and backfilling available slots.

Precision and backends

Supports Float16, BFloat16, Float32, Float64, and TF32. Tensor-core paths validate rank and storage constraints before scheduling, while incompatible grouped-GEMM members fall back to ordinary GEMMEx.

CPU and CUDA are supported. AMDGPU, oneAPI, and Metal hooks are present but untested.

Known limitations

Compressed-output GEMM currently requires a regular tile grid and executes on a single stream. Ragged final tiles remain supported for storage, compression, and dense-output GEMM.

[1] W. Boukaram, G. Turkiyyah, D. Keyes. Hierarchical Matrix Operations on GPUs. SIAM J. Sci. Comput. 41(4):C339–C366, 2019.