Vortex CUDA

GPU-accelerated exact vector search engine built from scratch with CUDA, C++20, Java JNI, and Spring Boot.

Vortex CUDA is an end-to-end vector-search system that starts at the CUDA kernel level and exposes persistent GPU indexes through a Java API and a hardened Spring Boot REST service.

Performance at a glance

Validated on an NVIDIA GeForce RTX 3050 6GB Laptop GPU using exact squared-L2 search:

Result	Validated workload
16.7× lower P50 latency	GPU exact Top-10 vs the project's scalar CPU reference on 500K × 128 vectors
~1.67–1.72 ms FP32 P50	Repeated live-demo single-query exact Top-10 on 500K × 128
~1,641–1,662 FP32 QPS	Repeated live-demo batch-32 exact Top-10 on 500K × 128
50% lower vector-index storage	FP16 storage vs FP32 storage
99.6875% Recall@10	FP16 storage + FP32 accumulation at batch 32 on 500K × 128
48/48 benchmark cases passed	Full reproducibility matrix
99.609375% minimum observed Recall@K	Across the M15 validation matrix

The CPU comparison is against this project's scalar reference implementation, not an optimized production vector database. The repeated live-demo numbers were captured with the Spring Boot service still running locally, so they demonstrate repeatability rather than isolated capacity-planning performance. FP16 is primarily a memory-efficiency tradeoff and is not claimed to provide a universal latency improvement.

Benchmark methodology → · RTX 3050 live benchmark snapshot →

Live RTX 3050 benchmark

For 500K × 128, exact Top-10, five repeated live-demo runs produced:

Batch	FP32 P50	FP16 P50	FP32 QPS	FP16 QPS	Recall@K
1	1.671–1.723 ms	0.987–1.012 ms	580–599	988–1,013	100%
8	4.874–4.939 ms	4.850–4.901 ms	1,620–1,641	1,632–1,649	100%
32	19.256–19.503 ms	19.553–19.804 ms	1,641–1,662	1,616–1,637	99.6875%

Vector storage for the same workload:

FP32 vector store: 244.14 MiB
FP16 vector store: 122.07 MiB
Reduction:         50.00%
Recall@1:          100%
Minimum Recall@K:  99.6875%
Validation:        PASS

Run the live benchmark locally:

.\DEMO_BENCHMARK.ps1 -Vectors 500000 -Dimension 128 -TopK 10

The wrapper runs the deterministic M15 harness and surfaces the requested workload with P50/P95/P99, QPS, FP32/FP16 comparison, storage reduction and recall validation.

Architecture

Client
  │
  │ HTTP / JSON / little-endian float32 binary
  ▼
┌──────────────────────────────────────────────┐
│              Spring Boot REST API            │
│ API key • validation • request IDs • metrics │
│ capacity limits • health • error handling    │
└──────────────────────┬───────────────────────┘
                       │
                       ▼
               VortexIndex Java API
                       │
                       │ JNI
                       ▼
                  vortex_jni.dll
                       │
                       ▼
                  GpuVectorIndex
                       │
          ┌────────────┴────────────┐
          │                         │
          ▼                         ▼
   Persistent FP32 store     Persistent FP16 store
          │                         │
          └────────────┬────────────┘
                       ▼
       CUDA exact distance + GPU Top-K kernels
                       │
                       ▼
                    NVIDIA GPU

Architecture details →

What makes it technically interesting

• Exact GPU Top-K search using squared-L2 distance.
• Persistent GPU-resident indexes instead of re-uploading vectors for every query.
• Hierarchical and fused GPU Top-K to avoid materializing a full production-path distance array.
• Warp-shuffle reductions, adaptive block sizing, and query-tile autotuning.
• Query-tiled batched kernels for improved amortized throughput.
• float4 vectorized memory access and an explicit FP32 FMA path.
• FP16 vector storage with FP32 accumulation for a validated 50% storage reduction.
• Pinned host memory + dual non-blocking CUDA streams for asynchronous transfer/compute plumbing.
• Nsight Compute-guided tuning based on measured kernel bottlenecks rather than guesswork.
• Reproducible benchmark harness with min/mean/stddev/P50/P95/P99, QPS, Recall@1, Recall@K, CSV, and JSON output.
• Java JNI integration with opaque native handles, shared_ptr lifetime protection, and lifecycle-safe AutoCloseable indexes.
• Spring Boot REST API supporting single-query, batch-query, JSON ingestion, and binary float32 ingestion.
• Production hardening with API-key protection, request correlation IDs, finite-value validation, capacity guards, health checks, metrics, graceful cleanup, and exception-based CUDA failure propagation.

Request path

POST /api/v1/indexes
        │
        ▼
Create persistent GPU index
        │
        ├──────────────► FP32 storage
        └──────────────► FP16 storage

POST /api/v1/indexes/{id}/search
        │
        ▼
Spring Boot
        ▼
Java VortexIndex
        ▼
JNI
        ▼
CUDA Top-K
        ▼
JSON neighbors

Precision modes

FP32_EXACT
FP16_STORAGE_FP32_ACCUMULATION

FP32_EXACT is the default API mode.

REST API

Method	Endpoint	Purpose
POST	/api/v1/indexes	Create a GPU index from JSON or binary float32 data
GET	/api/v1/indexes	List active indexes
GET	/api/v1/indexes/{id}	Read index statistics
POST	/api/v1/indexes/{id}/search	Exact single-query Top-K search
POST	/api/v1/indexes/{id}/search/batch	Batched exact Top-K search
DELETE	/api/v1/indexes/{id}	Release the index and GPU resources
GET	/api/v1/system	Native/CUDA runtime diagnostics
GET	/actuator/health	Service health
GET	/actuator/metrics	Operational metrics

REST API reference →

Binary ingestion

For larger uploads, Vortex supports a compact binary path:

POST /api/v1/indexes
Content-Type: application/octet-stream
X-Vortex-Vector-Count: <N>
X-Vortex-Dimension: <D>

The request body contains exactly N × D little-endian IEEE-754 float32 values in vector-major order.

Binary format →

Build

Requirements

Validated development environment:

• Windows 11 x64
• NVIDIA CUDA Toolkit 13.3
• Visual Studio 2022 Build Tools
• CMake 3.24+
• JDK 17+
• Maven 3.6.3+
• NVIDIA GPU compatible with the configured architecture

The current CMake configuration targets sm_86.

Native + Java build

cmake `
    -S . `
    -B build `
    -G "Visual Studio 17 2022" `
    -A x64

cmake --build build --config Release --clean-first

mvn --no-transfer-progress clean package

Expected artifacts:

build/Release/vortex_cuda.exe
build/Release/vortex_benchmark.exe
build/Release/vortex_jni.dll
server/target/vortex-server-1.0.0.jar

End-to-end validation

.\BUILD_AND_TEST_M18.ps1

The M18 test exercises the real native/JNI/REST stack, including:

CUDA exception path
native Release build
JNI shared library
Maven + JUnit
Spring Boot startup
health/readiness
API-key rejection
binary ingestion
capacity rejection
FP32 exact search
FP16 search
batch search
input validation
metrics
delete / 404 lifecycle

Expected final result:

============================================
 M18 PRODUCTION SMOKE TEST: PASS
============================================

Run the service

$env:VORTEX_API_KEY = "replace-with-a-strong-secret"

java `
    "-Dvortex.jni.path=D:\path\to\vortex_jni.dll" `
    -jar .\server\target\vortex-server-1.0.0.jar

When VORTEX_API_KEY is configured, /api/** requests require:

X-Vortex-Api-Key: <secret>

Health probes remain available separately:

GET /actuator/health

Security and deployment notes →

Release package

The tagged v1.0.0 release includes a Windows x64 distribution.

Download Vortex CUDA 1.0.0 →

To create the package locally from a clean worktree:

.\PACKAGE_M18_RELEASE.ps1

Output:

dist/vortex-cuda-1.0.0-windows-x64/
dist/vortex-cuda-1.0.0-windows-x64.zip

The package contains the JNI DLL, native executables, Spring Boot JAR, runtime configuration, documentation, build provenance, and a SHA-256 manifest.

Project evolution

M0–M5    CUDA exact search + GPU Top-K
M6–M12   fused, tiled, vectorized and FMA kernel optimization
M13      pinned host memory + CUDA streams
M14      FP16 storage + FP32 accumulation
M15      reproducible benchmark harness
M16      Java JNI integration
M17      Spring Boot REST backend
M18      production hardening + Vortex CUDA 1.0.0

Documentation

• Architecture
• REST API
• Benchmark methodology
• RTX 3050 live benchmark snapshot
• Binary ingestion format
• Security and deployment
• Changelog

Scope

Vortex CUDA is an exact-search engine and systems-engineering project, not a drop-in replacement for a distributed production vector database. Index data is process-resident; durable restart persistence, replication, multi-node sharding, and ANN graph/index structures are outside the 1.0.0 scope.