RVTensor

A RISC-V–controlled tensor accelerator, and the ML graph compiler that targets it. A small heterogeneous SoC in SystemVerilog — a scalar RV32I control core, a 4×4 systolic matrix engine, a software-managed scratchpad SRAM, and an explicit DMA data-movement engine — plus rvtc, an optimizing compiler that lowers a tensor graph (matmul/relu) into the accelerator's command stream with fusion, buffer-residency, and scheduling passes. Every program the compiler emits is verified bit-exact against NumPy on the RTL.

The same control-core + DMA + scratchpad + fixed-function-engine pattern is how storage controllers are organized — a RISC/ARM control core issuing commands to DMA and hardware engines (ECC, encryption, host interface) over a memory-mapped queue..

            RV32I core  (control / data movement only)
                 |
          memory-mapped command queue
                 |
        +--------+--------+
        |                 |
     DMA engine      matrix accelerator
        |                 |
   system RAM  <=DMA=>  scratchpad SRAM  <=>  4x4 systolic array
     ("DRAM")          (software-managed L1)   (16 int8 MAC PEs)

The design deliberately mirrors the ideas behind a Tenstorrent Tensix tile: a RISC-V processor for control, local SRAM used as an explicitly-managed scratchpad rather than a transparent cache, a dedicated matrix engine, and software-orchestrated data movement between DRAM, local memory, and compute.

Everything here is simulated and verified in RTL with Icarus Verilog — both sides of every benchmark number below are measured from simulation, not estimated.

---

Headline result

Same GEMM, scalar core vs. accelerator, both measured in RTL:

16x16 matrix multiply
  RV32I software GEMM :  103,227 cycles
  RVTensor accelerator:    2,080 cycles
  speedup             :     49.6x   (compute)  /  36.2x  (end-to-end incl. DMA)

N (matrix)	RV32I cycles	RVTensor cycles	+ DMA (end-to-end)	speedup	e2e speedup	MAC util.
4×4	1,807	46	94	39.3×	19.2×	8.7%
8×8	13,507	296	488	45.6×	27.7×	10.8%
16×16	103,227	2,080	2,848	49.6×	36.2×	12.3%
32×32	810,155	15,488	18,560	52.3×	43.7×	13.2%

_{RV32I = single-cycle core running sw/cpu_matmul.s (shift-add multiply, no M-extension). RVTensor = tiled 4×4 systolic engine. Reproduce with python bench/benchmark.py.}

One 4×4 GEMM, cycle by cycle

start → the engine goes busy, streams A and B through the array (feed_en), the MAC accumulator fills, and it raises done:

Open it interactively with the bundled GTKWave:

gtkwave sim/wave.vcd sim/wave.gtkw

---

The compiler (rvtc)

The scalar core in the diagram above is programmed by a command stream — the MMIO pokes that DMA operands in, launch the tiled matmul, and copy results back. sw/demo.s writes that stream by hand for one fixed GEMM. rvtc generates it from a high-level tensor graph, through an optimizing pipeline — which is the software half of a real accelerator stack.

  tensor graph        IR + passes            RV32I program       RTL
 ┌───────────┐  fuse ┌────────────┐ codegen ┌───────────┐  vvp  ┌──────┐
 │matmul/relu│ ─────►│ComputeNodes│ ───────►│ .s / .hex │ ─────►│ SoC  │
 │ over int8 │ alloc │+ buffer map│         │DMA+launch │       │+check│
 └───────────┘ sched └────────────┘         └───────────┘       └──────┘

Three passes (compiler/passes.py):

• Fusion — a ReLU that follows a matmul is folded into the engine's writeback (it has a ReLU flag). Without fusion the ReLU has no hardware unit and would fall back to the scalar core; with it, ReLU is free.
• Buffer residency — every activation/weight gets a scratchpad slot and is DMA'd in once; intermediates never spill to DRAM. A shared input is loaded once, not once per consumer.
• Scheduling — topological ordering of DMA-in → launch → DMA-out.

The IR (compiler/graph.py) is typed so it only accepts graphs the hardware can run bit-exactly (e.g. a matmul's int32 output cannot feed another matmul — there is no requantizer, so the IR rejects it instead of silently truncating).

It runs, and it's verified on the RTL

examples/mlp.py compiles a QKV projection — Q=relu(X·Wq), K=relu(X·Wk), V=X·Wv, three matmuls sharing one activation X, two of them fused with ReLU — then runs the generated program through Icarus Verilog and checks all three outputs against NumPy:

RTL (measured):  CPU cycles = 1,491   last-GEMM ACC cycles = 296
cross-check:     model busy(last N=8) = 296  -> OK
RESULT: PASS  (3 outputs bit-exact vs NumPy)

The compiler's passes are measured against a naive lowering (materialize every op to DRAM, run ReLU on the scalar core) with the same exact cycle model used in the hardware benchmark:

N	naive (cyc)	optimized (cyc)	speedup	DMA traffic cut
4×4	410	250	1.64×	1.29×
8×8	1,976	1,336	1.48×	1.29×
16×16	10,592	8,032	1.32×	1.29×
32×32	63,872	53,632	1.19×	1.29×

_{Fusion eliminates the scalar-core ReLU; residency cuts DMA traffic by reusing the shared activation. Reproduce with python bench/compiler_bench.py; verify on RTL with python examples/mlp.py or pytest tests/.}

See compiler/README.md for the full pipeline write-up.

---

Architecture

Modules (rtl/)

File	Role
rv32i_core.sv	Single-cycle RV32I core: LUI/AUIPC/JAL/JALR, all branches, LW/SW, full OP-IMM & OP groups, EBREAK. Programs the engines over MMIO.
systolic_array.sv	4×4 output-stationary array of int8×int8→int32 MAC PEs. Activations stream from the left, weights from the top, skewed diagonally.
matrix_accelerator.sv	Tiled GEMM FSM. Decomposes an N×N GEMM into 4×4 tiles, streams them through the array, accumulates partial products across the K dimension in-place, applies optional ReLU, writes int32 results.
scratchpad.sv	Software-managed local SRAM (dual read port + write port). The "L1" — not a cache.
dma.sv	Explicit copy engine: system RAM ⇄ scratchpad, one word/cycle, either direction.
accelerator_controller.sv	Memory-mapped register file (the command queue) the CPU pokes to configure and launch the DMA + accelerator, and polls for completion.
system_ram.sv	The "DRAM" the CPU sees.
top.sv	Ties it together with trivial software-serialised arbitration (RAM owned by DMA while busy else CPU; scratchpad owned by accelerator while busy else DMA).

The systolic array

Each processing element does one thing on every enabled cycle:

acc <= acc + a_in * b_in;   // and pass a_in right, b_in down

For C[i][j] = Σₖ A[i][k]·B[k][j], element A[i][k] is injected at the left of row i at cycle k+i and B[k][j] at the top of column j at cycle k+j. The diagonal skew makes the right operands meet at PE(i,j) on the same cycle, so all 16 PEs compute their dot products in parallel. See the derivation in the header of rtl/systolic_array.sv.

Why tiling matters

The array is 4×4, but it multiplies 8×8, 16×16, 32×32 by decomposing the problem into 4×4 tiles. Because the array is output-stationary, partial products along the K dimension accumulate in the PE accumulators themselves — the controller simply streams the next K-tile without clearing, which is exactly why tiling is cheap here and why real accelerators tile.

The programming model (what the CPU actually does)

sw/demo.s is the whole story — the scalar core never touches a matrix element:

1. DMA A : system RAM -> scratchpad
2. DMA B : system RAM -> scratchpad
3. write N / base addresses / ReLU, then ACC_CMD = start
4. poll ACC_STATUS until done
5. DMA C : scratchpad -> system RAM
6. ebreak

MMIO map (base 0x8000_0000) is documented in rtl/accelerator_controller.sv.

---

Results discussion (a.k.a. interview talking points)

• Where the 40–50× comes from. A scalar RV32I MAC is a multi-instruction subroutine (load, load, shift-add multiply, add) with no data reuse — an O(N³) instruction stream. The array does 16 MACs per cycle and reuses each operand across a row/column.
• Why MAC utilisation is only ~10–13%. With a 4×4 array and K=4 tiles, the systolic fill/drain (≈12 feed cycles for 4 useful MAC cycles per PE) and the per-tile scratchpad load (16 cycles) are not amortised over a long K stream. The obvious fixes are the interesting part: double-buffer the tile load behind compute, use larger tiles / longer K to amortise fill-drain, and pipeline the MAC so feed cycles are all useful. This is the classic memory-bandwidth vs. compute trade-off.
• What limits performance? Scratchpad bandwidth (one A + one B word/cycle) and the load/compute serialisation, not the MACs.
• What happens when the matrix exceeds SRAM? You tile the outer problem too and stream tiles from DRAM through the DMA — the scratchpad becomes a staging buffer, and DMA bandwidth becomes the ceiling.
• Cycle model is exact. bench/accel_model.py derives busy = nt²·(28·nt + 18) (nt = N/4) straight from the FSM, and benchmark.py asserts it matches the RTL at every size.

---

Build & run

Requirements: Icarus Verilog (iverilog 12+, ships with GTKWave) and Python 3 with numpy + matplotlib.

# everything: assemble, run all testbenches, benchmark, render figures
./run.sh            # bash / Git-Bash
# or
pwsh ./run.ps1      # PowerShell (Windows)

Individual pieces:

# one testbench
iverilog -g2012 -o sim/t.vvp rtl/systolic_array.sv tb/tb_systolic_array.sv && vvp sim/t.vvp

# the benchmark table + plots (drives the RTL across sizes)
python bench/benchmark.py

# the waveform figure from sim/wave.vcd
python bench/wave_plot.py

# the compiler: lower an ML graph and verify it bit-exact on the RTL
python examples/mlp.py
python -m pytest tests/ -q        # compiler unit + end-to-end tests

Test status

Testbench	Checks
tb_systolic_array	4×4 GEMM vs. golden — PASS
tb_matrix_accelerator	4/8/16/32 GEMM + ReLU vs. golden — PASS
tb_rv32i_core	runs a real program (Σ1..10, subtract) — PASS
tb_top	end-to-end: CPU drives DMA+accelerator for an 8×8 GEMM, result DMA'd back matches golden — PASS
tb_compiler + tests/	compiler end-to-end: rvtc lowers graphs (single FC layer, QKV, 16×16), each run through the SoC and checked bit-exact vs NumPy — PASS (pytest tests/)

---

Repository layout

rtl/       SystemVerilog RTL (8 modules)
compiler/  rvtc — the ML graph compiler (IR, passes, codegen, cost model, RTL verify)
examples/  compilable ML graphs (QKV projection, FC layer)
tests/     pytest: compiler unit tests + end-to-end bit-exact RTL verification
tb/        Testbenches (unit + system + compiler + benchmark harnesses)
sw/        RV32I assembler (Python) + assembly programs (demo, software GEMM)
bench/     Cycle/functional models, benchmark driver, compiler cost plot, waveform renderer
sim/       Build/run artifacts (git-ignored) + GTKWave save file
docs/      Figures

Scope & honesty notes

• Single-cycle CPU (1 instr/cycle) — this project is about the accelerator and the heterogeneous programming model, not a pipelined core.
• The scratchpad/RAM use combinational (register-file-style) reads; a synchronous-read SRAM is a drop-in refinement.
• int8 operands, int32 accumulation. Larger-than-scratchpad matrices, a real NoC across multiple tiles, and double-buffered tile loads are the natural next steps (see the discussion above).
• The compiler targets the accelerator's native shape (square int8 GEMM + fused ReLU). A requantization pass (int32→int8) to chain matmuls into true multi-layer MLPs, and a double-buffering pass to overlap DMA with compute, are the natural next passes — both map directly onto the utilization discussion above.

License

MIT — see LICENSE.