MAGIA-v4 Add a private cluster Event Unit and a direct L1 link for the control core

Bender.lock

@@ -81,7 +81,7 @@ packages:
     - fpnew
     - tech_cells_generic
   cv32e40x:
-    revision: a90101211048ba1a16cedbe4db963ab6e12569d7
+    revision: 8834f135214f8f25f710ebd77f48d9f03d9548e6
     version: null
     source:
       Git: https://github.com/pulp-platform/cv32e40x.git

Bender.yml

@@ -26,7 +26,7 @@ package:
 
 dependencies:
   redmule           : { git: "https://github.com/pulp-platform/redmule.git"           , rev: 2.2.1                                    }
-  cv32e40x          : { git: "https://github.com/pulp-platform/cv32e40x.git"          , rev: a90101211048ba1a16cedbe4db963ab6e12569d7 } # branch: vi/redmule_scaleup
+  cv32e40x          : { git: "https://github.com/pulp-platform/cv32e40x.git"          , rev: 8834f135214f8f25f710ebd77f48d9f03d9548e6 } # branch: lb/operand_forwarding
   cv32e40p          : { git: "https://github.com/FondazioneChipsIT/cv32e40p.git"      , rev: 193270df33f76f2728a364bd2d9b1865e0b7e4f9 } # branch: lb/PulpClusterCV32OHWFPU-divsqrt
   spatz             : { git: "https://github.com/pulp-platform/spatz.git"             , rev: 3228a9077cb33b4ee235029e8fcace52ed327334 } # branch: lb/magia-spatz_cc
   idma              : { git: "https://github.com/FondazioneChipsIT/iDMA.git"          , rev: 26ac2cd1c535851b14c9c508762203a180961cad } # branch: cc/mcast_magia

Makefile

@@ -577,7 +577,7 @@ hw-all: hw-clean hw-lib hw-compile hw-opt
 # Nonfree components
 MAGIA_NONFREE_REMOTE ?= $(GITLAB_UNIBO_SSH_STRING)/magia/nonfree.git
 MAGIA_NONFREE_DIR    ?= nonfree
-MAGIA_NONFREE_COMMIT ?= 5d887bcdf505c0949fa05dcd1b8e6ce3199b912c
+MAGIA_NONFREE_COMMIT ?= 3ac0a45bd398808a3357528608ff49a6d1bb6037
 MAGIA_NONFREE_DEPS   ?= 1
 
 .PHONY: magia-nonfree-init

README.md

@@ -181,17 +181,17 @@ To change the thread count, re-run `make verilate` with the new value.
 
 ## ⚙️ Architecture
 
-![](doc/MAGIA.png)
+![](doc/MAGIA_v4.png)
 
 ### Tile
-The central piece of the architecture is the MAGIA tile containing a GeMM accelerator, a Vector Processor, a DMA engine, a PULP cluster of RISC-V cores, a multi-banked L1 SPM, an Event Unit, and a lightweight control core. The L1 features interleaved memory banks that compose the Tightly-Coupled Data Memory (TCDM). Each tile has access to the global L2 and to a subset of other tiles' L1, accessing the latter via on-chip remote direct memory access (RDMA). Inter-tile and global communication is carried out through AXI-based narrow (32-bit) and wide (256-bit) NoC channels in [FlooNoC](https://github.com/pulp-platform/FlooNoC). External tiles and the core access the L1 through an OpenBus Interface ([OBI](https://github.com/pulp-platform/obi)) XBAR.
+The central piece of the architecture is the MAGIA tile containing a GeMM accelerator, a Vector Processor, a DMA engine, a PULP cluster of RISC-V cores, a multi-banked L1 SPM, an Event Unit, and a lightweight control core. The L1 features interleaved memory banks that compose the Tightly-Coupled Data Memory (TCDM). Each tile has access to the global L2 and to a subset of other tiles' L1, accessing the latter via on-chip remote direct memory access (RDMA). Inter-tile and global communication is carried out through AXI-based narrow (32-bit) and wide (256-bit) NoC channels in [FlooNoC](https://github.com/pulp-platform/FlooNoC). External tiles access the L1 through an OpenBus Interface ([OBI](https://github.com/pulp-platform/obi)) XBAR, the control core through its own HCI port.
 
 Each tile is controlled by a [CV32E40P](https://github.com/pulp-platform/cv32e40p) or [CV32E40X](https://github.com/openhwgroup/cv32e40x) main core (`core=`). Control of iDMA, RedMulE, FractalSync, Spatz CC, and the PULP cluster follows a memory-mapped model, with the Event Unit handling event aggregation for system control.
 
 `magia_tile` wraps `magia_isle` (cores, accelerators, L1, crossbars and Event Unit) with the FlooNoC network interface and router. What a tile contains is set by its `TileCfg` (`magia_tile_cfg_t` in `hw/mesh/magia_pkg.sv`).
 
 #### PULP Cluster
-Each tile embeds a cluster of `TileCfg.Cluster.NumCores` (8 by default) [CV32E40P](https://github.com/pulp-platform/cv32e40p) cores. Cluster cores share a Snitch instruction cache with an AXI refill path to L2, each core reaches the tile's L1 through its own HCI port and everything else (accelerator registers, PULP_CTRL, remote memory) through its own OBI master port into the tile crossbar. Cluster cores receive interrupts exclusively from the tile CSR (`PULP_CTRL`) — they are not connected to the Event Unit. The main core dispatches tasks to the cluster via the `PULP_CTRL` register block (`0x1740`), which provides: binary entry point (`PULP_BINARY`), per-core MEI dispatch (`PULP_START`), task function pointer (`PULP_TASKBIN`), data pointer (`PULP_DATA`), completion quorum (`PULP_NB_CORES_TO_WAIT`), and readiness/done handshake registers (`PULP_READY`, `PULP_DONE`). When the done quorum is reached, the tile CSR raises EU bit 12 on the main core's Event Unit, allowing the main core to sleep in WFE until the cluster finishes.
+Each tile embeds a cluster of `TileCfg.Cluster.NumCores` (8 by default) [CV32E40P](https://github.com/pulp-platform/cv32e40p) cores. Cluster cores share a Snitch instruction cache with an AXI refill path to L2; each core reaches the tile's L1, stack and reserved region through its own HCI port and everything else (accelerator registers, `PULP_CTRL`, remote memory) through its own OBI master port into the tile crossbar. The cluster has its own [Event Unit](https://github.com/pulp-platform/event_unit_flex), separate from the main core's, with a dispatch FIFO, hardware barriers and a hardware mutex. The main core dispatches a task to cluster core 0 through the `PULP_CTRL` register block (`0x1740`); core 0 can then spread it over the other cores through the cluster Event Unit. When the task ends, `PULP_CTRL` raises EU bit 12 on the main core's Event Unit.
 
 ### Mesh
 Replicating the MAGIA tile, we scale up to a two-dimensional (2D) mesh of compute tiles. Each tile takes its `TileCfg` from the `TILE_CFGS` parameter of `magia` (`HOMO_TILE_CFGS` by default). The NoC allows access to the global west-side L2 through row-side interfaces, while tiles exchange traffic through FlooNoC router. The mesh uses XY routing and carries both AXI narrow channels (32-bit) and AXI wide channels (256-bit), with protocol conversion handled by per-tile Network Interfaces (NIs).
@@ -271,11 +271,13 @@ Private map: the same range in every tile, each tile maps it to its own copy:
 | *RedMulE CTRL*    | `0x0000_0100-0x0000_01FF` |
 | *iDMA CTRL*       | `0x0000_0200-0x0000_05FF` |
 | *FractalSync CTRL*| `0x0000_0600-0x0000_06FF` |
-| *Event Unit*      | `0x0000_0700-0x0000_16FF` |
+| *Ctrl-core Event Unit* | `0x0000_0700-0x0000_16FF` |
 | *Spatz CTRL*      | `0x0000_1700-0x0000_173F` |
 | *PULP CTRL*       | `0x0000_1740-0x0000_17FF` |
 | *Collective CTRL* | `0x0000_1800-0x0000_18FF` |
-| *Reserved*        | `0x0000_1900-0x0000_FFFF` |
+| *Cluster Event Unit (direct)*   | `0x0000_1900-0x0000_28FF` |
+| *Cluster Event Unit (SoC-side)* | `0x0000_2900-0x0000_38FF` |
+| *Reserved*        | `0x0000_3900-0x0000_FFFF` |
 | *Stack*           | `0x0001_0000-0x0001_FFFF` |
 | *Spatz BootROM*   | `0x1000_0000-0x1000_00FF` |
 
@@ -294,7 +296,7 @@ Software/test utility addresses (used by SW runtime and testbench VIP):
 | Region            | Address                                    | Notes |
 |-------------------|--------------------------------------------|-------|
 | *Test End*        | `0xCCFF_0000`                              | Exit code location used by SW runtime/tests |
-| *String (utoa)*   | `0x0000_1900`                              | String scratch location (`RESERVED_START + STR_OFFSET`) |
+| *String (utoa)*   | `0x0000_3900`                              | String scratch location (`RESERVED_START + STR_OFFSET`) |
 | *Print (stderr)*  | `0xFFFF_0000`                              | Memory-mapped stderr sink in simulation VIP |
 | *Print (stdio)*   | `0xFFFF_0004`                              | Memory-mapped stdio sink in simulation VIP |
 | *Synch.*          | `tile_base + 0x000F_F000`                  | Top 4 KiB of L1 (`L1_BASE + SYNC_OFFSET`) |
@@ -343,17 +345,16 @@ A store is then tagged collective if it is written at `COLLECTIVE_ADDR_OFFSET` (
 3. A store at address `0xB000_0000` is then interpreted as a collective transaction
 
 ### PULP Cluster programming flow
-The PULP cluster uses a bare-metal dynamic dispatch model. The cluster binary is compiled as a position-independent ELF (origin `0x0`), converted to a flat binary, and embedded in the CV32 ELF as a byte array in the `.pulp_binary` section (see `sw/kernel_pulp/`).
+The PULP cluster uses a bare-metal, two-level dispatch model. The cluster binary is compiled as a position-independent ELF (origin `0x0`), converted to a flat binary, and embedded in the CV32 ELF as a byte array in the `.pulp_binary` section (see `sw/kernel_pulp/`).
 
-**Dispatch flow** (`sw/utils/cluster_utils.h`, `sw/utils/magia_pulp_utils.h`):
+**Main core → cluster core 0** (`sw/utils/cluster_utils.h`, `sw/utils/magia_pulp_utils.h`):
 
-1. `cluster_boot(binary_start)` — writes `PULP_BINARY`, asserts `CLK_EN`, polls `PULP_READY` until all cluster cores have armed their dispatcher loop.
-2. `cluster_arm_done_event()` — clears the CV32 Event Unit buffer and enables only EU bit 12 (cluster-done), avoiding spurious wakeups from stale RedMulE/iDMA events.
-3. `cluster_dispatch_task(task_addr, core_mask)` — writes `NB_CORES_TO_WAIT`, `TASKBIN`, then `PULP_START = core_mask`, which fires a per-core MEI to each selected core. Returns once all selected cores have ACK'd (i.e., `PULP_START` self-clears).
-4. `cluster_wait_done_eu()` — the control core sleeps on a blocking Event Unit load (`cv.elw` on CV32E40P, `lw` on CV32E40X) until EU bit 12 fires (PULP_DONE quorum reached).
-5. `cluster_stop()` — de-asserts `CLK_EN` to gate the cluster clock.
+1. `cluster_boot(binary_start)` — writes `PULP_BINARY`, sets `PULP_FETCH_EN`, polls `PULP_READY` until all cluster cores have booted.
+2. `cluster_arm_done_event()` — clears the CV32 Event Unit buffer and enables only EU bit 12 (cluster-done).
+3. `cluster_dispatch_task(task_addr)` — writes `PULP_TASKBIN`/`PULP_DATA` and rings `PULP_START`; returns once core 0 has taken the task.
+4. `cluster_wait_done_eu()` — the control core sleeps on a blocking Event Unit load (`cv.elw` on CV32E40P, `lw` on CV32E40X) until EU bit 12 fires.
 
-Each cluster core runs a dispatcher loop that waits in `WFI` for the MEI, reads `PULP_TASKBIN`/`PULP_DATA` from the trap handler, calls the task function, writes `PULP_DONE`, and re-enters `WFI`.
+**Core 0 → the other cores** (`sw/utils/cluster_utils.h`): `pi_cl_team_fork(n, entry, arg)` runs `entry(arg)` on `n` cores through the cluster Event Unit and ends on a hardware barrier, as in the pulp-sdk API.
 
 Cluster task sources live under `sw/tests/<test>/pulp_task/`. A test directory containing a `pulp_task/` subdirectory automatically triggers the dual-binary build flow in the Makefile.
 ## 🧰 Changing number of tiles

hw/mesh/magia.sv

@@ -217,12 +217,7 @@ module magia
 
         .irq_i               ( irq_i[i*N_TILES_X+j]                  ),
 
-        // Tile expects [N_CLUSTER_CORES:0] (1 main + 8 cluster cores).
-        // Replicate the single top-level debug_req_i bit across all cores;
-        // implicit name-based connection would leave bits [N:1] unconnected (X)
-        // and X-propagate into cv32e40p_controller.debug_req_pending, corrupting
-        // ctrl_fsm_cs (observed: ctrl_fsm_cs=0x1xx, debug_req_pending=x).
-        .debug_req_i         ( '0                               ),
+        .debug_req_i         ( debug_req_i                      ),
         .debug_havereset_o   ( debug_havereset_o[i*N_TILES_X+j] ),
         .debug_running_o     ( debug_running_o[i*N_TILES_X+j]   ),
         .debug_halted_o      ( debug_halted_o[i*N_TILES_X+j]    ),

hw/tile/cluster_event_map.sv

@@ -1,8 +1,8 @@
 /*
  * Copyright (C) 2023-2024 ETH Zurich and University of Bologna
  *
- * Licensed under the Solderpad Hardware License, Version 0.51 
- * (the "License"); you may not use this file except in compliance 
+ * Licensed under the Solderpad Hardware License, Version 0.51
+ * (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at
  *
  *     http://solderpad.org/licenses/SHL-0.51
@@ -16,8 +16,7 @@
  *
  * Authors: Luca Balboni <luca.balboni10@studio.unibo.it>
  *
- * Simple cluster event mapping module for MAGIA project
- * This module maps various event types to the final cluster event outputs
+ * Cluster event mapping: control-core layout for NB_CORES == 1, PULP cluster layout otherwise
 */
 
 module cluster_event_map #(
@@ -40,18 +39,31 @@ module cluster_event_map #(
   output logic [NB_CORES-1:0][31:0] events_mapped_o
 );
 
-  // Simple event mapping for each core
+  // NB_CORES == 1 selects the control-core layout, so a 1-core cluster would get it too
   for (genvar i = 0; i < NB_CORES; i++) begin : gen_event_mapping
-    assign events_mapped_o[i] = {
-      cluster_events_i[i][31:16],           // [31:16] Custom cluster events (upper 16 bits)
-      cluster_events_i[i][15:12],           // [15:12] PULP cluster events
-      acc_events_i[i],                      // [11:8]  Accelerator events
-      2'b0,                                 // [7:6]   Reserved
-      timer_events_i[i],                    // [5:4]   Timer events
-      dma_events_i[i],                      // [3:2]   DMA events
-      dispatch_events_i[i],                 // [1]     Dispatch event
-      barrier_events_i[i] | mutex_events_i[i] | periph_fifo_event_i  // [0] Combined sync/periph events
-    };
+    if (NB_CORES == 1) begin : gen_ctrl_core_map
+      assign events_mapped_o[i] = {
+        cluster_events_i[i][31:16],           // [31:16] Custom cluster events (pass-through)
+        cluster_events_i[i][15:12],           // [15:12] Custom cluster events (pass-through)
+        acc_events_i[i],                      // [11:8]  Accelerator events
+        sw_events_i[i][1:0],                  // [7:6]   Software events
+        timer_events_i[i],                    // [5:4]   Timer events
+        dma_events_i[i],                      // [3:2]   DMA events
+        dispatch_events_i[i],                 // [1]     Dispatch event
+        barrier_events_i[i] | mutex_events_i[i] | periph_fifo_event_i // [0] Combined sync/periph events
+      };
+    end else begin : gen_cluster_map
+      assign events_mapped_o[i] = {
+        cluster_events_i[i][31:19],           // [31:19] Custom cluster events (unchanged position)
+        dispatch_events_i[i],                 // [18]    PULP_DISPATCH_EVENT
+        mutex_events_i[i],                    // [17]    PULP_MUTEX_EVENT
+        barrier_events_i[i],                  // [16]    PULP_HW_BAR_EVENT
+        cluster_events_i[i][15:12],           // [15:12] Custom cluster events (unchanged position)
+        sw_events_i[i],                       // [11:4]  Software events (no acc/timer events in the cluster EU)
+        dma_events_i[i],                      // [3:2]   DMA events (tied to '0 in the cluster EU)
+        2'b0                                  // [1:0]   Reserved
+      };
+    end
   end
 
 endmodule : cluster_event_map

hw/tile/ctrl_demux.sv

@@ -14,11 +14,10 @@
  * limitations under the License.
  * SPDX-License-Identifier: SHL-0.51
  *
- * Authors: Luca Balboni, Fondazione Chips-IT
+ * Author: Luca Balboni, Fondazione Chips-IT
  *
- * MAGIA control demux: demuxes the single control port of the OBI crossbar towards
- * the control registers of each enabled unit
-*/
+ * MAGIA control demux: splits the control port of the OBI crossbar towards the enabled control units
+ */
 
 module ctrl_demux
   import magia_tile_pkg::*;