Skip to content

riscv: implement multikernel lifecycle and safe shutdown - #39

Draft
pro-utkarshM wants to merge 9 commits into
multikernel:multikernel-riscvfrom
pro-utkarshM:riscv/multikernel-23-25-v1
Draft

riscv: implement multikernel lifecycle and safe shutdown#39
pro-utkarshM wants to merge 9 commits into
multikernel:multikernel-riscvfrom
pro-utkarshM:riscv/multikernel-23-25-v1

Conversation

@pro-utkarshM

@pro-utkarshM pro-utkarshM commented Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown

Summary

  • implement the RISC-V multikernel CPU lifecycle with direct SBI HSM calls,
    physical hart IDs, bounded HART_STATUS polling, and parked-state checks
  • add an immutable entry stub that executes fence.i on every start while
    preserving the RISC-V a0/a1 boot ABI
  • contain spawn halt, power-off, reboot, panic, and SMP-stop paths with
    HART_STOP, and serialize lifecycle/resource changes against image teardown
  • retain a hot-added CPU with its instance when a lost ACK leaves ownership
    uncertain, so a possibly running hart is never exposed through the free pool

Addresses #23, #24, and #25.

Base and attribution

This draft targets multikernel-riscv at
fe31d7b0ca32b0f806c88059c88d70600fda45cc, the RISC-V foundation from #37
that Cong applied for follow-up work.

The previous temporary replay of Nikolay Nikolaev's foundation commit has been
dropped. His accepted foundation remains below this series with its original
authorship; every commit in this PR is above that branch point.

Scope

This PR keeps #23-#25 together because lifecycle, entry-cache coherency, and
safe shutdown share the same HSM handoff invariants.

#26 remains a separate follow-up branch. RISC-V instance-DTB device filtering
(#27) and firmware IPI receive support (#28) are out of scope.

Validation

Current head: eef3dcbad234

  • rv64 LLVM vmlinux Image build with CONFIG_MULTIKERNEL=y
  • rv64 LLVM vmlinux Image build with multikernel disabled
  • nommu_virt_defconfig check proving RISCV_M_MODE=y excludes multikernel
  • x86_64 GCC vmlinux build with CONFIG_MULTIKERNEL=y, CONFIG_OF=y, and
    CONFIG_KEXEC_FILE=y
  • linked-symbol check for every RISC-V multikernel architecture hook
  • entry-stub disassembly check: leading fence.i, no a0/a1 clobber, and no
    relocations
  • git diff --check and strict per-commit checkpatch.pl: zero reported
    errors, warnings, or checks
  • focused source-order assertions for reserve-before-handoff, uncertain-CPU
    accounting, RISC-V stopped-state confirmation, and halted-settlement locking

Prior QEMU/OpenSBI runtime results were produced on the pre-rebase series
stacked with #26. The accepted-base head still needs runtime revalidation
before this draft is ready for merge.

Remaining draft gates

  • rebase the separate riscv: KEXEC_TYPE_MULTIKERNEL Image loader, placement in the grant, no purgatory #26 loader branch onto this head
  • rerun QEMU virt + OpenSBI PLIC lifecycle tests on the accepted base
  • rerun halt, power-off, reboot, panic, secondary-hart stop, same-address
    respawn, different-Image respawn, and 50-cycle stress tests
  • inject lost CPU-add ACK and partial-start failures and verify conservative
    ownership/cleanup behavior

AIA runtime validation remains blocked on #27's per-hart IMSIC/APLIC and device
filtering. MKTTY/ring validation remains blocked on #28; neither is duplicated
here.

@pro-utkarshM
multikernel: require OF
ced11cc 
The instance restore and ring handoff paths consume the live OF
tree. Without CONFIG_OF, x86 multikernel configurations compile
references to OF globals that cannot link and cannot restore a spawn at
runtime. Reject that unusable configuration in Kconfig.

Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
@pro-utkarshM
riscv: add multikernel SBI HSM helpers and spawn state
eed1126 
RISC-V parks CPUs in firmware with SBI HSM, so multikernel must reuse
the architecture's HART_START, HART_STOP and HART_STATUS wrappers. Make
those helpers available to RISC-V architecture code while retaining the
existing SBI-to-Linux errno mapping.

Add only the per-instance context and entry-stub bookkeeping required by
the lifecycle implementation. Hart IDs remain physical firmware
identifiers and are never used as array indexes.

Link: multikernel#23
Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
@pro-utkarshM
riscv: add multikernel fence.i entry stub
d0b12c9 
OpenSBI does not invalidate a stopped hart's instruction cache when
HART_START restarts it, and an SBI remote fence cannot target a stopped
hart. Reusing an Image address can therefore execute stale instructions.

Add an immutable, relocation-free entry stub whose first instruction is
fence.i and whose target comes from the adjacent data page. Publish its
address through the multikernel manifest and route spawn-kernel secondary
starts through it as well as the primary start. The stub preserves a0 and
a1, so the primary receives its DTB and secondaries receive their normal
SBI boot data.

Flush the local instruction cache before every HART_STOP so a later start
cannot retain an older stub line.

Link: multikernel#24
Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
@pro-utkarshM
riscv: implement multikernel SBI HSM lifecycle
eb89a3a 
Implement the RISC-V multikernel CPU lifecycle with direct SBI HSM calls.
A spawn is allowed only after HART_STATUS reaches STOPPED; STARTED,
START_PENDING and STOP_PENDING are polled with a bounded timeout.

Allocate the immutable entry stub and mutable target context from the
instance control block, publish the stub in the manifest, and start the
physical hart with the Image entry and DTB boot ABI. Use physical hart IDs
for host doorbells and confirm firmware stop state before release or memory
reclaim. RISC-V deliberately reports force-stop as unsupported because HSM
has no remote HART_STOP operation.

Link: multikernel#23
Link: multikernel#24
Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
@pro-utkarshM
riscv: harden multikernel HSM start handoff
a684fa3 
Prime every assigned hart through an immutable host-text fence.i
trampoline before it fetches a rewritten instance stub or Image.
Preserve distinct HSM state errors for bounded lifecycle handling.

Link: multikernel#23

Link: multikernel#24

Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
@pro-utkarshM
multikernel: serialize instance resource changes
923987f 
Reuse the kexec lock for CPU, memory and device transfers so an
Image rewrite cannot race with resource mutation. Recover an active
instance only after every assigned hart is confirmed stopped.

Link: multikernel#23

Link: multikernel#26

Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
@pro-utkarshM
riscv: contain multikernel spawn shutdown paths
a85f409 
A spawned kernel must never invoke SBI SRST or a legacy shutdown
because those operations can reset the host. Detect the spawn handoff
before reset registration and suppress host-wide reset handlers.

Route halt, poweroff, restart, SMP stop and panic shutdown through
local HART_STOP. Cache the parent endpoint for a non-allocating panic
notification, set a safe spawn panic default, and prevent memory reuse
until every assigned hart is confirmed stopped.

Link: multikernel#25

Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
@pro-utkarshM
multikernel: retain CPUs after ambiguous hot-add
e8499f5 
Reserve instance tracking before handing a pool CPU to a running
kernel. If the response is lost and firmware cannot confirm the hart
stopped, conservatively keep the CPU assigned to that instance instead
of exposing a possibly running hart through the free pool.

Link: multikernel#23
Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
@pro-utkarshM
multikernel: serialize halted instance settlement
eef3dcb 
Take the kexec lock while confirming parked CPUs and moving an
instance back to the loaded state. This keeps asynchronous halt
notifications from racing CPU ownership changes or image teardown.

Link: multikernel#23

Signed-off-by: Utkarsh Maurya <projects.utkarshMaurya@gmail.com>
@pro-utkarshM
pro-utkarshM force-pushed the riscv/multikernel-23-25-v1 branch from 7fd9938 to eef3dcb Compare September 7, 2026 18:08
@pro-utkarshM
pro-utkarshM changed the base branch from master to multikernel-riscv September 7, 2026 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@pro-utkarshM