feat(BA-8594): add the kernel group tables and the kernel's group foreign key by fregataa · Pull Request #15919 · lablup/backend.ai

fregataa · 2026-10-20T09:52:18Z

Summary

Add kernel_groups: one row per session role, holding the execution spec its kernels share (image_id, shmem, startup_command, bootstrap_script, preopen_ports, requested_starts_at, batch_timeout_sec), unique on (session_id, role).
Add kernel_group_resource_slots (the request, one row per slot, id primary key and unique on (kernel_group_id, slot_name)) and kernel_group_environ (one row per key, (kernel_group_id, key) primary key, key format CHECK matching system_session_environ).
Add a nullable kernels.kernel_group_id foreign key, KernelGroupID / KernelGroupResourceSlotID / KernelGroupRole in common/data/entity, and NaturalKeyColumn so role and slot_name read back as their NaturalKey kind.

Schema only. The enqueue write that creates the group rows and the backfill of existing sessions follow in separate PRs. shmem is a column on the group, not a slot: it is a carve-out of the requested mem that the agent only applies as ShmSize.

Test plan

All model modules import and the three tables register with the shared metadata
alembic upgrade head / downgrade on a local halfstack DB: tables and the kernels column are created and dropped, existing kernels keep kernel_group_id = NULL
alembic check reports no drift for the new tables against the Row declarations
CI pants check / pants test

Resolves BA-8594

🤖 Generated with Claude Code

https://claude.ai/code/session_01RsGNdkS3wbsaS2isxoVGfU

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RsGNdkS3wbsaS2isxoVGfU

seedspirit · 2026-10-21T04:18:14Z

+    # runtime default. Not a slot: the agent sets it as ShmSize and allocates nothing.
+    shmem: Mapped[int | None] = mapped_column("shmem", sa.BigInteger, nullable=True)
+    startup_command: Mapped[str | None] = mapped_column("startup_command", sa.Text, nullable=True)
+    bootstrap_script: Mapped[str | None] = mapped_column("bootstrap_script", sa.Text, nullable=True)


How about using BootstrapScriptID here?

Alternatively, how about managing bootstrap_script and similar items as separate options rather than as part of the kernel group row?

We decided to add bootstrap script revision and bootstrap script revision id will be added

HyeockJinKim · 2026-10-21T02:21:37Z

+        ),
+        index=True,
+        nullable=True,
+    )


Shouldn't this be deleted along with the rest, rather than using SET_NULL?

HyeockJinKim · 2026-10-21T04:45:15Z

+        sa.ForeignKeyConstraint(
+            ["kernel_group_id"],
+            ["kernel_groups.id"],
+            name=op.f("fk_kernel_group_resource_slots_kernel_group_id_kernel_groups"),
+            ondelete="CASCADE",


Shouldn't we restrict it with RESTRICT and clean it up? Since we're linking to external resources.

…eign key Add kernel_groups (one row per session role with the shared execution spec), kernel_group_resource_slots (the request, one row per slot) and kernel_group_environ (one row per key, key format checked), plus a nullable kernels.kernel_group_id. Schema only; the enqueue write and the backfill of existing sessions follow separately. Register the new model modules for the component and scenario test sandboxes so the kernels foreign key resolves there. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RsGNdkS3wbsaS2isxoVGfU

Drop the bootstrap_script column; a group will reference a bootstrap script revision instead once revisions exist (BA-8658). Name the role key kernel_group_role like the other natural keys, and chain the migration after system_session_environ, which landed on main. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RsGNdkS3wbsaS2isxoVGfU

Move the environment variable name regex to manager/data/environ so system_session_environ and kernel_group_environ declare the same CHECK from one constant. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RsGNdkS3wbsaS2isxoVGfU

…rows reference kernels.kernel_group_id and kernel_group_resource_slots.kernel_group_id block the group's deletion; only its environ rows go with it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RsGNdkS3wbsaS2isxoVGfU

…me name constraint Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0128bKAiecc6WfVK9VBRjxBv

…ated key pattern main gained `ai.backend.manager.data.environ.types.ENVIRON_KEY_PATTERN` (#15919), so the copy this branch added under `common/data/environ` goes and both users import main's. main itself carries two heads, 51d33226c0a5 and c9e1b7a4d2f0, both on 3b8e1d6c9a47; an empty merge revision joins them and this branch's revision moves on top of it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

fregataa requested a review from a team as a code owner October 20, 2026 09:52

Copilot AI balanced review requested due to automatic review settings October 20, 2026 09:52

Copilot AI reviewed Oct 20, 2026

github-actions Bot added the size:L 100~500 LoC label Oct 20, 2026

github-actions Bot assigned fregataa Oct 20, 2026

github-actions Bot added comp:manager Related to Manager component comp:common Related to Common component require:db-migration Automatically set when alembic migrations are added or updated labels Oct 20, 2026

fregataa added a commit that referenced this pull request Oct 20, 2026

changelog: add news fragment for PR #15919

e57b36c

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RsGNdkS3wbsaS2isxoVGfU

fregataa marked this pull request as draft October 20, 2026 10:19

fregataa force-pushed the feat/BA-8594-kernel-group-tables branch 2 times, most recently from beeec31 to d0bce67 Compare October 20, 2026 10:36

fregataa marked this pull request as ready for review October 20, 2026 11:03

fregataa force-pushed the feat/BA-8594-kernel-group-tables branch 2 times, most recently from 2a729b8 to 690ac5e Compare October 20, 2026 11:19

seedspirit reviewed Oct 21, 2026

View reviewed changes

jopemachine reviewed Oct 21, 2026

View reviewed changes

Comment thread src/ai/backend/common/data/entity/kernel_group.py Outdated

fregataa force-pushed the feat/BA-8594-kernel-group-tables branch from 690ac5e to d89b453 Compare October 21, 2026 05:44

fregataa requested a review from a team October 21, 2026 06:31

HyeockJinKim requested changes Oct 21, 2026

View reviewed changes

fregataa force-pushed the feat/BA-8594-kernel-group-tables branch 2 times, most recently from 50560b2 to 47bcd33 Compare October 21, 2026 09:36

fregataa requested review from a team and HyeockJinKim October 21, 2026 09:37

fregataa and others added 5 commits October 22, 2026 11:26

fix(BA-8594): rebase the kernel group migration onto the storage volu…

a759460

…me name constraint Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0128bKAiecc6WfVK9VBRjxBv

fregataa force-pushed the feat/BA-8594-kernel-group-tables branch from 47bcd33 to a759460 Compare October 22, 2026 02:28

HyeockJinKim approved these changes Oct 22, 2026

View reviewed changes

HyeockJinKim merged commit f9ecf92 into main Oct 22, 2026
53 checks passed

HyeockJinKim deleted the feat/BA-8594-kernel-group-tables branch October 22, 2026 06:06

seedspirit mentioned this pull request Oct 22, 2026

chore: merge diverged alembic heads #15995

Merged

Navigation Menu

Repository navigation

feat(BA-8594): add the kernel group tables and the kernel's group foreign key - #15919

feat(BA-8594): add the kernel group tables and the kernel's group foreign key#15919
HyeockJinKim merged 5 commits into
mainfrom
feat/BA-8594-kernel-group-tables

fregataa commented Oct 20, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

seedspirit Oct 21, 2026

Uh oh!

seedspirit Oct 21, 2026

Uh oh!

fregataa Oct 21, 2026

Uh oh!

Uh oh!

HyeockJinKim Oct 21, 2026

Uh oh!

HyeockJinKim Oct 21, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

fregataa commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

seedspirit Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

seedspirit Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

fregataa Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

HyeockJinKim Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

HyeockJinKim Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

fregataa commented Oct 20, 2026 •

edited

Loading