feat(BA-8440): receive ingress pushes on a worker internal API port by jopemachine · Pull Request #15656 · lablup/backend.ai

jopemachine · 2026-10-15T09:03:14Z

📚 Stacked PRs

Part of the AppProxy worker Ingress stack (epic BA-8433). Merge in order (bottom-up):

~~feat(BA-8437): write circuit events to the worker ingress state #15650 — feat(BA-8437): write circuit events to the worker ingress state~~ (merged)
👉 feat(BA-8440): receive ingress pushes on a worker internal API port #15656 — feat(BA-8440): receive ingress pushes on a worker internal API port ← you are here
feat(BA-8443): push ingresses from the coordinator to api-delivery workers #15833 — feat(BA-8443): push ingresses from the coordinator to worker internal API ports

Resolves #15536 (BA-8440)

Summary

Add proxy_worker.internal_api_bind_addr, unset by default. When set, the worker opens an internal API port apart from the traffic and API ports; unset opens nothing.
Move /v2/ingress off the user-facing API port (global_subapp_pkgs) to that internal port. Requests are authenticated by X-BackendAI-Token against the shared API secret, and writes go to the worker's ingress state built in ingress_state_ctx.
The port opens in a cleanup context before worker_registration_ctx, so a push right after registration is received.
upsert and replace return IngressRevisionsResponse with the revision held for each pushed ingress. The writer applies every ingress or raises the 409 revision conflict, so the held revision is the one sent. remove stays 204.
Not in this PR:
- How the coordinator learns the internal address (BA-8443).
- Legacy circuit events write revision 0, so a frontend that also receives pushes rejects them with a revision conflict. A worker should not receive both for the same frontend; settle it when the coordinator starts pushing (BA-8443).

Test plan

tests/unit/appproxy/worker/api/internal/test_ingress.py — upsert and replace report the applied revisions
In-process over HTTP with the local app-proxy-worker.toml and the internal port set: no token → 401; upsert → 200 {"10205": 3}; older revision → 409; replace → 200 {"10206": 5}; remove → 204; revisions reflect each step; the user-facing API port answers 404 for /v2/ingress/revisions and 200 for /status

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The ingress API moves off the user-facing API port to an internal port set by proxy_worker.internal_api_bind_addr, authenticated by the shared API secret, and writes to the worker's ingress state. The port opens before the worker registers, so the coordinator can push right after registration. Upsert and replace report the revision held for each pushed ingress. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…PI package The handler is served only on the worker internal API port, so keep it out of the directory that holds the subapps mounted on the user-reachable API port. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…tion middleware The internal API answers only the coordinator, so it does not need the HTML error page and the jinja2 setup it requires. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot

🟢 Approval recommended

No unresolved issues were identified in listener isolation, authentication, startup ordering, or revision responses.

No open issues 🎉

What changed in this PR

Adds an optional internal API listener for AppProxy worker ingress pushes, separate from user-facing ports and backed by the existing ingress state.

Changes:

Adds a configurable internal listener that starts before worker registration.
Moves authenticated ingress endpoints off the public API.
Returns applied revisions for upserts and replacements, with updated tests.

File	Description
tests/unit/appproxy/worker/api/internal/test_ingress.py	Updates imports and verifies revision responses.
tests/unit/appproxy/worker/api/internal/BUILD	Registers the internal API tests.
src/ai/backend/appproxy/worker/server.py	Adds internal listener lifecycle and separates routing.
src/ai/backend/appproxy/worker/config.py	Adds the optional internal bind address.
src/ai/backend/appproxy/worker/api/internal/ingress.py	Connects handlers to ingress state and returns revisions.
changes/15656.feature.md	Documents the new capability.

_{Review effort: Balanced}

fregataa · 2026-10-19T04:38:11Z

    app = web.Application(middlewares=[coordinator_auth_middleware])
-    app["prefix"] = "v2/ingress"
-    # The ingress state is not built at startup yet.
-    handler = IngressHandler(state=None)
+    app["_root.context"] = root_ctx
+    handler = IngressHandler(root_ctx.ingress_state)


Just a question. Does it change the API endpoint prefix?

No, only the port is changed to use the internal API. The path prefix remains the same.

fregataa · 2026-10-19T04:39:15Z

+        Field(default=None),
+        BackendAIConfigMeta(
+            description=(
+                "The address where the worker receives ingress pushes from the coordinator, "


The coordinator uses this API address? And it should be a single one?

Yes, that's correct. Each worker runs as a single process and is assigned its own API address.

fregataa · 2026-10-19T04:40:51Z

+                ingress.frontend.key(): ingress.revision for ingress in body.parsed.ingresses
+            }


Does the update apply to an ingress instance? then it would be better to make a response from an ingress instance

There's no problem with having apply and replace_all return a value. However, since they would always return the same value as the ingresses argument, I don't think it would add much value at this point.

… a push apply and replace_all return the ingresses held after writing, and the upsert and replace responses are built from them instead of echoing the request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The validating writer rejects an older revision with 409 before writing, so the ingresses held after a push are always the ones sent, and returning them from the writer changed no response. This reverts commit bd4fa0c. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

jopemachine · 2026-10-20T01:21:29Z

Once the web UI handles the existing APIs, we plan to remove the parts of our API that call the App Proxy worker directly. Running a separate internal API port would only add cost, so this is closed as not planned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions Bot added the size:M 30~100 LoC label Oct 15, 2026

github-actions Bot assigned jopemachine Oct 15, 2026

github-actions Bot added the comp:app-proxy Related to App Proxy component label Oct 15, 2026

jopemachine added a commit that referenced this pull request Oct 15, 2026

chore: add news fragment for PR #15656

b5c0f2b

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

jopemachine mentioned this pull request Oct 15, 2026

feat(BA-8437): write circuit events to the worker ingress state #15650

Merged

2 tasks done

Base automatically changed from feat/BA-8437-circuit-events-to-ingress-state to main October 19, 2026 01:43

jopemachine and others added 3 commits October 19, 2026 12:56

chore: add news fragment for PR #15656

8ce0b35

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

jopemachine force-pushed the feat/BA-8440-internal-ingress-push-port branch from 964510a to 6cba6b1 Compare October 19, 2026 03:57

refactor(BA-8440): serve internal API errors through the common excep…

3dd1a6c

…tion middleware The internal API answers only the coordinator, so it does not need the HTML error page and the jinja2 setup it requires. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

jopemachine marked this pull request as ready for review October 19, 2026 04:09

jopemachine requested a review from a team as a code owner October 19, 2026 04:09

Copilot AI balanced review requested due to automatic review settings October 19, 2026 04:09

Copilot started reviewing on behalf of jopemachine October 19, 2026 04:10

View session

Copilot AI reviewed Oct 19, 2026

View reviewed changes

jopemachine mentioned this pull request Oct 19, 2026

feat(BA-8443): push ingresses from the coordinator to api-delivery workers #15833

Merged

2 tasks done

fregataa reviewed Oct 19, 2026

View reviewed changes

github-actions Bot added size:L 100~500 LoC and removed size:M 30~100 LoC labels Oct 19, 2026

github-actions Bot added size:M 30~100 LoC and removed size:L 100~500 LoC labels Oct 19, 2026

jopemachine requested review from a team and fregataa October 19, 2026 06:54

jopemachine closed this Oct 20, 2026

jopemachine added a commit that referenced this pull request Oct 20, 2026

chore: drop the news fragment of the closed PR #15656

dce8d98

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Navigation Menu

Repository navigation

feat(BA-8440): receive ingress pushes on a worker internal API port - #15656

feat(BA-8440): receive ingress pushes on a worker internal API port#15656
jopemachine wants to merge 6 commits into
mainfrom
feat/BA-8440-internal-ingress-push-port

jopemachine commented Oct 15, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

fregataa Oct 19, 2026

Uh oh!

jopemachine Oct 19, 2026

Uh oh!

fregataa Oct 19, 2026

Uh oh!

jopemachine Oct 19, 2026

Uh oh!

fregataa Oct 19, 2026

Uh oh!

jopemachine Oct 19, 2026

Uh oh!

jopemachine commented Oct 20, 2026 •

edited

Loading

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		ingress.frontend.key(): ingress.revision for ingress in body.parsed.ingresses
		}

Conversation

jopemachine commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📚 Stacked PRs

Summary

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

fregataa Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

jopemachine Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

fregataa Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

jopemachine Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

fregataa Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

jopemachine Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

jopemachine commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jopemachine commented Oct 15, 2026 •

edited

Loading

jopemachine commented Oct 20, 2026 •

edited

Loading