Skip to content

mason: expose app instance count and sticky routing - #515

Merged
shivam5 merged 1 commit into
databricks:mainfrom
shivam5:mason-routing-autoscaling
Sep 4, 2026
Merged

mason: expose app instance count and sticky routing#515
shivam5 merged 1 commit into
databricks:mainfrom
shivam5:mason-routing-autoscaling

Conversation

@shivam5

@shivam5 shivam5 commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add optional --instances to mason deploy for a fixed instance count
  • map it to equal compute_min_instances and compute_max_instances values required by the Apps API
  • support both new deployments and updates to existing deployments
  • document sticky routing and __Host-databricks-app-router in mason deploy --help
  • keep the default README deployment flow unchanged and simple

Demo

export PROFILE=<profile>
export BRIDGE=/home/shivam.mittal/databricks-ai-bridge-worktrees/mason-routing-autoscaling

rm -rf ~/horizontal-scaling
uv tool install --force --with 'mlflow[databricks]>=3.10.1' \
  --editable "$BRIDGE/integrations/mason"
mason login -p "$PROFILE"

mason init ~/horizontal-scaling
cd ~/horizontal-scaling
mason dev

Stop the local server, then create one instance and update the same deployment to two:

mason deploy horizontal-scaling --instances 1
mason deploy horizontal-scaling --instances 2

For the sticky-routing API demo, use a workspace-scoped OAuth profile:

export APP_AUTH_PROFILE=<workspace-oauth-profile>
export APP_URL=$(databricks apps get mason-horizontal-scaling -p "$PROFILE" -o json | jq -r .url)
export TOKEN=$(databricks auth token "$APP_AUTH_PROFILE" | jq -r .access_token)
export ROUTING_KEY=$(uuidgen)

for i in 1 2 3 4 5; do
  curl -fsS "$APP_URL/api/demo/config" \
    -H "Authorization: Bearer $TOKEN" \
    -b "__Host-databricks-app-router=$ROUTING_KEY" |
    jq '{session_id, instance_id}'
done

The same routing key should return the same session and process instance on every request.

Testing

  • uv run pytest -q — 306 passed
  • uv run ruff check
  • uv run ruff format --check
  • uv run ty check
  • fresh generated LangGraph project — 26 passed, 1 skipped
  • local cookie test — repeated requests returned the same non-null session_id and instance_id
  • live e2-dogfood test on September 4, 2026:
    • deployed with --instances 1
    • updated the same deployment with --instances 2
    • verified both API bounds were 2, active_instances=2, compute_status=ACTIVE, and app_status=RUNNING
    • cleaned up the temporary app and workspace source
  • the existing e2-dogfood account-routed OAuth token returned HTTP 401 for the App API; use a workspace-scoped OAuth profile for the curl demo above

Comment thread integrations/mason/src/databricks_mason/deploy.py Outdated

@elainewang-db elainewang-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

were you able to test this live and deploy updates the apps scaling properly? my claude hit some issues with this, not sure if it was an apps side blocker or my own setup

Comment thread integrations/mason/src/databricks_mason/deploy.py Outdated
Comment thread integrations/mason/src/databricks_mason/deploy.py Outdated
Comment thread integrations/mason/src/databricks_mason/deploy.py Outdated
@shivam5
shivam5 force-pushed the mason-routing-autoscaling branch from d1b1bf2 to 38ede8e Compare September 4, 2026 03:09
@shivam5 shivam5 changed the title mason: expose app scaling and sticky routing mason: expose app instance count and sticky routing Sep 4, 2026
Comment thread integrations/mason/README.md Outdated
@shivam5
shivam5 force-pushed the mason-routing-autoscaling branch from 91ca5ce to b4ddf54 Compare September 4, 2026 19:24
@shivam5
shivam5 force-pushed the mason-routing-autoscaling branch from b4ddf54 to 1d06ad8 Compare September 4, 2026 21:55
@shivam5
shivam5 enabled auto-merge (squash) September 4, 2026 21:56
@shivam5
shivam5 merged commit 418ea36 into databricks:main Sep 4, 2026
50 checks passed
jamesbxwu added a commit that referenced this pull request Sep 4, 2026
Reconcile main's #540 agent.toml store-binding model with the UC-only tracing
redesign so both stores and tracing are configured via agent.toml (no flags):

- Stores: adopt main's model - `mason memory/sessions bind` records bindings in
  agent.toml; dev/deploy read them via store_bindings and validate they exist
  (validate_stores). Dropped --memory/--session/--no-create-stores flags and the
  store-env-into-app.yaml path (stores are read from agent.toml at runtime).
- Tracing: keep UC-only via `mason tracing setup` + provision_trace_experiment.
  Dropped main's --with-traces/--traces-experiment flags and the trace-env branch
  of the old validator; removed the stale `mason tracing instrument` help example.
- agent_project.py: keep both additions - main's memory_store_id and our
  trace_location/trace_warehouse.
- Kept main's #515 --instances deploy feature and #538 UI changes as-is.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants