[yuechao-qin]

Problem

The cache key describes the work but not the authority it runs with, so a non-owner's identical submission matches the owner's entry and is served from it before the ownership check is reached. "Has anyone run this?" is the right question for ordinary work — but once a secret is involved, it has to become "has this user run this?".

The key is built from the component spec, the input hashes, and the arguments — and a secret argument appears there as the reference the pipeline wrote, never as the value it resolves to:

{"tok": {"secret": {"name": "tok"}}}

Secret's primary key is (user_id, secret_name), so that reference means a different value for every user, while hashing to the same key for all of them. The lookup then happens before the run's owner is loaded and well before ownership is checked:

key = hash(container spec, input hashes, {"secret": {"name": "tok"}})
hit? ──yes──▶ adopt the cached container execution, return     ← nothing below ever runs
   │no
owner  = pipeline_run.created_by
secret = session.get(Secret, (owner, "tok")) or raise "User ... does not have secret ..."

Fix

Add the run's owner to the hashed struct, but only when an argument references a secret:

# orchestrator_sql.py — before the key is calculated
secret_owner: str | None = None
if any(
    isinstance(dynamic_data_argument, dict)
    and DYNAMIC_DATA_SECRET_KEY in dynamic_data_argument
    for dynamic_data_argument in dynamic_data_arguments.values()
):
    secret_owner = session.scalar(...)   # this run's pipeline_run.created_by

# _calculate_container_execution_cache_key
if secret_owner is not None:
    cache_key_struct["secret_owner"] = secret_owner

The key becomes per owner for secret-consuming tasks, and byte-identical to today for everything else. It carries the owner's ID and never the secret's value: the key is stored on every execution node, indexed, and exported as a trace attribute.

Before / after

User A owns a secret named tok and has already run the task. Computed keys, for a task whose only argument is that secret:

	Before	After
key for A	md5=5aa99115…	md5=7468f0a0…
key for B	md5=5aa99115… — the same	md5=9cfd7ea9…
key for a task with no secret	md5=8c67cb8d…	md5=8c67cb8d… — unchanged

What each user gets when they submit that task:

Who	Before	After
A, the owner, running it again	cache hit, reuses their own execution	unchanged — cache hit on their own key
B, who owns no tok	❌ cache hit on A's entry: A's execution is attached to B's run and read as B's own; the ownership check is never reached	✅ miss, so the run reaches the check and ends in SYSTEM_ERROR — "User B does not have secret tok" — with no container launched
B, who owns their own tok with a different value	❌ cache hit on A's entry: B's results are computed with A's value	✅ miss, so B launches their own container execution with B's value
anyone, task with no secret	cache hit, shared across users	unchanged — still shared

Behaviour change

• One cache entry per owner for secret-consuming tasks. Tasks without secrets are unaffected.
• One-off at deploy: entries written earlier carry the old owner-free key, so the first secret-consuming run per owner misses once and relaunches. Normal from the second run on.
• Entries written earlier keep their owner-free keys. Those keys live only on nodes that launched — a node that reused stores none — so clearing the keys of secret-referencing nodes is a complete one-off pass.

Alternatives considered

	Why not
check ownership before the cache lookup	fixes the non-owner case only; two legitimate owners of the same name still share one execution, with the wrong value
hash the resolved secret value into the key	puts a value-derived digest into an indexed column and into traces, and still keys a user to a value rather than to their own grant
disable caching for secret-consuming tasks	loses the feature for exactly the pipelines that most need it

Tests

TestCacheKeySeparatesSecretOwners in tests/test_orchestrator_sql.py, 4 tests:

Test	Asserts
a non-owner is refused even though an identical execution exists	the refused run launches nothing, lands in SYSTEM_ERROR with "does not have secret", and does not point at the owner's container execution
a non-owner is refused when there is nothing to reuse	the control — unchanged behaviour with an empty cache
two owners of the same name each get their own execution	each run launches with its own owner's value, and the two keys differ
a task without a secret still shares one execution across users	the feature is intact: one launch, the second node reuses the first

Suite: 578 passed. Removing secret_owner from the key turns the first and third red and leaves the other two green.