Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,23 @@
What changed in PACT, newest first. Each entry says what an author can now write, or what now
holds that did not; the commits are listed under it for whoever wants the detail.

## 2026-11 — a hidden value belongs to the request, not the run (15 November)

`gets-hidden-values: yes` put a card back only for an action of the agent whose own redaction hid
it. A lead that hands the charge to a cashier hid the card one way, `[removed]`, because the lead
uses no such action itself, so the cashier's action could never get the number back; and a
placeholder the run's vault did not hold was sent to the tool as text.

### What holds now

- **The vault is the request's.** `AgentSpec.team_gets_hidden` says a member of the agent's team, at
any depth, uses such an action; such a lead keeps hidden values as placeholders, and the vault
rides the grant to every member (`run(vault=...)`, as `board=` does), so the member's action gets
back the card its lead hid.
- **A placeholder the request does not hold is never sent.** The call is not made; the model reads
`not done: ⟦card-number-7⟧ stands for no value this request holds — use only the placeholders this
conversation showed you, exactly as written` (`Vault.unknown`).

## 2026-11 — an agent's answer is held to what the agent says it gives (15 November)

`answers-with:` is a promise to whoever calls the agent. The reference harness put the shape to
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1000,11 +1000,11 @@ framework is an extra.
|---|---|
| **Design** | Thesis, 29 binding decisions, FRD (120 requirements), 60-row refusal ledger — complete |
| **Research** | 14 source-grounded studies, ~15,750 lines, over 140 repos (~15 GB) + 57 papers |
| **Code** | Loader, diagnostics, schema engine, CLI, harness, resolver, evals, SLO — **5195 tests (1696 Rust + 3499 adapter), clippy clean, TypeScript type-checked** |
| **Code** | Loader, diagnostics, schema engine, CLI, harness, resolver, evals, SLO — **5197 tests (1696 Rust + 3501 adapter), clippy clean, TypeScript type-checked** |
| **Adapters** | **All 7 named targets**, proven against one shared conformance suite |

```bash
./scripts/test-all.sh # 5195 tests, Rust + 7 adapters, fully offline
./scripts/test-all.sh # 5197 tests, Rust + 7 adapters, fully offline
```

**What is still owed** is not hidden — it lives in
Expand Down
38 changes: 32 additions & 6 deletions adapters/python/src/pact_adapters/harness.py
Original file line number Diff line number Diff line change
Expand Up @@ -604,6 +604,10 @@ async def _ask(
# authored (`shared-with:`, `admitted-when:`, `admitted-by:`); the entries
# written during a request, and running the admitting agent, are the host's.
"board", "board_facts", "admit",
# The request's hidden values (docs/98 WP-11). WHICH actions get them back is
# authored (`gets-hidden-values:`); the values a request hid are its own, and
# a lead hands its vault to its members as it does the board.
"vault",
# The circuit breaker's record (docs/98 WP-13). The RULE is authored
# (`rests-after:`, `rests-for:`); which tools failed lately is the host's.
"rests",
Expand Down Expand Up @@ -792,6 +796,11 @@ async def run(
#: Facts the LEAD shares that this member may read and write, handed down
#: with the board. A member does not declare them; the lead does.
board_facts: "tuple[Fact, ...]" = (),
#: The request's hidden values (docs/98 WP-11). The root run makes one when
#: an action it, or a member of its team, uses gets hidden values; every
#: member it starts gets the same object, riding the grant as the board
#: does, so a member's action restores the placeholder its lead minted.
vault: "Vault | None" = None,
#: Who judges an entry before it is written, when a fact says `admitted-by:`:
#: `admit(judge, fact, entry) -> (yes, because)`. The host's, because the
#: judge is a run and a run needs a transport. `None` refuses every such
Expand Down Expand Up @@ -1017,6 +1026,8 @@ async def sharing_the_meter(grant: Grant) -> str:
# its teammates do (docs/98 WP-8).
grant.board = board
grant.board_facts = shared
# The request's vault rides the same way (docs/98 WP-11).
grant.vault = chain.vault
# HOW this member was reached, which is what decides whether it owes
# a figure of its own (P4). It rides the grant for the reason the
# meter does: `Asker` takes one argument, and the check belongs
Expand Down Expand Up @@ -1046,11 +1057,14 @@ async def sharing_the_meter(grant: Grant) -> str:
# documents in the worked example loaded, validated and did nothing, and
# the card numbers they say they stop reached the model.
chain = spec.chain if chain is None else chain
# A vault of this run's own, when an action it uses gets hidden values back
# (docs/98 WP-11). A copy of the chain, so one run's placeholders are never
# The request's vault, when an action it or a member of its team uses gets
# hidden values back (docs/98 WP-11): the one a lead handed down, else a
# new one. A copy of the chain, so one request's placeholders are never
# another's: the spec's chain is shared by every run of the agent.
if spec.gets_hidden:
chain = dataclasses.replace(chain, vault=Vault())
if vault is None and (spec.gets_hidden or spec.team_gets_hidden):
vault = Vault()
if vault is not None:
chain = dataclasses.replace(chain, vault=vault)
# AND THE RUNNER THE CHAIN COULD NOT HAVE BEEN BUILT WITH. `AgentSpec.
# from_document` is handed a loaded document and nothing else (P-1), so it
# has no host to ask for one; `run()` has one and never passed it on. Through
Expand Down Expand Up @@ -3238,12 +3252,23 @@ def at_ceiling() -> "Reached | None":
# The one place a hidden value comes back (docs/98 WP-11): in the
# arguments of an action that says it gets them, after every rule
# ran, so the trace and any approval saw only the placeholder.
# A placeholder the request's vault does not hold (another request's,
# or one the model made up) never reaches the tool as text: the call
# is not made, and the model is told which placeholder it used.
unheld: list[str] = []
if chain.vault is not None and (call.name, str(sent.args.get("action") or "")) in spec.gets_hidden:
sent = ToolCall(sent.name, chain.vault.restored(sent.args))
unheld = chain.vault.unknown(sent.args)
if not unheld:
sent = ToolCall(sent.name, chain.vault.restored(sent.args))
# A tool resting after failing too often is not called (docs/98
# WP-13); the model is told until when, and can do something else.
back = rested.resting(call.name, clock()) if call.name in spec.rests else None
if back is not None:
if unheld:
out = (
f"not done: {', '.join(unheld)} {'stands' if len(unheld) == 1 else 'stand'} for no value "
f"this request holds — use only the placeholders this conversation showed you, exactly as written"
)
elif back is not None:
out = (
f"not done: {call.name!r} failed too often in a row and is resting "
f"for another {round(back - clock())}s — do without it for now"
Expand Down Expand Up @@ -4466,6 +4491,7 @@ async def ask(grant: Grant) -> str:
claims_workspace=claims_workspace,
board=getattr(grant, "board", None),
board_facts=tuple(getattr(grant, "board_facts", ()) or ()),
vault=getattr(grant, "vault", None),
admit=admit,
reacting=frozenset(getattr(grant, "reacting", ()) or ()),
)
Expand Down
37 changes: 37 additions & 0 deletions adapters/python/src/pact_adapters/ir.py
Original file line number Diff line number Diff line change
Expand Up @@ -799,6 +799,11 @@ class AgentSpec:
#: `gets-hidden-values: yes` (docs/98 WP-11): the only calls a hidden
#: value's placeholder is turned back into the value for.
gets_hidden: frozenset[tuple[str, str]] = frozenset()
#: Whether an action a member of this agent's team uses, at any depth, says
#: `gets-hidden-values: yes`. Such a lead keeps hidden values as
#: placeholders too, because the member's action can only get back a value
#: the lead's redaction kept: the vault belongs to the request, not the run.
team_gets_hidden: bool = False
#: The workspace's `detectors:`, each as one pattern (docs/98 WP-12), for a
#: `looks-like:` read inside this agent's run (a board's admission lines).
detectors: Mapping[str, str] = field(default_factory=dict)
Expand Down Expand Up @@ -1077,6 +1082,7 @@ def from_document(
for act, line in ((body.get("actions") or {}).items())
if isinstance(line, Mapping) and said_yes(line.get("gets-hidden-values"))
),
team_gets_hidden=_team_gets_hidden(doc, agent_key),
facts=Facts.from_document(doc, agent_key),
workspace_remembers=Facts.of(doc.get("remembers")),
template_language=_text(doc.get("template-language", "")) or "holes",
Expand Down Expand Up @@ -1612,6 +1618,37 @@ def _figure_as_text(v: Any) -> Any:
return str(v) if isinstance(v, int | float) and not isinstance(v, bool) else v


def _team_gets_hidden(doc: Mapping[str, Any], agent_key: str) -> bool:
"""Whether a member of `agent_key`'s team, at any depth, uses an action that
says `gets-hidden-values: yes`."""
agents = doc.get("agents") or {}
tools = doc.get("tools") or {}

def gets(key: str) -> bool:
body = agents.get(key) if isinstance(agents, Mapping) else None
uses = set(_as_list(body.get("uses"))) if isinstance(body, Mapping) else set()
return any(
isinstance(line, Mapping) and said_yes(line.get("gets-hidden-values"))
for t, tool in (tools.items() if isinstance(tools, Mapping) else ())
if t in uses and isinstance(tool, Mapping)
for line in (tool.get("actions") or {}).values()
)

seen: set[str] = set()
body = agents.get(agent_key) if isinstance(agents, Mapping) else None
pending = list((body.get("team") or {}) if isinstance(body, Mapping) else ())
while pending:
key = str(pending.pop())
if key in seen:
continue
seen.add(key)
if gets(key):
return True
member = agents.get(key) if isinstance(agents, Mapping) else None
pending.extend((member.get("team") or {}) if isinstance(member, Mapping) else ())
return False


def _as_list(v: Any) -> list[str]:
if v is None:
return []
Expand Down
18 changes: 18 additions & 0 deletions adapters/python/src/pact_adapters/placeholders.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,24 @@ def token(self, kind: str, original: str) -> str:
self._by_value[original] = placeholder
return placeholder

def unknown(self, value: Any) -> list[str]:
"""The placeholders in `value` this vault does not hold, sorted: another
request's, or one a model made up."""
found: set[str] = set()

def walk(item: Any) -> None:
if isinstance(item, str):
found.update(m for m in _TOKEN.findall(item) if m not in self.held)
elif isinstance(item, dict):
for v in item.values():
walk(v)
elif isinstance(item, (list, tuple)):
for v in item:
walk(v)

walk(value)
return sorted(found)

def restored(self, value: Any) -> Any:
"""`value` with every placeholder this vault holds put back."""
if isinstance(value, str):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,3 +133,72 @@ def test_a_workspace_detectors_placeholder_is_named_for_it() -> None:
finally:
DECLARED.reset(held)
assert _thing_of("(?:ORD-[0-9]{8})") == "", "with no detectors in scope, nothing to name it by"


def test_a_placeholder_the_request_does_not_hold_is_never_sent() -> None:
"""A guessed placeholder, or another request's, is not passed to the tool as text."""
got: list[dict] = []
model = Echo("payments", "charge")
spec = AgentSpec.from_document(_doc(True), "a")
out = asyncio.run(
run(spec, model, "please charge my card ⟦card-number-7⟧", {"payments": lambda a: (got.append(dict(a)), "ok")[1]})
)
assert got == [], got
said = out.steps[0].tool_results[0]
assert "⟦card-number-7⟧ stands for no value this request holds" in said, said


TEAM = {
"redaction": {"hide": ["anything that looks like a card number"]},
"agents": {
"desk": {
"description": "Takes the request and hands the charge to the cashier.",
"instructions": "Ask the cashier to charge the card.",
"team": {"cashier": "charges a card"},
},
"cashier": {"description": "Charges a card.", "instructions": "Charge it.", "uses": ["payments"]},
},
"tools": _doc(True)["tools"],
}


def test_a_member_restores_the_placeholder_its_lead_minted() -> None:
"""The vault is the request's (docs/98 WP-11): a lead whose team gets hidden
values keeps them as placeholders, and its member's action gets the card."""
from pact_adapters.harness import delegate_by_running
from pact_adapters.script import Script, Turn
from pact_adapters.transports.mock import ReferenceTransport

got: list[dict] = []
payments = {"payments": lambda a: (got.append(dict(a)), "charged")[1]}
seen: list[str] = []

class Lead:
async def model_call(self, system, history, tools): # type: ignore[no-untyped-def]
said = " ".join(str(m.get("content")) for m in history)
seen.append(said)
if any(m.get("role") == "tool" for m in history):
return "Charged.", []
card = str(history[0].get("content")).split("card ")[-1]
return "", [ToolCall("cashier", {"question": f"charge {card}"})]

def __getattr__(self, name: str) -> Any:
if name == "unenforced":
return lambda: ()
raise AttributeError(name)

def cashier_for(member: AgentSpec) -> ReferenceTransport:
return ReferenceTransport(Script([
Turn("", (ToolCall("payments", {"action": "charge", "card": "⟦card-number-1⟧"}),)),
Turn("done"),
]))

spec = AgentSpec.from_document(TEAM, "desk")
assert spec.team_gets_hidden and not spec.gets_hidden
out = asyncio.run(run(
spec, Lead(), f"please charge my card {CARD}", payments,
ask_member=delegate_by_running(TEAM, cashier_for, tool_impls=payments),
))
assert out.halted == "final", out.halted
assert got and got[0]["card"] == CARD, got
assert CARD not in " ".join(seen) and "⟦card-number-1⟧" in seen[0], seen
Original file line number Diff line number Diff line change
Expand Up @@ -318,6 +318,11 @@ def test_every_spec_field_that_holds_authored_state_is_filled_from_the_document(
"unbounded", # an owner's, and this example keeps its step limit
"detectors", # authored, and this example declares none
"rests", # authored, and this example rests no tool
# Authored (docs/98 WP-11), and no member of this example's team uses an
# action that gets hidden values. What proves the line reaches the spec
# and keeps a lead's placeholders for its member is
# `test_a_hidden_value_comes_back_only_where_it_is_needed.py`.
"team_gets_hidden",
# Authored (D29), and this example asks its runtime to load nothing: it
# demonstrates the no-code ceiling, and a plugin is code. What proves
# the line reaches the spec, refuses a run or is named on `unenforced`
Expand Down
7 changes: 6 additions & 1 deletion spec/schema.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3953,7 +3953,12 @@ groups:
an agent uses such an action, a hidden value becomes a placeholder
(`⟦card-number-1⟧`) instead of `[removed]`; the model, the logs and
anybody approving see only the placeholder, and the real value is put
back in this action's arguments and nowhere else.
back in this action's arguments and nowhere else. The placeholders
belong to the request: a lead whose team member uses such an action
keeps hidden values as placeholders too, and the member's action gets
back what the lead hid. A placeholder the request does not hold —
another request's, or a guessed one — stops the call instead of being
sent as text.
claim-lasts:
# docs/98 WP-7. The lease `same-request-key-across:` was waiting for
# (B13, docs/50-NOT-COPIED.md §6): a claim that expires frees the key,
Expand Down
Loading