feat(policy): add a resilient agent profile - #99
Conversation
📝 WalkthroughWalkthroughThe change adds an ChangesAgent policy
Bootstrap caller keys
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Caller
participant AgentProfile as profiles.agent
participant Policy as agent_policy_ir
participant Providers
participant Retry as retry_policies.agent
Caller->>AgentProfile: submit agent request
AgentProfile->>Policy: filter and rank candidates
Policy->>Providers: attempt selected candidate
Providers-->>AgentProfile: result or timeout
AgentProfile->>Retry: classify recoverable failure
Retry->>Providers: advance to next candidate
Providers-->>AgentProfile: fallback result
AgentProfile-->>Caller: return response
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)
policy_templates.py (2)
398-408: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider one shared fail-plan builder.
_agent_fail_planrepeats the fold that_balanced_fail_planalready performs, only over a different action map. Extract one helper that takes the action map and returns the plan. This keeps the plan shape identical for both templates and for the Luafail_planhelper inconfig.live.lua.♻️ Proposed refactor
-def _agent_fail_plan() -> list: - plan: list = ["always", dict(_AGENT_FAILURE_ACTIONS["unknown"])] - for reason in sorted(_AGENT_FAILURE_ACTIONS): - if reason != "unknown": - plan = [ - "override", - plan, - reason, - dict(_AGENT_FAILURE_ACTIONS[reason]), - ] - return plan +def _fail_plan(actions: dict[str, dict[str, Any]]) -> list: + plan: list = ["always", dict(actions["unknown"])] + for reason in sorted(actions): + if reason != "unknown": + plan = ["override", plan, reason, dict(actions[reason])] + return planThen call
_fail_plan(_AGENT_FAILURE_ACTIONS)and_fail_plan(_BALANCED_FAILURE_ACTIONS).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@policy_templates.py` around lines 398 - 408, Extract the shared fold logic from _agent_fail_plan and _balanced_fail_plan into a helper such as _fail_plan that accepts an action map and produces the same nested plan shape. Update both builders to delegate to this helper with their respective action maps, preserving the ordering and handling of the "unknown" entry used by the Lua fail_plan equivalent.
44-76: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThe agent rails are defined twice and can drift. Both files hold the same provider order, thresholds, scoring weights, timeouts, and the five trusted AntSeed peer identities. Only the fingerprint parity test in
tests/test_policy_templates.pycouples them, so any edit to one side breaks that test instead of updating the other side automatically.
policy_templates.py#L44-L76: add a comment that namesconfig.live.luaas the mirror of these constants, and state that both sides must change together.config.live.lua#L49-L76: add the same cross-reference topolicy_templates.py, or load the peer identities from a single shared data file that both the Lua config and the Python compiler read.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@policy_templates.py` around lines 44 - 76, Document the synchronization contract for the duplicated agent rails: in policy_templates.py around AGENT_PROVIDER_ORDER and the related constants, add a comment naming config.live.lua as the mirror and requiring both sides to change together; add the reciprocal cross-reference comment around the corresponding constants in config.live.lua (49-76). Do not alter the provider values, thresholds, weights, timeouts, or trusted peer identities.config.live.lua (1)
726-730: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRemove the profile-level
selectorfield. Whenpolicy_iris present, the router compiles that term directly and ignoresprofile.selector.agent_policy_ir()already embeds{"top_k", AGENT_TOP_K, ...}withAGENT_TOP_K = 8.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@config.live.lua` around lines 726 - 730, Remove the profile-level selector assignment from the agent configuration. In the agent table near policy_ir and retry_policy, delete selector = "top_k" and retain agent_policy_ir() as the source of the top_k behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@README.md`:
- Around line 146-152: Update the README `agent` template bullet to state its
actual $15 input and $30 output per-million-token price ceilings, matching
`AGENT_MAX_PRICE_IN` and `AGENT_MAX_PRICE_OUT` in `policy_templates.py`.
In `@tests/test_auth_proxy_dashboard_full.py`:
- Around line 1306-1314: Add an inline Ruff S105 suppression to the test fixture
assignment for token in test_gitops_bootstrap_keys_are_loaded_hash_only, keeping
the existing fixture value and assertions unchanged.
---
Nitpick comments:
In `@config.live.lua`:
- Around line 726-730: Remove the profile-level selector assignment from the
agent configuration. In the agent table near policy_ir and retry_policy, delete
selector = "top_k" and retain agent_policy_ir() as the source of the top_k
behavior.
In `@policy_templates.py`:
- Around line 398-408: Extract the shared fold logic from _agent_fail_plan and
_balanced_fail_plan into a helper such as _fail_plan that accepts an action map
and produces the same nested plan shape. Update both builders to delegate to
this helper with their respective action maps, preserving the ordering and
handling of the "unknown" entry used by the Lua fail_plan equivalent.
- Around line 44-76: Document the synchronization contract for the duplicated
agent rails: in policy_templates.py around AGENT_PROVIDER_ORDER and the related
constants, add a comment naming config.live.lua as the mirror and requiring both
sides to change together; add the reciprocal cross-reference comment around the
corresponding constants in config.live.lua (49-76). Do not alter the provider
values, thresholds, weights, timeouts, or trusted peer identities.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 37f59d03-8b83-4af5-a1fc-f7fc1835f79c
📒 Files selected for processing (8)
README.mdSKILL.mdauth_proxy.pyconfig.live.luadocs/DEPLOY.mdpolicy_templates.pytests/test_auth_proxy_dashboard_full.pytests/test_policy_templates.py
| - `agent` — the reusable `profile:agent` policy for autonomous tool users: | ||
| require tools, 128k context, top-ten measured intelligence and reliable, | ||
| priced routes; prefer healthy Codex/direct providers before gateways and | ||
| trusted AntSeed peers; cap the cascade at eight candidates. The policy also | ||
| sets a 10s first-token and 22s per-attempt timeout and moves immediately to a | ||
| different candidate on provider failures, so a stalled first route cannot | ||
| consume the complete request deadline. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
State the agent price ceilings in this bullet.
The paragraph below this list says all templates default to $5 input / $25 output per-million-token ceilings. The agent template uses $15 input and $30 output (AGENT_MAX_PRICE_IN, AGENT_MAX_PRICE_OUT in policy_templates.py). Add the actual rails here so the document stays accurate.
📝 Proposed fix
- `agent` — the reusable `profile:agent` policy for autonomous tool users:
require tools, 128k context, top-ten measured intelligence and reliable,
- priced routes; prefer healthy Codex/direct providers before gateways and
+ priced routes under $15 input / $30 output per million tokens; prefer
+ healthy Codex/direct providers before gateways and
trusted AntSeed peers; cap the cascade at eight candidates. The policy also📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - `agent` — the reusable `profile:agent` policy for autonomous tool users: | |
| require tools, 128k context, top-ten measured intelligence and reliable, | |
| priced routes; prefer healthy Codex/direct providers before gateways and | |
| trusted AntSeed peers; cap the cascade at eight candidates. The policy also | |
| sets a 10s first-token and 22s per-attempt timeout and moves immediately to a | |
| different candidate on provider failures, so a stalled first route cannot | |
| consume the complete request deadline. | |
| - `agent` — the reusable `profile:agent` policy for autonomous tool users: | |
| require tools, 128k context, top-ten measured intelligence and reliable, | |
| priced routes under $15 input / $30 output per million tokens; prefer | |
| healthy Codex/direct providers before gateways and | |
| trusted AntSeed peers; cap the cascade at eight candidates. The policy also | |
| sets a 10s first-token and 22s per-attempt timeout and moves immediately to a | |
| different candidate on provider failures, so a stalled first route cannot | |
| consume the complete request deadline. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@README.md` around lines 146 - 152, Update the README `agent` template bullet
to state its actual $15 input and $30 output per-million-token price ceilings,
matching `AGENT_MAX_PRICE_IN` and `AGENT_MAX_PRICE_OUT` in
`policy_templates.py`.
| def test_gitops_bootstrap_keys_are_loaded_hash_only(): | ||
| token = "llmr_reconciled" | ||
| loaded = auth_proxy._bootstrap_caller_key_hashes( | ||
| f'{{"{token}":"micromarkets-dev"}}') | ||
|
|
||
| assert loaded == { | ||
| hashlib.sha256(token.encode()).hexdigest(): "micromarkets-dev", | ||
| } | ||
| assert token not in loaded |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Silence the Ruff S105 finding on the test token.
Ruff flags token = "llmr_reconciled" as a possible hardcoded password (S105). The value is a test fixture, so add an inline suppression to keep the lint run clean.
🧹 Proposed fix
- token = "llmr_reconciled"
+ token = "llmr_reconciled" # noqa: S105 - test fixture, not a real secret📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def test_gitops_bootstrap_keys_are_loaded_hash_only(): | |
| token = "llmr_reconciled" | |
| loaded = auth_proxy._bootstrap_caller_key_hashes( | |
| f'{{"{token}":"micromarkets-dev"}}') | |
| assert loaded == { | |
| hashlib.sha256(token.encode()).hexdigest(): "micromarkets-dev", | |
| } | |
| assert token not in loaded | |
| def test_gitops_bootstrap_keys_are_loaded_hash_only(): | |
| token = "llmr_reconciled" # noqa: S105 - test fixture, not a real secret | |
| loaded = auth_proxy._bootstrap_caller_key_hashes( | |
| f'{{"{token}":"micromarkets-dev"}}') | |
| assert loaded == { | |
| hashlib.sha256(token.encode()).hexdigest(): "micromarkets-dev", | |
| } | |
| assert token not in loaded |
🧰 Tools
🪛 Ruff (0.16.1)
[error] 1307-1307: Possible hardcoded password assigned to: "token"
(S105)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/test_auth_proxy_dashboard_full.py` around lines 1306 - 1314, Add an
inline Ruff S105 suppression to the test fixture assignment for token in
test_gitops_bootstrap_keys_are_loaded_hash_only, keeping the existing fixture
value and assertions unchanged.
Source: Linters/SAST tools
1 participant
Summary
agenttemplate andprofile:agentfor autonomous tool clientsWhy
Micromarkets was carrying a frozen raw policy. Recent calls spent the complete 50s outer deadline on the first provider and returned 504, so the declared fallback cascade never ran. The shared profile keeps policy fixes centralized and preserves enough deadline for real failover.
Verification
python -m pytest tests -q: 737 passed, 2 skippedlua tests/run_lua.lua: 637 passedprofile:agentnormalize to the same policyDeployment dependency
Merge this first and wait for the Unhardcoded image bump and router rollout before merging the Micromarkets code PR or the devexp wiring PR.
Summary by CodeRabbit
New Features
Documentation