[devin-ai-integration]

Summary

Eight independent refactoring tasks to reduce duplication, clean up deprecated code, and improve modularity:

1. Unified retry logic (chat_api.py): Replaced inline retry loops in ChatModel.__call__() and AnthropicChatModel.__call__() with shared call_openai_api_with_retries() / call_anthropic_api_with_retries() from llm_utils.py.
2. Consolidated pricing registry (tracking.py): New get_pricing(model_name, provider) dispatcher. Provider-specific functions renamed to _get_pricing_* (private), with public aliases kept for backward compatibility.
3. Removed deprecated ChatModelArgs (chat_api.py): Class had no external consumers — removed entirely.
4. Split dynamic_prompting.py into prompting/ subpackage (flags.py, base.py, elements.py, __init__.py). Original file is now a thin re-export shim.
5. Cleaned up AssistantBench hack (loop.py): Added a generic pre_env_setup_hooks callback list on EnvArgs. The existing inline assistantbench.test check is preserved alongside the new hooks mechanism (with a TODO to migrate it to a hook in the future).
6. Deprecation warning for UI_COPILOT_RESULTS_DIR (exp_utils.py): Emits DeprecationWarning when the legacy env var is used.
7. Extracted shared test fixtures to tests/helpers/mock_llms.py: Moved CheatMiniWoBLLM_ParseRetry, CheatLLMArgs_LLMError, etc. out of test_agent.py.
8. Fixed Block.make() cloning (tool_use_agent.py): Replaced asdict(self) with copy.deepcopy(self) to handle non-serializable fields.

Bug fixes from post-review

After the initial implementation, four bugs were identified and fixed:

• OpenRouter usage=None crash: is_openai_response_valid() now returns False when response.usage is None, triggering a retry instead of crashing with AttributeError on completion.usage.prompt_tokens.
• ChatModel.retries always 0: generic_call_api_with_retries() now accepts an optional stats dict and populates it with the attempt count. ChatModel.__call__() passes this dict and updates self.retries from it, so get_stats() reports accurate retry counts.
• AnthropicChatModel tracker guard: Added isinstance(tracking.TRACKER.instance, tracking.LLMTracker) check, consistent with ChatModel, to prevent TypeError when TRACKER.instance is None.
• min_retry_wait_time passthrough: call_openai_api_with_retries() now accepts initial_retry_delay_seconds and ChatModel.__call__() forwards self.min_retry_wait_time through it, so the parameter is no longer silently ignored.

Review & Testing Checklist for Human

• Task 1 — Retry behavior parity: The four bugs above were patched, but verify the full set of error types handled by the old handle_error() in ChatModel.__call__ are still covered by the shared retry helpers (especially edge cases around openai.APIError subtypes and Anthropic-specific errors).
• Task 8 — deepcopy vs asdict semantics: asdict reconstructed a fresh instance from serialized field values; deepcopy clones the full object state. Verify that Block subclasses don't carry mutable state that _init() expects to be clean/absent.
• Task 4 — Re-export completeness: Verify all symbols previously importable from agentlab.agents.dynamic_prompting are still available (check __init__.py against the original file's public API).
• Task 5 — AssistantBench hook migration: The inline hack is preserved for now. Confirm the pre_env_setup_hooks field is serialization-safe (dataclass_json exclusion via metadata) and that the TODO to migrate the inline check to a hook is acceptable.

Recommended test plan: Run pytest -m 'not pricy' tests/ and verify all tests pass, especially tests/agents/test_agent.py. Local testing showed 60 passed, 8 pre-existing failures (missing MINIWOB_URL), 15 skipped, 20 deselected — all consistent with the main branch baseline.

Notes

• Backward-compatible public aliases are preserved for get_pricing_openai(), get_pricing_anthropic(), etc.
• dynamic_prompting.py remains importable with all original symbols re-exported.
• pre_env_setup_hooks on EnvArgs is excluded from DataClassJsonMixin serialization via metadata.
• Fixed darglint CI failures by adding proper Returns: / Args: docstring sections to the new and modified pricing functions in tracking.py.
• _get_pricing_openai() and _get_pricing_anthropic() return {} when langchain_community is not installed; this is pre-existing behavior, not a regression. The litellm provider works independently.

Link to Devin session: https://app.devin.ai/sessions/3b7524eb1b214fd3bc948b39d0d0e87d
Requested by: @lburgers

---

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring