Skip to content

test: cover TaskHint hint selection (dispatch, embedding ranking, encode retries) - #36

Open
devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1789677978-taskhint-tests
Open

devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1789677978-taskhint-tests

Conversation

@devin-ai-integration

Copy link
Copy Markdown

Summary

Closes the code scan coverage gap on TaskHint hint retrieval in tool_use_agent.py: choose_hints dispatch, embedding top-n ranking, and the _encode retry loop had no tests at all.

New tests/agents/test_tool_use_agent_hints.py (no network, no LLM):

  • choose_hints routes llm/direct/emb to the right helper with the right argument (goal vs task_name), and raises ValueError on an unknown mode.
  • choose_hints_emb with stubbed _encode/_similarity and similarities [0.1, 0.9, 0.5], top_n=2 must return the two most similar hints, pinning the argsort()[0][-top_n:] index math (result is ascending by similarity, so the best hint comes last).
  • _encode retries the requests.post call and succeeds on the third attempt, and re-raises after exhausting max_retries; time.sleep is monkeypatched so the retries are instant.

Note: pytest collection of anything importing agentlab.llm.response_api fails in this environment on from anthropic.types import Completion (anthropic 1.6.0 removed it) — preexisting on main and unrelated to this PR; tests were verified with that symbol shimmed in.

Devin-Org: engineering

Link to Devin session: https://app.devin.ai/sessions/c037048a6e0c41cc9794ec6579c3ec0f
Open in Devin Desktop: https://app.devin.ai/desktop/session/c037048a6e0c41cc9794ec6579c3ec0f?variant=devin

@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants