Skip to content

Fix/classify wrapped connection errors - #134

Open
lhumeau wants to merge 2 commits into
google:mainfrom
lhumeau:fix/classify-wrapped-connection-errors
Open

lhumeau wants to merge 2 commits into
google:mainfrom
lhumeau:fix/classify-wrapped-connection-errors

Conversation

@lhumeau

@lhumeau lhumeau commented Sep 18, 2026

Copy link
Copy Markdown

Summary

Fixes LLM failure classification when provider SDKs wrap transport-level
connection failures.

During an end-to-end Artemis Flash test against an OpenAI-compatible LM Studio
endpoint, Artemis recorded:

  • provider: openai
  • error: "Connection error."
  • max_retries: 2

The retry count showed that the exception was being classified as UNKNOWN,
even though the underlying failure was a connection error.

Provider SDKs such as OpenAI commonly wrap transport exceptions in types like
APIConnectionError, so checking only the outer exception loses the actual
failure category.

This change:

  • walks __cause__ / __context__ safely;
  • classifies wrapped transport failures as CONNECTION;
  • recognizes common provider/transport connection wrapper types;
  • preserves existing status-code and message-based classification;
  • adds regression tests.

Why this matters

A wrapped connection error currently receives the UNKNOWN retry policy
(2 attempts) instead of the CONNECTION retry policy (3 attempts), which can
prematurely pause an otherwise recoverable task.

Validation

Validated on Windows with Artemis Flash using:

  • OpenAI-compatible LM Studio endpoint over LAN
  • model: qwen/qwen3.8-27b
  • Android emulator running Android 16
  • no Gemini API key

End-to-end functional test completed successfully:

  • screenshot understanding
  • UI hierarchy processing
  • tool calling
  • swipe
  • tap
  • Settings navigation
  • About device navigation
  • Android version extraction
  • device model extraction
  • return to Home
  • successful task completion

The original failing trace recorded:

{
  "error": "Connection error.",
  "attempt": 1,
  "max_retries": 2,
  "provider": "openai"
}

@google-cla

google-cla Bot commented Sep 18, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant