fix(compiler): retry transient LLM API timeouts with bounded backoff #229

Problem

During batch document ingest (openkb add with multiple documents), the LLM compilation pipeline sometimes fails to generate complete concept/entity pages due to transient Gateway Timeouts from the Anthropic API.

Example output:

[147/148] Adding: jira-SSMPA-9592.md
  [WARN] 8 concept(s) planned but only 6 written for jira-SSMPA-9592 (Timeout).
  [WARN] 6 entity(ies) planned but only 5 written for jira-SSMPA-9592 (Timeout).

This results in an incomplete knowledge base where specific concepts/entities are missing entirely.

Reproduktion

# Run batch ingest with multiple documents
openkb add ~/documents/batch/*.md

# Observe logs: After ~150 seconds of concurrent API calls (concurrency=5),
# last few concept/entity generations fail with:
# litellm.Timeout: AnthropicException Timeout - Gateway Timeout

Occurs when:

• Document count is high (multiple parallel compilations)
• Anthropic API is under load
• Concurrency=5 sustained for ~150+ seconds

Workaround: Re-run the ingest command (transient nature means retry would succeed).

Kontext

• Python 3.12
• openkb add with high-concurrency compilation
• Anthropic Claude API timeout (503 Gateway Timeout)
• Not a client-side timeout — server-side load-induced

Root Cause

High-concurrency API requests (5 parallel via semaphore) over extended duration cause temporary gateway overload on Anthropic's infrastructure. Individual API calls are successful; only sustained parallel load triggers 503 responses on the N-th request.

Proposed Solution

Implement exponential backoff retry for transient errors (litellm.Timeout, 5xx API errors, rate limits) in the LLM call pipeline. Only retry errors that are stateless and temporary; skip permanent errors (4xx, validation, auth).

• Use LiteLLM's built-in retries parameter (exponential backoff by default)
• Filter exception types: only retry transient errors
• Bound retries to 2 attempts + 4s total wait
• Better logging to distinguish transient vs. permanent failures