IDLE-DEFER should re-arm IsProcessing when background tasks are active

[PureWeen]

Problem

When SessionIdleEvent arrives with active background tasks but IsProcessing is already false (cleared by a prior watchdog timeout, EVT-REARM cycle, or permission recovery), the IDLE-DEFER path is a no-op — it flushes text but doesn't re-arm processing. The background tasks (sub-agents, shells) continue running but PolyPilot thinks the turn is done.

Observed

19:32:19 [IDLE-DEFER] 'PP- IC Things-worker-1' session.idle received with active background tasks 
  — deferring completion (IsProcessing=False, response=0+0 chars)

IsProcessing=False means the response was already finalized (truncated). The orchestrator collects a short/empty response and reports the worker as failed.

Expected

When IDLE-DEFER detects IsProcessing=false AND background tasks are active, it should re-arm IsProcessing=true (similar to EVT-REARM) so the session stays alive until the background tasks complete and a final idle (without background tasks) arrives.

Considerations

• Must follow INV-2 (UI thread for mutations)
• Must restart the watchdog (INV-9)
• Should log [IDLE-DEFER-REARM] for diagnostics
• The existing EVT-REARM path handles a similar case (TurnStart with IsProcessing=false) — the IDLE-DEFER re-arm would be analogous

Related

• PR Fix premature idle: check BackgroundTasks before completing #399: Added IDLE-DEFER (BackgroundTasks check)
• INV-18 in processing-state-safety SKILL.md

[PureWeen]

Context update (April 2)

This issue is still valid and important. The IDLE-DEFER re-arm logic is needed regardless of the session.idle ephemeral finding.

Note: session.idle is ephemeral: true in the SDK — it is a live-stream-only event, never written to events.jsonl. This issue is about the LIVE event handler behavior, so it is unaffected by that finding. The bug described here (IDLE-DEFER seeing IsProcessing=false when background tasks are active) is a real PolyPilot issue that should be fixed.

[PureWeen]

Updated context (April 2 investigation)

This issue is now confirmed as the root cause of the "zero-idle" symptoms previously tracked in #299 (now closed).

Key finding: session.idle IS delivered — PolyPilot mishandles it

session.idle is ephemeral: true in the SDK — a live-stream-only event, intentionally never written to events.jsonl. The CLI DOES deliver it correctly. The problem is entirely in PolyPilot:

1. session.idle arrives with backgroundTasks active → IDLE-DEFER defers completion ✅
2. Background tasks finish → a second session.idle (without backgroundTasks) should arrive
3. But if IsProcessing was already cleared (by watchdog timeout, EVT-REARM cycle, or reconnect), the second idle is ignored → session appears stuck

Evidence: 208 IDLE-DEFER events in the diagnostics log, 100% followed by more session activity. The pattern described in this issue body matches exactly.

Impact on relaunch resilience

This issue also affects sessions after relaunch. PR #472 adds a poll-then-resume pattern that watches events.jsonl for session.shutdown. But session.idle is never on disk (by design), so the poller cannot detect normal turn completion. The 600s watchdog fallback handles it, but fixing this issue would mean the live-event path works correctly and the watchdog rarely needs to fire.

Broader scope than originally described

The original body describes the case where IsProcessing=false when IDLE-DEFER triggers. There is also a related case:

• IDLE-DEFER fires correctly (IsProcessing=true, defers)
• Background tasks complete, CLI emits second session.idle
• But the SessionState was replaced by a RECONNECT in the meantime → old handler is orphaned → second idle goes to orphaned handler → ignored

Both cases need fixing:

1. IDLE-DEFER-REARM: When IsProcessing=false + background tasks active → re-arm (as described in this issue)
2. RECONNECT idle forwarding: When a RECONNECT replaces the state, pending IDLE-DEFER on the old state should be transferred to the new state

References

• Zero-idle sessions: SDK never emits session.idle event #299 (closed) — was tracking this as a CLI/SDK bug, now known to be PolyPilot-side
• Investigate SDK premature session.idle — upstream fix or permanent workaround #389 — upstream SDK premature idle (separate but related)
• fix: session relaunch resilience + model selection fixes #472 — relaunch resilience workaround (poll + watchdog)

[PureWeen]

Diagnostic update: CLI backgroundTasks is reliable (April 2)

New [IDLE-DIAG] instrumentation confirms the CLI correctly populates backgroundTasks on every session.idle event. The count decreases as agents finish, and the final idle has null backgroundTasks.

This means the IDLE-DEFER logic works correctly for the happy path — it defers when agents are running and completes when they finish. The bug described in this issue (IDLE-DEFER seeing IsProcessing=false) is still real but may be rarer than thought, since it requires a prior watchdog timeout or reconnect to have cleared IsProcessing before the idle arrives.

The fix described in this issue is still the right approach: re-arm IsProcessing when IDLE-DEFER detects active backgroundTasks with IsProcessing=false.

[PureWeen]

Fixed in PR #472 (merged). The IDLE-DEFER-REARM pattern now re-arms IsProcessing when session.idle arrives with active backgroundTasks but IsProcessing is already false. This was the root cause of 'zero-idle' symptoms — the first idle event was correctly deferred, but when the second idle arrived (after background agents finished), IsProcessing was already false so the defer was a no-op.

The fix sets IsProcessing = true, HasUsedToolsThisTurn = true (for 600s tool timeout), and restarts the watchdog. Verified in live multi-agent tests.