Skip to content

fix: resolve unbounded get_response() hang on FakeBus - #130

Merged
JarbasAl merged 1 commit into
devfrom
fix/fakebus-tts-lifecycle
Aug 11, 2026
Merged

fix: resolve unbounded get_response() hang on FakeBus#130
JarbasAl merged 1 commit into
devfrom
fix/fakebus-tts-lifecycle

Conversation

@JarbasAl

@JarbasAl JarbasAl commented Aug 11, 2026

Copy link
Copy Markdown
Member

🤖 Auto-generated by Claude Fable 5 (claude-fable-5) via Claude Code — NOT human-reviewed. Verify before acting.

Revised after adversarial review — see "Concurrency defect found in review" below.

Field evidence

OpenVoiceOS/ovos-skill-alerts#138 ("Update 3") root-caused, via py-spy stack
captures on a real CI hang, two mechanisms by which real skill handlers
block forever under ovoscope's synchronous FakeBus:

  • (a) wait_while_speaking() (ovos_bus_client/session.py:710) burns
    its full 15s timeout for nested/cascading speak_dialog(..., wait=True)
    calls if recognizer_loop:audio_output_end never arrives. Already
    mitigated in ovoscope/__init__.py's _mock_tts mock-TTS handler
    (per-call threading.Timer unduck) — verified still correct for nested
    calls (see regression test below).
  • (b) OVOSSkill._wait_response() (ovos_workshop/skills/ovos.py
    ~1802-1809, ovos-workshop==7.0.6) has no deadline at all:
    while not ans: time.sleep(0.1). With the get_response()/ask_yesno()
    default num_retries=-1, the background _real_wait_response thread
    re-prompts forever if nothing ever answers it. A real voice satellite
    eventually emits mycroft.skills.abort_question on user silence;
    ovoscope's FakeBus never did. This is what leaked non-answered-thread
    state into ovos-skill-alerts' shared-MiniCroft e2e suite and made CI
    hang to the 30-minute job kill on 4 of 5 Python versions on PR fix: Han audit round 2 — SkillApi retention, accuracy gate, false-green assertions, race windows #123.

Fix

MiniCroft arms a short watchdog threading.Timer on
skill.converse.get_response.enable that emits the existing
mycroft.skills.abort_question bus message (no new message type — this is
the same message a real listener/GUI sends on cancel/silence, and
OVOSSkill._real_wait_response is already decorated with
@killable_event("mycroft.skills.abort_question", ...) to handle it) if
.disable doesn't fire first. .disable (emitted once get_response()
actually returns, answered or not) disarms the watchdog. Tracked/cancelled
in stop() the same way the existing mock-TTS timers already are, so no
orphaned timer can fire onto a closed bus and poison a later test.

Concurrency defect found in review

The first version of this fix kept ONE flat list of pending watchdog
timers. Adversarial review reproduced a real regression: with two
concurrent get_response() flows in flight (skill A and skill B, a
fleet-style MiniCroft running multiple skills), skill A's .disable
drained and cancelled the entire list, including skill B's still-armed
watchdog — if B's question was never answered, B hung forever again,
defeating the fix for exactly the multi-skill case it needs to hold up
under. Repro that pinned it: enable(A), enable(B), disable(A)0
timers left instead of 1.

Fixed by scoping each watchdog by (skill_id, session_id) — the same
two-part scope ovos_workshop's own
@killable_event("mycroft.skills.abort_question", check_skill_id=True)
already uses to decide which stalled thread an abort is actually for.
.disable now cancels only its own (skill_id, session_id) entry; the
abort emit carries the same skill_id/session context so only the
stalled question dies. stop() still drains and cancels every remaining
entry on teardown.

Smallest correct change: ovoscope/__init__.py only (MiniCroft),
mirroring the existing _mock_tts timer-tracking pattern already in the
file.

Red → green proof

test/unittests/test_tts_lifecycle_nested_and_get_response.py:

  • TestNestedSpeakDialogWait / TestUnansweredGetResponse — the original
    two regression tests. Before the fix: both hang (a 90s timeout kill was
    needed to get a shell back). After: both pass in ~3s.
  • TestConcurrentGetResponseWatchdogScoping (new, pins the review finding):
    • test_disabling_one_skill_does_not_cancel_another_skills_watchdog
      whitebox, drives the enable/disable protocol messages directly.
      On the flat-list version this FAILS: AssertionError: 1 != 0
      (skill B's watchdog is gone after only skill A disabled). On the fixed
      version it passes.
    • test_unanswered_skill_still_aborted_when_another_skill_answers_first
      — end-to-end: two skills, two sessions, concurrent get_response()
      calls; skill A is answered via an injected utterance, skill B is
      deliberately left unanswered. On the flat-list version this hangs
      (verified via a 20s timeout kill on the whole test class). On the
      fixed version: skill A's real answer survives untouched, skill B is
      aborted by its own watchdog and resolves to None — both complete in
      ~7s total for the 4-test file.

Full ovoscope suite (after fix)

1 failed, 626 passed, 13 skipped, 3200 warnings in 151.77s
FAILED test/unittests/test_tts_intelligibility.py::TestHarnessPlaybackMode::test_playback_captures_wav_and_scores

That one failure is a pre-existing order-dependent flake, unrelated to
this change: it passes in isolation and doesn't touch
get_response/speak_dialog mocking at all. Reproduces the same way with
the fix reverted.

Real-world gate: ovos-skill-alerts PR #123

Checked out OpenVoiceOS/ovos-skill-alerts PR #123
(feat/ovoscope-tests) into a separate throwaway clone/venv (Python
3.14 in this sandbox), installed this branch's ovoscope via local path,
and ran its test/end2end suite (ovos-workshop/ovos-core/padacioso
pipeline — ovos-padatious still not installable on this box, matching
the maintainer's own note in #138).

Note: this sandbox's exact pass/fail counts differ from the numbers seen
in review logs on a Python 3.11 + pytest-xdist setup (1 failed, 32 passed, 133 skipped in ~24s) — re-ran with pytest-xdist here too and
still saw the pre-existing vocab-mismatch failures, just distributed
differently across workers (30 failed, 2 passed, 52 skipped in ~19s, same
"terminates, no hang" outcome). The discrepancy tracks Python version /
worker distribution affecting which of the already-documented, unrelated
vocab-pinning tests execute — it does not affect the property this PR
claims: the suite reliably terminates instead of hanging.

Scope

Only ovoscope/__init__.py (MiniCroft) changed, plus the regression
test. No new bus message types. No changes to ovos-workshop (out of
scope for this repo — the _wait_response() no-deadline issue is a
separate upstream defect worth its own report there, not fixed here).

@github-actions github-actions Bot added the fix label Aug 11, 2026
@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 5ab5366a-f501-4151-b310-438894a6178d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown

I've gathered some intelligence on your latest changes. 🕵️‍♀️

I've aggregated the results of the automated checks for this PR below.

🔍 Lint

A detailed summary of the latest automation run. 📝

ruff: issues found — see job log

🏷️ Release Preview

Evaluating the impact of these changes on our release schedule. 📉

Current: 1.6.4a1Next: 1.6.5a1

Signal Value
Label fix
PR title fix: resolve unbounded get_response() hang on FakeBus
Bump build

✅ PR title follows conventional commit format.


🚀 Release Channel Compatibility

Predicted next version: 1.6.5a1

Channel Status Note Current Constraint
Stable Not in channel -
Testing Too new (must be <1.0.0) ovoscope>=0.7.2,<1.0.0
Alpha Compatible ovoscope>=1.6.4a1

🔒 Security (pip-audit)

Our digital defenses have been updated. 🛡️

✅ No known vulnerabilities found (79 packages scanned).

📋 Repo Health

Evaluating the longevity of the project. 🌳

✅ All required files present.

Latest Version: 1.6.4a1

ovoscope/version.py — Version file
README.md — README
LICENSE — License file
pyproject.toml — pyproject.toml
⚠️ setup.py — setup.py
CHANGELOG.md — Changelog
ovoscope/version.py has valid version block markers

⚖️ License Check

Scanning for any potential trademark infringements. ™️

✅ No license violations found.

Policy: Apache 2.0 (universal donor). StrongCopyleft / NetworkCopyleft / WeakCopyleft / Other / Error categories fail. MPL allowed.

🔨 Build Tests

From source to binary, let's see how it holds up. 🧱

✅ All versions pass

Python Build Install Tests
3.10
3.11
3.12
3.13
3.14

📊 Coverage

Measuring the reach of our test cases. 📏

59.0% total coverage

Files below 80% coverage (15 files)
File Coverage Missing lines
ovoscope/simple_listener.py 0.0% 63
ovoscope/tts_intelligibility.py 0.0% 200
ovoscope/version.py 0.0% 5
ovoscope/classic_listener.py 18.2% 117
ovoscope/intent_cases.py 21.4% 151
ovoscope/pytest_plugin.py 39.8% 237
ovoscope/cli.py 47.8% 132
ovoscope/ocp.py 47.9% 61
ovoscope/e2e.py 53.5% 66
ovoscope/media.py 57.0% 104
ovoscope/listener.py 57.1% 127
ovoscope/voice_loop.py 59.2% 118
ovoscope/audio.py 63.4% 126
ovoscope/__init__.py 64.2% 347
ovoscope/media_provider.py 67.6% 23

Full report: download the coverage-report artifact.


Generated with ❤️ by OVOS Automations

)

OVOSSkill.get_response()/ask_yesno() (ovos-workshop 7.0.6) spawn a killable
background thread that re-prompts forever when num_retries=-1 (the default)
and nothing ever answers it. On a real voice satellite the listener/GUI
eventually emits mycroft.skills.abort_question on user silence; ovoscope's
synchronous FakeBus never generates one, so any skill handler that calls
get_response()/ask_yesno() without a queued follow-up utterance hangs the
calling thread indefinitely (OVOSSkill._wait_response(),
ovos_workshop/skills/ovos.py ~1802-1809 - "while not ans: time.sleep(0.1)",
no deadline).

MiniCroft now arms a short watchdog timer on
"skill.converse.get_response.enable" that emits the existing
"mycroft.skills.abort_question" bus message (no new message type) if
".disable" hasn't fired first - mirroring what a real listener/GUI would
send on silence. This lets get_response()/ask_yesno() resolve to None
promptly instead of looping every get_response_timeout (20s default)
forever.

Watchdog timers are keyed by (skill_id, session_id) - the same two-part
scope ovos-workshop's own @killable_event("mycroft.skills.abort_question",
check_skill_id=True) already uses to decide which stalled thread an abort
is for. An adversarial review of an earlier version of this fix caught a
real regression here: a flat list of pending timers meant ANY skill's
".disable" cancelled EVERY other skill's still-pending watchdog too, so in
a fleet-style MiniCroft running multiple skills concurrently, one skill
finishing its (answered) get_response() would silently disarm a different
skill's watchdog - if that second question was never answered, it hung
forever again, defeating the fix for exactly the multi-skill case it needs
to hold up under. Scoping by (skill_id, session_id) and cancelling only the
matching entry on ".disable" fixes it; all remaining timers are still
cancelled in stop() the same way the existing mock-TTS timers already are.

The existing nested-speak_dialog(wait=True) mock-TTS handshake
(recognizer_loop:audio_output_start/end) already resolves correctly for
nested calls via its own per-call Timer; a regression test pins that
behavior alongside the get_response fix and its concurrency scoping.

Field evidence: OpenVoiceOS/ovos-skill-alerts#138 "Update 3" - py-spy
captures showing OVOSSkill._real_wait_response threads parked forever on
the ovoscope FakeBus, and PR #123's CI job silently hanging to the
30-minute kill on 4 of 5 Python versions after the last test finished.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@JarbasAl
JarbasAl force-pushed the fix/fakebus-tts-lifecycle branch from 85a5616 to 7f05d00 Compare August 11, 2026 03:32
@github-actions github-actions Bot added fix and removed fix labels Aug 11, 2026
@JarbasAl
JarbasAl marked this pull request as ready for review August 11, 2026 12:20
@JarbasAl
JarbasAl merged commit 200ab9e into dev Aug 11, 2026
14 checks passed
@JarbasAl
JarbasAl deleted the fix/fakebus-tts-lifecycle branch August 11, 2026 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant