Skip to content

services/hrm: enforce serve deadlines without reports - #2388

Open
jplexer wants to merge 1 commit into
coredevices:mainfrom
jplexer:codex/firm-4328-hrm-timeout
Open

jplexer wants to merge 1 commit into
coredevices:mainfrom
jplexer:codex/firm-4328-hrm-timeout

Conversation

@jplexer

@jplexer jplexer commented Oct 12, 2026

Copy link
Copy Markdown
Member

A silent HRM driver could keep periodic sampling running indefinitely because the unserved timeout was evaluated only on driver reports or unrelated state changes. Keep the manager's existing timer active for due, expiry, and unserved deadlines, so missing reports cannot bypass the cutoff.

Compute deadlines from existing timestamps so poor readings and unrelated updates do not extend the window. Timer callbacks reevaluate current state on KernelBG, respecting charging, run-level and preference gates, subscription renewal, and optical-path changes. Preserve continuous live/workout sampling and existing deferral timing. Remove expired subscriptions during the locked update pass so silent expiry also releases their allocations.

Fixes FIRM-4328

Validation:

  • All 42 HRM tests pass. Coverage includes silent-driver timeout/backoff, fixed deadlines, tick-zero activation, activity interval changes, stale callbacks, gates, optical-path scheduling, and expiry/allocation cleanup.
  • The silent-driver regression fails before the fix.
  • Activity, workout, and health test suites pass.
  • Normal Obelix PVT firmware builds with GH3X2X, HRV, Moddable XS, and speaker support enabled.
  • Formatting, gitlint, and git diff --check pass.

Hardware confirmation on the reporter's watch remains outstanding.

A silent driver could leave a periodic subscriber holding the sensor on
indefinitely: the unserved check ran only on driver reports or unrelated
state updates, while the manager timer was stopped during sampling.

Reuse the timer for the next due, expiry, or unserved deadline. Calculate
those deadlines from existing timestamps so callbacks cannot slide the
window, and reevaluate current state on KernelBG to respect gates, lease
renewals, and optical-path restarts. Preserve continuous live sampling.

Free expired subscribers during the locked update pass so silent expiry
does not retain their allocations after the sensor powers down. Enforce
the timeout at its boundary, including activation at tick zero.

Fixes FIRM-4328
This fixes a confirmed timeout gap found during investigation. Its role
in the customer's extended off-wrist incident remains unconfirmed.

Validation: all 42 HRM tests and the activity, workout, and health suites
pass. The silent-driver regression fails before the timer change.
Normal Obelix PVT firmware builds with GH3X2X, HRV, XS, and speaker support.

Co-authored-by: GPT-6 <noreply@openai.com>
Signed-off-by: Joshua Jun <lets@throw.rocks>
@jplexer
jplexer requested a review from gmarull as a code owner October 12, 2026 14:32
HRMSubscriberState *state = (HRMSubscriberState *)s_manager_state.subscribers;
while (state) {
HRMSubscriberState *next = (HRMSubscriberState *)state->list_node.next;
if (state->expire_utc && utc_now >= state->expire_utc) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pass now frees expired subscriptions even while the sensor is off/silent, but HRMEvent_SubscriptionExpiring is only sent from the data path (hrm_manager_new_data_cb / prv_system_task_hrm_handler), so the holder never gets the event first.

Kraepelin (walk/run hrm_session, 10 min expiry) and workout_service keep their now-dangling session refs. The next sys_hrm_manager_set_update_interval() on that ref returns false, and since the ref isn't HRM_INVALID_SESSION_REF the caller never re-subscribes: kalg_activity_hrm_active() keeps reporting true and activity HR never resumes. Before this change an expired-but-unreported session stayed in the list and a later set_update_interval revived it.

Either send the expiring event here before freeing, or keep the previous semantics for silent expiry.

const bool sensor_on = hrm_is_enabled(HRM);
if (sensor_on) {
remaining_ms = INT32_MAX;
} else if (remaining_ms <= 0) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a transient hrm_enable() failure (attempt 1 of HRM_MAX_ENABLE_FAILURES), the sensor is off with remaining_ms <= 0, so the timer is stopped here. With the sensor off no reports arrive either, so nothing triggers another pass: due subscribers (e.g. daily HR logging) can go unserved for hours, and the failure counter never reaches its limit because no further attempts happen.

Avoiding a busy retry makes sense, but this should re-arm with a backoff (e.g. after HRM_SENSOR_SPIN_UP_SEC) rather than stop.

@@ -377,8 +455,8 @@ static void prv_update_hrm_enable_system_cb(void *unused) {
// subscriber. Only counts continuous on-time, so subscribers that went overdue while the
// sensor was forced off (charging, run level) still get served first.
const bool serve_window_expired =

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The serve window is measured from the sensor's global on-time (sensor_on_since_ticks), not per subscriber. If a continuous reader (BLE HR relay, workout) has kept the sensor on > 120 s, a periodic subscriber that becomes due is marked served immediately (line ~496) without a single sample.

E.g. switch the activity SpO2 session to features=SpO2 while a workout is running: its stale last_valid_bpm_ticks gives age >= interval + unserved, serve_window_expired is already true, so it gets last_valid_bpm_ticks = cur_ticks before the path is selected and the SpO2 window is skipped with no IR measurement. The new timer now enforces this deadline even without reports, so it's more likely to hit.

// Schedule the next due, expiry, or unserved check even if the driver sends no reports.
// Must hold s_manager_state.lock.
static void prv_schedule_update_timer(int32_t remaining_ms) {
if (!prv_can_turn_sensor_on()) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While the gate is closed (charging, run level, prefs off) the timer is stopped, so expired subscriptions aren't freed until the gate reopens, despite the goal of releasing allocations on silent expiry. E.g. an app exits with a 5 min cleanup expiry and the watch goes on the charger for hours: the HRMSubscriberState stays allocated (and the ref still resolves) until unplugged or some unrelated event runs a pass. Consider still arming for the nearest expire_utc when the gate is closed.


// Schedule the next due, expiry, or unserved check even if the driver sends no reports.
// Must hold s_manager_state.lock.
static void prv_schedule_update_timer(int32_t remaining_ms) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This loop duplicates the per-subscriber deadline math of the main pass in prv_update_hrm_enable_system_cb (age, interval, spin-up, unserved window, expiry, prefs filter), which must be kept in sync by hand; a drift means arming the wrong deadline (1 ms busy re-arm via MAX(1, ...), or a missed unserved cutoff). It also re-reads rtc_get_time/rtc_get_ticks/prv_prefs_allowed_features and walks the list a second time. Folding the serve and expiry deadlines into the main loop's remaining_ticks and arming from that single value would be simpler.

deadline_ticks = MIN(deadline_ticks, due_ticks);
}
}
const int64_t deadline_ms = MAX(1, deadline_ticks * PBL_MSEC_PER_SEC / RTC_TICKS_HZ);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two things on this conversion:

  • It truncates, so the timer can fire just before the tick-based >= check in the main pass is met: serve_window_expired is false, the reschedule computes a 0/1-tick deadline and MAX(1, ...) arms a 1 ms timer for another full locked pass on KernelBG. Round up instead.
  • It hand-rolls ticks→ms with RTC_TICKS_HZ on a pbl_ms_to_ticks() value; it's only correct due to the PBL_TICK_HZ == RTC_TICKS_HZ static assert. The main pass already uses pbl_ticks_to_ms(); please use that (clamped to int64).

PBL_ASSERT_TASK(PebbleTask_KernelBackground);
pbl_mutex_lock(&s_manager_state.lock, PBL_FOREVER);
{
HRMSubscriberState *state = (HRMSubscriberState *)s_manager_state.subscribers;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This removal loop duplicates the walk-and-free in hrm_manager_new_data_cb (expired_state / prv_remove_and_free_subscription), and it makes the if (state->expire_utc && utc_now >= state->expire_utc) continue; check in the main loop below (~line 466) dead within the same locked pass. A shared prv_remove_expired_subscriptions(utc_now) helper used by both paths would remove the duplication and the dead check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants