Skip to content

fix: split week/month desktop queries by day to avoid 30s timeout - #951

Open
TimeToBuildBob wants to merge 2 commits into
ActivityWatch:masterfrom
TimeToBuildBob:bob/split-full-desktop-query-by-day
Open

fix: split week/month desktop queries by day to avoid 30s timeout#951
TimeToBuildBob wants to merge 2 commits into
ActivityWatch:masterfrom
TimeToBuildBob:bob/split-full-desktop-query-by-day

Conversation

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Problem

The month summary (/activity/.../month/.../view/summary) times out at Axios's 30s requestTimeout on large databases (years of data). The client call that actually waits is query_desktop_full, which sends the entire month as one TIMEINTERVAL.

query_category_time_by_period already splits week/month ranges into days "to avoid timeout on slow queries". The summary's first query does not.

The categorize-cache work in ActivityWatch/aw-server-rust#657 is a real win (~4× on heartbeat-shaped data) but is two orders of magnitude under 30s even on the naive path, so it cannot be this timeout. Issue ActivityWatch/aw-server-rust#629 closed on that merge; the month summary can still 30s-out until this query is split.

What this PR does

  • Split query_desktop_full and query_multidevice_full by day for week/month/multi-day/year periods. A single day stays one request.
  • Query one day at a time (abort-aware, same as query_category_time_by_period).
  • Merge daily app_events / title_events / cat_events / browser aggregates (sum durations, re-sort, re-apply the 100-event limit). active_events are concatenated so the period timeline keeps its shape.
  • Does not raise the global Axios timeout. Does not add another categorize cache.

Trade-off: merging per-day top-100s is not identical to a single month-wide top-100 (same trade-off query_category_time_by_period already accepts). An app that is #101 every day still will not appear; an app that is #1 on one day still will.

Tests

  • npx jest --selectProjects node --testPathPattern desktopQuerySplit --coverage=false — 9 passed
    • single-day stays one period; week → 7 days; February → 28; future days dropped
    • merge sums durations, keeps earliest timestamp, respects top-N
    • window/browser/stopwatch slices merge independently; missing slices are tolerated

Follow-up

Live confirmation on a 168 MB / 2y database that the month summary no longer hits the 30s Axios timeout. I do not have that reporter DB here.

query_desktop_full (and the multidevice variant) sent the whole week or
month as one TIMEINTERVAL, which Axios aborts at 30s on large databases.
query_category_time_by_period already splits those ranges into days.

Query one day at a time, then merge app/title/category/browser aggregates
and sum durations. A single day stays one request.
@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR prevents long desktop activity queries from exceeding the per-request timeout by splitting multi-day periods into sequential daily requests and recombining their results.

  • Adds period splitting and aggregation utilities for desktop, browser, and stopwatch results.
  • Updates desktop and multi-device activity loading to use the split-query path with cancellation checks.
  • Adds unit coverage for period generation, aggregation, limits, and missing result slices.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
src/queries.ts Exports the existing desktop event limit so client-side merged results can consistently reapply it.
src/stores/activity.ts Routes full desktop and multi-device queries through the abort-aware sequential period-query helper.
src/util/desktopQuerySplit.ts Implements period splitting and deterministic aggregation of daily desktop query results.
test/unit/desktopQuerySplit.test.node.ts Covers daily period generation, duration aggregation, event limits, independent result slices, and missing data.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Selected activity period] --> B[Split multi-day range into days]
  B --> C[Run daily requests sequentially]
  C --> D[Merge event groups and durations]
  D --> E[Update activity store]
  E --> F[Render summary and timeline]
Loading

Reviews (2): Last reviewed commit: "docs: record measured baseline for daily..." | Re-trigger Greptile

Comment thread src/util/desktopQuerySplit.ts Outdated
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 66.66667% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.47%. Comparing base (3cbe349) to head (190cb95).

Files with missing lines Patch % Lines
src/stores/activity.ts 5.00% 18 Missing and 1 partial ⚠️
src/util/desktopQuerySplit.ts 86.66% 8 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #951      +/-   ##
==========================================
+ Coverage   49.36%   50.47%   +1.10%     
==========================================
  Files          45       46       +1     
  Lines        2769     2841      +72     
  Branches      625      648      +23     
==========================================
+ Hits         1367     1434      +67     
- Misses       1322     1326       +4     
- Partials       80       81       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

CI-green and mergeable — waiting only on a maintainer click.

This PR is ready to merge, but the bot has pull-only access to this repo and can't self-merge — surfacing it here so it isn't lost. The monitoring loop will stop re-flagging it now that this note is posted.

Greptile 4/5 asked for profiling evidence on the week/month split.
Axios timeout is per-request (30s). On a 31 MB / 4-month aw-server,
July as one TIMEINTERVAL is 0.95s vs max daily 0.098s; sequential
31-day wall is 0.99s. The split cuts per-request time, not total
work — same shape as query_category_time_by_period.
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Addressed the Greptile 4/5 (missing profiling evidence) in 190cb95.

On a 31 MB / 4-month aw-server, a July fullDesktopQuery is 0.95s as one TIMEINTERVAL vs 0.098s max / 0.99s sequential wall as 31 daily requests. Axios timeout is per-request (30s); the split keeps each call under that budget. Re-requesting review.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

@greptileai review

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Automated merge handoff for 190cb95248ef:

  • CI gate: satisfied
  • Greptile: 5/5 (profiling P2 resolved)
  • Review threads: clear
  • Self-merge: blocked — pull-only access on this repo. Paths also sit outside the allowlist:
  • src/queries.ts
  • src/stores/activity.ts
  • src/util/desktopQuerySplit.ts

This head is ready for maintainer review and manual merge. Project monitoring marked it human_merge_required and will not spend retry sessions attempting an automated merge. A new head will be evaluated independently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant