Skip to content

feat(cache): add favorites cache warm-up strategy - #45873

Draft
eschutho wants to merge 6 commits into
masterfrom
cache-warmup-favorite-login
Draft

eschutho wants to merge 6 commits into
masterfrom
cache-warmup-favorite-login

Conversation

@eschutho

@eschutho eschutho commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

SUMMARY

Adds a favorites strategy to the existing cache-warmup Celery beat task. On each scheduled run it picks recently active users who have favorited dashboards and warms the charts on those dashboards, so their favorites load from cache. It is gated by the CACHE_WARMUP_FAVORITES config setting (default False; not a feature flag) and runs only when an operator schedules it.

A chart is only computed if its cached result is missing or expired; charts that are already cached are left alone, so no queries are repeated needlessly.

Design notes:

  • Each selected user gets their own task that runs as that user, not as SUPERSET_CACHE_WARMUP_USER, because cache keys depend on the requesting user (row-level security, per-user caching, impersonation). Access to each dashboard and chart is checked for that user.
  • It reuses ChartWarmUpCacheCommand. A new only_if_stale option makes it look up the data cache first and fill misses without bypassing the cache. The existing "always force refresh" behavior is unchanged when the option is not used.
  • Bounds: max_users and since on the strategy (most recent login first) and CACHE_WARMUP_FAVORITES_MAX_DASHBOARDS (default 5, newest favorites first) limit work per run. A per-result lock stops overlapping runs from computing the same result twice, and each per-user task has a 15 minute time limit and queued tasks expire after two hours. Favorites read per user are bounded in SQL. There is no per-user cooldown: the beat schedule is the throttle.
  • CACHE_WARMUP_FAVORITES_QUEUE (default None) can route these tasks to a dedicated queue so they do not compete with reports, alerts, or thumbnails.
  • Needs a Celery worker and a real DATA_CACHE_CONFIG backend.
  • Tests check that the cache key the warm-up looks up matches the one built on the request path, and that it changes with row-level security and the running user under per-user caching, and stays stable for relative time ranges.
  • Known limitation: dashboard default column filters (native filter defaults and legacy defaults) are applied, but other dashboard filter settings (time range, time column and grain, extras, ad hoc filters) are not, so a dashboard load that uses them can still miss the cache for those charts.

Example schedule:

"cache-warmup-favorites": {
    "task": "cache-warmup",
    "schedule": crontab(minute=0, hour="*/6"),
    "kwargs": {"strategy_name": "favorites", "since": "7 days ago", "max_users": 100},
}

TESTING INSTRUCTIONS

Unit tests: pytest tests/unit_tests/tasks/test_cache.py tests/unit_tests/commands/chart

Manual: configure Redis for DATA_CACHE_CONFIG and CACHE_CONFIG, run a Celery worker and beat with the schedule above and CACHE_WARMUP_FAVORITES = True. Favorite a dashboard as a user who has logged in recently, clear its cache, and confirm the worker computes its charts on the next run and computes nothing on a later run while the cache is warm.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration
  • Introduces new feature or API
  • Removes existing feature or API

🤖 Generated with Claude Code

Adds an opt-in CACHE_WARMUP_ON_LOGIN setting. After a successful login an
asynchronous Celery task runs as the user and computes only the missing or
expired chart query results for their most recently favorited dashboards.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added doc Namespace | Anything related to documentation preset-io labels Oct 30, 2026
@netlify

netlify Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for superset-docs-preview ready!

Name Link
🔨 Latest commit 4eeb6ca
🔍 Latest deploy log https://app.netlify.com/projects/superset-docs-preview/deploys/6ae4104cc2ee051ab5b4feb0
😎 Deploy Preview https://deploy-preview-45873--superset-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

eschutho and others added 3 commits October 30, 2026 03:08
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Replaces the login-triggered warm-up with a beat-scheduled `favorites`
strategy for the cache-warmup task. It enqueues one task per recently active
user that warms only missing or expired results on their favorited
dashboards, with a per-result lock, dashboard cap, optional dedicated queue
and a task time limit.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@eschutho eschutho changed the title feat(cache): warm favorited dashboards' cache on user login feat(cache): add favorites cache warm-up strategy Oct 30, 2026
eschutho and others added 2 commits October 30, 2026 03:25
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…docs

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 76.69173% with 31 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.70%. Comparing base (a8c13c6) to head (4eeb6ca).
⚠️ Report is 13 commits behind head on master.

Files with missing lines Patch % Lines
superset/commands/chart/warm_up_cache.py 60.97% 14 Missing and 2 partials ⚠️
superset/tasks/cache.py 83.14% 11 Missing and 4 partials ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           master   #45873       +/-   ##
===========================================
+ Coverage   65.62%   84.70%   +19.08%     
===========================================
  Files        3069     3069               
  Lines      205661   205788      +127     
  Branches    48336    48357       +21     
===========================================
+ Hits       134956   174315    +39359     
+ Misses      67624    28174    -39450     
- Partials     3081     3299      +218     
Flag Coverage Δ
hive 34.49% <21.05%> (-0.02%) ⬇️
mysql 52.55% <27.06%> (-0.04%) ⬇️
postgres 52.55% <27.06%> (-0.04%) ⬇️
presto 36.22% <21.05%> (-0.02%) ⬇️
python 87.74% <76.69%> (+34.87%) ⬆️
sqlite 52.32% <27.06%> (-0.04%) ⬇️
unit 82.47% <76.69%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc Namespace | Anything related to documentation preset-io review:draft size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants