Skip to content

feat(popularity): aggregate page views from redirected URLs - #696

Merged
caugner merged 3 commits into
mainfrom
popularities-including-redirects
Oct 7, 2026
Merged

caugner merged 3 commits into
mainfrom
popularities-including-redirects

Conversation

@caugner

@caugner caugner commented May 15, 2026 •

Copy link
Copy Markdown
Contributor

Description

Aggregate popularity across redirects: a page's effective popularity is now its own page views plus the page views of every URL that redirects to it. Applied to both the per-page popularity in built JSON and the ordering of search-index.json.

The development server uses the same aggregated popularity for search rankings.

Motivation

When a page is moved (e.g. Web/Foo/bar → Web/Reference/Foo/Bar), pageview history attributed to the old URL in popularities.json was previously lost, making the moved page appear less popular than it actually is and ranking it lower in search.

Additional details

  • Adds REDIRECTS_REVERSED (lowercase target URL → source URLs) and a popularity_for(url) helper in rari-doc/src/popularity.rs.
  • Lowercases popularities.json keys at load so the cross-lookup against the lowercase-keyed REDIRECTS map is consistent.
  • _redirects.txt files contain no chains, so a single hop suffices.
  • Real-world impact on en-US search-index.json is currently small: only 6 of 17.5k redirect sources appear in popularities.json (the upstream feed is keyed by current canonical URLs). Top-100 ordering is unchanged; the 5 pages that move up are:
main rank branch rank Δ URL
13358 3180 −10178 /en-US/docs/Glossary/Style_sheet
14372 10194 −4178 /en-US/docs/WebAssembly/Reference/Numeric/shr_s
14314 10531 −3783 /en-US/docs/WebAssembly/Reference/Definitions/global
14338 10705 −3633 /en-US/docs/WebAssembly/Reference/SIMD/load_store/load
154 143 −11 /en-US/docs/Learn_web_development/Core/Scripting

@caugner
caugner requested a review from a team as a code owner May 15, 2026 12:22
@caugner
caugner requested a review from LeoMcA May 15, 2026 12:22
@github-actions

github-actions Bot commented May 15, 2026 •

Copy link
Copy Markdown
Contributor

7ae9c94 was deployed to: https://rari-pr696.review.mdn.allizom.net/

@caugner
caugner marked this pull request as draft May 15, 2026 15:07
Comment thread crates/rari-doc/src/redirects.rs Outdated
@caugner
caugner marked this pull request as ready for review May 15, 2026 17:54
Comment thread crates/rari-doc/src/redirects.rs Outdated
Comment thread crates/rari-doc/src/redirects.rs Outdated
Comment thread crates/rari-doc/src/redirects.rs Outdated
Comment thread crates/rari-types/src/globals.rs Outdated
Aggregate page views from redirected URLs into each target page's popularity in built JSON and search-index order.

Keep redirect-aware aggregation in the popularity module, normalize URLs consistently, and cover pages without redirect sources.
@caugner
caugner force-pushed the popularities-including-redirects branch from 9afe85c to 60cb317 Compare September 24, 2026 16:39
@caugner
caugner marked this pull request as ready for review September 24, 2026 16:43

@LeoMcA LeoMcA left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! We should apply these changes to the server as well:

Comment thread crates/rari-doc/src/search_index.rs
@github-actions github-actions Bot added diff: 1-10 pages Build diff: 1 to 10 pages changed and removed diff: none Build diff: no pages changed labels Oct 6, 2026
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Build diff: 24797 changes in 1 of 14721 pages (0.0%)

Download report (diff.html)

@caugner
caugner added this pull request to stack #983 October 6, 2026 22:30
@caugner caugner changed the title feat(popularity): include redirected URLs in page popularity feat(popularity): aggregate page views from redirected URLs Oct 7, 2026
@caugner
caugner merged commit de0a034 into main Oct 7, 2026
19 checks passed
@caugner
caugner deleted the popularities-including-redirects branch October 7, 2026 13:38

This branch was successfully deployed

1 active deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diff: 1-10 pages Build diff: 1 to 10 pages changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants