Repository navigation
fix(search): match language tags against locales as language ranges - #919
Merged
Merged
Conversation
- Add matchLanguage to @lde/search/adapter: BCP-47 range matching, exact tag first, then truncations, extensions and sibling regions; und matches only und - Index a literal tagged nl-NL under a declared nl, exact tag first - Sort and boost a request in nl-NL on the nl keys instead of the first locale - Order a GraphQL language list by range, so nl-NL lists the nl value first - Document the matching in the Locales reference
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix #794
A language tag was matched against a field’s
localeswith===, so a browser’snl-NLmatched no declarednl. A Dutch reader’s sort fell back tolocales[0], a publisher’snl-NLtitles got no search or sort key, and the GraphQL language list could put another language first.matchLanguagein@lde/search/adapternow does the matching everywhere. It compares BCP-47 subtags case-insensitively and ranks the exact tag first, then truncations (nl-NL→nl), then extensions (nl→nl-BE), then sibling regions (nl-NL→nl-BE).undmatches onlyund.applyText): a literal is keyed under the declared locale its tag best matches; within a locale the exact tag’s values come first, so a document withnlandnl-BEtitles sorts on thenlone.compileSort):nl-NLsorts ontitle_sort_nl.collectSearchable): the same exact comparison decided which locale keeps full weight. Not in the issue, same cause.defaultLanguageOrder): each accepted language pulls in every available tag it matches, best first, without duplicates.The sibling rank goes beyond the issue. Without it, a request in
nl-NLagainstlocales: ['nl-BE', 'en']got no boost and sorted on the first locale only by accident, while the projection already routed a plainnlliteral intonl-BE. The catch: it also rankszh-Hansabove English for azh-Hantreader, though the scripts are not interchangeable.I tried
@formatjs/intl-localematcherinstead of a hand-written helper. It has the samezh-Hant→zh-Hansfallback, also matches across languages (af→nl,nb→no), throwsRangeErroron malformed tags from publisher data or headers, and returns one match wheredefaultLanguageOrderneeds a ranked list. Negotiator’s matcher, already a dependency, compares the primary subtag only. Neither saves code once wrapped.Lookups still fetch display fields only for declared locales, so a referenced document’s label tagged
nl-NL(orfr) renders without it, as it did before. Wideninginclude_fieldschanges what a lookup costs, so it stays out of this PR.