feat(lyrics): take Kugou's word-level timing instead of discarding it - #40
Open
Zakkaus wants to merge 14 commits into
Open
feat(lyrics): take Kugou's word-level timing instead of discarding it#40Zakkaus wants to merge 14 commits into
Zakkaus wants to merge 14 commits into
Conversation
This was referenced Aug 15, 2026
Zakkaus
force-pushed
the
feat/kugou-word-timing
branch
4 times, most recently
from
August 15, 2026 23:20
9a1180b to
6315e35
Compare
Searching a provider by title and artist is guesswork, and for a whole class of players it is unnecessary: NetEase clients publish the platform's own song id in their MPRIS metadata. If the player tells you which song it is, matching is not a problem you need to have — and as locez#14 puts it, lyrics are only reliable when they come from where the audio came from, because the first line's start, whether silence was trimmed, and which performance it is all differ between sources. The rules are waylyrics' (`src/sync/interop/mpris/hint.rs`), which locez#14 names as the reference, copied rather than guessed at: ElectronNCM, Qcm, musicfox, NeteaseCloudMusicGtk4 last segment of mpris:trackid feeluown fuo://netease|qqmusic/songs/<id> YesPlayMusic url /trackid/<id> anything with a file:// url the local path `xesam:url` is now parsed; it was not read at all before. A hint that names a supported, enabled provider fetches that id directly and the result is used without title, artist or duration scoring. Anything else — an unknown player, a disabled provider, a failed fetch — falls back to today's search path unchanged. A hint is an optimisation, never a dead end. QQ Music hints are represented but not acted on: this repository has no QQ Music provider, and inventing one is a separate change. `file://` yields only the path; reading tags needs a dependency that is not mine to add. Verified against the live NetEase API: given a deliberately wrong title and artist but the right id, the resolver returns the correct 58-line lyric.
Among candidates that pass the acceptance rules, kotonoha broke ties with a
chain of booleans ending in album equality. Album was exact equality after
normalisation, so a candidate whose album read 安和桥北 against a track's
安和橋北, or "X (Deluxe Edition)" against "X", counted for nothing — it ranked
level with a candidate carrying no album at all.
Ranking now uses the weighting waylyrics arrived at, which upstream recommended:
artist and album title*0.4 + artist*0.2 + album*0.4
artist only title*0.7 + artist*0.3
album only title*0.8 + album*0.2
neither title
Two things about it are deliberate. Album carries the same weight as title,
because it identifies the release a recording belongs to. Duration is absent
from the blend and stays the last tiebreak beneath it, because durations are
frequently inaccurate — the same reason it already sorted below album here.
Similarity is the Dice-Sørensen coefficient over character bigrams, matching the
reference. Python needs no decoding step for this: iterating a str already
yields code points, which is what the Rust builds explicitly. Bytes are never
involved, since byte bigrams would split multi-byte characters and make the
score meaningless for most of this library. Two empty strings score 0.0, not
1.0 — the trap SequenceMatcher already sprang here once.
The blend ranks and nothing else. It does not assign confidence and cannot
promote a candidate across a tier, because an instrumental shares its album, its
duration and nearly its title with the vocal version and would blend to an
excellent score. The variant veto still runs first: 甲乙丙丁 (你我怎麼兩清伴奏)
is still refused, 甲乙丙丁 (你我怎麼兩清) still accepted.
locez#32 already derives a local path from a `file://` url; nothing consumed it. A sidecar `.lrc` is the strongest source there is — it is the lyrics shipped with that exact audio, so there is no matching, no scoring, and no question of which recording it belongs to. It wins outright, before any network request. Only `<stem>.lrc` beside the audio is considered: no directory scan, no guessing at other names, and a sidecar whose resolved path leaves the audio file's own directory is refused, because this runs on whatever a player reports. Decoding tries UTF-8 then GB18030, since Chinese `.lrc` files in the wild are very often GB18030 or GBK. A file that decodes as neither, is missing, is empty, or parses to no timed lines is a miss, not an error: the existing search path runs and the user sees nothing unusual. A missing sidecar is the normal case. Parsing reuses `lrc_parser`; there is no second LRC parser. Reading embedded tags, the other half of locez#14, still needs a dependency and is not here.
A sidecar written against a different rip is commonly a second or two out, and the format's own shift tag is how the file says so. It was ignored, so the correction had to be re-entered by hand as a per-track offset. Per the format's wording a "+" value causes the lyrics to appear sooner, so it is subtracted from each timestamp. A value beyond a minute is treated as junk rather than an instruction, and a shifted line never goes negative.
The other half of locez#14's local case. locez#34 reads a `.lrc` beside the audio; this reads lyrics stored inside the file itself, which needs a tag parser. `mutagen` is that parser, and it is **optional**, declared under `[project.optional-dependencies]` as `embedded-lyrics` rather than in the required list. The import is lazy and an `ImportError` is an ordinary miss, so the application runs unchanged where it is absent — the feature simply does not exist there. That is deliberate: mutagen is GPL-2.0 while this project is MIT, so whether to ship it is the packager's decision and the user's, not a condition of running kotonoha. Debian gets `Suggests:`, Fedora `Recommends:`. Tags read, per format: `USLT` for ID3, `LYRICS` and `UNSYNCEDLYRICS` for Vorbis comments in FLAC/Ogg/Opus, and the `©lyr` atom for MP4. Almost all embedded lyrics in the wild are an LRC-formatted string sitting in an *unsynchronised* tag, so the text goes through the existing `lrc_parser` and is accepted only when it yields timed lines. Plain prose lyrics are a miss, not an empty result that would suppress the network path — this overlay cannot show untimed text. Precedence: a sidecar `.lrc` wins, because a user who put a file there did so deliberately; embedded tags come next; the network last. The suite passes with mutagen absent, which is the case that matters here: 460 passed, 6 skipped, and the skips are exactly the tests that need it.
_embedded_texts probed the MP4 key b"\xa9lyr" on whatever tag object the file carried. A Vorbis comment accepts only printable ASCII keys and raises ValueError on that one, and the caller turned the exception into a miss — so every FLAC and Ogg with embedded lyrics came back empty, after the LYRICS value had already been found. Isolate each lookup, and cover it with a real VCFLACDict instead of a plain dict, which tolerates any key and hid the failure.
Every embedded-tag test is guarded by importorskip, and CI installed only the test extra, so the whole feature was uncovered there — which is how a FLAC returning no lyrics reached a green build. Install the extra. ty needs the accompanying rule: the mutagen import requires a suppression when the extra is absent and has none to suppress when it is present, so one of the two states would always fail the gate.
locez#32 recognises `fuo://qqmusic/songs/<id>` from feeluown and produced a hint that nothing could act on, because there was no QQ Music provider. This adds the half that is actually possible without a login. Two anonymous calls, both verified from Australia and from a network inside China with identical results: music.pf_song_detail_svr / get_song_detail numeric song id -> songmid fcg_query_lyric_new.fcg songmid -> base64 LRC + translation The id matters: feeluown's URI carries the numeric song id, not the songmid the lyric endpoint wants — `QQSongSchema.identifier` is `fields.Int(data_key="id")` with `mid` as a separate field, so the conversion is not optional. The body is JSONP wrapped in `MusicJsonCallback(...)`, and `lyric` and `trans` are base64. Keyword search is deliberately absent. That is the half that genuinely needs a logged-in user's cookies: without them the search endpoint returns `code: 2001` and zero results, from both vantages. So `qqmusic` joins the known sources but NOT the default order — enabled, it can only answer when a player hands it an id, and querying it for an ordinary track would waste a request every time. No cookies, no login, no configurable API base url, and no self-hosted service.
Adding `qqmusic` to the valid sources without a `src.qqmusic` string left the settings list showing the raw key. Spotted in the running application, not by a test — so the fix comes with the test that would have caught it: every entry in `VALID_LYRICS_SOURCES` must have a display name in all four languages.
The karaoke sweep highlights the current word, but only when the lyrics carry per-word timing — and that came from exactly two places: netease's `yrc` and Cider's TTML. A song resolved from Kugou fell back to line-level highlighting and the user was never told why. Kugou has the timing. `kugou.py` asked its download endpoint for `fmt=lrc` and got plain LRC; `fmt=krc` returns KRC, which carries it. KRC decodes with the standard library alone: drop the four-byte `krc1` magic, XOR with a fixed 16-byte key, `zlib.decompress`, decode UTF-8. The result is `[start_ms,duration_ms]<offset,duration,flags>word…`, where word offsets are relative to the line — converted to the absolute timings `LyricWord` uses, which is the part that would look plausible and be wrong if it were missed, so a test pins it with a line that does not start at zero. The request falls back to `fmt=lrc` whenever KRC is missing, fails the magic check, fails to decompress, or yields no timed lines. Kugou already worked; this must not be able to make it worse. Verified against the live service: 成都 now returns 39 lines, all 39 with word timing, where before it returned 39 lines with none.
Kugou's lyric search answers 200/OK with an empty candidate list for a title it holds under a different keyword, and neither keyword form wins consistently. Measured over twelve tracks against the live endpoint: the title alone found candidates for 3, the title with the performer for 4, and the two together for 6. One extra request per track buys that recall; the ranking already merges candidates across keywords.
A word-timed Kugou hit is cached as base64 KRC, but parse_payload read only the LRC key. The cache deletes any row its parser cannot read, so every lookup of a word-timed track failed to parse, dropped the row, and refetched — the cache never served the path this feature added. Verified against a live fetch: 63 lines with word timing survive the round trip through the stored payload.
Zakkaus
force-pushed
the
feat/kugou-word-timing
branch
from
August 16, 2026 07:54
6315e35 to
880aecd
Compare
…r TLS Fetching and parsing shared one try block, so a network error left krc unbound while lines was set to (). The branch below then read krc: on the first candidate that raised UnboundLocalError, which no caller catches, and on any later one it silently reused the previous candidate's bytes. Measured by making download_krc raise aiohttp.ClientError, the whole lookup crashed rather than moving to the next candidate. The lyric endpoint serves the same request over https, so the song id and the lyrics it returns no longer travel in the clear.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The karaoke sweep highlights the current word, but only when the lyrics carry per-word timing. That came from exactly two places:
yrc.lrcSo the same song looks different depending on which source answered, and nothing tells the user why. Kugou is the interesting case, because it has the timing —
kugou.pywas asking its download endpoint forfmt=lrcand getting plain LRC.fmt=krcreturns KRC, which carries it.KRC, decoded with the standard library
krc1magiczlib.decompressThe result looks like:
[start_ms,duration_ms]for the line, then<offset,duration,flags>per word, where the offset is relative to the line start. Those are converted to the absolute timingsLyricWorduses. That conversion is the part that would look plausible and be wrong if it were missed, so a test pins it with a line that does not start at zero.No new dependency:
zlibandbase64are in the standard library, and the key is data, named and commented rather than obfuscated.Falls back, never worse
fmt=krcfirst,fmt=lrcwhenever KRC is absent, fails the magic check, fails to decompress, or yields no timed lines. Kugou already worked as a source; this change cannot make it worse than it was.Validation
QT_QPA_PLATFORM=offscreen uv run pytest— 471 passed, 6 skippeduv run ruff check .uv run ty checkTests build the KRC fixture by running the encoding backwards — zlib-compress, XOR, prepend the magic — so they exercise the real decode rather than a stub. No test makes a network call.
Checked against the live service, which is the point:
Before this change the same request returned 39 lines with none.
Follow-up: ask Kugou for the title with the performer as well
Fetching word timing is only worth as much as the search that finds the record, and Kugou's lyric search misses a lot. It answers
200 OKwith an empty candidate list for a title it holds under a different keyword, and neither keyword form wins consistently —海阔天空returns nothing alone and ten records withBeyondappended, while晴天is the other way round.Measured against the live endpoint over twelve tracks: the title alone found candidates for 3, the title with the performer for 4, the two together for 6. End to end,
夜に駆けるwent from no lyrics to 70 word-timed lines.The keyword list already flows into one ranking pass, so the second query costs one request and needs no other change. Kugou's own
singerfield stays out of matching, as before — it is the query that improves, not the identity check.