Add reviewed Linguist corpus conventions - #2
Merged
Merged
Conversation
andrew
added this pull request to stack #3
September 14, 2026 15:07
andrew
force-pushed
the
corpus/linguist-conventions
branch
from
September 14, 2026 15:16
6f1300a to
49c359e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add 52 rules adapted from GitHub Linguist, bringing the corpus to 254 rules. New coverage includes third-party directory spellings, CocoaPods and Yarn dependency trees, citation/manual documents, named minified files and source maps, .NET designer code, build wrappers and configuration files. The 128 new corpus examples cover nested packages, mixed case, inherited roles and misleading names.
The source audit used Linguist revision
16cac170f73c97eb4e4329d66fcb341ef25ef4f0: vendor patterns, documentation patterns and generated-file detection, including their callers and relevant tests. NOTICE records the revision, adaptations and MIT attribution.Linguist groups several conventions differently because its categories control file filtering. This import assigns roles by repository purpose:
Pods/is vendor,app.min.jsis generated,gradlewis build, and.gitattributesis configuration.testdata/retains test/fixture roles. Vendor regexes are narrowed to complete components;Carthage/Checkoutsand selected Yarn trees avoid classifying all adjacent project metadata as vendor. Documentation directories work inside monorepo packages, with document extensions restricted to avoid matching commands such as bare lowercaseinstall.Content-dependent generator detection, attribute overrides and broad library basename patterns remain outside this import. A filename such as
jquery.js, a plain.d.ts, or packaged data underinst/extdata/provides insufficient evidence for the added roles. Arbitrary.mapfiles and generator headers need bounded content inspection with separate spoofing and truncation cases. All manifest and lockfile names continue to come from git-pkgs/manifests.Add case-insensitive suffix matching for designer code and named source maps while retaining literal case rules for existing suffixes. Index directory-path rules by their final component to limit the cost of corpus growth. On the same machine, the million-path benchmark measured 253–258 ms before the additions, 307–308 ms with the initial implementation, and 270–277 ms after indexing, with zero allocations for label matching. These measurements exclude filesystem and blob I/O.
Allow CI on PRs targeting branches other than
mainso this stacked change receives its own checks. README examples and corpus documentation describe the added conventions and their limits.