feat(rfc_tools): enforce Git LFS tracking for media assets - #15
Conversation
There was a problem hiding this comment.
cleanup: sharing logging and process runner across other binaries, testing.
There was a problem hiding this comment.
cleanup for process runner sharing
There was a problem hiding this comment.
this is the source of truth for what should be placed into gitlfs.
6937ecb to
3d065b6
Compare
| - name: Checkout Code | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| fetch-depth: 0 |
There was a problem hiding this comment.
dropped any magic around fetch-depth because it was breaking assumptions. this isn't going to be a big tree, so silly optimization gone.
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| fetch-depth: ${{ github.event_name == 'pull_request' && 0 || 1 }} | ||
| fetch-depth: 0 |
There was a problem hiding this comment.
ditto: we need the full checkout.
5d99775 to
c1bd7b5
Compare
There was a problem hiding this comment.
I know what you're about to say "Codefu, this looks like you're adding an image" but I'm not:
❯ git cat-file -p HEAD:rfc/media/flutter_header.png
version https://git-lfs.github.com/spec/v1
oid sha256:9a14b0db34e1622cd3b2120f2661caaea7319992d1d15e10848764f653e8fc1e
size 6749
❯ git lfs ls-files
9a14b0db34 * rfc/media/flutter_header.pngWhich is exactly why I wanted this action.
| /// Verifier that enforces Git LFS pointer consistency for repository trees | ||
| /// using `git lfs fsck --pointers`. | ||
| /// | ||
| /// Rather than maintaining a duplicate whitelist of file extensions in Dart, |
There was a problem hiding this comment.
.gitattributes seems to contain both an allowlist of file name extensions that are presumed to be text, and a denylist of file name extensions that are presumed to be binary. Does the LfsVerifier fail if a file is not on the allowlist? Or does it fail if a file is on the denylist?
Either way, instead of using file name extensions, can you just ask grep instead, and if grep says that it's a binary file it has to be in lfs?
There was a problem hiding this comment.
.gitattributes is more of an enforcement about what we require in lfs, not an allowlist vs denylist!
Does the LfsVerifier fail if a file is not on the allowlist
Correct, if you write "foo.xyz" and its 10mb, I'm relying on the reviewers to say "Hey, don't do that".
can you just ask
grepinstead
Works if you're on a linux or mac machine, doesn't work if you are on a windows machine. Wouldn't be a bad thing if it was in the workflow file, but this is a dart util. Grep is still going to be brittle because I think its only using some simple heuristics (does the file contain NUL bytes or maybe some control characters...) or we could use file --mime-type <file>.
I'm not banning binary files. I'm requesting they are put in LFS. If someone tries to add a base64 encoded png to a document or file.
RFC media assets (such as screenshots) must be tracked via Git LFS. Committed raw binary blobs permanently bloat Git history—even if deleted in subsequent commits. This change introduces automated verification tooling and CI enforcement: - Configures case-insensitive Git LFS tracking in .gitattributes for media, fonts, archives, and binary artifacts. - Adds .github/CODEOWNERS gating .gitattributes and CI workflows. - Adds LfsVerifier and bin/lfs_verify.dart to ensure media files are tracked with 'filter=lfs' and backed by valid Git LFS pointers. - Uses 'git lfs fsck --pointers' directly against Git revision trees, eliminating false positives from smudged working-copy binaries and avoiding temp-file disk I/O. - Supports PR diff auditing (--base-branch) and full-tree auditing at HEAD, with an opt-in --audit-intermediate-commits flag for non-squash workflows. - Adds an automated GitHub Actions workflow (.github/workflows/lfs-verify.yml) reporting failures via inline PR annotations. - Adds comprehensive unit tests and real-git end-to-end tests.

RFC media assets (such as screenshots) must be tracked via Git LFS (Large File Storage). Committed raw binary blobs permanently bloat Git history, even when deleted in subsequent commits.
.gitattributesusing case-insensitive bracket globs (*.[pP][nN][gG], etc.), mapped tofilter=lfs.git lfs fsck --pointersdirectly against Git revision trees in the object database, eliminating false positives from smudged working-copy binaries and avoiding temp-file disk I/O.lfs-verify.ymlwith inline GitHub Actions error annotations (::error file=...::) and remediation instructions.Flow:
flowchart TD A["bin/lfs_verify.dart"] --> B["LfsVerifier.verify()"] B --> C{"Check Prerequisites"} C -->|"git-lfs missing"| Err["Fail Fast: Exit 1<br/>(Prompt 'git lfs install')"] C -->|"git-lfs present"| D{"Execution Mode"} D -->|"--base-branch (PR Mode)"| E["verifyPullRequest()"] D -->|"Default (Trunk Mode)"| F["verifyTrackedFiles()"] E --> G["git diff -z --name-only<br/>(Extracts PR-modified paths)"] G --> J{"Audit Scope"} F --> M["git lfs fsck --pointers HEAD<br/>(All tracked files)"] J -->|"Default PR Mode (Squash)"| K["git lfs fsck --pointers HEAD<br/>(Scoped to PR-modified files)"] J -->|"--audit-intermediate-commits"| L["git lfs fsck --pointers mergeBase..HEAD<br/>(Scoped to PR-modified files)"] K & L & M --> N["parseFsckOutput()<br/>(Extracts file path & diagnostic detail)"] N --> O["LfsVerificationResult<br/>(isSuccess, issues, checkedCount)"] O -->|"Issues Detected"| P["GitHub Actions Annotations<br/>::error file=...::<br/>& Actionable Remediation Guidance"] O -->|"Zero Issues"| Q["Exit 0: Verification Passed"]