Skip to content

feat(rfc_tools): enforce Git LFS tracking for media assets - #15

Merged
jtmcdole merged 2 commits into
ci_cleanupsfrom
git_lfs
Sep 17, 2026
Merged

jtmcdole merged 2 commits into
ci_cleanupsfrom
git_lfs

Conversation

@jtmcdole

@jtmcdole jtmcdole commented Sep 13, 2026

Copy link
Copy Markdown
Member

RFC media assets (such as screenshots) must be tracked via Git LFS (Large File Storage). Committed raw binary blobs permanently bloat Git history, even when deleted in subsequent commits.

  • Added Git LFS Configuration via .gitattributes using case-insensitive bracket globs (*.[pP][nN][gG], etc.), mapped to filter=lfs.
  • When users checkout the LFS files (smudge), this check uses git lfs fsck --pointers directly against Git revision trees in the object database, eliminating false positives from smudged working-copy binaries and avoiding temp-file disk I/O.
  • lfs-verify.yml with inline GitHub Actions error annotations (::error file=...::) and remediation instructions.

Flow:

flowchart TD
    A["bin/lfs_verify.dart"] --> B["LfsVerifier.verify()"]
    B --> C{"Check Prerequisites"}
    C -->|"git-lfs missing"| Err["Fail Fast: Exit 1<br/>(Prompt 'git lfs install')"]
    C -->|"git-lfs present"| D{"Execution Mode"}

    D -->|"--base-branch (PR Mode)"| E["verifyPullRequest()"]
    D -->|"Default (Trunk Mode)"| F["verifyTrackedFiles()"]

    E --> G["git diff -z --name-only<br/>(Extracts PR-modified paths)"]
    G --> J{"Audit Scope"}
    F --> M["git lfs fsck --pointers HEAD<br/>(All tracked files)"]

    J -->|"Default PR Mode (Squash)"| K["git lfs fsck --pointers HEAD<br/>(Scoped to PR-modified files)"]
    J -->|"--audit-intermediate-commits"| L["git lfs fsck --pointers mergeBase..HEAD<br/>(Scoped to PR-modified files)"]

    K & L & M --> N["parseFsckOutput()<br/>(Extracts file path & diagnostic detail)"]
    N --> O["LfsVerificationResult<br/>(isSuccess, issues, checkedCount)"]
    O -->|"Issues Detected"| P["GitHub Actions Annotations<br/>::error file=...::<br/>& Actionable Remediation Guidance"]
    O -->|"Zero Issues"| Q["Exit 0: Verification Passed"]
Loading

Comment thread lib/src/git_lister.dart

@jtmcdole jtmcdole Sep 13, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cleanup: sharing logging and process runner across other binaries, testing.

Comment thread lib/src/assigner.dart

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cleanup for process runner sharing

Comment thread .gitattributes

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the source of truth for what should be placed into gitlfs.

@jtmcdole
jtmcdole force-pushed the git_lfs branch 2 times, most recently from 6937ecb to 3d065b6 Compare September 13, 2026 16:13
- name: Checkout Code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dropped any magic around fetch-depth because it was breaking assumptions. this isn't going to be a big tree, so silly optimization gone.

uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: ${{ github.event_name == 'pull_request' && 0 || 1 }}
fetch-depth: 0

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto: we need the full checkout.

@flutteractionsbot
flutteractionsbot changed the base branch from main to ci_cleanups September 13, 2026 16:45
@flutteractionsbot
flutteractionsbot added this pull request to stack #17 September 13, 2026 16:45
@jtmcdole
jtmcdole removed this pull request from stack #17 September 13, 2026 16:52
@jtmcdole
jtmcdole added this pull request to stack #19 September 13, 2026 16:52
@jtmcdole
jtmcdole force-pushed the git_lfs branch 4 times, most recently from 5d99775 to c1bd7b5 Compare September 15, 2026 20:43

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know what you're about to say "Codefu, this looks like you're adding an image" but I'm not:

❯ git cat-file -p HEAD:rfc/media/flutter_header.png
version https://git-lfs.github.com/spec/v1
oid sha256:9a14b0db34e1622cd3b2120f2661caaea7319992d1d15e10848764f653e8fc1e
size 6749

❯ git lfs ls-files
9a14b0db34 * rfc/media/flutter_header.png

Which is exactly why I wanted this action.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also

Image

@jtmcdole
jtmcdole requested a review from zanderso September 15, 2026 20:49
@jtmcdole
jtmcdole marked this pull request as ready for review September 15, 2026 20:49

@eyebrowsoffire eyebrowsoffire left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment thread lib/src/lfs_verifier.dart Outdated
Comment thread lib/src/lfs_verifier.dart Outdated
/// Verifier that enforces Git LFS pointer consistency for repository trees
/// using `git lfs fsck --pointers`.
///
/// Rather than maintaining a duplicate whitelist of file extensions in Dart,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.gitattributes seems to contain both an allowlist of file name extensions that are presumed to be text, and a denylist of file name extensions that are presumed to be binary. Does the LfsVerifier fail if a file is not on the allowlist? Or does it fail if a file is on the denylist?

Either way, instead of using file name extensions, can you just ask grep instead, and if grep says that it's a binary file it has to be in lfs?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.gitattributes is more of an enforcement about what we require in lfs, not an allowlist vs denylist!

Does the LfsVerifier fail if a file is not on the allowlist

Correct, if you write "foo.xyz" and its 10mb, I'm relying on the reviewers to say "Hey, don't do that".

can you just ask grep instead

Works if you're on a linux or mac machine, doesn't work if you are on a windows machine. Wouldn't be a bad thing if it was in the workflow file, but this is a dart util. Grep is still going to be brittle because I think its only using some simple heuristics (does the file contain NUL bytes or maybe some control characters...) or we could use file --mime-type <file>.

I'm not banning binary files. I'm requesting they are put in LFS. If someone tries to add a base64 encoded png to a document or file.

RFC media assets (such as screenshots) must be tracked via
Git LFS. Committed raw binary blobs permanently bloat Git history—even if
deleted in subsequent commits.

This change introduces automated verification tooling and CI enforcement:
- Configures case-insensitive Git LFS tracking in .gitattributes for media,
  fonts, archives, and binary artifacts.
- Adds .github/CODEOWNERS gating .gitattributes and CI workflows.
- Adds LfsVerifier and bin/lfs_verify.dart to ensure media files are
  tracked with 'filter=lfs' and backed by valid Git LFS pointers.
- Uses 'git lfs fsck --pointers' directly against Git revision trees,
  eliminating false positives from smudged working-copy binaries and
  avoiding temp-file disk I/O.
- Supports PR diff auditing (--base-branch) and full-tree auditing at
  HEAD, with an opt-in --audit-intermediate-commits flag for non-squash
  workflows.
- Adds an automated GitHub Actions workflow (.github/workflows/lfs-verify.yml)
  reporting failures via inline PR annotations.
- Adds comprehensive unit tests and real-git end-to-end tests.
@jtmcdole
jtmcdole added this pull request to the merge queue Sep 17, 2026
Merged via the queue into main with commit 92afbf0 Sep 17, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants