Count /#refs# only when the file is opened for writing - #257
Open
Thenewmanator15 wants to merge 1 commit into
Open
Thenewmanator15 wants to merge 1 commit into
Thenewmanator15 wants to merge 1 commit into
Conversation
matopen counted the entries of /#refs# to number the references a write adds. A read-only open never writes, and /#refs# holds every cell element and object value in the file: 100,007 entries in a 1.6 GB acquisition file, where counting them took 1.3 s of a 2.6 s first open (the group's index is walked from disk). Opens for writing still count, so appended references keep unique names. Tests: a read-only open leaves the count alone and an "r+" open takes it; cells and a struct with a cell field appended, in one session and in a second, to a file written by MAT.jl (numbered references, where a reused name would collide), one written by MATLAB, and one with no /#refs#: existing variables stay intact and the appended ones read back as a plain write and read gives them. Skipping the count for "r+" as well fails all three.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
matopencounts the entries of/#refs#(fid.refcounter = length(g) - 1) so that a write can name the references it adds. A read-only open never writes, so this counts only when the file is opened for writing. MATLAB puts every cell element and object value in/#refs#: an acquisition file of 1.6 GB here has 100,007 entries, and counting them on the first touch of the file took 1.3 s (the group's index is walked from disk).Tests. A read-only open leaves the count alone and an
"r+"open still takes it. A cell and a struct with a cell field are appended, in one"r+"session and then a second, to three kinds of file: one written by MAT.jl (numbered references, where a name reused because the count was skipped would collide), one written by MATLAB (cell.mat, lettered references), and one with no/#refs#(simple.mat). In each, the variables already there read back unchanged and the appended ones read back as a plainmatwrite+matreadgives them. There was no"r+"test before; skipping the count for"r+"as well fails all three cases. Full suite passes (593).To be plain about the benefit: on the one unread file of that kind I could still test, a cold
matopenwith this change was no faster (4.05 s, against 3.4 s for a byte-identical copy onmaster): the first read of/#refs#metadata costs the same whichever step makes it, and on warm files the difference is within noise. So this removes work a read-only open never needs rather than delivering a measured speed-up; please close it if that isn't worth a change here.🤖 Generated with Claude Code