Integrate StructUtils.jl for typed struct round-tripping

[stephenberry]

StructUtils.jl Integration for Round-Tripping Julia Structs

Closes #235

Summary

MAT.jl can write arbitrary Julia structs to .mat files, but always reads them back as Dict{String, Any}. This PR integrates StructUtils.jl (the same engine behind JSON.jl's typed parsing) to enable opt-in typed reads and improved writes. The untyped matread(filename) behavior is unchanged.

New API

struct Person
    name::String
    age::Int
end

matwrite("data.mat", Dict("p" => Person("Alice", 30)))
matread("data.mat", "p", Person)   # Person("Alice", 30)

A typed read(handle, varname, T) is available on matopen handles for all three formats.

Because the struct write path now goes through StructUtils, its macros are re-exported and work with using MAT: @defaults (default field values), @tags (renaming and ignoring fields), @choosetype (abstract type dispatch), @noarg and @kwarg. Types MATLAB has no representation for are handled by extending StructUtils.lower on write and StructUtils.lift on read. The README and manual document all of this with examples.

Design Notes

These are the decisions a reader cannot get from the diff.

Array targets are constructed by MAT.jl, not by StructUtils' generic source iteration, which does not fit MAT data: a value read from a file is already a fully-typed Julia object, MATLAB has no scalars (a 1x1 array reads back as a scalar), MAT arrays are flat and carry their own shape (StructUtils infers dimensions by descending into nested sources), and the stored element type rarely matches the target. make(::MATReadStyle, ::Type{<:AbstractArray}, x) therefore converts element type and shape explicitly: a scalar fills a one-element array target, single/int32/logical/cell sources fill a Matrix{Float64}, trailing singleton dimensions are padded or dropped, a row or column vector fills a Vector, and a shape that cannot be matched raises DimensionMismatch rather than being silently flattened.

A one-element MatlabStructArray is unwrapped in prepare_source, ahead of dispatch, for a whole variable and for each element of an array being built, rather than only by a make method general in the target type: such a method is ambiguous with the per-type methods @choosetype generates, so unwrapping first is what keeps matread(file, var, SomeAbstractType) and Vector{SomeAbstractType} targets working against struct arrays. Two cases remain that MAT.jl cannot intercept, because StructUtils dispatches to @choosetype's method first; both are documented: a struct field of the abstract type reaches the chooser as a raw MatlabStructArray, and struct arrays stored as struct values cannot be read into Dict{String,SomeAbstractType}. Struct arrays are otherwise expanded to the equivalent array of Dicts at the point of construction, so they work as a nested field as well as a top-level variable, and can fill any collection target (Vector{T}, Matrix{T}, Set{T}, Tuple{...}, a narrowed Union); a struct array fills a single struct only when it holds exactly one element.

Four MAT-specific StructUtils hooks:

• structlike(::MATStyle, ::Type{<:Complex}) = false, because MATLAB stores complex numbers natively: they are leaf values, not two-field structs to rebuild from re/im.
• nulllike(::MATStyle, x::AbstractArray), because MATLAB spells "no value" as the empty matrix [], so an empty array narrows a Union{Nothing,T}/Union{Missing,T} field, and nothing/missing are written back as []. The ambiguity is inherent to the format: a genuinely empty numeric array in such a field is indistinguishable from a missing value. Empty character data is excluded, since that is how "" is stored and narrowing it would discard a value the file holds. v4 files store '' and [] identically, with element type Union{}, so there both narrow.
• lift from an empty character array to "" for AbstractString targets, the counterpart of the AbstractChar lift for one-character strings. Without this a String field holding "" cannot be read back at all. Empty numeric data is rejected rather than read as "".
• lift from a one-element numeric array for Number targets, the counterpart of a scalar filling a one-element array target. v4 files read a scalar back as a 1x1 matrix, so without this matread(file, "x", Float64) fails on v4.

On write, lower is applied at a single choke point (m_write_value), which every top-level variable, Dict value and cell or struct-array element passes through, so one overload covers a type wherever it appears. Struct fields are lowered by applyeach instead and so never pass through it, which is what keeps them from being lowered twice.

NamedTuple keeps its own write method rather than falling into the generic struct path, because a NamedTuple of vectors is a Tables.jl table and would otherwise be rejected by that path's "writing tables is not yet supported" check.

MATStyle <: StructUtils.StructStyle sets fieldtagkey = :mat, following JSON.jl's pattern, so tags can be namespaced per library:

@tags struct MyData
    x::Float64 &(mat=(name="matlab_x",), json=(name="json_x",))
end

Exports: the StructUtils macros above, the StructUtils module itself (@choosetype generates code referencing StructUtils.make, and extensions are written as qualified method definitions), the style types MATStyle/MATReadStyle/MATWriteStyle, and MATConstructionError. lower and lift are deliberately not exported: a function cannot be extended through an unqualified imported name, so an exported binding would not help.

Files Changed

File	Changes
Project.toml	Added StructUtils dependency (compat 2.5.1), bumped Julia compat 1.6 -> 1.9
.github/workflows/CI.yml	Test matrix 1.6 -> 1.9, matching the new Julia compat
src/MAT_types.jl	Defined MATStyle, MATReadStyle, MATWriteStyle, MATConstructionError; construct_from_raw helper; MAT-aware array construction; structlike/nulllike/lift hooks
src/MAT.jl	Added matread(filename, varname, T), re-exported StructUtils macros, styles and error type
src/MAT_HDF5.jl	Added read(f, name, T); CompositeKind write via applyeach; lower applied at every value entry point; nothing/missing written as []
src/MAT_v5.jl	Added read(matfile, varname, T)
src/MAT_v4.jl	Added read(matfile, varname, T); lower applied on write; nothing/missing written as []
README.md, docs/src/index.md	Documented typed reads, StructUtils macros, conversion rules and lower/lift extension
test/structutils.jl	New test file
test/runtests.jl	Added include("structutils.jl")

Breaking Changes

Julia compat bumped from 1.6 to 1.9, required by StructUtils.jl. Julia 1.6 is past its support lifecycle. This warrants a minor version bump; the PR leaves version untouched so the release is yours to time.

Bug Fixes

• isbits structs can now be written. The old isbits(s) guard in the CompositeKind write path rejected valid user structs whose fields happen to all be primitive (e.g. struct Point; x::Float64; y::Float64; end) with "This is the write function for CompositeKind, but the input doesn't fit". The guard is now !StructUtils.structlike(MATWriteStyle(), T), which accepts user structs and rejects non-struct types. No existing type changes dispatch path: the specific m_write methods (Symbol, NamedTuple, Dict, MatlabOpaque, FunctionHandle, ...) are more specific and still match first.
• Single-character string fields round-trip. MAT.jl reads a one-character string back as a Char, since MATLAB has no scalar char type distinct from a 1x1 char array; it is now lifted for AbstractString fields.
• Empty string fields round-trip. MAT.jl reads "" back as an empty array, which no String field could previously accept.
• nothing and missing can be written, as MATLAB's [].
• Clearer write errors: the generic write path reports cannot write a value of type `T` to a MAT file.

Future Work

• Performance: typed reads currently build a Dict first, then construct the target type. A future optimization could construct directly from HDF5.
• Columnar struct-array reads: struct arrays are expanded to Array{Dict{String,Any}} before constructing typed elements. Defining StructUtils.applyeach for MatlabStructArray would construct T elements from columns without the intermediate Dicts.

Test Plan

test/structutils.jl adds 45 testsets covering typed reads of structs, nested structs and struct arrays; each StructUtils macro; array element-type and shape conversion, including the mismatch errors; Union narrowing on [] and the empty-string cases above; the lower/lift extension pattern; write-path type handling; and the v4, v5, v7.3 and compressed formats. Backward compatibility is covered by the pre-existing suite, which is unchanged.

The full suite (768 tests) passes on Julia 1.9.4 with StructUtils 2.5.1 (the declared minimums) and on Julia 1.12.7 with StructUtils 2.9.1 (current). 2.5.1 is the compat floor because Set targets fail on 2.5.0, where StructUtils.initialize for AbstractSet is missing.

[ViralBShah]

@stephenberry Is it possible to fix the failing invalidations CI?

[ViralBShah]

@mkitti Would you be the right person to review and merge?

[mkitti]

I can review

[mkitti]

I'm also suggesting @matthijscox for reviewer since they participated in the original issue.

[matthijscox]

well this is quite a large PR, will have to schedule some review time soon

[Copilot]

Copilot review overview

🟡 Changes recommended

Three moderate issues remain in typed conversion and dispatch behavior.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

• Restrict empty-array string lifting to character arrays · New

What changed in this PR

Integrates StructUtils.jl to add opt-in typed MAT reads and improved struct round-tripping.

Changes:

• Adds typed matread/read APIs across MAT formats.
• Adds MAT-specific StructUtils styles, conversions, and macros.
• Updates documentation, CI, dependencies, and tests.

File	Summary
test/​structutils.jl	Adds StructUtils integration tests.
test/​runtests.jl	Includes the new test suite.
src/​MAT.jl	Exposes typed reads and StructUtils APIs.
src/​MAT_v5.jl	Adds typed v5 reads.
src/​MAT_v4.jl	Adds typed v4 reads and lowering support. Moderate finding (1 vote): scalar targets do not normalize 1×1 numeric variables.
src/​MAT_types.jl	Implements typed construction and MAT-specific hooks. Moderate findings: empty numeric arrays can lift to strings (2 votes); nested MatlabStructArray dispatch remains ambiguous for abstract targets (1 vote).
src/​MAT_HDF5.jl	Adds typed reads and StructUtils-aware writing.
README.md	Documents typed struct round-tripping.
Project.toml	Adds StructUtils and updates Julia compatibility.
docs/​src/​index.md	Adds user-facing documentation.
.github/​workflows/​CI.yml	Updates the Julia CI matrix.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[matthijscox]

@stephenberry Is it possible to fix the failing invalidations CI?

I see the same number of failing invalidations on my simple PR #251 . I think it's due to the upgrade to Julia 1.13.

Though I just ran it locally and it says 678 invalidations on 1.12, and here on 1.13 it says 161 invalidations. So not sure why it considers the PRs to increased the number of invalidations.

Edit: it compares to default branch. It says our new PRs have 161 invalidations and the main branch has 160. I doubt my PR added an extra invalidation, so I'm tempted to ignore this jump in 1 invalidation.

[stephenberry]

@ViralBShah The Invalidations failure is a measurement artifact rather than a new invalidation; #251 shows the same 160 -> 161. Measuring both branches warm under equal conditions, the counts match (159 vs 159 on Julia 1.13, 429 vs 429 on 1.12).

I've pushed an update (squashed) addressing the review:

• An empty numeric [] no longer reads into a String field as "".
• v4 1x1 matrices now fill scalar targets, and v4 [] narrows to nothing as documented.
• @choosetype types now work as the element type of a cell array of struct arrays.
• Docstrings for MATStyle, MATReadStyle and MATWriteStyle.
• Two remaining @choosetype edge cases with struct arrays are documented (see the description).