Rewrite CSV.jl internals for 1.0

[quinnj]

Summary

This PR replaces the CSV.jl parsing and writing internals for 1.0. A quote-aware structural index drives eager, lazy, row, chunk, transpose, and Tables.Scan reads. One column plan carries selection, types, missing-value intent, and field options. The writer renders bounded blocks in deterministic order. CSV keeps a small namespaced API and exports no names.

The rewrite uses released dependencies: Parsers 3, InlineStrings 2, Tables 1.14, DataStrings 1, and Durations 1.1. DataDecimals 1 is a weak dependency.

Implementation and behavior

• Eager tables own their text. File, read, and Chunks copy long cells into column-owned buffers. Chunk buffers are adopted by reference. A finished eager read releases the mapped input; changing a source byte vector or rewriting the file cannot change the table. Rows and lazy remain documented views.
• One structural index and column plan. Readers share quote handling, selection, field options, explicit type intent, and diagnostics. The vector scanner is selected automatically; fastindex=false retains the scalar reference path. Generic CPU checks cover both ARM and x86.
• Compatible field-start quoting. A quote inside an unquoted field remains content, such as Pipe 3" long. Indexing detects this case and rebuilds under the field-start rule. Fuzz tests cover dialects and chunk boundaries.
• Visible, structured problems. Eager readers retain CSV.problems(file) and print one summary warning by default. on_error=:collect records silently; :error throws CSV.ParseError. Chunks warns once, and Rows keeps :collect.
• Explicit inference and missing-value contracts. Timestamps infer with nanosecond precision, widening to microsecond precision when needed for range. types=DateTime and typemap remain available. Declared Union{Missing,T} string requests retain their union. Pooling is opt-in; unquoted empty fields are missing. Decimal types are explicit and require exact parsing.
• Shared writer kernels. Column descriptors feed one row loop, with uncommon types staged once per block. IO sinks start at their current position; writing CSV.Chunks streams under one header. RowWriter, transforms, custom quoting, date formats, and float formats preserve their existing contracts.
• Faster string, Time, and float output. String and DataString bytes use one quote/escape policy with direct access to contiguous storage. A shared clock renderer handles Time and Timestamp without a temporary String per Time cell. A bounded exact-fraction shortcut handles short binary fractions; other floats still use Ryu. The default float notation decision no longer computes a redundant floating remainder.
• Less duplicate code. Pointer-backed strings and byte vectors share the quote policy. Interior field starts no longer test for a row-end event that cannot occur there. These cleanups remove 31 lines relative to the separately committed writer optimization.
• Executable documentation. Reading, writing, examples, migration, release notes, and public docstrings describe the ownership and option contracts. Examples and doctests run in the strict documentation build.

The migration guide lists the changed and removed 0.10 options. Julia 1.10 remains the minimum supported runtime.

Performance

The release benchmark pass profiles CPU and allocations for all twelve approximately 200 MiB read inputs and all seven writer inputs, each at one and eight tasks: 38 cases. The final comparison loads each revision as a normal package in two balanced fresh-process pairs, with five warm calls per case per process. The table reports the geometric mean of the paired speed ratios for ce58a6a → 35bf7d8, on an Apple M3 Max and Julia 1.12.6. Higher is faster.

Writer input	1 task	8 tasks
Numeric	1.19×	1.15×
Mixed	1.22×	1.17×
Strings	1.26×	1.32×
Quoted	1.37×	1.58×
Temporal	3.24×	4.21×
Wide	2.23×	2.02×
Longtext	1.26×	1.03×

Temporal writing removes about 1.25 GB of temporary allocation at either task count. Read timings range from 0.99× to 1.03×; no material reader gain is claimed. An unrelated one-core workload remained active, so small differences, including the threaded long-text result, are not treated as established gains.

The writer optimization is isolated in 23e7e5e; 35bf7d8 contains only the cleanup. The hosted cleanup-only comparison on AMD EPYC / Julia 1.13 covers 23 cases over four rounds: 22 minimum-run times are within 5%, and DataString writing takes 6.3% less time. Allocations are unchanged and all fingerprints match. This supports the code reduction, not a broad new throughput claim for the cleanup itself.

Exploratory in-process prototypes used a nested candidate module. A same-source control exposed substantial loading-mode bias on some small cases, so those timings are not used for the final table. The general-float check covers uniform, normal, and broad-exponent random values; a larger custom-date check matches all values and stays within 1% at both task counts.

The investigation rejected general writer-buffer reuse, an extra reader reservation pass, explicit string-capacity state, extra quote-scanner branches, and bulk-copy escaping. Their isolated wins came with throughput or memory regressions elsewhere. Worker scheduling and string ownership remain unchanged.

Validation

Validated source: 35bf7d8.

• Julia 1.10.11 full checked-bounds suite: 330,576 / 330,576.
• Julia 1.13.0 full checked-bounds suite: 331,022 / 331,022.
• Aqua: 11 / 11. Trim compilation and execution: 5 / 5. Generic ARM CPU dispatch: 4 / 4.
• Strict Documenter build passed, including doctests, public-doc checks, cross-references, and rendering.
• All 24 read comparisons matched schemas and every column value. All 14 writer comparisons matched SHA-256 output hashes.
• Writer parity covers every Float16 bit pattern, randomized Float32/Float64 values, fraction limits and neighboring floats, Unicode and quote boundaries, missing values, 10,000 generated Time values, custom formats, transforms, RowWriter, and threaded writes.
• Both revisions completed all 130 public API benchmark cases twice; eighteen outliers and controls received focused follow-up checks. A nine-process missing-heavy check matched all bytes and placed the final median within 3% of the prior PR at one and eight tasks.
• CI run 34971986741: all twelve substantive jobs passed on 35bf7d8, including Linux ARM/x86/x64, Windows, macOS ARM, Julia 1.10/current/nightly, trim, quality, and docs. The manual performance job is intentionally skipped on PR CI.
• Hosted performance run 34972026888: passed, with an empty value/output fingerprint diff and unchanged allocations across the 23 selected cases.

Gates before a 1.0 release

• Maintainer review of the rewritten API and implementation.
• PkgEval and reverse-dependency review beyond the existing DataFrames probes. Packages bounded to CSV 0.10 will not select 1.0 automatically.

The version is already 1.0.0. This PR does not tag or register CSV.

Contribution disclosure

Claude authored the rewrite and review fixes. Codex contributed review fixes, latency and performance work, adversarial tests, benchmark tooling, CPU guards, and release validation.

Co-authored by Codex