perf: reduce peak memory on string categoricals

[arthur-priorlabs]

Issue

RES-2592: optimize TabPFNEnsemblePreprocessor.fit_transform_ensemble_members_iterator RAM usage — stacked on #1186, which left the half-categorical case as the one mix it could not help.

Motivation and Context

What

• Two more rebuilds of the feature matrix removed, both on the path a table with categorical columns takes.
• Takes the half-string mix from 3.33 GB to 2.80 GB of transient RSS, and 4.40 GB -> 2.80 GB cumulatively against main.

Why

• perf: remove redundant copies in ensemble preprocessing #1186 skips the reshape rebuild only when the column order is untouched, and the encoder rebuild only when nothing is categorical. A half-categorical table meets neither condition, so it kept paying three full-size arrays where the numeric mixes now pay one.
• That mix is not a corner case: it is what any table with a categorical column looks like to this step.

How

• Reshape: when the transform is the identity and every input column is used exactly once, take the permutation in one gather instead of a ColumnTransformer's block-plus-hstack.
• Encode: write the codes and the passthrough columns straight into one preallocated output, the pattern clean_data already uses in its own _encode_into_preallocated.

Both stages had to go: they were tied at the peak, so removing either alone moved nothing. Measured at 66,667 x 400 with the profiler, this step's transient goes 0.422 GB -> 0.213 GB for the reshape and 0.427 GB -> 0.318 GB for the encoder.

	numeric	half-string	numeric-object
input	10.67 GB (666,667 x 2,000)	1.07 GB (333,333 x 400)	1.07 GB (333,333 x 400)
categoricals	0 of 2,000	200 of 400	0 of 400
transient RSS, highest	12.00 -> 12.00 GB (+0.0%)	3.33 -> 2.80 GB (-16.0%)	1.07 -> 1.07 GB (+0.0%)
transient RSS, lowest-direct	12.00 -> 12.00 GB (+0.0%)	2.67 -> 2.67 GB (+0.0%)	1.07 -> 1.07 GB (+0.0%)
median wall, highest	11.015 -> 10.980 s (-0.3%)	6.698 -> 6.741 s (+0.6%)	0.695 -> 0.683 s (-1.7%)
median wall, lowest-direct	10.976 -> 10.859 s (-1.1%)	13.684 -> 13.975 s (+2.1%)	0.696 -> 0.686 s (-1.5%)

Six complete A/B comparisons against this PR's base, run by scripts/bench_matrix.py on dedicated cpuhighmem16spot nodes, with the ensemble members compared cell by cell: bit-identical in all six. Only the half-string RSS cell is outside the ±3% this benchmark carries; every other number here says "unchanged", including all six wall times.

Two results worth reading before the diff

Nothing is won on the dependency floor. On python 3.10 / numpy 1.22 / sklearn 1.2 the base already peaks at 2.67 GB where current libraries peak at 3.33 GB, and this change leaves the floor exactly where it was. Profiling both environments at a shape small enough to profile (66,667 x 400) shows identical stage ladders, so the floor's cheaper baseline only appears at the full shape and I did not chase the cause further. What the table does establish is that the floor is not made worse.

Wall time is a wash, and the gate is stricter than the measurement. The two half-string cells read +0.6% and +2.1%; a nine-repeat run of the same comparison put the medians 2.3% apart while the fastest repeats were 0.3% apart, with the spread inside one run reaching 8.9%. The assembly is doing slightly more work than sklearn's two block copies, but I could not measure it above the noise. The benchmark's zero-tolerance default therefore rejects these cells; --tolerance 0.03 passes them.

What remains on this mix is one gather output, one assembled output and the block of codes, so ~2.5x the table it returns. Going below that means letting the encoder write into its input in place — safe inside the pipeline, which copies once up front precisely so steps have a private array, but a change to the mutation contract of a step that can also be called directly. Deliberately left out of this PR.

---

Public API Changes

• No Public API changes
• Yes, Public API changes (Details below)

---

How Has This Been Tested?

suite	result
tests/test_preprocessing, tests/test_torch_preprocessing	772 passed
A/B vs this PR's base: 3 mixes x 2 dependency sets, plus --no-gpu-preprocessing	members bit-identical, no RSS regression

• New: _encode_into_preallocated is pinned against sklearn as the oracle, over nine input shapes — values, dtype and memory layout, compared with a ColumnTransformer fitted the ordinary way. Two more tests cover the step end to end and the one-row fit the assembly relies on.
• That layout assertion earned its place immediately: the first draft of this branch pinned the output to Fortran order, and the test caught that hstack returns row-major for two real blocks and Fortran order only when every block it stacks already is. A layout drift here is not cosmetic — the sklearn SVD downstream converges to a different basis on a C- than on a Fortran-contiguous input, which is exactly what perf: remove redundant copies in ensemble preprocessing #1186's _pin_layout exists to stop.

---

Checklist

• The changes have been tested locally.
• Documentation has been updated (if the public API or usage changes).
• A changelog entry has been added (see changelog/README.md), or "no changelog needed" label requested.
• The code follows the project's style guidelines.
• I have considered the impact of these changes on the public API.

---

[arthur-priorlabs]

This change is part of the following stack:

• perf: remove redundant copies in ensemble preprocessing #1186
  • perf: reduce peak memory on string categoricals #1187 ◀

_{Change managed by git-spice.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit a681d94. Configure here.}

[cursor]

Empty remainder promotes output dtype

Medium Severity

_encode_into_preallocated always sets the output dtype to np.result_type(codes.dtype, X.dtype), including when every column is categorical and there is no passthrough block. A ColumnTransformer with empty remainder returns the encoder output as-is, so its dtype is only codes.dtype. For an object-dtyped all-categorical table that becomes object instead of float, which can break numeric steps that follow.

 

^{Reviewed by Cursor Bugbot for commit a681d94. Configure here.}