Sum the total on the axis it is plotted against

[gustavdelius]

Stacked on #539, which is stacked on #538 — this PR's diff is against resource-length-params.

Summary

A total can only be formed once every line sits on the same coordinate. On a weight axis they do: every species shares the model's weight grid. On a length axis they do not — each species, and now the resource, converts weight to length with its own allometric relationship, so at a given length the lines sit at different weights and their grids no longer coincide. That is why the Total line was dropped from length-based plots.

It is now formed after the conversion, at equal length rather than at equal weight, which is the quantity that means something there: the number of organisms per unit length, whatever they are.

How the sum is formed

add_total_line() interpolates each series onto the sorted union of all the size coordinates, linearly in the logarithm of size since the grid is logarithmic, with a series contributing nothing outside its own range.

The important property: where the series already share a grid, the union is that grid and interpolation reproduces the values exactly. That is always the case on a weight axis, and on a length axis whenever the weight-length parameters agree — so nothing is approximated in the cases where nothing needs to be. Interpolation only does real work when species genuinely differ in a or b, and there it is the only correct answer.

The weight-axis total is verified unchanged against the previous implementation for powers 0, 1 and 2.

This moves the total out of prepare_ArraySpeciesBySize_plot_data() and past the conversion in every path that draws one: the array plot(), addPlot(), plot2() and plotRelative() via the two data-frame helpers, plotSpectra(), and both animate() methods. plotRelativeDataFrame() now converts before joining rather than after, so a total exists to join on; the relative difference itself is unaffected, since a common Jacobian cancels.

Two corrections that come with it

Both on the weight axis, both recorded in NEWS.md and the upgrade skill with symptom-index rows:

• plotSpectra(total = TRUE, resource = FALSE) now leaves the resource out of the total. The resource argument used to control only whether the resource line was drawn — the total counted it either way, so a consumers-only total was not available. This is the fix agreed when we chose to keep the resource in the length-axis total: it gives a way out for anyone who does not want the two length conventions mixed.
• plot(<array>, total = TRUE) now sums the lines that are drawn rather than the whole array, so it no longer counts abundance outside a species' size range. On the North Sea model this moves the total at about 2% of sizes, by up to a factor of three where a species holds abundance beyond its w_max — the old total sat visibly above the sum of the lines there. all.sizes = TRUE reproduces the old sum exactly.

Testing

Full suite: 4313 passing, 0 failures. New tests cover add_total_line() on a shared grid (exact), on grids that differ (checked against hand-computed approx() values at each union point), and with a grouping column; that the spectrum total is unchanged on a weight axis for all three powers; that it appears on a length axis and equals the sum of the lines drawn there; that a length-axis total on species with differing a/b needs more points than the weight grid and reduces to the single surviving species at the largest length; and both weight-axis corrections above.

🤖 Generated with Claude Code

[gustavdelius]

Third commit (018dd99): total now means the total of everything the object holds, everywhere.

To answer the question that prompted this: plotSpectra()'s total never excluded the resource on master — it is total_n <- n_pp; total_n[fish_idx] + colSums(n), unconditional in both resource and species. I introduced that regression earlier today, in a2a35b1 in this PR, following the "include, and fix resource" option in the design question. Your memory of the old behaviour was right and it is now restored.

The array plots were the ones genuinely out of step, since plot() for arrays was introduced (fb1a203, April). They summed only the species selected for display, and only the sizes inside each species' own range. That made the total a property of the plot rather than of the model, so you could not plot two species against the community total.

Both now follow one rule — the total is the total of everything the object holds:

	before	now
plotSpectra(total = TRUE)	resource + all species	unchanged
plotSpectra(total = TRUE, resource = FALSE)	resource + all species	unchanged (a2a35b1 briefly changed this; reverted)
plot(<array>, total = TRUE)	selected species, trimmed to size ranges	whole array
plot(<array>, species = "Cod", total = TRUE)	Cod only	whole array
animate(<array>, total = TRUE)	selected species	whole array

The contributors are assembled by total_contributors() — just the usual preparation of the whole array — and summed after the size conversion by append_total_line(), so the length-axis machinery is unchanged.

The two "corrections" I listed in the PR description are therefore gone: the resource = FALSE one is reverted outright, and the size-range one is superseded, since the total now sums the whole array rather than what is drawn. What remains for the changelog is the single clarification above, plus the length-axis total itself. NEWS, the upgrade skill and the cheatsheet are updated to match, and the tests now assert the invariance directly: the total does not move when species, all.sizes, background or resource change.

Suite: 4317 passing, 0 failures.

[gustavdelius]

Fourth commit (ba8cb0a): plotSpectra2() and plotSpectraRelative() keep their total on a length axis.

Both assembled the two spectra on the weight axis and converted afterwards, inside the data-frame helper. So the total they had been handed was already summed at equal weight, and reached the conversion with no species to convert it by — it was silently dropped, and a length-axis comparison lost its total.

The fix turned out to need no new plumbing: they now ask plotSpectra() for the axis they actually want, so the frames come back already converted, with the total in them formed the same way as in a single spectrum plot. plotComparisonDataFrame() and plotRelativeDataFrame() are then called without size_axis, so nothing is converted twice. plotSpectraRelative() had the identical bug and gets the identical fix.

Verified: for every combination of axis (w, l) and power (0, 1, 2), each model's series in the comparison — total included — is exactly what plotSpectra() draws for that model on its own. The weight axis is unchanged.

The data_ylim treatment is deliberately untouched: on a length axis the inner call still gets c(0, NA), because the user's ylim applies to the converted values and is imposed at the end. The existing test that the length-axis comparison data keeps values below ylim still passes.

Suite: 4333 passing, 0 failures.

[gustavdelius]

Fifth commit (1fed987): the data_ylim special case is gone.

plotSpectra() applies ylim twice over — as the axis limits and as a filter on the data, the latter carrying the hard 1e-20 floor that keeps a spectrum's numerically-zero tail from stretching a log axis. That double duty is a package convention, not a quirk: prepare_ArrayTimeBySpecies_plot_data() does the same and plotBiomass() documents it.

plotSpectra2() could not follow it on a length axis. It assembled the two spectra on the weight axis, so the values it would have filtered were a Jacobian away from the ones the user's limits described. It passed c(0, NA) to the inner calls instead — disabling the upper cut and lowering the floor from 1e-20 to 0 — and left the limits to the scale alone. That workaround came in with fa3a80af, four days ago, as part of the Jacobian work.

The previous commit removed the reason for it: the inner call now converts to the requested axis, so the filter sees exactly the values the limits describe. ylim is handed straight over.

The inconsistency this removes, with ylim = c(median, NA) and size_axis = "l":

	rows returned	rows below the limit
plotSpectra()	512	0
plotSpectra2() before	1024 per model	1024
plotSpectra2() after	512 per model	0

The drawn plot is unchanged — the axis limits hid those points either way. What changes is return_data = TRUE, and a series dipping below the lower limit mid-range now breaks into segments instead of running off the bottom, which is already how plotSpectra() draws it.

The test from fa3a80af flips as predicted, from any(data < limit) to all(data >= limit); I added an assertion alongside it that the comparison and the single plot now return identical values for the same ylim, which is the property that was actually missing.

Suite: 4334 passing, 0 failures.