Skip to content

Measure the performance numbers of the article, and build its table - #12

Merged
gouarin merged 1 commit into
mainfrom
performance-metrics
Sep 13, 2026
Merged

gouarin merged 1 commit into
mainfrom
performance-metrics

Conversation

@gouarin

@gouarin gouarin commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

The three numbers the tables of the article are made of, measured by the solver
instead of reconstructed from a log, and the script that turns them into the
table.

What is measured

euler/metrics.hpp, wired into the three binaries:

sparsity index cells of the adapted mesh over the cells of the uniform mesh at --max-level, at the initial and at the final time. A Riemann problem fills its mesh up as the waves spread, and one number taken at one end would flatter or damn it.
cell updates, Mcu/s one cell advanced by one time step, counted once per cell per step whatever the integrator does inside — otherwise Strang's 2*dim - 1 sweeps would read as more work done.
time to solution the time loop. The adaptation is in it and is reported apart; writing files is not, --nfiles being a choice of whoever runs the solver.

Reduced over the ranks once, in a collective call: cells and updates summed,
times taken at their maximum. --metrics-file writes the same numbers as JSON.

The table

python python/performance.py --levels 6 7 8 9 --uniform, on configuration 3
of Lax & Liu to t_f = 0.8 at second order, one core:

 l_min  l_max   resolution    mr-eps    Mcu/s  time (s)    AMR           cells ti/tf    sparsity ti/tf
     3      6         64^2   default      2.2      0.39  25.1%           1720 / 3784     42.0% / 92.4%
     3      7        128^2   default      2.4      2.49  29.6%          3916 / 13522     23.9% / 82.5%
     3      8        256^2   default      2.5     15.41  28.1%          8416 / 45394     12.8% / 69.3%
     3      9        512^2   default      2.5    102.92  28.4%        17524 / 151480      6.7% / 57.8%
     9      9        512^2   default      4.7    147.64   0.0%       262144 / 262144   100.0% / 100.0%

The last row is the uniform mesh at the same resolution, the reference the
article puts at the bottom of its own table: the adapted run does 2.7 times
fewer cell updates and is 1.4 times faster
, running at a little more than half
the throughput of the uniform one. About a third of what is lost is the
adaptation itself; the rest is what a cell of an adapted mesh costs.

--mr-eps takes several values and measures each. At level 9, two decades of
threshold take the final sparsity from 58% to 16% and the time to solution from
115 s to 59 s, with the initial mesh unmoved — the details of a piecewise
constant state are of order one at a discontinuity, far above every threshold in
that range. Those rows say nothing about accuracy, which is what
python/error_analysis.py measures.

Reading it against the article

The readme says it at length, and the short version is that the sparsity column
is where the comparison goes wrong most easily. What the multiresolution keeps
is a neighbourhood of the discontinuities, which are curves in a plane, so their
share of the mesh falls with the resolution: 92%, 83%, 69%, 58% over the four
rows. A sparsity index quoted without the max-level it was measured at compares
nothing.

Tests

Held against what the run computed, not against themselves: 100% sparsity on a
uniform mesh whose cell count is arithmetic on a unit domain, final cell count
equal to the cell count of the HDF5 file the run wrote, time to solution holding
the adaptation and not the output. One slow test holds the fall of the sparsity
index with the resolution.

136 fast tests, 23 slow.

The article this repository reproduces is first of all a performance paper, and
its tables are made of three numbers this solver did not measure: the sparsity
index of the adapted mesh, the cell updates per second, and the time to
solution. Without them the repository reproduces pictures and compares nothing.

euler/metrics.hpp measures them where they happen rather than leaving them to
be reconstructed from a log, and the conventions are the article's:

  sparsity index    cells of the adapted mesh over the cells of the uniform
                    mesh at max-level, reported at the initial and at the final
                    time. A Riemann problem fills its mesh up as the waves
                    spread, and one number taken at one end would flatter or
                    damn it.

  cell updates      one cell advanced by one time step, counted once per cell
                    per step whatever the integrator does inside. SSP-RK2
                    evaluates the flux twice and Strang sweeps 2*dim - 1 times;
                    counting those would make the more expensive integrator
                    look faster.

  time to solution  the time loop. The mesh adaptation is part of it and is
                    reported apart, which is the number the comparison of a
                    multiresolution against a gradient criterion turns on.
                    Writing files is not part of it: --nfiles is a choice of
                    whoever runs the solver, and a table of times that moved
                    with it would compare nothing.

Everything is accumulated on the local subdomain and reduced once, at report
time, which is a collective call: cells and updates summed over the ranks,
times taken at their maximum.

--metrics-file writes the same numbers as JSON, and python/performance.py makes
the table out of one run per resolution. On the reference case of the article,
configuration 3 of Lax & Liu to t_f = 0.8 at second order, on one core:

  l_min  l_max  resolution   Mcu/s  time (s)    AMR       cells ti/tf   sparsity ti/tf
      3      6        64^2     2.2      0.39  25.1%       1720 / 3784   42.0% / 92.4%
      3      7       128^2     2.4      2.49  29.6%      3916 / 13522   23.9% / 82.5%
      3      8       256^2     2.5     15.41  28.1%      8416 / 45394   12.8% / 69.3%
      3      9       512^2     2.5    102.92  28.4%    17524 / 151480    6.7% / 57.8%
      9      9       512^2     4.7    147.64   0.0%   262144 / 262144  100.0% / 100.0%

The last row is the uniform mesh at the same resolution, the reference the
article puts at the bottom of its own table: the adapted run does 2.7 times
fewer cell updates and is 1.4 times faster, running at a little more than half
the throughput of the uniform one. About a third of what is lost is the
adaptation itself; the rest is what a cell of an adapted mesh costs.

The sparsity column is where a comparison with the article goes wrong most
easily. What the multiresolution keeps is a neighbourhood of the
discontinuities, which are curves in a plane, so their share of the mesh falls
with the resolution -- 92%, 83%, 69%, 58% over the four rows above. A sparsity
index quoted without the max-level it was measured at compares nothing. The
threshold is the other half: --mr-eps takes several values and measures each,
and two decades of it take the final sparsity at level 9 from 58% to 16% and
the time to solution from 115 s to 59 s, with the initial mesh unmoved, the
details of a piecewise constant state being of order one at a discontinuity and
far above every threshold in that range. Those rows say nothing about accuracy,
which is what python/error_analysis.py measures.

The metrics are held against what the run computed rather than against
themselves: the sparsity index is 100% on a uniform mesh whose cell count is
arithmetic on a unit domain, the final cell count is the cell count of the HDF5
file the run wrote, and the time to solution holds the adaptation and not the
output. One slow test holds the fall of the sparsity index with the resolution,
which is the reading every comparison with the article depends on.
@gouarin
gouarin merged commit b5eeb01 into main Sep 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant