I have just moved the fulmine entry from tuned to standard (#1102), and it will cost me score. I
think it is the right trade, and it made me look at how the two modes are shown on the site.
Today a tuned entry is marked with a ring on the leaderboard. Someone who does not know what the
ring means reads the row as "this framework does this". It does not. To get that number they would
have to open the benchmark implementation, read it, and copy whatever is in there into their own
application, if their application can even take it. And once they do that, the honest question is
the next one: why does the framework not do this itself?
That is the gap I would like the site to close. A tuned row is a promise the user cannot collect by
choosing the framework, only by working for it.
Some things that would help, in order of how cheap they are:
- Write the mode in words on the row, not only as a ring: a
tuned badge next to the name.
- Show the mode as a column that can be sorted and filtered, so the two can be read apart at a
glance.
- Consider making standard the default view, with tuned opt-in, the way experimental already is.
I am not asking for the tuned category to go away. It answers a real question: how far this stack
can be pushed when someone who knows it is allowed to push. That is worth measuring, and worth
publishing. But the reader has to understand which of the two numbers they get by installing the
framework and which one they get by sweating for it.
TechEmpower is the cautionary tale here. It slid into being a contest of tuning, and a lot of people
stopped reading it as a guide to what a framework does. HttpArena has the two modes already, which
is more than TechEmpower ever had. I would just make the difference impossible to miss.
I have just moved the fulmine entry from tuned to standard (#1102), and it will cost me score. I
think it is the right trade, and it made me look at how the two modes are shown on the site.
Today a tuned entry is marked with a ring on the leaderboard. Someone who does not know what the
ring means reads the row as "this framework does this". It does not. To get that number they would
have to open the benchmark implementation, read it, and copy whatever is in there into their own
application, if their application can even take it. And once they do that, the honest question is
the next one: why does the framework not do this itself?
That is the gap I would like the site to close. A tuned row is a promise the user cannot collect by
choosing the framework, only by working for it.
Some things that would help, in order of how cheap they are:
tunedbadge next to the name.glance.
I am not asking for the tuned category to go away. It answers a real question: how far this stack
can be pushed when someone who knows it is allowed to push. That is worth measuring, and worth
publishing. But the reader has to understand which of the two numbers they get by installing the
framework and which one they get by sweating for it.
TechEmpower is the cautionary tale here. It slid into being a contest of tuning, and a lot of people
stopped reading it as a guide to what a framework does. HttpArena has the two modes already, which
is more than TechEmpower ever had. I would just make the difference impossible to miss.