Repository navigation
sort: read lowercase SI units with -h and -f - #15644
Conversation
With -f, GNU sort -h reads 2m as 2M and 1e as 1E. uutils ignored -f on this path and treated a lowercase unit other than k as no unit. The key parser now accepts lowercase units when the key folds case, and get_unit maps them to the uppercase ones. --debug still marks them as outside the key, as GNU does. Fixes uutils#15577.
Merging this PR will not alter performance1 warning, 1 note
Comparing Footnotes
|
| let mut parse_settings = settings | ||
| .numeric_locale | ||
| .num_info_settings(selector.settings.mode == SortMode::HumanNumeric); | ||
| .num_info_settings(selector.settings.mode == SortMode::HumanNumeric, false); |
There was a problem hiding this comment.
could you please add a --debug -h -f case to the test so this stays aligned with GNU?
There was a problem hiding this comment.
@sylvestre Added test_human_numeric_ignore_case_debug, which checks the --debug -h -f output against GNU's.
|
Binary size comparison: |
|
Thanks for your PR |
Fixes #15577
With
-h,-fhad no effect. GNU sort folds case before the human-numeric compare, so2mis read as2Mand1eas1E. uutils read a lowercase unit other thankas no unit, both for the global-h -fand for a key like-k2,2hf.NumInfoParseSettingsgets afold_si_unitsflag. When it is set,NumInfo::parsealso includes a lowercase unit in the number's range.FieldSelector::get_selectionsets it from the key'signore_case.get_unitreads the lowercase units like the uppercase ones. A lowercase unit other thankis only in the range when the key folds case, so nothing changes without-f. I wrote it as amatchinstead of thepositionsearch. With only an uppercase conversion added,sort -hon 5M lines was about 3% slower. With thematchit takes the same time asmain.--debugkeeps marking a lowercase unit as outside the key, even with-f. GNU 9.12 does the same (2mis underlined as_, not__).I added
test_human_numeric_ignore_case. It checks-h -f,-hfand-k1,1hf, and that-hand-f -k1h(where the key has its own options and does not inherit-f) still don't read lowercase units. It fails without the change, and alltest_sorttests and theuu_sortunit tests pass.I also compared 18 option combinations with GNU 9.12 on input with lowercase and uppercase units, negative values,
0m,-0mand a barem:-h,-h -f,-hf,-fh, with-r,-u,-s,-c,--debug, and keys like-k1,1hf,-f -k1,1hand-k2,2hf. The output was the same for all of them.The
sort_benchbenchmarks have no-hcase, and their results only moved by run-to-run noise. I also timed a release build ofsort -h --parallel=1on 5M lines againstmain: 3.32–3.40 s onmainand 3.33–3.37 s with the change.I used an AI assistant to help draft parts of the code and tests; I reviewed the changes and ran the tests myself.