Skip to content

tr: complement, truncate, classes, repeats and error paths - #240

Open
metif12 wants to merge 1 commit into
vlang:mainfrom
metif12:tr-parity
Open

metif12 wants to merge 1 commit into
vlang:mainfrom
metif12:tr-parity

Conversation

@metif12

@metif12 metif12 commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #239, which got basic translation, delete, squeeze and the lower/upper/digit classes in.

What this adds, each measured byte-for-byte against GNU 9.4 (WSL) and uutils 0.0.17 on this machine:

  • -c is a real complement now: SET1 is replaced by all 256 bytes outside it, in byte order, then translated positionally (tr -c a-z A-Z maps \n to K, spaces/digits to Z). The old code translated SET1 and stuffed every other byte with set2[0].
  • -t/--truncate-set1, applied after complementing.
  • -s with two sets translates first, then squeezes SET2 (-s a-z A-Z turns hello into HELO).
  • Short SET2 repeats its last character; all POSIX classes (alnum blank cntrl graph print punct space xdigit, with space now including VT/FF); octal escapes plus \a \b \f \v; [c*n] and bare [c*] repeats; the two operand-error paths exit 1 with empty stdout.
  • Three bugs that made the old tests pass for the wrong reason: V .. ranges exclude the end (a-z expanded to a-y), [:digit:] parsed as digit: (trailing colon kept), and set2[0] used where the last character belongs.

Residuals (also in code comments, deliberate):

  • tr -ct follows GNU (identity here); uutils 0.0.17 maps every byte to the last character of SET2 instead. Untested against the rig reference for that reason.
  • Extra operands with -d/-ds are ignored like uutils; GNU rejects them.
  • [=c=] equivalence classes are not implemented.
  • On the two error paths uutils prints the diagnostic with a trailing blank line; GNU prints it once. This port matches GNU and pins those bytes in the tests.

v test src/tr green (14 test functions, differential against uutils), v fmt -l clean.

@JalonSolov

Copy link
Copy Markdown
Contributor

Conflict with recent push of formatted tr.v

Follow-up to vlang#239. Parse and behavior work measured against GNU 9.4 and uutils 0.0.17 on this machine: proper -c complement (positional over 256 bytes), -t truncation after complementing, -s with two sets (translate then squeeze SET2), short-SET2 last-char repetition, all POSIX classes, octal and alert/backspace/formfeed/vertical-tab escapes, [c*n]/[c*] repeats, and the two operand-error paths (exit 1, empty stdout). Known residuals are noted in code comments: -ct follows GNU where uutils differs, extra operands are ignored with -d like uutils rather than rejected like GNU, and [= =] equivalence classes are not implemented.
@metif12

metif12 commented Oct 17, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main to clear the conflict, and the conflict was worth looking at rather than resolving mechanically.

7f9bef0d37 ("all: fmt files") reformatted the same character-class lines this branch changes, so the rebase collided on all of them. The two sides are not equally right: main has the spacing right and the arithmetic wrong, this branch has both right. So this branch's side wins, and the result is fmt-clean — which I checked rather than assumed, since taking the wrong side would have looked like a normal resolution and silently reintroduced the bug.

The bug, measured

V's .. is exclusive. Confirmed on the compiler this repo builds with:

>>> for i in 0 .. 10 { n++ }
10
>>> print(vals)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

main expands every class and range with for c in u8(a) .. u8(z), which therefore stops at y. Running main's own expansion code verbatim:

[:alpha:] -> 50 chars (GNU=52), ends "WXY"     contains z? false  Z? false
[:digit:] ->  9 chars (GNU=10), ends "8"       contains 9? false

Every class dropped its last character.

There is a second bug, and it is the bigger one

main extracts the class name as:

end := s.index_after(']', i + 2) or { s.len }
class := s[i + 2..end]

index_after returns the index after the ], so for [:digit:] the slice is s[2..9] — "digit:", trailing colon included. match class compares against 'digit', never matches, and falls through to else {}. So every character class on main expands to nothing, and the off-by-one above is masked: the loops never run.

$ tr '[:digit:]' X   # main
0123456789  ->  0123456789      # nothing translated

That is why I did not open a one-line hotfix for this. A + 1 on its own would have been dead code, and would have looked like a fix.

Differential against GNU

Since the repo's own tr test rig needs a GNU tr on PATH and reports failures on main too, I compared directly against tr (GNU coreutils) 8.32 over 27 invocations:

differences vs GNU
main 18 / 27
this branch 2 / 27

Both remaining cases are the same thing and not a defect: GNU prints its own argv[0], so its diagnostics read /usr/bin/tr: missing operand where this reads tr: missing operand. Exit codes, stdout and the other stderr text all agree. The repo's rig normalises exactly this before comparing, which is why its failure here is a harness artifact on Windows rather than a finding.

Functional spot check after the rebase:

$ tr 'a-z' 'A-Z'  <<< abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ          # includes Z
$ tr -d '[:digit:]' <<< abcXYZ019-_
abcXYZ-_                            # 0 and 9 removed

Not verified here: the full v test src/tr rig on Windows, and the Vinix cross-build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants