Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement tr #147

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

feat: implement tr #147

wants to merge 1 commit into from

Conversation

revosw
Copy link

@revosw revosw commented May 18, 2024

This pull request implements the tr command, which is documented in section 9.1 tr: Translate, squeeze, and/or delete characters.

One aspect of this implementation of tr that differs from GNU coreutils is how each character is handled. GNU coreutils processes byte-for-byte, not taking into account multibyte codepoints. This implementation, however, processes codepoint-for-codepoint. Here are some examples.

echo old | tr old new: Translating a single-byte codepoint to another single-byte codepoint works the same between this implementation and GNU coreutils' implementation.

Input Output (V coreutils) Output (GNU coreutils)
Codepoints old new new
Bytes

111 108 100

110 101 119

110 101 119

echo ! | tr ! ❗: Translating a single-byte codepoint into a multibyte codepoint. The GNU implementation only replaces the first byte (226), which means programs that expect UTF-8 encoded data might render the replacement codepoint � (U+FFFD) in its place to represent incomplete data

Input Output (V coreutils) Output (GNU coreutils)
Codepoints !
Bytes

33

226 157 151

226

If replacing entire codepoints is undesirable, it's very easy to revert to the byte-for-byte behavior.

As of now, this is what is implemented

  • tr <arg1> <arg2>
  • tr -d <arg>, tr --delete <arg>
  • tr -t <arg1> <arg2>, tr --truncate-set1 <arg1> <arg2>
  • tr -s <arg1> <arg2>, tr --squeeze-repeats <arg1> <arg2>
  • tr -c <arg1> <arg2>, tr --complement <arg1> <arg2>

@spytheman
Copy link
Member

The project goal is to re-implement the GNU coreutils tools behaviors as closely as possible, including error messages etc.

@JalonSolov
Copy link
Contributor

However, a new option could be added, to allow proper handling of multi-byte chars.

Nothing says we can't have options they don't, just that the default, and any options they do have, should act as close to theirs as possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants