Commit Graph
31 Commits
Author SHA1 Message Date
pantonshire e90b77256a version bump 2023-06-15 14:57:18 +01:00
pantonshire 402da43bc1 fix utfdump_wasm Cargo.toml 2023-06-15 14:53:09 +01:00
pantonshire 673364873d specify msrv 2023-06-15 14:52:59 +01:00
pantonshire 0d1872b902 wasm library 2023-06-15 14:25:02 +01:00
pantonshire 9d6eae0bd0 make lib no-std compatible 2023-06-05 14:46:39 +01:00
pantonshire effe6916d3 move unicode_data_encoded.gz to lib 2023-06-05 11:12:48 +01:00
pantonshire c274ba6f01 🐛 only show consumed bad bytes for invalid characters
Previously, the bytes displayed for invalid characters included bytes
from the byte stream that were peeked rather than consumed. This
resulted in certain bytes being displayed multiple times, since the
peeked byte could appear in the following character.

For example, `printf '\xce\x61' | utfdump_bin` would result in the byte
0xce being displayed twice, once at the end of the invalid character and
once as the valid character `a`.

This patch modifies `utfdump::utf8::Utf8Error` so it also stores the
number of consumed bad bytes, enabling the binary to output only the
consumed bad bytes.
2023-06-05 11:06:28 +01:00
pantonshire ecf2abbdad retrieve data from unicode.org in data.py 2023-06-05 10:57:19 +01:00
pantonshire 6b911caa15 update Cargo.lock 2023-06-05 10:26:31 +01:00
pantonshire 6e8d197ae4 remove core 2023-06-05 10:26:13 +01:00
pantonshire f9430db2f9 update binary to use new lib 2023-06-05 10:25:08 +01:00
pantonshire cc47c90f50 include compressed encoded data in repo 2023-06-04 18:18:33 +01:00
pantonshire 25dae48064 refactoring 2023-06-04 18:18:08 +01:00
pantonshire a3750b5732 working rust decoder for new encoded data format 2023-06-04 18:12:30 +01:00
pantonshire dc4650262d packed struct for character data 2023-06-03 21:35:55 +01:00
pantonshire 24abc7ed79 work on rust side of new encoded data format 2023-06-02 20:50:47 +01:00
pantonshire d9b0c049ab gzip compress the unicode data 2023-06-01 16:51:11 +01:00
pantonshire a427b7f58f output encoded data 2023-06-01 16:23:51 +01:00
pantonshire 6791ac8d35 character data encoding 2023-06-01 14:24:33 +01:00
pantonshire d213c81bf6 🎉 start new encoder for unicode data 2023-05-30 18:51:34 +01:00
pantonshire ef6765e037 ♻️ refactoring 2023-05-19 09:29:37 +01:00
pantonshire 96d328c829 ✨ support for invalid utf8 2023-05-18 17:04:30 +01:00
pantonshire 26ee43af2e make Combining column human-readable
This patch introduces a `CombiningClass` type to represent the canonical
combining class of a codepoint. The `fmt::Display` implementation of
this type writes the human-readable name of the combining class, if
there is one. This replaces the previous behaviour, which was to just
print the raw byte value of the combining class.
2022-09-20 16:39:27 +01:00
pantonshire f1355b5fe3 update README to include link to releases page 2022-09-19 13:36:47 +01:00
pantonshire edb418216f update gitignore 2022-09-19 13:11:13 +01:00
pantonshire 9c1a98adb7 add help text for -f flag
This patch adds a documentation comment to `Args.full_category_names`,
which `clap` uses to generate help text for the `-f` flag.
2022-09-19 13:09:42 +01:00
pantonshire 22d7e6a2a7 add README.md 2022-09-19 12:20:55 +01:00
pantonshire 885cee0e1f add LICENSE file 2022-09-19 12:02:09 +01:00
pantonshire 6ed0de3c7c update Cargo.toml information
This patch updates the Cargo.toml files for both crates in this
repository, adding the author, license, description and repository
fields.
2022-09-19 12:00:09 +01:00
pantonshire 6cb3b0a412 add command line option to show full category name
This patch adds `clap` as a dependency for command-line argument
parsing, and introduces a `-f` flag to allow toggling whether or not the
full names of character classes are displayed.
2022-09-18 21:59:23 +01:00
pantonshire 172d1a14fe Initial commit 2022-09-18 18:05:00 +01:00