aboutsummaryrefslogtreecommitdiffstats
path: root/text
Commit message (Collapse)AuthorAgeFilesLines
* Break text into lines, graphemes, words and sentences as Unicode's tests requireeunhhu2026-09-288-2/+5659
| | | | | | | | | UAX #14 line breaking (LB1-LB31, numbers tailored as Unicode's tests are) and UAX #29's grapheme cluster, word and sentence boundaries, each from generated property tables, the rules taken in order, in linear time. They pass all of Unicode 15.1's LineBreakTest (10,274 cases), GraphemeBreakTest (1,187), WordBreakTest (1,826) and SentenceBreakTest (512).
* Shape text as HarfBuzz does, from a symbol file, checked against HarfBuzzeunhhu2026-09-284-0/+3011
| | | | | | | | | | | | | The build tool compiles a font's shaping plans the way HarfBuzz's planner does (features, stages, masks, lookups per script and direction) and carries them with the font's character map, advances and GDEF, GSUB and GPOS tables in a section of the symbol file (VEGASYM4); glyphs shaping reaches become symbols of their own. A small engine ports HarfBuzz 10's default, Arabic and Hebrew shaping step for step and reads the tables in place, every read bounds-checked. The build refuses a file whose shaping differs from HarfBuzz's on random runs: DejaVu Sans agrees on 600,000 runs in six scripts both ways, and every installed font the tool accepts agrees too, at 1.05-1.18 times HarfBuzz's time.
* Generate the Unicode properties shaping needseunhhu2026-09-283-2/+6353
| | | | | | Joining types, general categories and scripts (with their ISO 15924 codes) as two-stage tables, canonical decompositions one step at a time, and Extended_Pictographic, from the UCD by text/unicode.py.
* Normalize text to NFD and NFC, passing all of Unicode's normalization testseunhhu2026-09-285-2/+4373
|
* Resolve bidirectional text as UAX #9 does, passing all of Unicode's bidi testseunhhu2026-09-285-0/+2436