Surface runtime projecting Pure views across pixel, terminal, CLI, and web. http://git.yeowoo.org/vega/atom?h=master 2026-09-28T15:31:53Z 2026-09-28T15:31:53Z eunhhu necpition205@gmail.com 2026-09-28T15:31:53Z urn:sha1:30a730c54298f6abafc622045cc2e6542126f735 Run against HarfBuzz 14.5.0 and Unicode 18.0.0, text/unicode.py met a line breaking class it does not know (HH) and text/syllabic.py a table laid out otherwise, and the tool had already emptied both files. A generator's output now replaces its file only when it succeeds, and the tool stops before building when one fails: those are what to port first. 2026-09-28T15:27:28Z eunhhu necpition205@gmail.com 2026-09-28T15:27:28Z urn:sha1:96c35e7919e5e25187bd5adedda516a32cfc8639 text/upstream.py brings in a HarfBuzz release (default: the one the build links) and the UCD of the Unicode version its tables came from, builds that HarfBuzz as one static library from its amalgamated source (no system package, any version), regenerates the Unicode tables and the syllabic ones, holds the syllable machines to HarfBuzz's compiled ones, and runs the text checks built against it: Unicode's conformance tests on that UCD, random shaping against that HarfBuzz, the layout's promises, the kern class tests, HarfBuzz's own suite. --corpus adds every installed font (text/fonts.py, the corpus runner, now in the tree). What a new version breaks shows as failing checks with the font and the input to port a fix from; the generated files are left changed for review. make check-text is those checks alone; UCD and HARFBUZZ_SRC default to build/upstream's when the tool fetched them. Its first run found that text/shape-build.c used clock_gettime without asking for POSIX, building only because the system HarfBuzz's headers did. Against HarfBuzz 10.2.0 built from source and the 16.0.0 UCD, every check passes in about a minute. 2026-09-28T15:27:03Z eunhhu necpition205@gmail.com 2026-09-28T15:27:03Z urn:sha1:fb8e49a31a7ba7f333965c93b08bd0c58120494e HarfBuzz 10.2's tables are Unicode 16.0, Vega's were the installed UCD's 15.1: characters new in 16.0 could be read two ways. The tables now come from the 16.0 UCD, and the line breaking rules are revision 53's: LB13's 'x IS' split into LB15c (SP / IS NU, as in 'subtract .5') and LB15d; the numbers of section 8.2's Example 7 made LB25 (new: IS x NU, PR x OP IS NU); LB19 split, with LB19a letting quotation marks between East Asian characters break; LB20a keeping a word-initial hyphen to its word; LB21a narrowed to Hebrew, hyphen, then not Hebrew. vega_east_asian now covers all of East_Asian_Width F, W and H. Unicode 16.0's conformance tests all pass (bidi 770241 and 91707, line breaks 16672, graphemes 1093, words 1826, sentences 512, normalization 19965). test-text reads NormalizationTest.txt plain or compressed, and passes over a UCD of another version than the tables' (the installed one here is 15.1). HarfBuzz's suite (5018 the same), the installed fonts' and Noto's random runs, and the surfaces' checks agree as before. 2026-09-28T15:15:37Z eunhhu necpition205@gmail.com 2026-09-28T15:15:37Z urn:sha1:e30e3c2f17fb26cddc91e9f950f6f26948fa8e0e A font whose GSUB grows the buffer past its bounds (text-rendering-tests' GSUB-3) made vega_shape fail. HarfBuzz goes on with the glyphs of the last pass that finished: every later pass does nothing, positions are the default advances. Vega's buffer operations already stand still once the buffer is unsuccessful, so it now goes on too, and a broken font shows what HarfBuzz shows rather than nothing. HarfBuzz's suite: 5018 of 5199 the same, none failing. 2026-09-28T15:13:58Z eunhhu necpition205@gmail.com 2026-09-28T15:13:58Z urn:sha1:ea6dbcf627e9a440590d0d35ff16ca7ae60d5719 The kern table's class subtables, OpenType's and Apple's, as HarfBuzz reads them (hb-aat-layout-kerx-table.hh, hb-ot-kern-table.hh): format 2's class tables give byte offsets into a kerning array, format 3's byte classes index a value list. Both go through the same pass as pairs (marks skipped, the kerning split between the pair). Format 2 filters the glyphs by the class tables' digests, as HarfBuzz's accelerator does, so a digest's false positive reads class 0 there too; what a subtable reads must lie within it, the last one's within the table. The build checks the subtables as HarfBuzz's sanitizer does, and still refuses state machines (format 1) and cross-stream kerning. text/test-kern.py makes fonts with format 2 and 3 subtables in every order, with the edge cases that reading has (value 1, glyphs outside a table, offsets below the array or odd, values past a subtable's or the table's end, classes and indexes past their counts): 24 fonts, 96 plans, the same as HarfBuzz. make check runs it where fontTools is. HarfBuzz's suite: 5017 of 5199 the same. 2026-09-28T15:07:05Z eunhhu necpition205@gmail.com 2026-09-28T15:07:05Z urn:sha1:cf6bcf8697589d2a6fc3c6d98bc52bf7eb1ea4ae The section now carries each sequence the font maps, a base and a selector, with the glyph HarfBuzz gives the pair (a default one's the base's own). As HarfBuzz's normalizer does, a sequence the font has becomes that one glyph, the two characters' clusters merged; one it lacks still passes on as two, the selector hidden. The header grows to 88 bytes for the table. The random check now mixes in the font's own sequences and stray selectors: 30000 runs over Noto Sans CJK's 14787 ideographic variation sequences are HarfBuzz's, and so are the installed fonts' and Noto's. HarfBuzz's suite: 5014 of 5199 the same, none different; what is left is what the build refuses and one run without a plan. 2026-09-28T15:04:05Z eunhhu necpition205@gmail.com 2026-09-28T15:04:05Z urn:sha1:960fa21762484d3014a6778a7673f15acf2d34bb The shaping section's character map came from hb_face_collect_unicodes. HarfBuzz's own suite showed where that list and its mapping part: it cuts a format 13 map short (AdobeBlank2's whole Unicode to one glyph), and a symbol map (3,0) it searches through the private use area, Latin and, in an old Arabic font, Arabic too. Now every code point is asked with hb_font_get_nominal_glyph (about 30 ms a font) and the map carried as groups: codes going to consecutive glyphs, or all to one (12 bytes each, so a format 13 map stays small). HarfBuzz's suite: 5011 of 5199 cases the same (1355 before). The installed fonts (213) and the Noto ones (21) still shape as HarfBuzz does on random runs. 2026-09-28T14:58:32Z eunhhu necpition205@gmail.com 2026-09-28T14:58:32Z urn:sha1:ddbbc39cc65946f59298eab546c906ad3dea6841 text/harfbuzz-suite.py reads a HarfBuzz release's test/shape/data (in-house, text-rendering-tests, aots: fonts and inputs, many added when a font or a shaper once went wrong) and shapes every case both ways with the new vega-symbols shape-cases: it compiles a font's section for the scripts its cases need (guessed as HarfBuzz guesses them), then compares each run with HarfBuzz's, glyph for glyph and unit for unit. The oracle is HarfBuzz run live, not the recorded outputs, which depend on options Vega has no counterpart for (font functions, sizes, output formats). Cases asking for what Vega does not do (user features, variations, languages, context, cluster levels, buffer flags, vertical text) and fonts of other systems are left out and counted by why. text/harfbuzz-suite.known lists what differs today, and why: AAT, the character map built from what HarfBuzz lists rather than how it maps (format 13, symbol maps into the private use area), variation selectors, Arabic fallback with shared glyphs, kern format 2, Zawgyi, and one run without a plan. Any other difference fails the run; a listed case that agrees is named. make check-harfbuzz HARFBUZZ_SRC=... runs it (make check too, when given): 5929 cases, 5199 compared, 1355 the same. 2026-09-28T14:52:13Z eunhhu necpition205@gmail.com 2026-09-28T14:52:13Z urn:sha1:a4ca275ad1b5edc15474407e2852f8fe9abaac4e vega_layout takes fonts in order of preference (VegaFonts). A grapheme cluster goes to the first font that has all its characters (default ignorables aside: shaping hides them); a cluster of Common and Inherited characters stays in the font before it when that one has it, so the spaces between Arabic words stay in the Arabic font; failing that, the first font with its first character, else the first font. Runs split where the font changes too. Positions are in the first font's units, a run's pen scaled as a whole so rounding does not add up. Callers can narrow what a font has (a symbol file holds only the glyphs its codes reach). vega_default_ignorable is shape.c's HarfBuzz 10.2 set, now public. 2026-09-28T14:20:08Z eunhhu necpition205@gmail.com 2026-09-28T14:20:08Z urn:sha1:bafb710f0047751e26b87897eaf2a05609708536 The rest of HarfBuzz 10.2's Arabic shaper. Joining types now come from HarfBuzz's own table (Unicode 16, with Syriac's Alaph and Dalath-Rish groups the state machine's last two columns use), the general category deciding where it has none, so Syriac shapes with fin2, fin3 and med2. What stch multiplies is recorded and, after positioning, stretched under the word before it, repeating pieces copied as often as fit. A font with presentation forms but no Arabic features gets HarfBuzz's fallback lookups, which the build tool writes as OpenType and the runtime applies at their pause like any GSUB lookup. The generator now reads HarfBuzz's generated tables through their lookup functions' ranges and offsets: rows whose range starts mid-row have blank cells in front, which the row comments alone miscounted (Arabic Supplement); the Indic table, read again this way, did not change. Noto Sans Syriac, FreeSans's Syriac, Unifont's Arabic and Noto Sans Arabic shape as HarfBuzz does at 3,000 runs a plan; no installed font is refused any more (213 fonts, 1,822 plans).