Surface runtime projecting Pure views across pixel, terminal, CLI, and web. http://git.yeowoo.org/vega/atom?h=master 2026-09-28T15:35:18Z 2026-09-28T15:35:18Z eunhhu necpition205@gmail.com 2026-09-28T15:35:18Z urn:sha1:434e8a9930d954494a462690739ea4ead21ff4e8 Bringing in HarfBuzz 14.5 and Unicode 18 broke both generators and showed over ten changes to shaping output since 10.2. Following them is cloning someone else's implementation, bugs and all, release after release: a cost without end, and no compression of abstractions. So the text subsystem stops here: no more chasing HarfBuzz's bugs and habits (AAT, Zawgyi, Arabic fallback over shared glyphs stay refused), and upgrades only when needed (a new script, a security problem), with text/upstream.py. Left as a design task for later: compiling a font's whole behaviour into Vega's own format at build time, so that shaping at run time no longer tracks HarfBuzz. 2026-09-28T15:27:28Z eunhhu necpition205@gmail.com 2026-09-28T15:27:28Z urn:sha1:96c35e7919e5e25187bd5adedda516a32cfc8639 text/upstream.py brings in a HarfBuzz release (default: the one the build links) and the UCD of the Unicode version its tables came from, builds that HarfBuzz as one static library from its amalgamated source (no system package, any version), regenerates the Unicode tables and the syllabic ones, holds the syllable machines to HarfBuzz's compiled ones, and runs the text checks built against it: Unicode's conformance tests on that UCD, random shaping against that HarfBuzz, the layout's promises, the kern class tests, HarfBuzz's own suite. --corpus adds every installed font (text/fonts.py, the corpus runner, now in the tree). What a new version breaks shows as failing checks with the font and the input to port a fix from; the generated files are left changed for review. make check-text is those checks alone; UCD and HARFBUZZ_SRC default to build/upstream's when the tool fetched them. Its first run found that text/shape-build.c used clock_gettime without asking for POSIX, building only because the system HarfBuzz's headers did. Against HarfBuzz 10.2.0 built from source and the 16.0.0 UCD, every check passes in about a minute. 2026-09-28T15:27:03Z eunhhu necpition205@gmail.com 2026-09-28T15:27:03Z urn:sha1:fb8e49a31a7ba7f333965c93b08bd0c58120494e HarfBuzz 10.2's tables are Unicode 16.0, Vega's were the installed UCD's 15.1: characters new in 16.0 could be read two ways. The tables now come from the 16.0 UCD, and the line breaking rules are revision 53's: LB13's 'x IS' split into LB15c (SP / IS NU, as in 'subtract .5') and LB15d; the numbers of section 8.2's Example 7 made LB25 (new: IS x NU, PR x OP IS NU); LB19 split, with LB19a letting quotation marks between East Asian characters break; LB20a keeping a word-initial hyphen to its word; LB21a narrowed to Hebrew, hyphen, then not Hebrew. vega_east_asian now covers all of East_Asian_Width F, W and H. Unicode 16.0's conformance tests all pass (bidi 770241 and 91707, line breaks 16672, graphemes 1093, words 1826, sentences 512, normalization 19965). test-text reads NormalizationTest.txt plain or compressed, and passes over a UCD of another version than the tables' (the installed one here is 15.1). HarfBuzz's suite (5018 the same), the installed fonts' and Noto's random runs, and the surfaces' checks agree as before. 2026-09-28T15:15:37Z eunhhu necpition205@gmail.com 2026-09-28T15:15:37Z urn:sha1:e30e3c2f17fb26cddc91e9f950f6f26948fa8e0e A font whose GSUB grows the buffer past its bounds (text-rendering-tests' GSUB-3) made vega_shape fail. HarfBuzz goes on with the glyphs of the last pass that finished: every later pass does nothing, positions are the default advances. Vega's buffer operations already stand still once the buffer is unsuccessful, so it now goes on too, and a broken font shows what HarfBuzz shows rather than nothing. HarfBuzz's suite: 5018 of 5199 the same, none failing. 2026-09-28T15:13:58Z eunhhu necpition205@gmail.com 2026-09-28T15:13:58Z urn:sha1:ea6dbcf627e9a440590d0d35ff16ca7ae60d5719 The kern table's class subtables, OpenType's and Apple's, as HarfBuzz reads them (hb-aat-layout-kerx-table.hh, hb-ot-kern-table.hh): format 2's class tables give byte offsets into a kerning array, format 3's byte classes index a value list. Both go through the same pass as pairs (marks skipped, the kerning split between the pair). Format 2 filters the glyphs by the class tables' digests, as HarfBuzz's accelerator does, so a digest's false positive reads class 0 there too; what a subtable reads must lie within it, the last one's within the table. The build checks the subtables as HarfBuzz's sanitizer does, and still refuses state machines (format 1) and cross-stream kerning. text/test-kern.py makes fonts with format 2 and 3 subtables in every order, with the edge cases that reading has (value 1, glyphs outside a table, offsets below the array or odd, values past a subtable's or the table's end, classes and indexes past their counts): 24 fonts, 96 plans, the same as HarfBuzz. make check runs it where fontTools is. HarfBuzz's suite: 5017 of 5199 the same. 2026-09-28T15:07:05Z eunhhu necpition205@gmail.com 2026-09-28T15:07:05Z urn:sha1:cf6bcf8697589d2a6fc3c6d98bc52bf7eb1ea4ae The section now carries each sequence the font maps, a base and a selector, with the glyph HarfBuzz gives the pair (a default one's the base's own). As HarfBuzz's normalizer does, a sequence the font has becomes that one glyph, the two characters' clusters merged; one it lacks still passes on as two, the selector hidden. The header grows to 88 bytes for the table. The random check now mixes in the font's own sequences and stray selectors: 30000 runs over Noto Sans CJK's 14787 ideographic variation sequences are HarfBuzz's, and so are the installed fonts' and Noto's. HarfBuzz's suite: 5014 of 5199 the same, none different; what is left is what the build refuses and one run without a plan. 2026-09-28T15:04:05Z eunhhu necpition205@gmail.com 2026-09-28T15:04:05Z urn:sha1:960fa21762484d3014a6778a7673f15acf2d34bb The shaping section's character map came from hb_face_collect_unicodes. HarfBuzz's own suite showed where that list and its mapping part: it cuts a format 13 map short (AdobeBlank2's whole Unicode to one glyph), and a symbol map (3,0) it searches through the private use area, Latin and, in an old Arabic font, Arabic too. Now every code point is asked with hb_font_get_nominal_glyph (about 30 ms a font) and the map carried as groups: codes going to consecutive glyphs, or all to one (12 bytes each, so a format 13 map stays small). HarfBuzz's suite: 5011 of 5199 cases the same (1355 before). The installed fonts (213) and the Noto ones (21) still shape as HarfBuzz does on random runs. 2026-09-28T14:52:35Z eunhhu necpition205@gmail.com 2026-09-28T14:52:35Z urn:sha1:0b554f2ea62a921e1c8a43ffb24ad75f428af2d6 vega_draw keeps what it wrote as spans (VegaSpan: a row, the columns it was given, where its cells reached, its text and tone). A span written where the same pen stopped follows the one before it (a field's name and value, the help line, the prompt); one written after the pen moved (a table's column, a list's label, a badge) starts afresh at its column, and a badge sits flush with its end. When the first font shapes, the pixels set type: a span and those following it are one line of text, laid out as one paragraph in the font stack (bidi and kerning cross the spans; each glyph takes the tone of the span its character came from), from its column up to the nearest of its spans' ends and the next span on the row, cut at a grapheme cluster boundary with an ellipsis if it must be. The prompt gets a text cursor. --symbols repeats: fonts in order of preference (at most 8); the cells, when the first font does not shape, also fall back through them. The check sets a label in one script, alone in its list, and holds it to what the font that has it sets by itself (vega-symbols paragraph, which now sizes itself to its text with width 0), pixel for pixel: Latin kerning, Hangul, Arabic and Devanagari over DejaVu Sans, Noto Sans CJK KR and FreeSans. 2026-09-28T14:52:13Z eunhhu necpition205@gmail.com 2026-09-28T14:52:13Z urn:sha1:a4ca275ad1b5edc15474407e2852f8fe9abaac4e vega_layout takes fonts in order of preference (VegaFonts). A grapheme cluster goes to the first font that has all its characters (default ignorables aside: shaping hides them); a cluster of Common and Inherited characters stays in the font before it when that one has it, so the spaces between Arabic words stay in the Arabic font; failing that, the first font with its first character, else the first font. Runs split where the font changes too. Positions are in the first font's units, a run's pen scaled as a whole so rounding does not add up. Callers can narrow what a font has (a symbol file holds only the glyphs its codes reach). vega_default_ignorable is shape.c's HarfBuzz 10.2 set, now public. 2026-09-28T14:20:08Z eunhhu necpition205@gmail.com 2026-09-28T14:20:08Z urn:sha1:bafb710f0047751e26b87897eaf2a05609708536 The rest of HarfBuzz 10.2's Arabic shaper. Joining types now come from HarfBuzz's own table (Unicode 16, with Syriac's Alaph and Dalath-Rish groups the state machine's last two columns use), the general category deciding where it has none, so Syriac shapes with fin2, fin3 and med2. What stch multiplies is recorded and, after positioning, stretched under the word before it, repeating pieces copied as often as fit. A font with presentation forms but no Arabic features gets HarfBuzz's fallback lookups, which the build tool writes as OpenType and the runtime applies at their pause like any GSUB lookup. The generator now reads HarfBuzz's generated tables through their lookup functions' ranges and offsets: rows whose range starts mid-row have blank cells in front, which the row comments alone miscounted (Arabic Supplement); the Indic table, read again this way, did not change. Noto Sans Syriac, FreeSans's Syriac, Unifont's Arabic and Noto Sans Arabic shape as HarfBuzz does at 3,000 runs a plan; no installed font is refused any more (213 fonts, 1,822 plans).