aboutsummaryrefslogtreecommitdiffstats
path: root/research
Commit message (Collapse)AuthorAgeFilesLines
* 030: the numbers from one build (the runtime 40,847 and the hand path 813, ↵HEADmastereunhhu22 hours1-3/+3
| | | | cycles 15,735 and 348, beside Vega's 251 and 93; other builds 1-2% apart) and 602 named as the substituted algorithm (373 cycles); 9.43x said to be the emitted chain's factor, the executor's share step 1's first measurement; 009's tension stated once (sparse updates in the log's delta, whole numbers zigzag there: a second shape, the cheaper one); the caveats of 047 together; model.py's import where imports go
* 047: "the leaf is the commit" measured against the bars set before: appends ↵eunhhu23 hours4-7/+177
| | | | pass, sparse updates do not, as said beforehand that means two units, not one: list bytes a message at 1 / 16 / 256 a commit, entries' appends today 65.5 / 15.7 / 12.4, leaves cow 1,636 / 102 / 14.5, leaves with appends in place past every kept snapshot's count (tail) 25.9 / 10.9 / 10.0; the bank's transfers today 122 / 35.6 / 30.1 (bars 244 / 71 / 60), cow 7,428 / 2,098 / 996, tail 4,826 / 1,975 / 992, cells written in place 186 / 126 / 122 but seen by kept snapshots (933,888 / 542,208 / 10,848 times); after the run, a leaf with room for changes (delta, 16 B each, the leaf written anew when full): 739 / 545 / 292 with room 8, 363 / 268 / 176 with 32, 249 / 161 / 136 with 128, every one over (today's log puts a change in about 7.5 B: varint places, zigzag wholes, one frame a commit); no kept snapshot sees a write of cow, tail or delta; rows as leaves (packet.c): 44.4 machine instructions a row to check and take 32-row column chunks against pure_list_read's 1,318 (cycles 14.2 against 372), every cell the same
* 047: "the leaf is the commit" (feedback 009) set up before it runs: today's ↵eunhhu23 hours5-3/+288
| | | | store measured as the base (baseline.py, 046's shim and journal: list bytes a message, the log and images, at 1 / 16 / 256 messages a commit: entries' appends 65.5 / 15.7 / 12.4, the bank's transfers 122 / 35.6 / 30.1, the tree apart), and a byte-accounting model (model.py) of commits writing the leaves they changed in memory's own bytes (leaf 296 to 1,320 B, inner node 680 B, cow; tail appends in place past every kept snapshot's count; inplace, which may not be, to show why), its bars written before the run (appends at most today's, sparse updates at most twice, no write a kept snapshot sees) and what a pass for appends with a miss for updates would mean (two units, not one); 046's Run takes pure-run's arguments; 045's keyed file says 24 refused, not the 26 its commit said
* 045: the app keyed and its range found by a seek, emitted as it says (no ↵eunhhu23 hours4-8/+373
| | | | algorithm the app does not state, nothing copied out of the tree): aup takes ranges as a list keyed by last and first(x in ranges where x.last >= code) as its range, the executor answers every line and is itself 40,258 to 27,934 machine instructions a point; the hand rungs on it (the seek by each level's guide keys and the leaf's key column, the found item's fields read from it, the other ranges read by key): keyed_shared 788 a point, end to end 813 with 348 cycles (the position-halving seek of a1382de, another program, 604 with 373 cycles; Vega's C 251 with 93)
* 045: feedback 009's zero copy measured: ranges read where its tree keeps it ↵eunhhu23 hours3-14/+163
| | | | instead of copied out (stored_flat, which 009 forbids): through pure_list_run 7,931 machine instructions a point, the tree's layout read directly (the descent by the guides' counts) 7,135, both above the copy's 6,136, and on them the later rungs worse (shared 1,576 against the copy's 564): a positional halving descends the tree each probe; the halving done by the tree's own order (each inner node's children by their first items' starts, read in place, then within the leaf) 6,137 and shared 578, the copy's within 2.5% (cycles 388 against 312), end to end 602 a point (2.5x Vega's C, cycles 4.1x); that needs ranges ordered by start (the data is; a keyed list's seek would say it); the list's code built into the harness for it (-O2, inlinable: exact 8,532, the runtime 40,745)
* 046, 029: the taken-over store sent a message too (opened() works on a copy: ↵eunhhu23 hours6-14/+59
| | | | show, add, show; 20 more kill -9s: all open and take it as owners; 667 prefixes again with it: all pass); the price measured again without the fold that inflated it (entries.aup's summed view folded every entry each message): 48.5 to 232 us a message on tmpfs, 146 us of the 183 waiting for acks; the shim never stopped across the runs
* 046: 029's principle 2 put to its breaking experiment on one Pi, the tools ↵eunhhu23 hours13-0/+2066
| | | | and the numbers: an owner (pure-run --store) with shim.so preloaded ships every change its store makes to its files (ops.h: create, write, truncate, allocate, rename, unlink, mkdir), cut at each fsync or fdatasync, over pure link (shipper.c, follower.c), and its sync returns once the follower has the cut (synchronous: one cut in flight, no log); identity: 100,000 messages (200,081 cuts; 34 compactions, an image written again) and 200,000 (400,149 cuts; 67 compactions, a second log segment), the stores compared byte for byte every 1,999 and 3,001 cuts and at the end: 0 differences, the follower's store opened as owner holding every message; takeover: 70 kill -9s (owner alone, the whole node, the node with cuts of 512 and 64 B records): all open, hold every message the owner said done, none not sent, the entries in order (and equal to the owner's own store each time); prefixes: 867 op boundaries of a 100,000-message stream (568 within cuts, 266 around installs) opened as stores: all pass; the price: 204 to 400 us a message on tmpfs (155 us waiting for acks, two cuts a message), 903 bytes shipped a message, 42% of them zeros written ahead (a compaction's new file and the tree's tail)
* 045, 029: a message end to end, the event's border written by hand too ↵eunhhu24 hours2-2/+149
| | | | (app.c's BORDER: rows whole and finite, the budget and memory begun again, the one state the handler changes written apart and made the state's at a faultless end into a buffer no reader holds, events and changes counted; every end answers all 18,672 lines, the broken-rule build refused at both hand ends): machine instructions a code point, the runtime 40,365, its empty handler 1,595, the hand border with shared 592 and with plain 379, Vega's C given the same payload 240: 68x, 2.5x C (cycles 3.2x), the border 1,595 to about 22; what went: the rows rebuilt into a tree each message (pure_list_read's make_tree), 54 states walked five times an event (40 of them t.X's registers), the last event's nested lists, a bytes state set whole through a text's piece tree; no new fact (N2, today's proof, the language, forms chosen)
* 045: UAX #14's whole app emitted by hand, the chain (app.c, app-way.h: on ↵eunhhu24 hours4-0/+703
| | | | line, decide and its fns written once against a rung's macros; every rung answers all 18,672 lines as the test and exactly as exact; with LB18 made wrong every rung is refused): machine instructions a code point, the handler: exact 8,785 (registers and plain calls; doubles, fmod, whole and bounds checks twice, the event's lists FLAT PureLists with list_held and grows at each append, Text's lists copied, ranges the stored tree, the budget), integers 7,979, ranges' columns flat 6,561, the event's lists typed buffers 696, Text sharing them 564, no bounds checks 502, no charges 362 (the same algorithm in C), Vega's C 225; the executor 40,189 and stitched 28,089 hold the event's own 1,566 besides (an empty handler: decoding the points into a list, clearing and freeing the event's lists, malloc, the breaks state through the store tree)
* 045: range() emitted by hand, Pure's meaning kept or dropped where the proof ↵eunhhu25 hours3-0/+297
| | | | could license it (range.c; every way's answers the executor's and the stitched build's, for all 61,285 points): a code point's machine instructions in the executor 14,488, stitched 8,144, exact (registers, a plain call; doubles, fmod, the finite and whole checks, the tree read, the budget) 2,474, integers 1,587, a flat column 1,083, both 226, C 129; a round of halving exact 217, both 19.8, C 11.3: the bar (3x C's round) missed by exact, passed by both
* 045: Pure's answers checked for real (cost.c and where.c compared only the ↵eunhhu25 hours3-23/+33
| | | | C's with the test before; now Pure's boundaries are read from its view, and a wrong variant is refused with 0 lines right); the executor and the stitched build, rebuilt at adccf58, both answer all 18,672 lines, and the stitched build's numbers stand (40,189 to 28,089 machine instructions a point, 1.43x; cycles 1.32x); 029 says so
* 029: principle 1's break by mechanism (045's samples): dispatch 30.6%, lists ↵eunhhu25 hours1-0/+25
| | | | built 17.3%, numbers 13.5%, lists read 13.0%, moves 9.0%, calls 7.1%, malloc 4.0%: no layer holds most of the cliff, three generic ones (interpretation, the lists' form, the numbers') 1.4-1.5x each, the cliff their product; range()'s binary search 36% of the machine instructions and 269.5 of 549.9 executed (an algorithm Vega's C does not use); a candidate broken by arithmetic before code (let lists held again as versioned states: 13,000 of 40,236 at most, 1.5x); the bar's arithmetic (one machine instruction an instruction is still 2.5x C: the instructions must fall too); "the general JIT 008 forbids" corrected (008's task A asks for one execution principle from the proof; it forbids the executors' split and scaled-down techniques)
* 045: where the machine instructions go, by mechanism (where.c samples the ↵eunhhu25 hours6-0/+514
| | | | address every 100,003 user instructions from perf_event_open's ring, the answers checked first; where.py names each by addr2line's inlined chain: the step() case its line is in, or the function outside): the app's 40,236 a code point are dispatch 30.6% (the switch and run_calls' loop), lists built 17.3%, numbers 13.5%, lists read 13.0%, moves 9.0%, calls 7.1%, malloc 4.0%; no mechanism near most; range()'s binary search alone 14,489 a point (36%, variant v0r): 11.4 rounds of about 1,270, of which floor division with fmod 186 and a stored read 240
* 029: principle 1 broken on UAX #14 (045), written as its own rule asks: (a) ↵eunhhu25 hours1-0/+35
| | | | 2.19x against 4x, its premise wrong (the source already keeps t.cls[i] in locals: the reads only the proof merges are 16.4 of 549.9), (b) 1.31x against 3x, and the path capped (interpretation is 22 of an instruction's 74 machine instructions: stitching bodies gives at most 1.42x); what was tested (step() on constant instructions, frame and checks as they are) and what was not (an emitter given the proof, the general JIT 008 forbids); where it goes (runtime machinery: a call 1,375, a stored read 331, the lists 22,651 a point) and the 7x the batch lanes give that a fn in a block loses; the 1,774 and 3.2-3.9 ns corrected (the budget's count: 549.9 run, 11.7 ns each) here and in 140; the op counts kept
* 045: 029's principle 1 broken on UAX #14 before any emitter, the tools and ↵eunhhu25 hours16-0/+607
| | | | the numbers: (a) the executor runs 549.9 instructions a code point (the budget's 1,774 counts blocks as written), moves 205.5, constants 76.7 and reads of the same list, version, field and place in an event 16.4, so merging leaves 251.3: 2.19x, the bar 4x (ops-count.patch); (b) the executor's own steps stitched on constant instructions, a function a basic block (stitch.c, native-hook.patch), answers checked: 40,703 to 28,601 machine instructions a point, 15,566 to 11,850 cycles: 1.31x, the bar 3x (cost.c); Vega's C 220 and 82; where it goes: dispatch 22 machine instructions an instruction of 74, an empty handler 1,583 a point, the lists built 22,651 more, the Text record 2,951, decide 12,988 (events.c, variants); a stored list's read 331, a block fn's call 1,375, an arithmetic step 248 in the executor and 35 in batches (unit.c, units)
* 115: the modules replaced are freed (0 bytes a change over 4,000, was ↵eunhhu27 hours1-0/+4
| | | | 19,284), how, what checks it (586 of 600 loads at a freed address, answers right; ASan clean), what a mutant comparing addresses showed (nothing: requests hold their module and freeing waits a pass), the push group woken again, the bindings left (24 B an input a module); the bank's queries within 0.4% (s50, s51); CHECKPOINT's remaining list without it
* The modules an app ran are freed (research 115's next step): every app's ↵eunhhu27 hours7-4/+323
| | | | module a Loaded with holds (each snapshot of it, each request read with it); one replaced is retired and freed on the apps thread once no app runs it, every connection thread has begun a pass since (woken for it), and nothing holds it; who keeps a module to ask later whether it is the same asks its serial (core: PureModule.serial, a reader's module_serial, pure_snapshot_module, pure_snapshot_release says it freed; serve: routes, groups, links, bindings, queries, the published snapshot), as an address may come back; a link binds under its receiver's lock, a link naming an input under binding_lock; 030's memory.py: 4,000 changes 0 bytes a change (was 19,284), 4,100 freed; stress.py: readers, view workers, push streams and a body across changes, 586 of 600 loads at a freed module's address, every answer right, ASan 1,000 changes clean; a push group that looked while its app resumed missed the new snapshot (published before the app took requests) and waited for its next save: woken again once it takes them (server_view_workers had failed once under load)
* 140: determinism across builds at 027, 028 and 140's state: record_all.py's ↵eunhhu29 hours2-0/+972
| | | | 486 runs (41 modules) the same with gcc and clang on the Pi (Pure 0896b38, aup 1efe1c7; aup's modules the same bytes under clang); the Mac and WSL2 not reached (ssh timed out, no route), the bundle and r140.sh ready for them
* 029's record_all.py: the pure-runs of the checks 027, 028 and 140 added, ↵eunhhu29 hours1-0/+67
| | | | recorded run by run for machines to compare (aup's 018 checks as 033's record.py does, its 027 stages', 028's bytes state, 140's frictions; Pure's module_specs, the nine specs on the three paths, Pure's specs copied under the temporary directory so no machine's paths are in an input): 486 runs of 41 modules on the Pi
* 140: LIST_VALUE's copy reviewed before building (measured by spec: UAX #14 ↵eunhhu29 hours3-0/+32
| | | | about 10% of its time, JSON tree 6%, HTTP 2%, inflate 0.06%, the rest none; the cost a tree built item by item, 64 ns an item and 250 ns a copy): lend and detach (CoW) noted as the decision point (a branch on every event list's change for 10/6/2/0/0/0), a compile-time move ruled out (it changes LIST_VALUE's contract), the arrays copied whole built (UAX #14 -6.8%, its share to about 3%); 041's five works within 0.03% (b49, b50), the bank's queries within 0.3% (s49, s50), noise between builds larger
* 140: aup's frictions written away and the apps measured again (no core ↵eunhhu30 hours2-2/+13
| | | | change, no new opcode): lists declared together, targets together (straight when no value can read what a target changes), a list given a list value whole, abs, tick.period, hex where numbers go; lines to Python 1.47-2.04x (were 1.47-2.15x): UAX #14 176 to 169, layout 116 to 110, HTTP 148 to 146 and its last workaround gone; instructions of the four whose modules changed within 0.2%, inflate's and UAX #14's modules the same; the six specs' checks pass on the three paths, module_specs nine; RTO's checks no longer rewrite G (tick.period); the suite table's rows 5, 8 and 9
* 140's 8, box layout, written in aup with no new opcode (aup's ↵eunhhu31 hours8-0/+625
| | | | examples/layout.aup: a subset of CSS Flexbox 1 written down first, one line, row or column, padding, gap, six justify-content, four align-items, grow and shrink with 9.7's loop, cross sizes stretched, nothing sized by content): 2,000 random trees (29,388 nodes; 7,659 items frozen at their padding by the loop) are Taffy 0.15's within 0.01 (layout_taffy, built by cargo; the worst 8.3e-5) and layout.py's bit for bit on the three paths, the paths the same; the single-pass subset first tried refused 66% of trees, so the loop came in; Taffy given min sizes equal to the padding (given 0 it floors at 0 while resolving lengths, where 9.7 floors the content box), the reference adding from the left (sum compensates since Python 3.12); module_specs in make check: 100 trees (tests/specs/layout); bars: 116 lines against Python's 54, 4,754 ns a node, Taffy's 644 (7.4x), pure Python 5,396 (Pure 1.14x faster), 1,494 instructions a node (9.7's loop 44%); four frictions; aup's rows read by position no longer copy (99,067 ns a node before); the suite table's row 8; CHECKPOINT's direction row
* 140's 10, RFC 6298's retransmission timer, written in aup with no new opcode ↵eunhhu31 hours3-0/+186
| | | | (aup's examples/rto.aup, the timer a deadline in the state seen at each periodic tick): 20 scenarios (10 seeds, ticks every 10 and 100 ms; a closed loop: first sendings lost one in five, the receiver acknowledging cumulatively) and their 241,612 events right against rto.py, the model of the same rules (no known implementation takes given events), on the three paths, the paths the same; module_specs in make check: one scenario at both periods (tests/specs/rto, 2,297 events); bars: one workaround (the periodic tick: 2,795.8 ticks a timeout at 10 ms, 281.6 at 100, seen 4.27 and 49.23 ms late on average; ticks 93% and 57% of the counted work), 56 lines against Python's 30, three frictions (an input's period not readable, no abs, send a keyword); the one-shot timer's shape as a decision point, one spec needing it: noted, not made; the suite table's row 10
* 140's 5, line breaking (UAX #14, Unicode 16.0), written in aup with no new ↵eunhhu32 hours4-0/+396
| | | | opcode (aup's examples/linebreak.aup, the properties as data: 2,787 ranges from the UCD 16.0 Vega's text is frozen at, built by Vega's text/unicode.py, loaded once): every line of LineBreakTest.txt (16,672) and 2,000 random lines (their boundaries as linebreak.py, the Python reference, which passes the test whole, works them out) right on the three paths, the paths' outputs the same; module_specs in make check: the ranges and the lines kept in tests/specs/linebreak (derived from the UCD, the Unicode License v3's notice), run by pure-run --views; bars: written, no workaround (three frictions: hex literals always i64, number no let's name, a let's list one a line), 176 lines against Python's 99, 6,920 ns a code point against Vega's C 34.7 (199x), pure Python 2,523 (Pure 2.7x slower), 1,774 instructions a code point (setup 260, the ranges' binary search 360, decide about 1,150 a boundary); the decisions before 8 (box layout: Taffy can be fetched by cargo, a subset to choose first; as a handler) and 10 (a one-shot timer is a core primitive one spec needs: noted, not made); the suite table's row 5; CHECKPOINT's direction row
* 140 measured again without the workarounds: the three specs' checks pass on ↵eunhhu32 hours7-7/+30
| | | | the three paths as before (zlib's 640 streams and 240 damaged, dnspython's 400 messages and 300 damaged, h11's 200 streams on every cut and 200 damaged); lines inflate 132 to 121, DNS 115 to 106, HTTP 159 to 148 (Python's 1.5-1.9x, was 1.6-2.0x); time inflate 1,152 to 1,139 ns a byte (102x C), DNS 244 to 221 (61x, instructions 63.6 to 49.9 a byte: == in its compression), HTTP 278 to 270 (83x, with the free empty join); left: DNS's view calls no block fn, HTTP's let list takes no list value; module_specs, the checks and the timing tools read the bytes states (inflated, pending); the suite table's rows for 1, 3 and 9
* CONCAT with an empty side gives the other as it is, uncopied (values do not ↵eunhhu32 hours2-0/+22
| | | | change), so a bytes state joined while empty (HTTP's pending + packet.data) no longer copies the packet: HTTP a packet at a time 288 to 270 ns a byte at the same instructions, as the old app (270.0); counts: b48 against b47 knapsack without batches -0.42%, join -0.16%, BFS with batches -0.12%, the rest within 0.03%; the bank's queries (s47 and s48 in turns, six each) in_credit -0.34%, field -0.34%, found -0.22%, per_branch -0.20%, large -0.18%, ranks -0.13%, none the same, the seven answers the same bytes
* 140: small block fns inlined in aup, made and measured, not merged (the ↵eunhhu32 hours1-0/+276
| | | | patch kept): a block fn whose only return is its last line, with no loop or called where the stack is empty, written out where it is called, its parameters slots of its own (one set a fn and kinds of arguments in a handler); the three specs' checks passed, but inflate 191.2 instructions a byte as before and HTTP 61.4 to 62.1 (+1.1%, 277.7 to 274.9 ns): the hot fns (inflate's decode, HTTP's crlf) return early inside a loop, a fn with a loop cannot be written out where values are on the stack (Pure opens no loop above values), and storing a short body's parameters costs what CALL and PICK did; inflate's 23% was the expression fn bit's (a loop, a let and conversions gone with the call); what would make it pay is Pure's (loops above values, or a cheaper frame), one spec behind it; the decision point's list of what was made after it
* 028's bytes alone as a state, made: module state type 6 read as a text state ↵eunhhu33 hours2-0/+22
| | | | flagged PURE_TEXT_BYTES in state_flags (so serve's same_states compares it for queries and replaces), its TEXT_LOAD giving bytes and its TEXT_SET taking them, a cursor's ops on it refused (anchor in bytes) and an anchor in it refused at load; one store file text-NAME, the values record marking each bytes state (kind 7, its name, 8 zero bytes), a text state's file that is no UTF-8 its initial value when marked and store_error when not (as before); a replace carrying bytes to a text only when its draft is UTF-8, else opening the store's draft anew; pure-run shows a bytes state's hex; unit test bytes_states (seven starts from a store: 0xff and 0x00 kept, bytes to text to bytes to text with no UTF-8 and with it; three replaces; four refused); counts: b47 against b46 BFS with batches +0.12%, the rest within 0.001%; the bank's queries (s45 and s47 in turns, six each, bytes compared among them) in_credit -0.31%, per_branch -0.18%, ranks -0.11%, the rest within 0.01%, the seven answers the same bytes; Pure, Vega and aup checks green; 008, 028
* Bytes compared (140's decision point: DNS and HTTP both met it): ↵eunhhu33 hours1-0/+10
| | | | TEXT_COMPARE's argument takes PURE_COMPARE_BYTES (bit 8) beside its relation, its two then bytes (UTF-8 asked of neither), as CONCAT's argument says bytes; verified as A_ORDER (a relation, and the bit), the executor relating by the argument's relation alone; the JIT, the tier and the batch executor leave TEXT_COMPARE to the executor as before; unit test bytes_compared: eight relations of a constant's bytes, none, 0xff and 'A' (0xff after 'A', a longer after its first byte), four refused (the bit on texts, no bit on bytes, a text with bytes, a relation past NE); counts: b46 against b45 the same on all five works, with and without batches; 008
* 140's 9, HTTP/1.1 requests a packet at a time (RFC 9112), written in aup ↵eunhhu34 hours4-0/+578
| | | | with no new opcode (aup's examples/http.aup: each packet an event, its bytes added to what the last left, every request they complete read, h11's grammar): 200 streams (132,623 bytes, 578 requests: methods, targets, HTTP/1.1 and a last 1.0, mixed-case names, spaces around values, Content-Length and chunked bodies with extensions and trailers, some ending in part of a request) fed whole, at random boundaries and a byte at a time (the 50 under 400 bytes), the requests read h11's on every cut and the bytes left those after them, the three paths' outputs the same bytes; 200 damaged streams fed whole as http_requests.py (the Python reference) reads them, 35 refused by both and 165 the same; module_specs in make check: 80 streams and their reading (tests/specs/http, no h11 at check time) whole on the three paths and cut 88 ways; bars: written, three workarounds (the bytes left in a list of one: bytes are no state; names and codings compared as text: bytes have no ==; field lines returned in a record: a fn returns no bare list), 159 lines against Python's 79, 278 ns a byte against the same algorithm in C 3.27 (85x), pure Python 67 (its bytes methods are C: Pure 4.1x slower), h11 284; 140's decision point after 027: bytes alone as a state (inflate, HTTP) and bytes' == (DNS, HTTP) now each met by two written specs, so the two-spec rule lets them into the core (their design not begun); a view calling a block fn met by one (DNS); aup's three its compiler's; CHECKPOINT's direction row
* 140's 1, DNS messages (RFC 1035), written in aup with no new opcode (aup's ↵eunhhu35 hours4-0/+527
| | | | examples/dns.aup: a message read by a handler into its header and records, names uncompressed in their wire form, those in NS, CNAME, PTR, MX, SOA and SRV data too, each pointer pointing before the last; written back by the view wire, the owners compressed): 400 messages dnspython made (148,582 bytes, 3,466 records: A, AAAA, NS, CNAME, PTR, MX, SOA, TXT, SRV, EDNS, shared suffixes, 63-byte labels, bytes above 0x7f, 245-byte names) read on the three paths as dnspython reads them (each section's records as a set) and written back to bytes dnspython reads as the same message, the paths' outputs the same bytes; 300 damaged messages as dns_message.py (the Python reference) reads them, 185 refused by both and 115 the same records and bytes; module_specs in make check: 200 of those messages (tests/specs/dns, no dnspython at check time) read on the three paths, written back and read again to the same state and bytes, 23 cut ones failing; dnspython shuffles a set's records unless want_shuffle is off, and merges a set's differing TTLs to the least; bars: written, two workarounds both the core's (bytes have no ==: TEXT_COMPARE takes text alone; a view calls no block fn: CALL is in handlers and functions alone), 115 lines against Python's 72, 244 ns a message byte read and written against the same algorithm in C 3.65 (67x), glibc's resolver reading alone 4.4, pure Python 131 (Pure 1.9x slower), 218 compiled; 63.6 instructions a byte, reading 42%, writing 58%; CHECKPOINT's direction row
* 140's 3, inflate (RFC 1951), written in aup with no new opcode (aup's ↵eunhhu35 hours8-8/+458
| | | | examples/inflate.aup, codes read a bit at a time from canonical counts as zlib's puff.c): 640 streams zlib made (6,488,480 bytes: random bytes, words, runs and repeats 32,768 back; levels 0, 1, 6, 9 and zlib's five strategies, so stored, fixed and dynamic blocks) inflated to the original's bytes on the three paths, the paths' outputs the same bytes; 240 damaged streams (cut, a bit flipped) read as inflate.py (the Python reference) reads them, 165 refused by both and 75 the same bytes; module_specs in make check: 144 streams on the three paths, 21 cut ones failing and leaving the last bytes; bars: written (no new opcode), three workarounds (lists passed in records: a fn takes no list; the output a list of one: a bytes value is no state; a block's tables built in a fn: a let's list lives in the handler's own block), 132 lines against Python's 81, 1,152 ns an output byte against the same algorithm in C 10.7 (108x) and zlib 3.5 (329x), pure Python 1,214; 191 instructions a byte (a block fn called for each bit was 246 and 1,553 ns: small block fns' inlining is 2c's leftover); the four timing tools' message calls fixed for 4b's API (no keys); CHECKPOINT's direction row
* 140 after 027: the ten probes compiled again (run.sh writing to a temporary ↵eunhhu35 hours12-3/+83
| | | | directory: 10-13's -o /dev/null made a compiled probe say Permission denied): three compile as written, the other seven's diagnostics now mostly naming the form, so the same needs written in 027's forms (probes-027/): a byte an i64, bits on i64s, a record holding itself through a list, a recursive fn in a handler with its result type, a character by for and char, a variant in a list's item all compile; three gaps left, each a change of the core: a view calling a recursive fn or a fn with a block, a record or a variant as a state by itself, a character at a cursor; the suite table's column after 027 (2 and 4 written, 6 and 7 work, the first needs of 1, 3, 5 and 9 compiling, 8's layout as a view blocked by the first gap)
* 027's stage 5 closed: two numbers (number, binary64, the default of literals ↵eunhhu35 hours10-0/+39
| | | | and of counts; i64 the exact integers, bits and money), / a number's division and // the floored quotient (FLOORDIV, 95 opcodes now 96), an i64 sum exact, money i64 minor units shown by .Nd, a replace and stored values moved when exact; the bar: TPC-C's money and rates integers in Pure and SQLite alike, every row of every table exactly the same after 20,000 transactions; core 22,766 lines to 22,791, serve.c 5,058 as it was, aup's compile.c 5,720 to 5,726; counts: b45 against stage 4's end (b42) within 0.04% on all five works; the bank's queries (s42 and s45 in turns, six each) per_branch -0.58%, in_credit +0.56%, ranks +0.20%, the rest within 0.01%, the seven answers the same bytes; TPC-C in RAM (W=1, 64 terminals, 20 s, in turns, three each): s42 on its own app (money numbers) 46,092 transactions a second, s42 on stage 5's (money i64) 47,303, s45 on it 47,287: the runtime the same (-0.03%), the app 2.6% faster with i64 money (the server's CPU a transaction 42.3 to 39.6-39.8 us, not taken apart), new-order p99 3,584 to 3,456 us; 027's status: the five stages made; CHECKPOINT's direction row through stage 5
* 027's 5c: money as i64 minor units shown by .Nd: NUM_TEXT's decimal form ↵eunhhu35 hours2-0/+22
| | | | takes digits (bits 20-24 as FIXED's), an integer's last N digits written after a point (1234 at two is 12.34, -5 -0.05, 0 0.00; width and zeros after the sign), verification allowing digits on FIXED and DECIMAL alone; unit test: ten rows of formats_and_held (-2^63 at two, 25 digits, a number's whole 1234, its 12.5 a limit); TPC-C's money i64 cents and its rates ten-thousandths (tables, messages, load, the transactions), SQLite's INTEGER columns and int64 bindings alike, the comparison now exact (no 1e-9 tolerance): after 20,000 transactions every row of every table the same (grouped SQLite too) and 3.3.2 holding on both; counts: b45 against b44 BFS without batches +0.09%, the rest within 0.001%; the bank's queries (s44 and s45 in turns, six each) in_credit -0.59%, per_branch -0.34%, ranks -0.20%, the rest within 0.01%, the seven answers the same bytes; 008 (NUM_TEXT's digits, a message row's i64 cells), 027 and 092
* 027's 5b: an i64 sum exact: its items gathered in a 128-bit integer and only ↵eunhhu36 hours2-0/+22
| | | | the result checked against the range (overflow past it), so a sum past the range midway and back in it at the end is the answer and an i64 sum's order is none of its business; the batch executor's folds are numbers' lanes alone, summed in the items' order, as before; 008's FOLD says so; counts: b44 against b43 within 0.04% (BFS without batches -0.03%); the bank's queries (s43 and s44 in turns, six each) per_branch -1.32% (all six below s43's lowest: a larger Fold lays the number sum's loop otherwise), in_credit -0.72%, ranks -0.26%, the rest within 0.01%, the seven queries' answers the same bytes on both; 008 and 027
* 027's stage 5 designed and 5a made: two numbers (number, binary64, the ↵eunhhu36 hours2-0/+22
| | | | default of literals and of counts, positions, lengths and u32 fields, whole to 2^53 exactly; i64 the exact integers past it, bits and money, never mixed, moved by number() and i64() when exact), / a number's division, // the floored quotient, i64 folds exact (only the result in range, so in any order), number folds in order unless the batch executor's 2^53 test says any order is exact, money as i64 minor units shown by .Nd, a replace carrying number, bool and i64 when exact; 5a: opcode FLOORDIV (95, 96 now): a number's floored quotient as Python's float //, exactly (a less its floored remainder, divided, to the nearest whole; a zero signed as a / b; by zero divide), a new opcode so the JIT, the tier and the batch executor leave it to the executor; unit test floor_division (7 // 2, -7 // 2, 7 // -2, 1 // 0.1 = 9, 0 // -3 = -0, 1e300 // 7, by zero); counts: b43 against b42 within 0.001% but BFS without batches -0.09%; the bank's queries (s42 and s43 in turns, six each) -0.20% to 0.00%; 008 and 027
* 027's stage 4 closed: 106 opcodes now 95, 7 state kinds 6, 6 input kinds and ↵eunhhu36 hours8-0/+28
| | | | the frame and pointing 3 kinds and their bindings, a view's type a record field's, the fixed limits the format's and the list engine's widths alone; core 22,398 lines to 22,766, serve.c 5,086 to 5,058, aup's compile.c 5,923 to 5,720; TPC-C in RAM (W=1, 64 terminals, 20 s, in turns with stage 3's end, three each) 45,893 transactions a second to 46,063 (+0.37%), new-order p99 3,712 to 3,456 us; the bank's queries (stage 3's on its aup's module, five each in turns) field -1.23% to in_credit +1.15% (the noisy three within their 1-3% spread); CHECKPOINT's direction row through stage 4, its nesting as it is now
* 027's 4f-4: loops nested to any depth: the load counts a module's deepest ↵eunhhu37 hours2-0/+22
| | | | nesting (nesting), a run keeping PURE_MODULE_NESTING (8) on its stack as before and a module nesting deeper running in the executor that calls, its loops in its app's room (PureApp.deep, grown to it), so run()'s hot path is as it was; the JIT compiles modules nesting 8 at most; a function's nesting counted without a cap (u32), its frame sized to it; unit test: 40 loops nested, run to their end (2^20 rounds) and stopped by the work budget (2^40), 8 still on the stack; counts: b42 against b41 with batches BFS -0.12%, without BFS +0.09%, the rest within 0.001%; the bank's queries (s41 and s42 in turns, six each) in_credit +1.04%, per_branch +0.60%, ranks +0.36%, the rest within 0.01% (their spread: the bank has no functions, so none runs the changed executor, and s42's in_credit is within s41's own earlier six); 008 and 027
* 027's 4f-3b: no fixed count of records or of their fields in all: the ↵eunhhu37 hours2-0/+22
| | | | header's u16s alone (16 and 128 before); a stored list's record graph (PureLayout) holds a few records itself and more on the heap, 256 at most (an image numbers them in a byte), an image or a checkpoint not written then without room for it; a record's 16 fields kept, the list engine's width (a leaf's columns, a run's columns and narrow cells, the arrays' and packs' columns), to be widened with the lists' shapes sized to their fields; unit tests: 20 records of 10 fields (200 in all), an item of the last appended; a stored list whose six records each hold a list of the next saved and read back after a restart (its graph on the heap); counts: b41 against b40 with batches BFS +0.12%, without BFS -0.09%, the rest within 0.001%; the bank's queries (s40 and s41 in turns, six each) within 0.09% (the first three's in_credit +1.16% its spread: 2.75% between runs of one build); 008 and 027
* 027's 4f-3a: no fixed count of inputs, views (the load's columns among them, ↵eunhhu37 hours2-0/+16
| | | | in u16s), handlers or message fields: the header's u16s alone; a message's fields at most 64 (PURE_MESSAGE_FIELDS: a send pushes them all, as the stored sends' shapes hold them), outputs at most 255 (the store names them in a byte's count: the links and the outbox's image), the numbers of outputs a module dropped carried within it; an app's numbers of each input (last sent, saved, where the event began) in a block sized to its module's inputs (by_input), a replace laying the new one first and filling it by output name; image headers sized to the module (header_room); a view's arguments u32, a made column's field u16; pure-serve's timers grow to the module's inputs, its outlets and known senders its own count (OUTLETS, 32), a list view's JSON keys its columns on the stack for 64, else on the heap; unit test: 40 inputs and 70 views, the last of each used; counts: b40 against b39b within 0.01% with batches and without but BFS +0.09%; the bank's queries (s39b and s40 in turns, three each) in_credit -0.64%, per_branch -0.37%, field +0.34%, the rest within 0.22%; 008 and 027
* 027's 4f-2: no fixed count of states: the header's u16 alone ↵eunhhu38 hours3-0/+32
| | | | (PURE_MODULE_STATES gone); a module's transient and counted states are flags of each, the lists views and functions change or name sets of states (state_words words, a bit each), the verifier's walked lists a list with its length, duplicate names found by sorting them; an app's eighteen arrays of each state (values, texts, lists, the stored lists' slots, the last save's and the burst's, a reader's, an event's values and marks) in one block sized to its module, a replace laying the new module's apart and taking it once nothing can fail; the runtime's lists placed past any state (PURE_LIST_HEARD 0x10000), their slots after the states'; a snapshot's arrays sized so too; pure_app_read false without room (pure-serve then shows none); a save's drafts, values and the tree's revisions on the stack for 64 states, else on the heap; the JIT compiles modules of 32 states at most (its sets a word), the app's lists read through a pointer; a view reads the app's values as they are (no state changes in one), no copy of 32 a evaluation; unit test: 300 states loaded and the last counted; a hole closed: pure_app_stop read the arrays a failed start never laid; PureApp 4,256 bytes, not 8,504; counts: b39b against b38 with batches within 0.06%, without batches knapsack +0.49%, join +0.23%, the rest within 0.17%; the bank's queries (s38 and s39b in turns, three each) field -1.99%, found -1.18%, in_credit -1.10%, large -0.99%, the rest -0.58% to +0.11% (the first build, s39, copied a view's values at run time's size: in_credit +2.4%); 008 and 027
* 027's 4f-1: a module's tables sized to it: PureModule's thirty-four tables ↵eunhhu39 hours7-5/+21
| | | | of states, inputs, message fields, views (the columns the load makes among them, counted from the view and record entries first), handlers, records, record fields, functions and signature items point into one block the load lays out (lay_tables) and pure_module_release gives back, so PureModule is 2,752 bytes, not 24,320; a sanitized build poisons 16 bytes between tables, so reading past one is caught (none was); limits as they were; a test and five of research 044's timers gave the load a module never zeroed (it gives back what one holds), now zeroed; counts: b38 against b37 with batches within 0.25%, without batches matmul -0.45%, join +0.36%, knapsack +0.28%, BFS -0.25%; the bank's queries (s37 and s38 on one aup module in turns, three each) field -0.68%, found -0.51%, large -0.47%, the rest -0.11% to -0.20%; 027's 4f designed in four steps and 4f-1 as made
* Module format v7, 027's 4e: a view of any value: a view's and a column's ↵eunhhu39 hours6-0/+47
| | | | type a record field's (number, text, bool, i64, list, bytes; PURE_VIEW_TABLE now PURE_VIEW_LIST), a view entry 20 bytes (its table + 1, its items' record + 1, its list state + 1); a list view's rows a list state's items as they are or filtered (its record's fields its columns, made at load: each reads its item's field as it is), a list state's map (its columns following it, as tables were), or the items of a list its block leaves (copied into a kept list once a change, its rows and cells kept until the next); a list cell lent in its items' stored form (pure_app_cell_items, pure_items_next reading it), bytes as a text is; a pointing only at a list view over a state; pure-run writes bytes as hex and a list cell as an array of objects, pure-serve a bytes view as application/octet-stream and, in JSON, bytes as hex strings and list cells as arrays of objects; a hole closed: a view without a body never began its growth anew, so in a snapshot's reader (pure-serve's views) reading a kept list's list failed as memory (unreachable before 4e: no such view could be written), now marked a body at load; unit test list_views (as it is, filtered, computed, a map's list and bytes columns, a bytes view, six refused); check server_trees keeps its forests in a stored list shown by list views over HTTP against the model, a restart too; counts: the first build's bank tables rose (field +5.9%: view_of left inline at its seventh caller, serve's put_bytes too), so view_of is inlined, the made columns' and computed lists' paths and serve's new writers kept apart, a computed list's rows held as a filter's; b37 against b35 within 0.001% with batches and without; the bank's queries (s35 on the 4d aup's module, s37 on 4e's, six each in turns) field +0.67%, found +0.43%, large +0.31%, the rest within 0.20%; 008 and 027
* Module format v7, 027's 4d: every input a message: a module's inputs are ↵eunhhu40 hours3-0/+22
| | | | messages, outputs and arguments (kinds 1 to 3), a message bound to what makes it (PureBinding: a key's press or its release, a text input, a timer, a frame, a pointing at a table), each binding's fields fixed (none; one text; one number, dt; one u32, the item's position, the table its view), one frame input at most; a handler's event its input's number; no DT or TEXT_INPUT (97 opcodes now 95, FIELD reads dt and the text), pure_app_input, frame, tick, text and point, pure_module_position, PureEdge or an app's ticks: every event is pure_app_message, saved after it unless deferred; pure-run sends each command's message to the input bound so (a release or a frame no input takes still an event, clearing the fault, so the workbench parity holds) and defers a stored app's saves past frames as before; pure-serve posts a press with its key's release, refuses timers, frames and pointings (no_input) and ticks timers as messages; holes closed: the tier found a handler's input by the old numbering, reading a message's number field as another input's (aup's numbers caught it), and serve's replies_borrow did too; module-probe presses as messages; unit tests: the test modules' builder (expand) writes the old shapes as bound messages, pointing over a pointing input and its table, stored frames deferred as a host defers them, a press's handler refused a FIELD; counts: b35 against b34 with batches BFS +0.03%, the rest within 0.001%, without batches BFS +0.12%, join +0.01%; the bank's queries (s34 on the 4c aup's module, s35 on 4d's, six each in turns) field -0.23%, found -0.14%, large -0.12%, in_credit +0.11%, the rest within 0.06%; 008 and 027
* Module format v7, 027's 4c: cursors and indices one anchor: a module's state ↵eunhhu41 hours3-0/+22
| | | | type 2 is a place in the text or the list it names (a bool 4, an i64 5), and seven anchor instructions take the place of the thirteen of cursors and indices (103 opcodes now 97): LOAD and STORE an anchor state, START and END of a text or a list, MOVE, COUNT and AT naming it and a unit (a character, a line, an item); the load holds an anchor in a text as a cursor and in a list as an index (a mark in a list refused) and each anchor instruction as the runtime's own one for it (PURE_OP_CUR_*, PURE_OP_INDEX_*, numbered past a module's in module-run.h with the kept lists' steps; a module holding one refused), so the executor's, the batches' and the JIT's steps are those they were; the verifier holds MOVE, COUNT and AT to the text or list they name; new: AT in a text (a cursor so many characters or lines from its start, not_found past the end); COUNT's lines from 0; unit test anchors (AT and COUNT in a text, AT in a list, four refused); the tests' modules written with the anchor instructions; counts: b34 against b33 with batches matmul -1.36%, the rest within 0.08%, without batches within 0.04%; the bank's queries (six each, s33 on the 4b-2 aup's module) in_credit +0.28%, per_branch +0.23%, ranks +0.10%, the rest 0.00%; 008 and 027
* Module format v7, 027's 4b-2: keyed documents gone, a keyed list of records ↵eunhhu42 hours5-4/+28
| | | | holding bytes doing their work: no collections table, DOC, DOC_SIZE or REPLACE (106 opcodes now 103, those from LEN on moved up), keyed messages or a base revision (pure_app_message takes an input and a payload), no PureData document callbacks (a data host opens states and saves them together), no burst by document key; the header 38 bytes; pure-edit-module gone; pure-serve without /objects, ETag, If-Match and If-None-Match, 412 or an app a thread for a module of documents alone (one without states or outputs runs so); a hole closed: an instruction in no handler's, view's or function's block was never verified yet the load's later passes read its arguments (a damaged module read out of bounds once the header moved what the mutation test damaged), now blanked to CONST 0; checks moved: server_documents (DOCS_SOURCE, keyed byte documents with the app's revision, against a Python model, one and three connection threads, a restart) in place of the parity with editor.pure, pipelining and bursts through kill -9 on its inputs and views, the lent budget on a text view (lending a document's read is server_lent_views' already), clocks and tokens on it; message tests without a document host; the HTTP benchmark's document suite runs the earlier document server and the baselines alone; the count harness builds against either message API (runs.sh build: -DPURE_KEYED_MESSAGES before 4b); counts: b33 against b32 with batches within 0.002% but BFS -0.33%, without batches BFS +0.63%, knapsack +0.52%, join +0.38%; the bank's queries (s32 on a v6 module, s33 on v7) field +0.68%, found +0.44%, large +0.42%, the rest within 0.13%; 008 as v7, 027's 4b-2 as made and 4c designed
* Module format and executor: 027's 4b-1, a record field holding bytes (type ↵eunhhu42 hours3-0/+28
| | | | 5): held as a text is but for UTF-8, its default a constant; a function's bytes in a signature this type too (PURE_SIGNATURE_BYTES gone); no list keyed by bytes; replaced, a text field becomes a bytes field keeping its value (pure_cell_as, kinds_carry); images' headers take its kind; the JSON edge reads it from a string; pure-run shows it as hex; an append asks a record's fields held as bytes from a mask the load makes (record_held) instead of each field's type twice, so b32 runs fewer instructions than b30 (BFS without batches -1.05%, join -0.25%, BFS with batches -0.36%, the rest within 0.04%; the bank's queries s30 against s32 -0.01% to +0.30%); unit test bytes_fields (bytes not UTF-8, a text field replaced by a bytes field, three refused); server_trees with a bytes field; 008 and 027
* Module executor and pure-serve: 027's 4a, a message's rows holding lists: ↵eunhhu43 hours2-0/+14
| | | | rows whose records hold lists are read, as the message arrives, by stage 3's decoder into the event's lists (value_take into the rows' list), the event beginning with them; the JSON edge reads an object's array as its record's objects, 64 lists deep at most (items_of: an item's lists encoded first and the item moved down over them); the load no longer refuses them; a hole closed: the decoder read past the end where fewer than 4 bytes were left for a length (size - at - 4 wrapped around), unseen while it read only what was stored; unit test nested_rows (three events, four rows not whole changing nothing); check server_trees (300 random forests over HTTP against a Python model walking them in the same order, a require refusing some, a list field not an array, a field the record lacks and 65 lists deep refused, a restart); counts: b30 against b29 within 0.001% but BFS with batches +0.12%, the bank's queries -0.00% to +0.23%; 027's stage 4 planned in six steps (4a to 4f, the counts it starts from, 4b in two commits) and 4a as made; 008 on nested rows
* Research 140, 027's stage 3 measured: a board of columns of cards of tags ↵eunhhu43 hours7-0/+82
| | | | (aup's stored_nested) alike in memory and on a store restarting eight times, every step's fault, states and a view (2,400 snapshots, 54 restarts); killed at the tree's four stages and the log's two, all or nothing; replacement moving nested lists by name at depth; appending into a stored item's list costs about 170 ns an item it already holds (append_cost.c: 42 µs an add on average over the first 500, 738 µs over 8,000), building one by appends quadratic; instruction counts: the first commit's BFS +1.45% with batches and +1.68% without, from branches in the hot steps, gone with the load's internal ops (b29 against b27: BFS 0.000%/+0.034%, the others within 0.008%; the bank's queries -0.25% to +0.22%); what remains (a message's rows holding lists, a list field's .count in a fold, for over xs[i].f); 027's stage 3 as made (the representation chosen against the plan, its cost); 008's wording (kind 5 in images' headers alone); CHECKPOINT's direction row through stage 3