aboutsummaryrefslogtreecommitdiffstats
path: root/docs/design
Commit message (Collapse)AuthorAgeFilesLines
* 029: two framings in the 009 section made plain: there is no disk leaf for a ↵eunhhu26 hours1-2/+8
| | | | list today (090's tree is in memory, a list on disk is images and a log), so the zero copy measured is memory's; and the seek rung is another program, not an emission (it halves by the tree's order, charges the budget once a call): 578 holds only with a keyed list, its seek and the order, an emitter alone gets 1,576; cycles beside instructions (388 against 312, end to end 368 against 292)
* 029: feedback 009's rule (one byte shape for the same data) applied to both ↵eunhhu26 hours1-0/+44
| | | | principles' results: packet to execution with no decoding holds in the hand-written border (rows read where the payload has them; checks stay), not in the runtime (a tree each message); the leaf as the execution array with no copy costs nothing only when the search follows the tree's order (578 against the copy's 564; a positional halving descends each probe: 1,576 and 2,114), which needs ranges ordered by start (a keyed list would say it; the app's own halving rebuilds the index the tree is); commit extent = replication record = follower bytes holds as files (046), but an item has more than one shape there: packet = TREE leaf column = image body for one field, the log re-frames each change (64 B a message, 56 of them frame), several fields go by rows in packets and images and by columns in leaves, FLAT event lists narrow; the next question put, not built: the leaf as the commit
* 046, 029: the taken-over store sent a message too (opened() works on a copy: ↵eunhhu26 hours1-6/+9
| | | | show, add, show; 20 more kill -9s: all open and take it as owners; 667 prefixes again with it: all pass); the price measured again without the fold that inflated it (entries.aup's summed view folded every entry each message): 48.5 to 232 us a message on tmpfs, 146 us of the 183 waiting for acks; the shim never stopped across the runs
* 029: principle 2 passed its breaking experiment (046), written with what it ↵eunhhu26 hours1-1/+37
| | | | showed against its own words: identity (233 samples and the ends of 100,000 and 200,000 messages: 0 bytes differ), taking over (70 kill -9s, all open with every message done), torn tails (867 op boundaries opened, 568 within cuts); "every write a save makes is a self-delimiting, self-verifying extent" false as written (zeros ahead, installs by rename under process-numbered names, directory syncs, 8 MB segments taken): file operations shipped, still no replication layer (the store's own ops cut at its own syncs, heard the ack); the price on tmpfs (204 to 400 us, 903 B a message, 42% zeros the follower could make); what was not tried (a follower started again and heard from its files, pure-serve's checkpoint thread, a real medium and two machines, a segment let go, asynchronous, choosing the owner)
* 045, 029: a message end to end, the event's border written by hand too ↵eunhhu27 hours1-1/+25
| | | | (app.c's BORDER: rows whole and finite, the budget and memory begun again, the one state the handler changes written apart and made the state's at a faultless end into a buffer no reader holds, events and changes counted; every end answers all 18,672 lines, the broken-rule build refused at both hand ends): machine instructions a code point, the runtime 40,365, its empty handler 1,595, the hand border with shared 592 and with plain 379, Vega's C given the same payload 240: 68x, 2.5x C (cycles 3.2x), the border 1,595 to about 22; what went: the rows rebuilt into a tree each message (pure_list_read's make_tree), 54 states walked five times an event (40 of them t.X's registers), the last event's nested lists, a bytes state set whole through a text's piece tree; no new fact (N2, today's proof, the language, forms chosen)
* 029: the facts counted again: two new (N1 whole numbers, N4 unchanged after ↵eunhhu27 hours1-7/+11
| | | | LIST_VALUE) and one analysis widened (N2 the handler's changes), not three new as 7ffb114 says: the 9.43x of the event's lists as typed buffers needs no new fact (keyed and indexed are declared, an event's list is one by its place; a typed buffer does every operation an unkeyed, unindexed one gets), only the form chosen and the memory charge's contract; seven fns (range and six), not six
* 029: UAX #14's whole chain and the facts it needs: the handler 38,623 ↵eunhhu27 hours1-7/+45
| | | | machine instructions a point (event aside) to 564 emitted by hand, 2.5x Vega's C (225; cycles 3.6x), factors 1.46 (dispatch), 3.02 (memory slots and calls), 1.10 (whole numbers), 1.22 (ranges flat), 9.43 (the event's lists typed buffers), 1.23 (Text sharing), the contract left (bounds 1.12, budget and memory 1.39) and the algorithm (1.61); the facts: today's proof licenses exact (each fn's registers, lists and changes: none of the six has lists or changes), then three new (N1 whole numbers, N3 a let list's operations, N4 unchanged after LIST_VALUE) and one analysis widened (N2 the handler's changes); the event itself now the largest layer (1,566 a point: the points decoded into a list, lists cleared and freed, malloc, breaks through the store tree), outside this chain; a premise corrected (the event's lists are FLAT arrays since 021, not trees: the 9.43x is the generic list path); the let lists' factor, broken by a share before, measured as the chain's largest
* 029: range()'s chain written, and two of its own claims taken back: the ↵eunhhu28 hours1-14/+47
| | | | executor 14,488 machine instructions a point, stitched 8,144, emitted exact 2,474, with integers 1,587, a flat column 1,083, both 226, C 129 (dispatch 1.78x, memory slots and the call's machinery 3.29x, the numbers' and lists' form 10.9x, bounds and budget 1.75x C); the bar set before (3x C's round) missed by exact (217) and passed by both (19.8); the premises checked: an unchanged list proven (function_changes), a whole-number index not (number slots, INDEX_AT checks it: a fact for the proof to take on), a local's lifetime not tested here; "no layer holds most, each 1.4-1.5x" and the let lists' "broken by arithmetic" were shares inside the executor, not factors; "one machine instruction an instruction is 2.5x C" was wrong (both does 269.5 in 226); what is bundled (memory slots with the call), the flat column's cost moved to changes (unmeasured), and 36% of the app only
* 045: Pure's answers checked for real (cost.c and where.c compared only the ↵eunhhu28 hours1-1/+4
| | | | C's with the test before; now Pure's boundaries are read from its view, and a wrong variant is refused with 0 lines right); the executor and the stitched build, rebuilt at adccf58, both answer all 18,672 lines, and the stitched build's numbers stand (40,189 to 28,089 machine instructions a point, 1.43x; cycles 1.32x); 029 says so
* 029: principle 1's break by mechanism (045's samples): dispatch 30.6%, lists ↵eunhhu28 hours1-1/+27
| | | | built 17.3%, numbers 13.5%, lists read 13.0%, moves 9.0%, calls 7.1%, malloc 4.0%: no layer holds most of the cliff, three generic ones (interpretation, the lists' form, the numbers') 1.4-1.5x each, the cliff their product; range()'s binary search 36% of the machine instructions and 269.5 of 549.9 executed (an algorithm Vega's C does not use); a candidate broken by arithmetic before code (let lists held again as versioned states: 13,000 of 40,236 at most, 1.5x); the bar's arithmetic (one machine instruction an instruction is still 2.5x C: the instructions must fall too); "the general JIT 008 forbids" corrected (008's task A asks for one execution principle from the proof; it forbids the executors' split and scaled-down techniques)
* 029: principle 1 broken on UAX #14 (045), written as its own rule asks: (a) ↵eunhhu28 hours1-2/+31
| | | | 2.19x against 4x, its premise wrong (the source already keeps t.cls[i] in locals: the reads only the proof merges are 16.4 of 549.9), (b) 1.31x against 3x, and the path capped (interpretation is 22 of an instruction's 74 machine instructions: stitching bodies gives at most 1.42x); what was tested (step() on constant instructions, frame and checks as they are) and what was not (an emitter given the proof, the general JIT 008 forbids); where it goes (runtime machinery: a call 1,375, a stored read 331, the lists 22,651 a point) and the 7x the batch lanes give that a fn in a block loses; the 1,774 and 3.2-3.9 ns corrected (the budget's count: 549.9 run, 11.7 ns each) here and in 140; the op counts kept
* 029 second draft from a blank page (the first, five techniques listed, ↵eunhhu29 hours1-189/+95
| | | | rejected by outside feedback 008): two principles, each a layer deleted because another already holds what it holds, each with one experiment that can break it before code; 1 the proof is the compiler (the verifier proves stack places and types, loop bounds, each fn's change set, no aliasing between lists, and the three executors throw it away; a C compiler must guess what Pure has proven, so verified aup should be easier to compile to C's speed than C; delete the executor as a path, the batch lanes and the AArch64 backend, emit by stitching the executor's own op bodies; break it on UAX #14: the merging the proof licenses 4x fewer instructions, no dispatch 3x less a instruction), 2 the commit is the replica (a save's extents self-delimiting and self-verifying, the link's heard their durable ack: no replication layer, the follower's files the leader's byte for byte; break it with two hosts on one Pi, cmp at sampled saves, kill -9 and open as owner; checkpoints and compaction to ship or make deterministic; the owner-choosing rule deferred); schema evolution an open question (one shape in the tree, projected on read: unmeasured, and leaves are columns); effects and screens already one boundary, backends
* 029: spreading fns at load is a reason to try, not a result (an early return ↵eunhhu30 hours1-1/+3
| | | | in a loop's body becomes a branch out of it, unmeasured): ③'s first work settles it
* 029 (draft, not a 1.0 freeze): past one node, outside feedback 007's five ↵eunhhu30 hours1-0/+194
| | | | gaps as architecture and order, each hung on an invariant already there (deterministic events, the chained commit log, ordered once delivery, views as pure functions) with a first proof on the Pi, its pass number and a stop rule: 1 the computing cliff and views' fns (the slow specs all runner 2: fns send whole blocks to the executor; fns spread at load below the language, bytes as columns, fewer instructions, CALL in a view whose callees change nothing), 2 the living store's evolution (a migration a pure fn of the new module, rewritten in batches while the app answers; 030's items-wide at 1M under 5 ms unanswered, the store the same bytes), 3 distribution (committed bursts replayed by a follower, an epoch fenced owner chosen by two of three, topology as a directory app's views; two hosts on one Pi first), 4 Vega past the grid (a display list as a view type, layout in aup with text measured by the host, IME and clipboard as input events, native windows last), alongside 5 effects whose results come back as events, one entry, a C ABI v0; the app contract gathered as draft v0, the fixed limits deferred
* 028 and 140: list values' copying measured a second time (UAX #14's Text ↵eunhhu35 hours1-0/+2
| | | | record copies its nine lets' lists a line: with building them, 260 of 1,774 instructions a code point, after inflate's code()), and a keyed list's floor seek (the last item keyed x or below) first wanted (UAX #14's ranges, a binary search by position, 360 instructions a code point), one spec behind it
* 028's bytes alone as a state, made: module state type 6 read as a text state ↵eunhhu36 hours2-8/+30
| | | | flagged PURE_TEXT_BYTES in state_flags (so serve's same_states compares it for queries and replaces), its TEXT_LOAD giving bytes and its TEXT_SET taking them, a cursor's ops on it refused (anchor in bytes) and an anchor in it refused at load; one store file text-NAME, the values record marking each bytes state (kind 7, its name, 8 zero bytes), a text state's file that is no UTF-8 its initial value when marked and store_error when not (as before); a replace carrying bytes to a text only when its draft is UTF-8, else opening the store's draft anew; pure-run shows a bytes state's hex; unit test bytes_states (seven starts from a store: 0xff and 0x00 kept, bytes to text to bytes to text with no UTF-8 and with it; three replaces; four refused); counts: b47 against b46 BFS with batches +0.12%, the rest within 0.001%; the bank's queries (s45 and s47 in turns, six each, bytes compared among them) in_credit -0.31%, per_branch -0.18%, ranks -0.11%, the rest within 0.01%, the seven answers the same bytes; Pure, Vega and aup checks green; 008, 028
* 028: bytes alone as a state (140's decision point: inflate and HTTP both met ↵eunhhu36 hours1-0/+77
| | | | it): state type 6 a module's bytes, held as a text is (one draft, app->texts) with state_bytes marking it, no new opcode (TEXT_LOAD gives and TEXT_SET takes bytes for it), no anchor in it; one store file text-NAME, the values record (PVS1) marking each bytes state (kind 7, its name) so a text file that is no UTF-8 is a move when the record says bytes (the initial value) and corruption when it does not (store_error, as before); text and bytes carried over each other when exact (text to bytes always, bytes to text when UTF-8), on a replace and on a start, A to B to A raising no stale value; aup's state x: bytes; the checks planned, two names for the file and a new runtime kind turned down, list sharing and emptying left for when their cost shows
* Bytes compared (140's decision point: DNS and HTTP both met it): ↵eunhhu36 hours1-1/+1
| | | | TEXT_COMPARE's argument takes PURE_COMPARE_BYTES (bit 8) beside its relation, its two then bytes (UTF-8 asked of neither), as CONCAT's argument says bytes; verified as A_ORDER (a relation, and the bit), the executor relating by the argument's relation alone; the JIT, the tier and the batch executor leave TEXT_COMPARE to the executor as before; unit test bytes_compared: eight relations of a constant's bytes, none, 0xff and 'A' (0xff after 'A', a longer after its first byte), four refused (the bit on texts, no bit on bytes, a text with bytes, a relation past NE); counts: b46 against b45 the same on all five works, with and without batches; 008
* 027's stage 5 closed: two numbers (number, binary64, the default of literals ↵eunhhu38 hours1-2/+15
| | | | and of counts; i64 the exact integers, bits and money), / a number's division and // the floored quotient (FLOORDIV, 95 opcodes now 96), an i64 sum exact, money i64 minor units shown by .Nd, a replace and stored values moved when exact; the bar: TPC-C's money and rates integers in Pure and SQLite alike, every row of every table exactly the same after 20,000 transactions; core 22,766 lines to 22,791, serve.c 5,058 as it was, aup's compile.c 5,720 to 5,726; counts: b45 against stage 4's end (b42) within 0.04% on all five works; the bank's queries (s42 and s45 in turns, six each) per_branch -0.58%, in_credit +0.56%, ranks +0.20%, the rest within 0.01%, the seven answers the same bytes; TPC-C in RAM (W=1, 64 terminals, 20 s, in turns, three each): s42 on its own app (money numbers) 46,092 transactions a second, s42 on stage 5's (money i64) 47,303, s45 on it 47,287: the runtime the same (-0.03%), the app 2.6% faster with i64 money (the server's CPU a transaction 42.3 to 39.6-39.8 us, not taken apart), new-order p99 3,584 to 3,456 us; 027's status: the five stages made; CHECKPOINT's direction row through stage 5
* 027's 5d: a replace's rule and 008: the code was already one function ↵eunhhu38 hours2-6/+18
| | | | (pure_cell_as) for a replace while running, the stored values at a start and a changed record's list moved (in memory and from its image): a whole number in [-2^63, 2^63) to an i64 (-0 as 0), an i64 within 2^53 to a number, 0 and 1 to and from a bool, all else the new default; aup's test number_kinds_carried runs it on states and a stored list's field, on a replace and on starts from a store; 008's replace and start say so for i64, and the values and a list image's i64 cells (two's complement, little-endian 8 bytes); no code changed, so no counts; 027
* 027's 5c: money as i64 minor units shown by .Nd: NUM_TEXT's decimal form ↵eunhhu38 hours2-3/+15
| | | | takes digits (bits 20-24 as FIXED's), an integer's last N digits written after a point (1234 at two is 12.34, -5 -0.05, 0 0.00; width and zeros after the sign), verification allowing digits on FIXED and DECIMAL alone; unit test: ten rows of formats_and_held (-2^63 at two, 25 digits, a number's whole 1234, its 12.5 a limit); TPC-C's money i64 cents and its rates ten-thousandths (tables, messages, load, the transactions), SQLite's INTEGER columns and int64 bindings alike, the comparison now exact (no 1e-9 tolerance): after 20,000 transactions every row of every table the same (grouped SQLite too) and 3.3.2 holding on both; counts: b45 against b44 BFS without batches +0.09%, the rest within 0.001%; the bank's queries (s44 and s45 in turns, six each) in_credit -0.59%, per_branch -0.34%, ranks -0.20%, the rest within 0.01%, the seven answers the same bytes; 008 (NUM_TEXT's digits, a message row's i64 cells), 027 and 092
* 027's 5b: an i64 sum exact: its items gathered in a 128-bit integer and only ↵eunhhu39 hours2-1/+10
| | | | the result checked against the range (overflow past it), so a sum past the range midway and back in it at the end is the answer and an i64 sum's order is none of its business; the batch executor's folds are numbers' lanes alone, summed in the items' order, as before; 008's FOLD says so; counts: b44 against b43 within 0.04% (BFS without batches -0.03%); the bank's queries (s43 and s44 in turns, six each) per_branch -1.32% (all six below s43's lowest: a larger Fold lays the number sum's loop otherwise), in_credit -0.72%, ranks -0.26%, the rest within 0.01%, the seven queries' answers the same bytes on both; 008 and 027
* 027's stage 5 designed and 5a made: two numbers (number, binary64, the ↵eunhhu39 hours2-1/+28
| | | | default of literals and of counts, positions, lengths and u32 fields, whole to 2^53 exactly; i64 the exact integers past it, bits and money, never mixed, moved by number() and i64() when exact), / a number's division, // the floored quotient, i64 folds exact (only the result in range, so in any order), number folds in order unless the batch executor's 2^53 test says any order is exact, money as i64 minor units shown by .Nd, a replace carrying number, bool and i64 when exact; 5a: opcode FLOORDIV (95, 96 now): a number's floored quotient as Python's float //, exactly (a less its floored remainder, divided, to the nearest whole; a zero signed as a / b; by zero divide), a new opcode so the JIT, the tier and the batch executor leave it to the executor; unit test floor_division (7 // 2, -7 // 2, 7 // -2, 1 // 0.1 = 9, 0 // -3 = -0, 1e300 // 7, by zero); counts: b43 against b42 within 0.001% but BFS without batches -0.09%; the bank's queries (s42 and s43 in turns, six each) -0.20% to 0.00%; 008 and 027
* 027's stage 4 closed: 106 opcodes now 95, 7 state kinds 6, 6 input kinds and ↵eunhhu39 hours1-0/+10
| | | | the frame and pointing 3 kinds and their bindings, a view's type a record field's, the fixed limits the format's and the list engine's widths alone; core 22,398 lines to 22,766, serve.c 5,086 to 5,058, aup's compile.c 5,923 to 5,720; TPC-C in RAM (W=1, 64 terminals, 20 s, in turns with stage 3's end, three each) 45,893 transactions a second to 46,063 (+0.37%), new-order p99 3,712 to 3,456 us; the bank's queries (stage 3's on its aup's module, five each in turns) field -1.23% to in_credit +1.15% (the noisy three within their 1-3% spread); CHECKPOINT's direction row through stage 4, its nesting as it is now
* 027's 4f-4: loops nested to any depth: the load counts a module's deepest ↵eunhhu40 hours2-2/+16
| | | | nesting (nesting), a run keeping PURE_MODULE_NESTING (8) on its stack as before and a module nesting deeper running in the executor that calls, its loops in its app's room (PureApp.deep, grown to it), so run()'s hot path is as it was; the JIT compiles modules nesting 8 at most; a function's nesting counted without a cap (u32), its frame sized to it; unit test: 40 loops nested, run to their end (2^20 rounds) and stopped by the work budget (2^40), 8 still on the stack; counts: b42 against b41 with batches BFS -0.12%, without BFS +0.09%, the rest within 0.001%; the bank's queries (s41 and s42 in turns, six each) in_credit +1.04%, per_branch +0.60%, ranks +0.36%, the rest within 0.01% (their spread: the bank has no functions, so none runs the changed executor, and s42's in_credit is within s41's own earlier six); 008 and 027
* 027's 4f-3b: no fixed count of records or of their fields in all: the ↵eunhhu40 hours2-2/+15
| | | | header's u16s alone (16 and 128 before); a stored list's record graph (PureLayout) holds a few records itself and more on the heap, 256 at most (an image numbers them in a byte), an image or a checkpoint not written then without room for it; a record's 16 fields kept, the list engine's width (a leaf's columns, a run's columns and narrow cells, the arrays' and packs' columns), to be widened with the lists' shapes sized to their fields; unit tests: 20 records of 10 fields (200 in all), an item of the last appended; a stored list whose six records each hold a list of the next saved and read back after a restart (its graph on the heap); counts: b41 against b40 with batches BFS +0.12%, without BFS -0.09%, the rest within 0.001%; the bank's queries (s40 and s41 in turns, six each) within 0.09% (the first three's in_credit +1.16% its spread: 2.75% between runs of one build); 008 and 027
* 027's 4f-3a: no fixed count of inputs, views (the load's columns among them, ↵eunhhu40 hours2-3/+18
| | | | in u16s), handlers or message fields: the header's u16s alone; a message's fields at most 64 (PURE_MESSAGE_FIELDS: a send pushes them all, as the stored sends' shapes hold them), outputs at most 255 (the store names them in a byte's count: the links and the outbox's image), the numbers of outputs a module dropped carried within it; an app's numbers of each input (last sent, saved, where the event began) in a block sized to its module's inputs (by_input), a replace laying the new one first and filling it by output name; image headers sized to the module (header_room); a view's arguments u32, a made column's field u16; pure-serve's timers grow to the module's inputs, its outlets and known senders its own count (OUTLETS, 32), a list view's JSON keys its columns on the stack for 64, else on the heap; unit test: 40 inputs and 70 views, the last of each used; counts: b40 against b39b within 0.01% with batches and without but BFS +0.09%; the bank's queries (s39b and s40 in turns, three each) in_credit -0.64%, per_branch -0.37%, field +0.34%, the rest within 0.22%; 008 and 027
* 027's 4f-2: no fixed count of states: the header's u16 alone ↵eunhhu41 hours2-3/+23
| | | | (PURE_MODULE_STATES gone); a module's transient and counted states are flags of each, the lists views and functions change or name sets of states (state_words words, a bit each), the verifier's walked lists a list with its length, duplicate names found by sorting them; an app's eighteen arrays of each state (values, texts, lists, the stored lists' slots, the last save's and the burst's, a reader's, an event's values and marks) in one block sized to its module, a replace laying the new module's apart and taking it once nothing can fail; the runtime's lists placed past any state (PURE_LIST_HEARD 0x10000), their slots after the states'; a snapshot's arrays sized so too; pure_app_read false without room (pure-serve then shows none); a save's drafts, values and the tree's revisions on the stack for 64 states, else on the heap; the JIT compiles modules of 32 states at most (its sets a word), the app's lists read through a pointer; a view reads the app's values as they are (no state changes in one), no copy of 32 a evaluation; unit test: 300 states loaded and the last counted; a hole closed: pure_app_stop read the arrays a failed start never laid; PureApp 4,256 bytes, not 8,504; counts: b39b against b38 with batches within 0.06%, without batches knapsack +0.49%, join +0.23%, the rest within 0.17%; the bank's queries (s38 and s39b in turns, three each) field -1.99%, found -1.18%, in_credit -1.10%, large -0.99%, the rest -0.58% to +0.11% (the first build, s39, copied a view's values at run time's size: in_credit +2.4%); 008 and 027
* 027's 4f-1: a module's tables sized to it: PureModule's thirty-four tables ↵eunhhu42 hours1-0/+21
| | | | of states, inputs, message fields, views (the columns the load makes among them, counted from the view and record entries first), handlers, records, record fields, functions and signature items point into one block the load lays out (lay_tables) and pure_module_release gives back, so PureModule is 2,752 bytes, not 24,320; a sanitized build poisons 16 bytes between tables, so reading past one is caught (none was); limits as they were; a test and five of research 044's timers gave the load a module never zeroed (it gives back what one holds), now zeroed; counts: b38 against b37 with batches within 0.25%, without batches matmul -0.45%, join +0.36%, knapsack +0.28%, BFS -0.25%; the bank's queries (s37 and s38 on one aup module in turns, three each) field -0.68%, found -0.51%, large -0.47%, the rest -0.11% to -0.20%; 027's 4f designed in four steps and 4f-1 as made
* Module format v7, 027's 4e: a view of any value: a view's and a column's ↵eunhhu42 hours2-8/+60
| | | | type a record field's (number, text, bool, i64, list, bytes; PURE_VIEW_TABLE now PURE_VIEW_LIST), a view entry 20 bytes (its table + 1, its items' record + 1, its list state + 1); a list view's rows a list state's items as they are or filtered (its record's fields its columns, made at load: each reads its item's field as it is), a list state's map (its columns following it, as tables were), or the items of a list its block leaves (copied into a kept list once a change, its rows and cells kept until the next); a list cell lent in its items' stored form (pure_app_cell_items, pure_items_next reading it), bytes as a text is; a pointing only at a list view over a state; pure-run writes bytes as hex and a list cell as an array of objects, pure-serve a bytes view as application/octet-stream and, in JSON, bytes as hex strings and list cells as arrays of objects; a hole closed: a view without a body never began its growth anew, so in a snapshot's reader (pure-serve's views) reading a kept list's list failed as memory (unreachable before 4e: no such view could be written), now marked a body at load; unit test list_views (as it is, filtered, computed, a map's list and bytes columns, a bytes view, six refused); check server_trees keeps its forests in a stored list shown by list views over HTTP against the model, a restart too; counts: the first build's bank tables rose (field +5.9%: view_of left inline at its seventh caller, serve's put_bytes too), so view_of is inlined, the made columns' and computed lists' paths and serve's new writers kept apart, a computed list's rows held as a filter's; b37 against b35 within 0.001% with batches and without; the bank's queries (s35 on the 4d aup's module, s37 on 4e's, six each in turns) field +0.67%, found +0.43%, large +0.31%, the rest within 0.20%; 008 and 027
* Module format v7, 027's 4d: every input a message: a module's inputs are ↵eunhhu43 hours2-96/+124
| | | | messages, outputs and arguments (kinds 1 to 3), a message bound to what makes it (PureBinding: a key's press or its release, a text input, a timer, a frame, a pointing at a table), each binding's fields fixed (none; one text; one number, dt; one u32, the item's position, the table its view), one frame input at most; a handler's event its input's number; no DT or TEXT_INPUT (97 opcodes now 95, FIELD reads dt and the text), pure_app_input, frame, tick, text and point, pure_module_position, PureEdge or an app's ticks: every event is pure_app_message, saved after it unless deferred; pure-run sends each command's message to the input bound so (a release or a frame no input takes still an event, clearing the fault, so the workbench parity holds) and defers a stored app's saves past frames as before; pure-serve posts a press with its key's release, refuses timers, frames and pointings (no_input) and ticks timers as messages; holes closed: the tier found a handler's input by the old numbering, reading a message's number field as another input's (aup's numbers caught it), and serve's replies_borrow did too; module-probe presses as messages; unit tests: the test modules' builder (expand) writes the old shapes as bound messages, pointing over a pointing input and its table, stored frames deferred as a host defers them, a press's handler refused a FIELD; counts: b35 against b34 with batches BFS +0.03%, the rest within 0.001%, without batches BFS +0.12%, join +0.01%; the bank's queries (s34 on the 4c aup's module, s35 on 4d's, six each in turns) field -0.23%, found -0.14%, large -0.12%, in_credit +0.11%, the rest within 0.06%; 008 and 027
* Module format v7, 027's 4c: cursors and indices one anchor: a module's state ↵eunhhu44 hours2-57/+86
| | | | type 2 is a place in the text or the list it names (a bool 4, an i64 5), and seven anchor instructions take the place of the thirteen of cursors and indices (103 opcodes now 97): LOAD and STORE an anchor state, START and END of a text or a list, MOVE, COUNT and AT naming it and a unit (a character, a line, an item); the load holds an anchor in a text as a cursor and in a list as an index (a mark in a list refused) and each anchor instruction as the runtime's own one for it (PURE_OP_CUR_*, PURE_OP_INDEX_*, numbered past a module's in module-run.h with the kept lists' steps; a module holding one refused), so the executor's, the batches' and the JIT's steps are those they were; the verifier holds MOVE, COUNT and AT to the text or list they name; new: AT in a text (a cursor so many characters or lines from its start, not_found past the end); COUNT's lines from 0; unit test anchors (AT and COUNT in a text, AT in a list, four refused); the tests' modules written with the anchor instructions; counts: b34 against b33 with batches matmul -1.36%, the rest within 0.08%, without batches within 0.04%; the bank's queries (six each, s33 on the 4b-2 aup's module) in_credit +0.28%, per_branch +0.23%, ranks +0.10%, the rest 0.00%; 008 and 027
* Module format v7, 027's 4b-2: keyed documents gone, a keyed list of records ↵eunhhu45 hours2-122/+151
| | | | holding bytes doing their work: no collections table, DOC, DOC_SIZE or REPLACE (106 opcodes now 103, those from LEN on moved up), keyed messages or a base revision (pure_app_message takes an input and a payload), no PureData document callbacks (a data host opens states and saves them together), no burst by document key; the header 38 bytes; pure-edit-module gone; pure-serve without /objects, ETag, If-Match and If-None-Match, 412 or an app a thread for a module of documents alone (one without states or outputs runs so); a hole closed: an instruction in no handler's, view's or function's block was never verified yet the load's later passes read its arguments (a damaged module read out of bounds once the header moved what the mutation test damaged), now blanked to CONST 0; checks moved: server_documents (DOCS_SOURCE, keyed byte documents with the app's revision, against a Python model, one and three connection threads, a restart) in place of the parity with editor.pure, pipelining and bursts through kill -9 on its inputs and views, the lent budget on a text view (lending a document's read is server_lent_views' already), clocks and tokens on it; message tests without a document host; the HTTP benchmark's document suite runs the earlier document server and the baselines alone; the count harness builds against either message API (runs.sh build: -DPURE_KEYED_MESSAGES before 4b); counts: b33 against b32 with batches within 0.002% but BFS -0.33%, without batches BFS +0.63%, knapsack +0.52%, join +0.38%; the bank's queries (s32 on a v6 module, s33 on v7) field +0.68%, found +0.44%, large +0.42%, the rest within 0.13%; 008 as v7, 027's 4b-2 as made and 4c designed
* Module format and executor: 027's 4b-1, a record field holding bytes (type ↵eunhhu45 hours2-3/+16
| | | | 5): held as a text is but for UTF-8, its default a constant; a function's bytes in a signature this type too (PURE_SIGNATURE_BYTES gone); no list keyed by bytes; replaced, a text field becomes a bytes field keeping its value (pure_cell_as, kinds_carry); images' headers take its kind; the JSON edge reads it from a string; pure-run shows it as hex; an append asks a record's fields held as bytes from a mask the load makes (record_held) instead of each field's type twice, so b32 runs fewer instructions than b30 (BFS without batches -1.05%, join -0.25%, BFS with batches -0.36%, the rest within 0.04%; the bank's queries s30 against s32 -0.01% to +0.30%); unit test bytes_fields (bytes not UTF-8, a text field replaced by a bytes field, three refused); server_trees with a bytes field; 008 and 027
* Module executor and pure-serve: 027's 4a, a message's rows holding lists: ↵eunhhu46 hours2-2/+65
| | | | rows whose records hold lists are read, as the message arrives, by stage 3's decoder into the event's lists (value_take into the rows' list), the event beginning with them; the JSON edge reads an object's array as its record's objects, 64 lists deep at most (items_of: an item's lists encoded first and the item moved down over them); the load no longer refuses them; a hole closed: the decoder read past the end where fewer than 4 bytes were left for a length (size - at - 4 wrapped around), unseen while it read only what was stored; unit test nested_rows (three events, four rows not whole changing nothing); check server_trees (300 random forests over HTTP against a Python model walking them in the same order, a require refusing some, a list field not an array, a field the record lacks and 65 lists deep refused, a restart); counts: b30 against b29 within 0.001% but BFS with batches +0.12%, the bank's queries -0.00% to +0.23%; 027's stage 4 planned in six steps (4a to 4f, the counts it starts from, 4b in two commits) and 4a as made; 008 on nested rows
* Research 140, 027's stage 3 measured: a board of columns of cards of tags ↵eunhhu46 hours2-1/+17
| | | | (aup's stored_nested) alike in memory and on a store restarting eight times, every step's fault, states and a view (2,400 snapshots, 54 restarts); killed at the tree's four stages and the log's two, all or nothing; replacement moving nested lists by name at depth; appending into a stored item's list costs about 170 ns an item it already holds (append_cost.c: 42 µs an add on average over the first 500, 738 µs over 8,000), building one by appends quadratic; instruction counts: the first commit's BFS +1.45% with batches and +1.68% without, from branches in the hot steps, gone with the load's internal ops (b29 against b27: BFS 0.000%/+0.034%, the others within 0.008%; the bank's queries -0.25% to +0.22%); what remains (a message's rows holding lists, a list field's .count in a fold, for over xs[i].f); 027's stage 3 as made (the representation chosen against the plan, its cost); 008's wording (kind 5 in images' headers alone); CHECKPOINT's direction row through stage 3
* Module format and executor: 027's stage 3, nested stored state: a stored or ↵eunhhu47 hours1-3/+13
| | | | a transient list's records may hold lists (a message's rows not, nor as a key); such a list's cell holds the value in its items' stored form (PURE_CELL_LIST, held as bytes are: each item's fields in order, 8 bytes a number, bool or i64, a u32le length and bytes a text or a list, a list's in this form again), read into one of the event's lists by ITEM_GET (counted in the event's room) and written from one by LIST_APPEND, LIST_INSERT and ITEM_SET, APPEND_IN reading, appending and writing back (undone with the event): images, the log, undoing, readers' shares and commits hold it as a text's bytes, so nothing new is stored or recovered; a list image's header names its record graph (a list field's items' record by its number, the graph's other records after the fields: a record holds itself through a list), and starting or replacing moves the items of the lists in fields by field name down the graph (PureLayout, pure_list_migrate); an event's lists keep the event's lists' numbers; a view lets go the lists a view before it read in; no FIND joined to a list field's read; pure-run shows a kept list's lists; unit tests (an event changing a number and a stored item's list, the process dying at the tree's four stages and the log's two, failed commits changing nothing; a replacement moving a stored and a transient list's lists, their items' record gaining a field and the board's fields changing places; under ASan and UBSan too)
* Research 140, 027's stage 2c measured: JSON read into a tree by recursive ↵eunhhu48 hours1-3/+17
| | | | descent (aup's examples/json_tree.aup: value(d, i) -> Json, number giving the value and where it ends, each call's own let's lists, string's escapes joined by char(u), a depth-first walk passing items as values) against Python's reading walked so: JSONTestSuite's 95 y_ taken alike, no n_ taken, 2b's 6 i_ taken (a lone surrogate refused by char as encoding); 100,000 and 50,000 deep refused as memory (the frames fill the event's room, 0.2 s), 30,000 deep taken (past Python's json); 300 random documents alike; in make check (module_specs' digest now depth first, three paths); 142 lines against the same design's 105 in Python (json_rec.py; the bar missed: 24 lines of declarations, 23 of braces), 46 times cJSON's time and 1.6 times plain Python's (600 ns a byte, 248 Pure instructions; 71% in the steps, about 10% in calls, 66,000 calls a document, most of them small block fns like skip); characters against Python's ord in aup's check; probes: p5-chars compiles, p4-rec (a view calling itself) refused; instruction counts: the workloads' modules the same, batches within 0.001% of 2b (BFS -0.21%), the executor alone -0.001% to +0.30%, the bank's queries -0.30% to +0.11% (four runs each; bar 1%); json_rec_time.c (cJSON, a depth-first digest compared first); CHECKPOINT's direction row through 2c
* Module executor: a call's frame counted in the event's room by a table, as ↵eunhhu48 hours1-0/+3
| | | | gas is (PURE_CALL_*: 256 a frame, 64 a value of its stack's height, 8 a local, 128 a loop it nests, 1024 a list of its own, 128 a held value, 16 a register; each at least what it stands for, checked as it compiles), not by the implementation's sizes: a module's calls fail at the same depth on every machine and build; a call's list let go is kept for the next calls unless what it keeps empty passes what a frame counts for it
* Module format and executor: 027's stage 2c, functions and characters: a ↵eunhhu2 days1-2/+33
| | | | function is a block CALL runs (102), its parameters the stack it begins with and RETURN (103) its results on top, which take the arguments' place; each call has its own stack and locals and its own of what the function's code names (held values empty, the event's lists empty, registers bound to none), the caller's as they were when it returns; a function's code is a handler's but for a message, CALL in handlers and functions alone, and a walk around a call keeps to the lists the callee (or what it calls) changes; calls nest as deep as the event's room allows, each frame's bytes counted there (memory) and its block's length as work; the functions' table in the header's last u16 and after the handlers (a block, parameters and results) with their signature items (a record field's type, 5 bytes); run_calls, run_batches' steps with frames in the app's blocks that never move (a CALL opens one, a RETURN closes it, a stop closes all), an event's lists calls let go kept for the next; a value held by a slot the callee has its own of copied to the arena as it crosses; held values of the payload, the arguments or the constants held as they are (nothing copied until appended to); the JIT leaves blocks with calls or list locals to the executor; local type 4 (a list value, its record in bits 16-31); CHAR_AT (104, the code point beginning at a byte position of a text) and CHAR (105, a code point's text: limit outside 0-0x10FFFF, encoding for a surrogate); and a hole closed: LOOP and EACH open on a stack holding their operands alone (a loop's block begins at the bottom of its run's stack: a verified module with values below a loop crashed the executor, 62 below and 64 in its block); unit tests (fib, a tree summed by a walk that calls itself around its register, each call's own list and held text, a list through a local, characters and their faults, a call without end failing as memory and as work with nothing changed, 11 refused; under ASan and UBSan too)
* Research 140, 027's stage 2b measured: JSON read into a tree (aup's ↵eunhhu7 days1-1/+9
| | | | examples/json_tree.aup: a variant, lists in records, match, appends into a field's list, number(t)) and walked for a digest, against Python's reading: JSONTestSuite's 95 y_ taken alike, no n_ taken, 6 i_ taken; 300 random nested documents alike; in make check (module_specs, the suite and 100 random documents, three paths); 167 lines against the same design's 133 in Python (the bar missed: the type declarations), 40 times cJSON's time and 1.3 times plain Python's (556 ns a byte, 525 instructions); json_tree.py, json_tree_time.c (cJSON linked, digests compared first)
* Module format and executor: 027's stage 2b, lists in records: a record field ↵eunhhu7 days1-2/+19
| | | | of type LIST (its items' record in its default) holds a list the event made, its number an i64 in the cell (0 an empty one), so lists keep their inside, batches and the JIT as they were (the JIT's append fast path leaves such records alone); LIST_VALUE (a copy of an event's list, nothing made for an empty one), LIST_BIND (a register, an event's list flagged PURE_LIST_BOUND, read and walked, never changed by code), APPEND_IN (an item into the list in a field of an event's list's item, copied first when another value shares it or a register is bound to it), sharing counted where cells take them; such records only in event lists (no stored, transient or message list, no key); the event's lists go as each event or view's body begins, registers given back their own; record field names may be paths (at.x: records in records, flattened by aup); NUM_PARSE (a text written as a JSON number, its nearest double, core's pure_parse_number); unit tests (values copied, shared then appended, a bound register seeing what it was bound to, three events alike, 6 refused; under ASan and UBSan too, which found the record kinds' missing LIST)
* Research 140, 027's stage 2a measured: whether bytes are JSON (aup's ↵eunhhu7 days1-0/+11
| | | | examples/json.aup, 80 lines against the same design's 90 in Python): JSONTestSuite's 95 y_ cases taken and 188 n_ refused, 21 of the 35 i_ taken, the three paths alike, in make check (module_specs, the 316 small cases in tests/specs/json under their MIT license, the two large ones made and hashed); 116 times C's time and 1.2 times plain Python's (299 ns a byte, 339 instructions); && and || made to decide alone (writing it, i < n && d[i] read past the end) with AND and OR kept where the right side cannot fail, every other module byte for byte the same; SHA-256 again with formats and joined bytes, 42 lines and 159 times C
* Module format and executor: 027's stage 2a: BYTES_SLICE (bytes from a up to ↵eunhhu7 days1-3/+8
| | | | before c, whole positions inside, numbers or i64s), BYTE (one byte of an i64 0-255), BYTES_LOAD and BYTES_STORE (an event's 32 values held by name, 0-15 bytes and 16-31 texts so a text's hold UTF-8 alone, emptied as an event or a view's body begins, appended to in place, counted in the room: MEMORY past it), CONCAT of bytes, NUM_TEXT in a format (a width padded with spaces or with zeros after the sign, d x X b o of an i64 or a whole number, f with 0-30 digits; -0 as 0), a reply of a held value kept by value, LEN wherever a value is read (it had been for handlers alone); pure-run's message NAME @PATH (the file's bytes, of any size); unit tests (16 formats, 9 slices, held values set, appended, set to a part of themselves, emptied each event, a doubling stopped by the room, a reply kept, 7 refused)
* Research 140, 027's stage 1 measured: SHA-256 written in aup (its ↵eunhhu7 days1-0/+10
| | | | examples/sha256.aup) right for FIPS 180-2's examples and 138 more messages on every path, in make check against hashlib; 52 lines against Python's 28 (the bar missed: no formatting, no bytes joined or sliced, no lists as values, moved into stage 2), 176 times C's time and 0.43 of plain Python's (1,591 ns a byte; a block's 19,129 instructions, 39% of them the inlined fn's literals not folded); 044's check, time.c against the same algorithm in C (digests compared first) and the Python reference
* Module format: bits and bytes (design 027's stage 1): I64_BITS (and, or, ↵eunhhu7 days1-0/+6
| | | | xor, a left shift as a times 2^b failing as overflow past the range, a right shift as floor(a / 2^b), a count outside 0-63 failing as limit: integers in two's complement without end), BYTE_AT (the byte of any bytes value at a whole position, a number or an i64, else limit), BYTES_OF (a list of one i64 field as bytes, a value outside 0-255 failing as limit) and INDEX_AT (a list's item at a position, not_found where there is none); the fold-shape and batch analyses count their arity and the lists they read; unit tests of the ends (27 bit cases, 11 reads of a constant's and a text state's bytes, 7 refused)
* Design 027: the computation core inside 026's contract: an app's state one ↵eunhhu7 days1-0/+189
| | | | typed value built of bool, i64, number, bytes (text the UTF-8 ones), records, variants and lists of anything (an unkeyed list keyed by position), anchors for cursors and indexes; functions that may recur and while, ending by the budget as EVM's gas (the module the unit of determinism); bit operations on integers as infinite two's complement within i64; folds left folds in meaning, reordered by the implementation only for the same answer; meaning apart from representation; answers to 026's suspects; five stages (bytes, bits and lists of numbers for SHA-256; nesting, variants, recursion and characters for JSON; nested state stored; patterns gathered; the number model) with bars
* Design 026: the app contract written as mathematics, the part to freeze at ↵eunhhu7 days1-0/+185
| | | | 1.0: an event a total, atomic, deterministic function step(S, E) -> (S, Out*) or a fault (budgets in instructions and bytes), time an input, views pure functions of the committed snapshot, the history seen a prefix of commits, delivery in order and once with the transport's state in the store, values the same on every path, replacement joined by names; the layers it removes; its own suspects (input and view kinds, cursors and indexes, keyed documents, the number model, the budget's unit, fixed limits); the computation language rebuilt before 1.0, no compatibility kept before it (the user's word)
* Research 138: the batch executor's stage 5, design 025 made: sums into cells ↵eunhhu11 days2-4/+7
| | | | at once in integers (join 116.0 to 110.5 ms, short of 108: what is left is the same cells' memory, read, add and write again), decide's bit found once (BFS 23.2, short of 21.5: the tail's time was its loads more than its instructions), and the batch executor's loops aligned to 64-byte lines (the hot loops' placement had swung knapsack 6% at the same instructions; the lists' loops aligned too cost the bank's queries 1-2% more instructions, so not they); against b9 matmul -36%, knapsack -28%, roots -7.6%, join -6.2%, BFS -3.9%, the executor alone, the bank queries and TPC-C (+0.7%) as they were; 025's bars and what each stage measured; 024 and CHECKPOINT say so
* Design 025: stage 5's bars measured again on b16: BFS's candidate tail about ↵eunhhu11 days1-2/+8
| | | | 5.7 ms of 24.1 (decide, keep_filtered, apply_filters and the list side), halved 21.5 ms; join's sums into cells about 17 ns a round, a chain through the same cells, in four-byte integers with the width once a batch about 5 ns, 108 ms; neither moves BFS from about 2.7x the compiled code, and join's recurrence (about 40 ms) stays outside