aboutsummaryrefslogtreecommitdiffstats
path: root/core/module.c
Commit message (Collapse)AuthorAgeFilesLines
* A list value of numbers copied whole (140's LIST_VALUE review, no new ↵eunhhu36 hours1-3/+4
| | | | opcode, no change to what a module means): nest() made every list value item by item into a new tree (64 ns an item, 250 ns a copy fixed); an event's list in FLAT arrays, not keyed, its cells numbers, bools and integers, is now copied column by column (pure_list_copied: 600 items 38,268 to 985 ns, 60 items 3,926 to 324); UAX #14's nine lists a line 6,917 to 6,445 ns a code point (-6.8%, the same instructions); JSON tree's, HTTP's and inflate's (trees: texts or lists in their records, or few) as they were; copied_lists in test-module (random lists against item by item, then each changed apart; trees, keys and texts not copied so)
* CONCAT with an empty side gives the other as it is, uncopied (values do not ↵eunhhu39 hours1-0/+2
| | | | change), so a bytes state joined while empty (HTTP's pending + packet.data) no longer copies the packet: HTTP a packet at a time 288 to 270 ns a byte at the same instructions, as the old app (270.0); counts: b48 against b47 knapsack without batches -0.42%, join -0.16%, BFS with batches -0.12%, the rest within 0.03%; the bank's queries (s47 and s48 in turns, six each) in_credit -0.34%, field -0.34%, found -0.22%, per_branch -0.20%, large -0.18%, ranks -0.13%, none the same, the seven answers the same bytes
* 028's bytes alone as a state, made: module state type 6 read as a text state ↵eunhhu39 hours1-8/+40
| | | | flagged PURE_TEXT_BYTES in state_flags (so serve's same_states compares it for queries and replaces), its TEXT_LOAD giving bytes and its TEXT_SET taking them, a cursor's ops on it refused (anchor in bytes) and an anchor in it refused at load; one store file text-NAME, the values record marking each bytes state (kind 7, its name, 8 zero bytes), a text state's file that is no UTF-8 its initial value when marked and store_error when not (as before); a replace carrying bytes to a text only when its draft is UTF-8, else opening the store's draft anew; pure-run shows a bytes state's hex; unit test bytes_states (seven starts from a store: 0xff and 0x00 kept, bytes to text to bytes to text with no UTF-8 and with it; three replaces; four refused); counts: b47 against b46 BFS with batches +0.12%, the rest within 0.001%; the bank's queries (s45 and s47 in turns, six each, bytes compared among them) in_credit -0.31%, per_branch -0.18%, ranks -0.11%, the rest within 0.01%, the seven answers the same bytes; Pure, Vega and aup checks green; 008, 028
* Bytes compared (140's decision point: DNS and HTTP both met it): ↵eunhhu40 hours1-5/+7
| | | | TEXT_COMPARE's argument takes PURE_COMPARE_BYTES (bit 8) beside its relation, its two then bytes (UTF-8 asked of neither), as CONCAT's argument says bytes; verified as A_ORDER (a relation, and the bit), the executor relating by the argument's relation alone; the JIT, the tier and the batch executor leave TEXT_COMPARE to the executor as before; unit test bytes_compared: eight relations of a constant's bytes, none, 0xff and 'A' (0xff after 'A', a longer after its first byte), four refused (the bit on texts, no bit on bytes, a text with bytes, a relation past NE); counts: b46 against b45 the same on all five works, with and without batches; 008
* 027's 5c: money as i64 minor units shown by .Nd: NUM_TEXT's decimal form ↵eunhhu42 hours1-5/+10
| | | | takes digits (bits 20-24 as FIXED's), an integer's last N digits written after a point (1234 at two is 12.34, -5 -0.05, 0 0.00; width and zeros after the sign), verification allowing digits on FIXED and DECIMAL alone; unit test: ten rows of formats_and_held (-2^63 at two, 25 digits, a number's whole 1234, its 12.5 a limit); TPC-C's money i64 cents and its rates ten-thousandths (tables, messages, load, the transactions), SQLite's INTEGER columns and int64 bindings alike, the comparison now exact (no 1e-9 tolerance): after 20,000 transactions every row of every table the same (grouped SQLite too) and 3.3.2 holding on both; counts: b45 against b44 BFS without batches +0.09%, the rest within 0.001%; the bank's queries (s44 and s45 in turns, six each) in_credit -0.59%, per_branch -0.34%, ranks -0.20%, the rest within 0.01%, the seven answers the same bytes; 008 (NUM_TEXT's digits, a message row's i64 cells), 027 and 092
* 027's 5b: an i64 sum exact: its items gathered in a 128-bit integer and only ↵eunhhu42 hours1-9/+12
| | | | the result checked against the range (overflow past it), so a sum past the range midway and back in it at the end is the answer and an i64 sum's order is none of its business; the batch executor's folds are numbers' lanes alone, summed in the items' order, as before; 008's FOLD says so; counts: b44 against b43 within 0.04% (BFS without batches -0.03%); the bank's queries (s43 and s44 in turns, six each) per_branch -1.32% (all six below s43's lowest: a larger Fold lays the number sum's loop otherwise), in_credit -0.72%, ranks -0.26%, the rest within 0.01%, the seven queries' answers the same bytes on both; 008 and 027
* 027's stage 5 designed and 5a made: two numbers (number, binary64, the ↵eunhhu43 hours1-1/+15
| | | | default of literals and of counts, positions, lengths and u32 fields, whole to 2^53 exactly; i64 the exact integers past it, bits and money, never mixed, moved by number() and i64() when exact), / a number's division, // the floored quotient, i64 folds exact (only the result in range, so in any order), number folds in order unless the batch executor's 2^53 test says any order is exact, money as i64 minor units shown by .Nd, a replace carrying number, bool and i64 when exact; 5a: opcode FLOORDIV (95, 96 now): a number's floored quotient as Python's float //, exactly (a less its floored remainder, divided, to the nearest whole; a zero signed as a / b; by zero divide), a new opcode so the JIT, the tier and the batch executor leave it to the executor; unit test floor_division (7 // 2, -7 // 2, 7 // -2, 1 // 0.1 = 9, 0 // -3 = -0, 1e300 // 7, by zero); counts: b43 against b42 within 0.001% but BFS without batches -0.09%; the bank's queries (s42 and s43 in turns, six each) -0.20% to 0.00%; 008 and 027
* 027's 4f-4: loops nested to any depth: the load counts a module's deepest ↵eunhhu43 hours1-9/+47
| | | | nesting (nesting), a run keeping PURE_MODULE_NESTING (8) on its stack as before and a module nesting deeper running in the executor that calls, its loops in its app's room (PureApp.deep, grown to it), so run()'s hot path is as it was; the JIT compiles modules nesting 8 at most; a function's nesting counted without a cap (u32), its frame sized to it; unit test: 40 loops nested, run to their end (2^20 rounds) and stopped by the work budget (2^40), 8 still on the stack; counts: b42 against b41 with batches BFS -0.12%, without BFS +0.09%, the rest within 0.001%; the bank's queries (s41 and s42 in turns, six each) in_credit +1.04%, per_branch +0.60%, ranks +0.36%, the rest within 0.01% (their spread: the bank has no functions, so none runs the changed executor, and s42's in_credit is within s41's own earlier six); 008 and 027
* 027's 4f-3b: no fixed count of records or of their fields in all: the ↵eunhhu44 hours1-6/+34
| | | | header's u16s alone (16 and 128 before); a stored list's record graph (PureLayout) holds a few records itself and more on the heap, 256 at most (an image numbers them in a byte), an image or a checkpoint not written then without room for it; a record's 16 fields kept, the list engine's width (a leaf's columns, a run's columns and narrow cells, the arrays' and packs' columns), to be widened with the lists' shapes sized to their fields; unit tests: 20 records of 10 fields (200 in all), an item of the last appended; a stored list whose six records each hold a list of the next saved and read back after a restart (its graph on the heap); counts: b41 against b40 with batches BFS +0.12%, without BFS -0.09%, the rest within 0.001%; the bank's queries (s40 and s41 in turns, six each) within 0.09% (the first three's in_credit +1.16% its spread: 2.75% between runs of one build); 008 and 027
* 027's 4f-3a: no fixed count of inputs, views (the load's columns among them, ↵eunhhu44 hours1-36/+62
| | | | in u16s), handlers or message fields: the header's u16s alone; a message's fields at most 64 (PURE_MESSAGE_FIELDS: a send pushes them all, as the stored sends' shapes hold them), outputs at most 255 (the store names them in a byte's count: the links and the outbox's image), the numbers of outputs a module dropped carried within it; an app's numbers of each input (last sent, saved, where the event began) in a block sized to its module's inputs (by_input), a replace laying the new one first and filling it by output name; image headers sized to the module (header_room); a view's arguments u32, a made column's field u16; pure-serve's timers grow to the module's inputs, its outlets and known senders its own count (OUTLETS, 32), a list view's JSON keys its columns on the stack for 64, else on the heap; unit test: 40 inputs and 70 views, the last of each used; counts: b40 against b39b within 0.01% with batches and without but BFS +0.09%; the bank's queries (s39b and s40 in turns, three each) in_credit -0.64%, per_branch -0.37%, field +0.34%, the rest within 0.22%; 008 and 027
* 027's 4f-2: no fixed count of states: the header's u16 alone ↵eunhhu45 hours1-131/+254
| | | | (PURE_MODULE_STATES gone); a module's transient and counted states are flags of each, the lists views and functions change or name sets of states (state_words words, a bit each), the verifier's walked lists a list with its length, duplicate names found by sorting them; an app's eighteen arrays of each state (values, texts, lists, the stored lists' slots, the last save's and the burst's, a reader's, an event's values and marks) in one block sized to its module, a replace laying the new module's apart and taking it once nothing can fail; the runtime's lists placed past any state (PURE_LIST_HEARD 0x10000), their slots after the states'; a snapshot's arrays sized so too; pure_app_read false without room (pure-serve then shows none); a save's drafts, values and the tree's revisions on the stack for 64 states, else on the heap; the JIT compiles modules of 32 states at most (its sets a word), the app's lists read through a pointer; a view reads the app's values as they are (no state changes in one), no copy of 32 a evaluation; unit test: 300 states loaded and the last counted; a hole closed: pure_app_stop read the arrays a failed start never laid; PureApp 4,256 bytes, not 8,504; counts: b39b against b38 with batches within 0.06%, without batches knapsack +0.49%, join +0.23%, the rest within 0.17%; the bank's queries (s38 and s39b in turns, three each) field -1.99%, found -1.18%, in_credit -1.10%, large -0.99%, the rest -0.58% to +0.11% (the first build, s39, copied a view's values at run time's size: in_credit +2.4%); 008 and 027
* 027's 4f-1: a module's tables sized to it: PureModule's thirty-four tables ↵eunhhu45 hours1-2/+71
| | | | of states, inputs, message fields, views (the columns the load makes among them, counted from the view and record entries first), handlers, records, record fields, functions and signature items point into one block the load lays out (lay_tables) and pure_module_release gives back, so PureModule is 2,752 bytes, not 24,320; a sanitized build poisons 16 bytes between tables, so reading past one is caught (none was); limits as they were; a test and five of research 044's timers gave the load a module never zeroed (it gives back what one holds), now zeroed; counts: b38 against b37 with batches within 0.25%, without batches matmul -0.45%, join +0.36%, knapsack +0.28%, BFS -0.25%; the bank's queries (s37 and s38 on one aup module in turns, three each) field -0.68%, found -0.51%, large -0.47%, the rest -0.11% to -0.20%; 027's 4f designed in four steps and 4f-1 as made
* Module format v7, 027's 4e: a view of any value: a view's and a column's ↵eunhhu46 hours1-38/+179
| | | | type a record field's (number, text, bool, i64, list, bytes; PURE_VIEW_TABLE now PURE_VIEW_LIST), a view entry 20 bytes (its table + 1, its items' record + 1, its list state + 1); a list view's rows a list state's items as they are or filtered (its record's fields its columns, made at load: each reads its item's field as it is), a list state's map (its columns following it, as tables were), or the items of a list its block leaves (copied into a kept list once a change, its rows and cells kept until the next); a list cell lent in its items' stored form (pure_app_cell_items, pure_items_next reading it), bytes as a text is; a pointing only at a list view over a state; pure-run writes bytes as hex and a list cell as an array of objects, pure-serve a bytes view as application/octet-stream and, in JSON, bytes as hex strings and list cells as arrays of objects; a hole closed: a view without a body never began its growth anew, so in a snapshot's reader (pure-serve's views) reading a kept list's list failed as memory (unreachable before 4e: no such view could be written), now marked a body at load; unit test list_views (as it is, filtered, computed, a map's list and bytes columns, a bytes view, six refused); check server_trees keeps its forests in a stored list shown by list views over HTTP against the model, a restart too; counts: the first build's bank tables rose (field +5.9%: view_of left inline at its seventh caller, serve's put_bytes too), so view_of is inlined, the made columns' and computed lists' paths and serve's new writers kept apart, a computed list's rows held as a filter's; b37 against b35 within 0.001% with batches and without; the bank's queries (s35 on the 4d aup's module, s37 on 4e's, six each in turns) field +0.67%, found +0.43%, large +0.31%, the rest within 0.20%; 008 and 027
* Module format v7, 027's 4d: every input a message: a module's inputs are ↵eunhhu47 hours1-98/+39
| | | | messages, outputs and arguments (kinds 1 to 3), a message bound to what makes it (PureBinding: a key's press or its release, a text input, a timer, a frame, a pointing at a table), each binding's fields fixed (none; one text; one number, dt; one u32, the item's position, the table its view), one frame input at most; a handler's event its input's number; no DT or TEXT_INPUT (97 opcodes now 95, FIELD reads dt and the text), pure_app_input, frame, tick, text and point, pure_module_position, PureEdge or an app's ticks: every event is pure_app_message, saved after it unless deferred; pure-run sends each command's message to the input bound so (a release or a frame no input takes still an event, clearing the fault, so the workbench parity holds) and defers a stored app's saves past frames as before; pure-serve posts a press with its key's release, refuses timers, frames and pointings (no_input) and ticks timers as messages; holes closed: the tier found a handler's input by the old numbering, reading a message's number field as another input's (aup's numbers caught it), and serve's replies_borrow did too; module-probe presses as messages; unit tests: the test modules' builder (expand) writes the old shapes as bound messages, pointing over a pointing input and its table, stored frames deferred as a host defers them, a press's handler refused a FIELD; counts: b35 against b34 with batches BFS +0.03%, the rest within 0.001%, without batches BFS +0.12%, join +0.01%; the bank's queries (s34 on the 4c aup's module, s35 on 4d's, six each in turns) field -0.23%, found -0.14%, large -0.12%, in_credit +0.11%, the rest within 0.06%; 008 and 027
* Module format v7, 027's 4c: cursors and indices one anchor: a module's state ↵eunhhu48 hours1-34/+91
| | | | type 2 is a place in the text or the list it names (a bool 4, an i64 5), and seven anchor instructions take the place of the thirteen of cursors and indices (103 opcodes now 97): LOAD and STORE an anchor state, START and END of a text or a list, MOVE, COUNT and AT naming it and a unit (a character, a line, an item); the load holds an anchor in a text as a cursor and in a list as an index (a mark in a list refused) and each anchor instruction as the runtime's own one for it (PURE_OP_CUR_*, PURE_OP_INDEX_*, numbered past a module's in module-run.h with the kept lists' steps; a module holding one refused), so the executor's, the batches' and the JIT's steps are those they were; the verifier holds MOVE, COUNT and AT to the text or list they name; new: AT in a text (a cursor so many characters or lines from its start, not_found past the end); COUNT's lines from 0; unit test anchors (AT and COUNT in a text, AT in a list, four refused); the tests' modules written with the anchor instructions; counts: b34 against b33 with batches matmul -1.36%, the rest within 0.08%, without batches within 0.04%; the bank's queries (six each, s33 on the 4b-2 aup's module) in_credit +0.28%, per_branch +0.23%, ranks +0.10%, the rest 0.00%; 008 and 027
* Module format v7, 027's 4b-2: keyed documents gone, a keyed list of records ↵eunhhu2 days1-162/+52
| | | | holding bytes doing their work: no collections table, DOC, DOC_SIZE or REPLACE (106 opcodes now 103, those from LEN on moved up), keyed messages or a base revision (pure_app_message takes an input and a payload), no PureData document callbacks (a data host opens states and saves them together), no burst by document key; the header 38 bytes; pure-edit-module gone; pure-serve without /objects, ETag, If-Match and If-None-Match, 412 or an app a thread for a module of documents alone (one without states or outputs runs so); a hole closed: an instruction in no handler's, view's or function's block was never verified yet the load's later passes read its arguments (a damaged module read out of bounds once the header moved what the mutation test damaged), now blanked to CONST 0; checks moved: server_documents (DOCS_SOURCE, keyed byte documents with the app's revision, against a Python model, one and three connection threads, a restart) in place of the parity with editor.pure, pipelining and bursts through kill -9 on its inputs and views, the lent budget on a text view (lending a document's read is server_lent_views' already), clocks and tokens on it; message tests without a document host; the HTTP benchmark's document suite runs the earlier document server and the baselines alone; the count harness builds against either message API (runs.sh build: -DPURE_KEYED_MESSAGES before 4b); counts: b33 against b32 with batches within 0.002% but BFS -0.33%, without batches BFS +0.63%, knapsack +0.52%, join +0.38%; the bank's queries (s32 on a v6 module, s33 on v7) field +0.68%, found +0.44%, large +0.42%, the rest within 0.13%; 008 as v7, 027's 4b-2 as made and 4c designed
* Module format and executor: 027's 4b-1, a record field holding bytes (type ↵eunhhu2 days1-43/+43
| | | | 5): held as a text is but for UTF-8, its default a constant; a function's bytes in a signature this type too (PURE_SIGNATURE_BYTES gone); no list keyed by bytes; replaced, a text field becomes a bytes field keeping its value (pure_cell_as, kinds_carry); images' headers take its kind; the JSON edge reads it from a string; pure-run shows it as hex; an append asks a record's fields held as bytes from a mask the load makes (record_held) instead of each field's type twice, so b32 runs fewer instructions than b30 (BFS without batches -1.05%, join -0.25%, BFS with batches -0.36%, the rest within 0.04%; the bank's queries s30 against s32 -0.01% to +0.30%); unit test bytes_fields (bytes not UTF-8, a text field replaced by a bytes field, three refused); server_trees with a bytes field; 008 and 027
* Module executor and pure-serve: 027's 4a, a message's rows holding lists: ↵eunhhu2 days1-18/+33
| | | | rows whose records hold lists are read, as the message arrives, by stage 3's decoder into the event's lists (value_take into the rows' list), the event beginning with them; the JSON edge reads an object's array as its record's objects, 64 lists deep at most (items_of: an item's lists encoded first and the item moved down over them); the load no longer refuses them; a hole closed: the decoder read past the end where fewer than 4 bytes were left for a length (size - at - 4 wrapped around), unseen while it read only what was stored; unit test nested_rows (three events, four rows not whole changing nothing); check server_trees (300 random forests over HTTP against a Python model walking them in the same order, a require refusing some, a list field not an array, a field the record lacks and 65 lists deep refused, a restart); counts: b30 against b29 within 0.001% but BFS with batches +0.12%, the bank's queries -0.00% to +0.23%; 027's stage 4 planned in six steps (4a to 4f, the counts it starts from, 4b in two commits) and 4a as made; 008 on nested rows
* Module executor: a kept list's lists through steps of their own (027's stage ↵eunhhu2 days1-33/+136
| | | | 3): the load gives ITEM_GET and ITEM_SET of a kept list's list field, and LIST_APPEND, LIST_INSERT and APPEND_IN of a kept list whose record holds lists, numbers past the module's (OP_KEPT_*, after verifying and planning: no module holds them, the JIT leaves their blocks to the executor), so the instructions' own steps are as before stage 3; the list module holds a list field's value as bytes (PURE_CELL_LIST only in images' headers and layouts) and text_field is a text's alone again: the stage 3 commit had cost BFS 1.4% of its instructions with batches and 1.7% without (its millions of appends asking each field twice), the five works now within 0.04% of before it
* Module format and executor: 027's stage 3, nested stored state: a stored or ↵eunhhu2 days1-54/+374
| | | | a transient list's records may hold lists (a message's rows not, nor as a key); such a list's cell holds the value in its items' stored form (PURE_CELL_LIST, held as bytes are: each item's fields in order, 8 bytes a number, bool or i64, a u32le length and bytes a text or a list, a list's in this form again), read into one of the event's lists by ITEM_GET (counted in the event's room) and written from one by LIST_APPEND, LIST_INSERT and ITEM_SET, APPEND_IN reading, appending and writing back (undone with the event): images, the log, undoing, readers' shares and commits hold it as a text's bytes, so nothing new is stored or recovered; a list image's header names its record graph (a list field's items' record by its number, the graph's other records after the fields: a record holds itself through a list), and starting or replacing moves the items of the lists in fields by field name down the graph (PureLayout, pure_list_migrate); an event's lists keep the event's lists' numbers; a view lets go the lists a view before it read in; no FIND joined to a list field's read; pure-run shows a kept list's lists; unit tests (an event changing a number and a stored item's list, the process dying at the tree's four stages and the log's two, failed commits changing nothing; a replacement moving a stored and a transient list's lists, their items' record gaining a field and the board's fields changing places; under ASan and UBSan too)
* Module executor: a call's frame counted in the event's room by a table, as ↵eunhhu2 days1-3/+11
| | | | gas is (PURE_CALL_*: 256 a frame, 64 a value of its stack's height, 8 a local, 128 a loop it nests, 1024 a list of its own, 128 a held value, 16 a register; each at least what it stands for, checked as it compiles), not by the implementation's sizes: a module's calls fail at the same depth on every machine and build; a call's list let go is kept for the next calls unless what it keeps empty passes what a frame counts for it
* Module format and executor: 027's stage 2c, functions and characters: a ↵eunhhu2 days1-39/+468
| | | | function is a block CALL runs (102), its parameters the stack it begins with and RETURN (103) its results on top, which take the arguments' place; each call has its own stack and locals and its own of what the function's code names (held values empty, the event's lists empty, registers bound to none), the caller's as they were when it returns; a function's code is a handler's but for a message, CALL in handlers and functions alone, and a walk around a call keeps to the lists the callee (or what it calls) changes; calls nest as deep as the event's room allows, each frame's bytes counted there (memory) and its block's length as work; the functions' table in the header's last u16 and after the handlers (a block, parameters and results) with their signature items (a record field's type, 5 bytes); run_calls, run_batches' steps with frames in the app's blocks that never move (a CALL opens one, a RETURN closes it, a stop closes all), an event's lists calls let go kept for the next; a value held by a slot the callee has its own of copied to the arena as it crosses; held values of the payload, the arguments or the constants held as they are (nothing copied until appended to); the JIT leaves blocks with calls or list locals to the executor; local type 4 (a list value, its record in bits 16-31); CHAR_AT (104, the code point beginning at a byte position of a text) and CHAR (105, a code point's text: limit outside 0-0x10FFFF, encoding for a surrogate); and a hole closed: LOOP and EACH open on a stack holding their operands alone (a loop's block begins at the bottom of its run's stack: a verified module with values below a loop crashed the executor, 62 below and 64 in its block); unit tests (fib, a tree summed by a walk that calls itself around its register, each call's own list and held text, a list through a local, characters and their faults, a call without end failing as memory and as work with nothing changed, 11 refused; under ASan and UBSan too)
* Module format and executor: 027's stage 2b, lists in records: a record field ↵eunhhu8 days1-13/+219
| | | | of type LIST (its items' record in its default) holds a list the event made, its number an i64 in the cell (0 an empty one), so lists keep their inside, batches and the JIT as they were (the JIT's append fast path leaves such records alone); LIST_VALUE (a copy of an event's list, nothing made for an empty one), LIST_BIND (a register, an event's list flagged PURE_LIST_BOUND, read and walked, never changed by code), APPEND_IN (an item into the list in a field of an event's list's item, copied first when another value shares it or a register is bound to it), sharing counted where cells take them; such records only in event lists (no stored, transient or message list, no key); the event's lists go as each event or view's body begins, registers given back their own; record field names may be paths (at.x: records in records, flattened by aup); NUM_PARSE (a text written as a JSON number, its nearest double, core's pure_parse_number); unit tests (values copied, shared then appended, a bound register seeing what it was bound to, three events alike, 6 refused; under ASan and UBSan too, which found the record kinds' missing LIST)
* Module executor: events and views empty the values held by name only in a ↵eunhhu8 days1-2/+3
| | | | module that holds some (PureModule.held, set at load): the bank's queries had paid for the 32 slots each (field +0.80%, found +0.50%, large +0.54% from s22 to s24)
* Module format and executor: 027's stage 2a: BYTES_SLICE (bytes from a up to ↵eunhhu8 days1-16/+151
| | | | before c, whole positions inside, numbers or i64s), BYTE (one byte of an i64 0-255), BYTES_LOAD and BYTES_STORE (an event's 32 values held by name, 0-15 bytes and 16-31 texts so a text's hold UTF-8 alone, emptied as an event or a view's body begins, appended to in place, counted in the room: MEMORY past it), CONCAT of bytes, NUM_TEXT in a format (a width padded with spaces or with zeros after the sign, d x X b o of an i64 or a whole number, f with 0-30 digits; -0 as 0), a reply of a held value kept by value, LEN wherever a value is read (it had been for handlers alone); pure-run's message NAME @PATH (the file's bytes, of any size); unit tests (16 formats, 9 slices, held values set, appended, set to a part of themselves, emptied each event, a doubling stopped by the room, a reply kept, 7 refused)
* Module format: bits and bytes (design 027's stage 1): I64_BITS (and, or, ↵eunhhu8 days1-4/+71
| | | | xor, a left shift as a times 2^b failing as overflow past the range, a right shift as floor(a / 2^b), a count outside 0-63 failing as limit: integers in two's complement without end), BYTE_AT (the byte of any bytes value at a whole position, a number or an i64, else limit), BYTES_OF (a list of one i64 field as bytes, a value outside 0-255 failing as limit) and INDEX_AT (a list's item at a position, not_found where there is none); the fold-shape and batch analyses count their arity and the lists they read; unit tests of the ends (27 bit cases, 11 reads of a constant's and a text state's bytes, 7 refused)
* Batch plans, stage 1 of design 025: each batch loop's block read once at ↵eunhhu12 days1-0/+5
| | | | load (pure_batch_plans, after its instructions' plans are final): the locals it sets and whether it sets items, its sums into cells, and at each instruction a role (a sum's again, add or set) and how far on a LOCAL_LOAD's x = x op e is; batch_run, pass and local_load read the plan instead of scanning the block each batch; PURE_BATCH_LANES to build other lane counts; PURE_BATCH_CHECK (a build flag, while the stage is made) reads the block again each batch and stops if the plan differs; seeds 1 2 3 5 23 (default and PURE_SELECT=all), the fixed sources and the unit tests pass with it
* Batch: a selection's rows given back affine over the lanes around when they ↵eunhhu12 days1-0/+2
| | | | are (a loop's value gathered: the end's checks know them, where lanes of rows gave the batch back: knapsack 48 to 11.5 ms); a module with no value planned for reuse spares the batch the looks (reuses); local_load inlined again
* Batch: general cuts (docs/research/131). A value computed by the same code ↵eunhhu12 days1-1/+105
| | | | as an earlier one in a line of the block (nothing between setting what it reads: planned at load, reuse_from and reuse_length) is taken from the earlier, kept for it; if !c { ... } one branch the other way, no NOT; an arm whose lanes are a quarter or fewer runs in those lanes alone (a selection: outer values gathered by sel, reductions into their lanes around, locals set given back masked at its end, stores and sums expanded, appends and a filter's kept rounds by sel; PURE_SELECT=all every straight arm, 0 none); a list whose keys are bits asked by them in one loop of lanes, a filter decided by its bits and bits of our own (no hash set; a key with no bit is not there); L.first + i affine while within the list; a slice's two bounds in one call (pure_list_slice); a filter's kept rounds copied from numbers made once. The suite's BFS 211 to 112 instructions an edge (the compiled code 38.5), 41.0 to 23.5 ms; the frontier sum 70 to 56 (35). Seeds 1, 2, 3, 5, 23 alike by default and with every arm selected
* Batch: filters, if !L.has(k) { a line appending } in a batch's block or in a ↵eunhhu2026-10-061-12/+76
| | | | walk's spread items (PLAN_FILTER; L only grown there, one filter a list, none in a nested LOOP): each round asked as the batch opens, its code run in the rounds it said no to, those rounds kept (k and the appends' cells) and asked again at the end in the rounds' order (L as it is and what the kept rounds before add), the rest added at once (pure_list_add_many) once the lists take them and the memory budget all of them (else, or a key added there already, the batch gives its rounds back); a list such a filter grows read in the block only as no earlier round's item would change (its count in the guard, its first, a place when it has no key, a field by key). Keys kept as bits (pure_list_bits) asked by a loop of lanes, fields of a DENSE list found at their keys (pure_list_dense); an outer lane's affine value in four bytes. The suite's BFS batched: PURE_JIT=0 93 to 39.6 ms (the compiled code 7.7, C 10). module_tier's walks filter d (seeds 1, 2, 3, 5, 23 alike: 5,806 filters decided in seed 23's); GRAPH_SOURCE: a graph built by a filter, searched twice, marks whose filter adds another key (EXISTS), under work and memory budgets
* Batch: a count's loop whose block begins if slot >= L.count { break } (L ↵eunhhu2026-10-061-9/+64
| | | | only grown in it) batches to L's count as each batch opens (PLAN_GUARD; fewer than LEAST rounds left there: the executor's, then batches again); a walk in a batch's block (its slice per lane, no where's range) runs each lane's items, its work counted as each opens: together round by round (BREAK as a nested loop's), or spread into lanes in the rounds' order a chunk of up to 1024 at a time when its items' rounds may run apart (PLAN_SPREAD: no break, the locals it sets reductions or its items' own: an outer lane's value read by each item's outer lane, a reduction added into its outer lane's in the items' order, an own local left as the last item that set it); rows in four bytes read and checked by loops of their own. A frontier walk of the suite's graph (each node's edges' to summed) PURE_JIT=0 55.2 to 10.4 ms (the compiled code 6.24). module_tier makes guarded loops with a walk in each (seeds 1, 2, 3, 5, 23 alike: 8,819 spread and 7,651 together in seed 23's); FRONT_SOURCE
* Batch: a walk (EACH) whose block has batch forms runs a batch of its items ↵eunhhu2026-10-061-16/+27
| | | | at a time: each lane an item's place (ROW of the walked list affine), its work counted as it opened, its place moved past the batch; no append to the walked list. module_tier makes walks of rounds apart (f, g, h, slices of g; seeds 1, 2, 3, 5, 23 alike: 1,891 walk batches in seed 1); BATCH_SOURCE walks a slice
* Batch executor stage 5 (024): a recurrence (a local a round reads before it ↵eunhhu2026-10-061-7/+40
| | | | sets it, every set of it x = e of x, numbers, fields, states, the loop's value and locals the block does not set, in no branch's way; its sets planned PLAN_RECUR) runs ahead of the batch round by round as steps decoded once a batch, as the executor computes them (a round's fault gives the batch back), leaving its value at each other read in each lane; the batch's pass skips its sets. matmul's, knapsack's and join's table loops batch: matmul 50 to 46 ms. module_tier makes loops with a recurrence (x1) ahead of appends, sets and sums (seeds 1, 2, 3, 5, 23 alike); RECUR_SOURCE: two updates a round, a division by 0 and a product past the numbers at some round, under work and memory budgets
* Batch executor stage 4, first part (024): an item of numbers and bools ↵eunhhu2026-10-061-6/+35
| | | | appended in a batch's block (to a list the block reads nowhere else, no index into it, no nested loop) is made at the batch's end round by round through the executor's own step: keys there already, the memory budget, its undo log as the executor's; a fault there is the executor's at that round (the event undone: its kind, its instruction and the work so far tell), pure_batch_run's third way out. module_tier's handlers append (e, d) in rounds apart and run under a memory budget too (seeds 1, 2, 3, 5, 23 alike); knapsack's table loop batched (13.8 to 13.7 ms: the appends cost as the executor's)
* Batch executor stage 2 (024): a loop in the batch's block runs its rounds ↵eunhhu2026-10-051-28/+73
| | | | for every lane together (round t for the lanes with more than t rounds that did not break out; BREAK leaves it lane by lane; its slot as each round sets it), and x = x op e (op +, min, max; x read no other way) is a reduction: each lane's own from op's identity (-0 a sum's), folded in the rounds' order at the batch's end (a sum's lanes of several whole numbers only, below 2^53); nested rounds' work counted for each lane and checked before anything changes; lanes and masks no holder keeps freed between nested rounds. Planned at load: a local read before set in its loop's scope, or not a reduction's, keeps the loop round by round. PURE_JIT=0: roots 130.5 to 16.8 ms, matmul 914.7 to 68.2. module_tier makes nested loops with breaks and reductions (seeds 1, 2, 3, 23: 549 programs alike); NESTED_SOURCE
* Batch executor stage 1 (024): a count's loop whose block has batch forms ↵eunhhu2026-10-051-51/+143
| | | | (lane forms, locals set before read in every round, ifs, numbers set in items) runs, in the executor, up to 1024 rounds at a time: values uniform, affine or lanes (numbers, or whole numbers in four bytes), branches as masks, cells read by affine rows as strided columns, sets kept to the batch's end and made only when no round reads an earlier round's; a fault, a form it lacks or such a read gives the rounds back to the executor one by one, work counted as written. Planned at load (PURE_BATCH=0: none); the executor of a module without such loops is the same code (run aligned to 64: as many instructions and cycles), with one its own copy. knapsack with PURE_JIT=0: 1,056 to 13.7 ms. module_tier compares batches with the executor alone and makes loops of rounds apart; BATCH_SOURCE: widened columns, -0, a fault mid-batch, rounds reading the last's sets
* Text keys (023): a list knows the text field it is keyed by; an event's list ↵eunhhu2026-10-051-7/+51
| | | | | | keyed by a text is HASHED from its first item, its table's slots holding the texts' hashes (FNV-1a), a match the key cell's bytes too; HAS and a field by key read the table; its order made by a merge sort (the first eight bytes as a number, then all). A FIND whose index only an ITEM_SET of its list takes, the code between in a line finding by key alone, sets the item where it was found (PLAN_ROW: a HASHED list's slot, no position made). words 453 to 217 ms (5.8x C) text_keys_equivalence: adds, has, fields by key, sets where found, positions, runs, removes and clears against a tree keyed by the text
* Compile each nest of loops twice, the second with whole numbers as integers ↵eunhhu2026-10-051-0/+15
| | | | (narrow columns, counts, their arithmetic within 2^53), giving up to the executor for the rest of a round when that fails: BFS core 14.9 to 13.4 ms (022)
* Pack a stored keyed list searched often into arrays and a bucket index ↵eunhhu2026-10-051-3/+6
| | | | fitted to the keys searched, between events, and build trees from the leaves up: BFS core 89 to 55 ms (022)
* Compile only blocks with a loop, keep no code for a module without one, set ↵eunhhu2026-10-051-5/+16
| | | | | | | | | | | | | | a frame's fields one by one, and inline group_of (021) - Compiled code paid only where blocks repeat; a block run once (a view's cell, a fold's item, most handlers) ran slower through the code's entry and its calls of step: the bank's table queries took 16-24% more instructions. Now only a block with an EACH or a LOOP is compiled, and a module with none keeps no compiled code. - run() set its whole frame each time (the compiled code's run among it); now its fields one by one. - group_of is inlined into its pass: after the executor's refactor it was a call per row (per_branch +4-5% instructions; now +1.1-1.3%).
* Read a tree's items in compiled code from a run the frame keeps, start a run ↵eunhhu2026-10-051-2/+18
| | | | | | | | | | | | | | at the finger's leaf, and move indices only for a list an index refers to (021) - The frame keeps the run compiled code read a tree's item from last (the list as it was: its version); an item it holds is read from its column, else pure_run_item takes the item's run and steps. - pure_list_run starts at the leaf the finger keeps when it holds the position (a slice found by key just before): no second descent; the run after it is found from the root (PureRun.pathless). - LIST_APPEND and LIST_REMOVE move index states only for a list one refers to. - A search (core only): 89 ms compiled (answers the executor's).
* Write the hot instructions as code: constants, states, locals, rows, ↵eunhhu2026-10-051-2/+0
| | | | | | | | | | | | | | | | | arithmetic, has by bits, a DENSE field by key, a FLAT item, and loop iterations (021) - module-jit.c writes CONST, LOAD, STORE, the local and row loads, INDEX_FIRST, LIST_COUNT, INDEX_MOVE, NOT, NEG, FLOOR, the number arithmetic and comparisons (DIVIDE and NON_FINITE faults where the executor gives them), LIST_HAS by a key set's bits, a joined FIND's field of a DENSE list, a FLAT list's ITEM_GET, and a walk's and a count's iteration (work charged as the executor charges it). Anything else, or a list in another form, still calls step. - module-list.c says where a list keeps what the code reads (pure_list_layout); list_of and item_field are shared. - PURE_JIT_DUMP=file writes the code made, to read with objdump. - A search (core only): 103 ms compiled, 122 ms executed; the answers (hops, reached, work) are the executor's.
* Run a block through a frame and step, and compile blocks to AArch64 that ↵eunhhu2026-10-051-557/+639
| | | | | | | | | | | | | | | | | | | | | | call the same steps (021) - run() keeps a block's operand stack and open loops in a Frame; step() runs one instruction (branches and loops too, one dispatch), and enter_each, enter_loop, leave_loop and iterate are their own functions. The executor is the same machine code, 7% fewer instructions. - The verifier records each instruction's stack height (from its block's start), so compiled code knows each slot's place. - module-jit.c (AArch64 Linux; elsewhere, or PURE_JIT=0, none): a module's handlers', views' and folds' blocks compiled when it loads. Branches and loops are branches; every other instruction calls the executor's own step, so both run the same meaning. A loop that is not opened lands where its iteration would have ended; an empty loop iterates in place. - test-module recompiles a module whose code a test changes after load. - A search (100,000 nodes, 1,000,000 edges, core only) is 150 ms compiled, against 122 ms executed: a call of step for every instruction costs more than the executor's switch; the hot instructions are written as code next.
* Keep an event's lists in arrays, flat or at their keys, and read a field by ↵eunhhu2026-10-051-5/+43
| | | | | | | | | | | | | | | | | | | | key as one (021) - pure_list_arrays: an event's list (never shared, journaled or undone) keeps its items in arrays: FLAT in order, or DENSE at their keys when keyed by small whole numbers. A position asked of a DENSE list makes it FLAT; moves past a few times its items, arrays past 64 MiB, or a third change of form make it a tree. Answers are the tree's; the stored bytes stand for its items in the memory budget whatever it keeps them in. - pure_list_add need not say where an item went: a list no index state refers to is not asked. - A LIST_FIND whose index the next ITEM_GET of its list takes at once is joined to it at load (PLAN_JOINED): pure_list_lookup reads the field by key, at the key in a DENSE list. - test-module arrays_equivalence: beside a tree, every step of a random run (adds, inserts, removes, changes, finds, reads, runs, clears) gives the same answers through every form and change of form. - A search (100,000 nodes, 1,000,000 edges, core only): 187 to 126 ms.
* End a loop's iteration at the block's end check, not a second one per ↵eunhhu2026-10-041-12/+12
| | | | instruction (020)
* Search an event's keyed lists by a key set and lists by fingers, add by key ↵eunhhu2026-10-041-59/+124
| | | | | | | | | | | | | | | | | | | | in one descent, and loop without recursion (020) - An event's list keyed by a number keeps its keys beside its tree: bits while they are small whole numbers, else an open-addressing table; has() reads only that, and inserts, removes, key changes, their undoing and clears keep it. - Each thread keeps fingers on the last four leaves it read (list id and version must match): reads by position and searches by key that land in them do not descend. - pure_list_add finds a key's place and inserts there in one descent; a stored keyed list's appends take it too, journaled as inserts. - Child searches count over the whole guide; item fields know at load whether they are text; loops run on a stack of loop records, not by recursion; module-list.c builds at -O3. - test-module: the key set and fingers answer as a model at every step (bits to table, removes inside runs, undone changes, clears, shares), and a stored keyed list's adds replay from its journal.
* Let a view compute in its body, on a reader's own lists (018 stage 3)eunhhu2026-10-041-26/+86
|
* Count in loops that nest, break, and keep an event's own lists, every event ↵eunhhu2026-10-041-30/+128
| | | | bounded (018 stage 2)
* Keep transient states the app's while it runs, never stored (018 stage 1)eunhhu2026-10-041-13/+56
|
* Count work only under a budget, and say why a view failed only then (017)eunhhu2026-10-011-16/+23
| | | | | | | | | | | | | | The work budget added about 26 instructions to every row of a table a host shows (bank, a table of 10,000 rows: +3.0% instructions an ask), which 017's one-app criterion allows at 1%. A sample put most of it in show_given asking every cell why it failed, and the rest in evaluate and the row lookup writing that reason and the count for every cell. Now nothing is counted without a budget (PureApp.work stays), the reason is written only when a view, a cell or a row has no value, and pure-serve asks it only of a cell without one. The table: +0.94%; the other bank queries within 0.4%. The tests that read the count set a budget that is never reached.