| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
| |
carry streams across module changes
|
| |
|
|
| |
must show (019)
|
| |
|
|
| |
research 119)
|
| |
|
|
| |
measure instead (018)
|
| | |
|
| | |
|
| |
|
|
| |
bounded (018 stage 2)
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
Requests to an app that checks tokens go 2.92% slower than to an open
one on 64 keep-alive connections, 2.2% of it the head the token makes
longer (an open app sent the same head is as slow); hashing every
request on a new connection costs 0.91%. 031 measures the three in
turns.
|
| |
|
|
|
|
|
|
|
| |
An app given a tokens file (tokens=FILE, --tokens FILE) takes requests
bearing a token whose SHA-256 the file lists: read for its views, queries
and documents, write for all, input=NAME for one input alone; 401
without one, 403 past its grant. A delivery another node sealed needs
none: its key told who sent it. A connection keeps its last token's
grant; a reading swaps the tokens without stopping the app.
|
| |
|
|
|
|
|
|
|
| |
The use after free in local_done and the owner's lock made after another
app's outlet used it were found reading the code, not by the check,
which passed with both before it was made to change senders (115, 017).
The usage now says changed lines change in place unless their store
moved; and 115 names the path no check takes: a module refused while
changing in place.
|
| |
|
|
|
|
|
|
|
|
| |
Changing an app in place keeps a million items' app answering but for
1.12 ms (a host restart: 150 ms; stopping and starting it again from its
store, as first built, 150.4 ms), 171.5 ms when its records change shape
(332 ms). Another app's p99 stays 0.97 times as it was while one changes
every 0.2 s; a host of one app is as it was (TPC-C +0.7%, instructions
at most +0.02%); each module change leaves 19 KB. 030 holds the windows
of both builds, the isolation, the pairs, the counts and the memory.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
(017, stage 3)
The --apps file is the host's shape: on SIGHUP the host reads it again
and starts the apps of new lines, stops those of lines gone, starts
again those whose line changed (states carried by name from the store)
and those stopped or uncertain. A request carries the module it was
resolved with, and is not applied under another (503 stopped); link
streams and local outlets look their input up again. Route
authentication (3b) is designed apart.
|
| |
|
|
|
|
| |
The same events make byte-identical stores on the x86_64 WSL2 (4 KiB
pages) as on the Pi and the Mac, and a store made on either of the Pi
and the WSL2 opens on the other and goes on the same.
|
| |
|
|
|
|
|
|
|
| |
The same modules and inputs give the same outputs, failures and store
bytes on the Pi, an x86_64 WSL2 and an M3 Mac; other compilers found an
argument read before it was set, and the core and store build on macOS
again. WSL2's wall clock steps back about 2 s every 32 s. The Pi froze
twice under a measuring tool that held its output: it now streams it,
and experiments run under research/guard.py.
|
| |
|
|
|
|
|
| |
The loop alone slowed four links to half: woken by every record queued,
it chased the owner a record at a time. Pushing once a turn made links
1.6 times as fast, with threads or the loop alike; the loop's own cost
is within the noise. Twenty outputs now add two threads, not 21.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Each link and each HTTP wire sends on a thread of its own: a host with
twenty outputs runs twenty-one threads more than one without. 2b moves
them into one sending loop: an epoll over the links' UDP sockets and
the wires' TCP sockets with one wake, waiting until the soonest link
timer or wire pause. Links (2b-1) do in the loop what their threads
did each round; wires (2b-2) become a non-blocking HTTP/1.1 client
state machine, with non-blocking steps added to the crypto table for
TLS. A name looked up again still waits where it is, and says so.
Before building: every link, wire, answer and seal check green; a host
of ten links and ten wires two threads above one without; and link and
wire throughput, alternated before and after, no more than 5% lower at
the median (four links together: 10%).
|
| |
|
|
|
|
|
|
| |
Stage 2 is built and measured: what was learned (the receiving side
needed nothing new, an output goes one way only) and the numbers. The
checkpoint adds sending to the many-apps row, and keeps 2b (one loop
for the send threads) and stage 3 (apps added and replaced while the
host runs) among what is left.
|
| |
|
|
|
|
|
|
|
|
| |
Two apps of one host, src sending one message for each of 1,000 items
in a burst event, 2,000 bursts a run, five rounds with the ways in
turns: the local outlet stores 1,141,642 sends a second at 1.17 us of
the host's CPU each, a loopback pure:// link to the host's own port
578,879 at 2.93 us (1.97 times the sends, 0.40 times the CPU). 017's
second stage asked for at least the link's rate at half its CPU or
less: it goes in. Every run checks dst's sum: each send came once.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
show (017)
Stage 2: each app of --apps takes its wiring (send=, seal=, reply=), its
sends' source being APP.OUTPUT; one --link-port takes every app's
inputs, a stream's input name APP/INPUT (012 unchanged); and send=
OUTPUT=APP/INPUT hands an app's saved sends to another app of the same
host with no socket, seal or packet: an outlet with the window links
have, a batch of records as a link delivers them put in the receiving
app's queue, applied after its last number heard, and the number heard
after its save trimming the sender's outbox. Merging the send threads
into one loop is split off as 2b.
Before building: it goes in if the inner outlet stores at least as many
sends a second as a loopback pure:// link between the same two apps,
at half the host's CPU a stored send or less; checked by a host sending
to an app of its own and to one of another host, exactly once and in
order across a kill -9, and held back by a full receiver.
|
| |
|
|
|
|
|
|
|
|
| |
017's first stage is built and measured: the shape now says what was
built where it differs from the design (work counted as written,
nothing counted without a budget, every app of --apps run by workers,
a busy app kept on its worker, a home thread for each app's
connections), and what was learned on the way. The checkpoint lists
many apps on one host among what Pure shows, and stages 2 and 3 among
what is left.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
017's first stage met its four criteria on the Pi (RAM stores, five
rounds, the order of the ways turned): app B's 99th percentile while
app A is flooded by 64 connections is 168 us, half of two processes'
336 us; the two apps busy together write 146.1k a second against
146.2k, with the same 1.4 ms tail; an app costs a host 120 KiB of its
own memory (a process: 1,311 KiB PSS) and 100 apps still run on three
threads; a host of one app keeps its speed (TPC-C +1.2% at the median
of six pairs, two of them interfered and kept; the bank's queries
within 2.6%) and its instructions (+0.94% at most).
On the way: apps hopped between workers each turn (fixed in db4ee9c),
connections of one app spread over all connection threads made the
host a third more expensive a request and its tail a kernel tick
(fixed by home threads, d24881d), and the work budget's counting cost
every table row 26 instructions (fixed in be297ea). The earlier runs
are kept apart (db4ee9c's isolation, d24881d's pairs and counts, the
interrupted first try).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve runs one app a process: a module, a store, its own threads,
port and key, scheduled by Linux. Within one process nothing bounds an
event's run (it grows with its lists), the requests waiting for the
owner, or one store's sync, which holds every other store's commit
since the tree store has one writer's turn per process.
017 makes an app an execution domain (007): a host of many apps
(--apps FILE, routes /APP/...) runs them by turns on a fixed number of
worker threads, none per app; budgets per app in deterministic units
(memory in stored bytes as before, work in instructions counted per
block and per item visited, queue in requests: 422 work, 503 busy);
writer turns per store; and an app that cannot start or whose save
became uncertain stops alone. Stage 1 is all that; links, sends between
apps and an app's lifetime come after.
Before building: it goes in if an app B's p99 under app A's flood is
within 1.5 times what two processes give, an app costs under a quarter
of an idle process's own memory with no thread of its own, the pair's
throughput is within 10% of two processes', and a one-app host keeps
its speed (3% in time, 1% in instructions).
|
| |
|
|
|
|
|
|
|
|
| |
The six pairs rerun once the Pi had been quiet for a minute, on the same
binaries. Two pairs were still interfered (the first's before at an
owner share of 0.54, the sixth at 0.73 on both sides); the four clean
ones are -1.7%, -0.4%, +1.8% and +1.1%, CPU per transaction 43.25 to
42.9 us. The bank's queries are as fast or faster after, though the
interference fell more on the before runs; the instruction counts stay
the comparison to trust.
|
| |
|
|
|
|
|
|
|
|
|
| |
016 is built: bool and i64. The second stage's findings: an integer
literal takes the other side's type, found by compiling it on trial
(linear once the wanted type is passed down); a literal with no context
stays a number, so no program changes meaning and modules without i64
are the same bytes; one rule carries values between numbers, bools and
i64s; and the number paths passed the timing criterion, with what i64
had added to them counted and halved (110). The checkpoint lists bool
and i64 in the app model and drops i64 from what is left.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
016's second stage passed its criterion: six interleaved pairs against
ee21aec gave TPC-C (RAM, W=1) -0.6% at the median and the bank's
queries 0.0% to +2.2%, no run interfered. The ranks query was slower
all six times, and chasing it showed that on this Pi timing cannot
separate 1-3%: two servers asked in turns disagree with themselves
(ee21aec to e02bfc9 +1.6%, e02bfc9 to 481642f -1.2%, ee21aec to 481642f
+2.8%), and a query whose instructions grew 0.6% ran 3.5% faster.
Instructions do not move so: pcount.c counts the server's user-mode
instructions with perf_event_open (no perf here), and psample.c samples
where they go. ranks had grown 0.4% in instructions (the rest was where
code and data landed), per_branch 3.6% and tables 20 instructions a row,
which 07047c3 halved. The six pairs of 07047c3 were interfered (another
session's build, load 4-12) and are kept apart, unused.
|
| |
|
|
|
|
|
|
|
| |
server_meaning gives every value an i64 field, 13 values from 0 and -0
to both ends of the range and just inside them, in over JSON, rows and
a query, over the link (where B takes id, a u32, as an i64) and HTTP,
and out at all seven exits with every digit. 32 more refusals, each
with its reason on JSON, a query and rows; an i64 view's text at both
ends and an i64 argument that counts exactly and refuses the rest.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An i64 is an int64_t in a value's 8 bytes, its own stack type: CONST and
NUM_TEXT take argument 2 for one, eleven opcodes are appended (ADD, SUB
and MUL fail as overflow past its range, DIV and MOD floor and fail as
divide by zero, NEG, COMPARE with a relation, MIN, MAX, and I64_FROM and
NUM_FROM, which convert only where the value fits and else fail as
limit), and moves (LOAD, STORE, locals, items, FIELD, SEND) copy it as
they copy a number. Type codes: state 6, record field 3, message field 7
(8 bytes), view 4, local 3. v6 stays: all of it is added.
- Folds with PURE_FOLD_INTEGER sum (overflow fails), keep the most or
least of, or count the different i64s; distinct's set keeps an i64's
-1 apart from its free mark. Blocks with an i64 run item by item.
- A list's kinds are its stored header's (number, text, bytes, bool,
integer): integer columns are int64_t, ordered as integers as keys
(found without guide keys for now), stored in 8 bytes and logged as
zigzag varints of any size. Records name their fields' type in one
array (record_field_type) in place of text and bool flags.
- A number, a bool and an i64 carry over each other by one rule,
pure_cell_as, wherever one becomes another: a module replaced, a stored
value or item read by another module, a waiting send recast. A u32
field delivers to an i64 one.
- The host reads i64 fields from JSON, rows and queries as integer digits
in range and nothing else, and shows every digit in views, tables,
deliveries and pure-run. Faults are named by one function,
pure_fault_name, in place of three tables; overflow is new.
test-module: 28 arithmetic cases at the range's ends, relations,
conversions both ways, 14 refusals, a keyed i64 list app (order, a sum
that overflows, a failed add undone, messages and sends of 8 bytes, every
digit as text), recasts, replacement between numbers, bools and i64s, and
three stored runs against memory with restarts (46 images written).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Before building i64: a value slot holds a number or an i64 in the same 8
bytes (the stack, states, list cells, locals, constants), moves copy the
bits and only computing opcodes look at the type; eleven opcodes are
appended (arithmetic that fails as overflow, floor division and modulo,
compare, min, max, the two conversions), CONST and NUM_TEXT take
argument 2, and type codes are added, so v6 stays. i64-keyed lists compare
as integers without guide keys for now; folds over i64 run item by item.
JSON and queries take plain integer digits in range; views, tables and
deliveries show every digit. u32 widens to i64; i64, numbers and bools do
not deliver to each other; replacing converts when the value fits.
It is in when every check is green and the number paths keep their
speed: bank queries and TPC-C (RAM) no more than 3% slower at the median
of six interleaved pairs.
|
| |
|
|
|
|
|
| |
016's first stage is built and checked: no opcode was new, a stored
send's bool was refused on the way back in (found by server_meaning's
restart, fixed), -0 becomes false as 015 says zero has no sign, and
True in JSON is no JSON at all. i64 is what is left of 016.
|
| |
|
|
|
|
|
| |
server_meaning's origin gains a bool view and a view with a bool
argument: GET /views/any_ok answers true as text, with_ok?want=true and
?want=false count the kept values each way, and 1, True or nothing for
the argument is 400 invalid.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
server_meaning gives each of its 67 values a bool field, true or false:
in as JSON, a query, a message's rows; over the link to B (taking it by
name among reordered fields) and over HTTP to C; out through A's view and
query, B's and C's views, a restart, a restart after a checkpoint and B
with its module replaced. A bool must come out as true or false, never 1
or 0. Refused on every way in, with the reason checked: 1, 0, "true",
null or an array as json_type, True as json (no JSON at all), a query's
1, 0, True, TRUE, "true " or nothing as query, a row's 1 as json_type:
52 refusals, 39 before. 015 says what a bool is at each boundary.
This check found the stored outbox refusing bool fields (a node could not
start again with a bool send waiting), fixed in the commit before.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A condition was a number, non-zero being true: a flag was a number field,
a view showed 1 for true, and nothing kept a number from standing for a
bool or a bool from being added to. Now the verifier has a bool among its
stack types, and a verified module never mixes the two. No opcode is new:
- CONST with argument 1 is a bool (+0 or 1), NUM_TEXT with argument 1
writes one as true or false; LOAD and STORE take a number or a bool
state, locals may hold a bool (2).
- Comparisons (LT to NE, TEXT_COMPARE) and LIST_HAS give a bool; AND, OR
and NOT take and give them; JZ, REQUIRE, filters and folds' tests take
one (count and first leave a bool, sum, max, min and distinct a number
and a bool); EQ and NE compare two bools too.
- Type codes: state 5, record field 2, message field 6 (one byte in the
prefix, 0 or 1, or the message fails as limit), view 3. A bool's
constant, initial value and default are +0 or 1; a list is not keyed by
one. PUREMOD6: v5 modules are built again.
At run time a bool is a number's 0 or 1, so leaf-at-a-time folds run as
before. A number that becomes a bool (a module replaced, a stored state
or item read again, a waiting send recast) keeps 0 (either sign) and 1
and takes the default otherwise; a bool that becomes a number is 0 or 1.
The host carries bools on every boundary: JSON true and false only (a
row's field too), a query's true and false, true and false in views,
table JSON and HTTP deliveries; a link takes a bool only into a bool.
A stored outbox's field types were read back with a check that refused
bool (at or past rows): a node whose output had a bool field and a send
waiting could not start again. test-module now stores such a send and
opens it (it fails on the old check), beside 28 bool refusals of the
verifier and bool states, fields, views, sends, links and migration.
The lane generator makes numbers and bools apart.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Fetching a found item's cells, a leaf's keys and each level's keys ahead
made finding and reading items 20% faster on 100,000 items in a micro
benchmark, but TPC-C's clean pairs moved 1% at most and finding alone got
48% slower, so it is not in. The owner's finding samples were the descent,
not the fields: a write right after a read copies the leaf whole anyway,
and what costs is walking structures larger than the caches at random.
|
| | |
|
| |
|
|
| |
(015, 108)
|
| | |
|
| |
|
|
| |
check (108)
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Most of it was the allocator: leaves past glibc's thread cache, freed by
whichever thread let the last snapshot go. jemalloc was 11-14% faster
at more memory; reusing the nodes in Pure gave a median +9.4% over six
pairs, 4-8% less CPU a transaction in all six, and 1.97x the state bytes
resident where it was 2.18x. TPC-C in RAM is now 4.0x a tuned SQLite at
W=1 and 3.5x at W=4; finding items is the owner's largest part now.
|
| |
|
|
|
|
|
|
| |
transaction
The 90x of 092 was against SQLite's defaults committing each
transaction; tuned for the card and grouped as Pure groups, SQLite is
4-11x behind, the bytes it writes a transaction being the difference.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
warehouses (106)
SQLite tuned first, by the criteria committed before (and, beyond them,
checkpoints each 40 MB of WAL, in its favour). On the SD card Pure stays
4.2-10.8x ahead at equal terminals, SQLite writing 8-18x the bytes a
transaction; in RAM 3.2-3.5x. All tables equal at W=1 and W=4, and
every run consistent. In RAM the one owner is the limit (0.94 of a core
at W=4, no gain from 16 to 64 terminals), a third of its busy time
copying nodes the snapshots share; checkpoints and memory are within
their criteria.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
The board is Vega's examples/shared-board.aup, and Vega's shared_board
check runs its two nodes: changes on either agree, and a partition
healed agrees on the later change. Reading a named field list once a
delivery would save part of 310 ns a record during upgrades alone: not
worth an API. What review 004 asked that is left is 103's third item,
broader performance comparisons under conditions stated first.
|
| |
|
|
|
|
|
|
|
| |
aup spreads an item or a message into a send or a record, and sets an
item whole (c69682e): the board sends { ..at } and applies cards[k] =
Card { ..remote }, and a new field is written in two places, the record
and the output. v6 on both nodes, cut apart and joined again, agrees on
the later change; its module is byte for byte the one with every field
written.
|
