| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A worker put its app back at the queue's tail after every turn, and
the queue woke another worker for it: an app with steady work hopped
between workers (and cores) each turn, and every turn woke a thread.
Two busy scoreboard apps on two workers wrote 84k a second where two
processes wrote 117k.
Now an app takes its turns one after another on the same worker while
it has work and no other app waits; it goes to the tail only when one
does, and then without waking another worker, since this one takes the
head. Due timers are looked at between its turns, so a timer of another
app still gets a turn. A worker wakes another only when it leaves apps
waiting behind the one it took. The two apps now write about as much as
the two processes (measurements in research/experiments/026).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve --apps FILE hosts an app a line: NAME MODULE STORE and its
budgets (memory=, work=, queue=). Each app's routes start with its name
and reach its own module, states, snapshot and queries; a view is shown
on the connection thread from its own app's snapshot, read once a round.
No app has a thread of its own. Its owner is what used to be a thread's
loop, now a turn: its due timers, what its wires delivered, and the
requests waiting for it applied and saved once, its answers handed back.
Workers (--workers N; one for a host of one app, else one a CPU at most)
take apps from a queue in turn; a worker sleeps until the soonest timer.
A turn takes every waiting request when no other app waits, else 256;
it gathers the clients its last save answered only when alone.
An app's queue budget refuses what would wait past it (503 busy) on the
connection thread, before its owner sees it. An app whose module does
not load, whose store does not open, or whose store is another app's
stops alone: the start line says why, and its routes answer 503
stopped. The stop line counts each app's turns, events and refusals.
A host of one app (--module) runs as before, its owner on one worker
named pure-owner: every check passes unchanged. server_apps runs six
apps of which two stop, floods one past its queue while another
answers, refuses a walk past one app's work budget that another runs,
holds an app's store from another process (that app answers 503
store_busy and applies nothing, the others save), and restarts.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An event's run was bounded only by its lists: a walk or a fold is its
block's length for each item, and a list holds up to 16 million. One
event could hold the owner for seconds, one view a connection thread.
pure_app_work(app, n) budgets one unit of work: an event, or one
evaluation of a view, a cell or a table's filter. It is counted as
written, as each part begins: a block's length, and a fold's or walk's
block length for every item of its list (or slice), whatever the plan
runs (lanes, a group or rank pass, a range). Nothing is counted inside
the interpreter's loop. Past the budget an event fails with the new
fault work and changes nothing (PURE_LIMIT, so 422 work over HTTP); a
view, a cell or a table's rows have no value and pure_app_view_fault
says work, which pure-serve answers 422 work, for a whole table when
a cell of it ran past. pure-serve --work N sets it (0: none).
test-module counts exact amounts on the shop's walk, the tasks' fold
view and a filtered table, and lane_folds now checks that a planned
fold counts what the same fold run item by item counts (3,600 folds,
every view, cell, filter and handler). server_work grows a list over
HTTP until the walk, the table and the fold each pass a budget of 400.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Threads of one process took turns as the tree store's writer for every
store at once, through one mutex held for a whole session, syncs
included, and commits grouped in one queue whose leader published each
store's requests in turn. One store's slow sync held every other
store's commit, so apps of one host would wait on each other's device.
The turns and the group are now a store's, found by its directory's
identity (two paths to one directory still share them, since the file
lock arbitrates only between processes). The temporary files' sequence,
which that mutex guarded, is atomic. A test holds a commit in its sync
and commits to another store meanwhile, while a second commit to the
held store, by another path, waits; before this change it hangs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve runs one app a process: a module, a store, its own threads,
port and key, scheduled by Linux. Within one process nothing bounds an
event's run (it grows with its lists), the requests waiting for the
owner, or one store's sync, which holds every other store's commit
since the tree store has one writer's turn per process.
017 makes an app an execution domain (007): a host of many apps
(--apps FILE, routes /APP/...) runs them by turns on a fixed number of
worker threads, none per app; budgets per app in deterministic units
(memory in stored bytes as before, work in instructions counted per
block and per item visited, queue in requests: 422 work, 503 busy);
writer turns per store; and an app that cannot start or whose save
became uncertain stops alone. Stage 1 is all that; links, sends between
apps and an app's lifetime come after.
Before building: it goes in if an app B's p99 under app A's flood is
within 1.5 times what two processes give, an app costs under a quarter
of an idle process's own memory with no thread of its own, the pair's
throughput is within 10% of two processes', and a one-app host keeps
its speed (3% in time, 1% in instructions).
|
| |
|
|
|
|
|
|
|
|
| |
The six pairs rerun once the Pi had been quiet for a minute, on the same
binaries. Two pairs were still interfered (the first's before at an
owner share of 0.54, the sixth at 0.73 on both sides); the four clean
ones are -1.7%, -0.4%, +1.8% and +1.1%, CPU per transaction 43.25 to
42.9 us. The bank's queries are as fast or faster after, though the
interference fell more on the before runs; the instruction counts stay
the comparison to trust.
|
| |
|
|
|
|
|
|
|
|
|
| |
016 is built: bool and i64. The second stage's findings: an integer
literal takes the other side's type, found by compiling it on trial
(linear once the wanted type is passed down); a literal with no context
stays a number, so no program changes meaning and modules without i64
are the same bytes; one rule carries values between numbers, bools and
i64s; and the number paths passed the timing criterion, with what i64
had added to them counted and halved (110). The checkpoint lists bool
and i64 in the app model and drops i64 from what is left.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
016's second stage passed its criterion: six interleaved pairs against
ee21aec gave TPC-C (RAM, W=1) -0.6% at the median and the bank's
queries 0.0% to +2.2%, no run interfered. The ranks query was slower
all six times, and chasing it showed that on this Pi timing cannot
separate 1-3%: two servers asked in turns disagree with themselves
(ee21aec to e02bfc9 +1.6%, e02bfc9 to 481642f -1.2%, ee21aec to 481642f
+2.8%), and a query whose instructions grew 0.6% ran 3.5% faster.
Instructions do not move so: pcount.c counts the server's user-mode
instructions with perf_event_open (no perf here), and psample.c samples
where they go. ranks had grown 0.4% in instructions (the rest was where
code and data landed), per_branch 3.6% and tables 20 instructions a row,
which 07047c3 halved. The six pairs of 07047c3 were interfered (another
session's build, load 4-12) and are kept apart, unused.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Counting user-mode instructions per query (bank, 100,000 accounts)
against ee21aec found what i64 had added to the number paths, and an
instruction sampler found where:
- fold_in took a union and asked whether the fold was of i64s for every
item, in the group and leaf-at-a-time passes too, where a fold of
i64s never runs (it is never planned). Those passes now call
fold_number, which takes a double; fold_in keeps the i64 branch for
folds run item by item. per_branch: 27.83M instructions an ask back
to 27.05M (26.72M before i64).
- cell_at's third kind made it too big to inline into pure_list_get,
so every item read became a call. A number's, bool's or integer's
cell is 8 bytes read as they are (memcpy, so no aliasing rule is
bent), one branch as before, and it inlines again. A table of 10,000
rows: 8.78M instructions an ask to 8.70M (8.58M before i64).
Behaviour is unchanged; every check passes.
|
| |
|
|
|
|
|
|
|
| |
server_meaning gives every value an i64 field, 13 values from 0 and -0
to both ends of the range and just inside them, in over JSON, rows and
a query, over the link (where B takes id, a u32, as an i64) and HTTP,
and out at all seven exits with every digit. 32 more refusals, each
with its reason on JSON, a query and rows; an i64 view's text at both
ends and an i64 argument that counts exactly and refuses the rest.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An i64 is an int64_t in a value's 8 bytes, its own stack type: CONST and
NUM_TEXT take argument 2 for one, eleven opcodes are appended (ADD, SUB
and MUL fail as overflow past its range, DIV and MOD floor and fail as
divide by zero, NEG, COMPARE with a relation, MIN, MAX, and I64_FROM and
NUM_FROM, which convert only where the value fits and else fail as
limit), and moves (LOAD, STORE, locals, items, FIELD, SEND) copy it as
they copy a number. Type codes: state 6, record field 3, message field 7
(8 bytes), view 4, local 3. v6 stays: all of it is added.
- Folds with PURE_FOLD_INTEGER sum (overflow fails), keep the most or
least of, or count the different i64s; distinct's set keeps an i64's
-1 apart from its free mark. Blocks with an i64 run item by item.
- A list's kinds are its stored header's (number, text, bytes, bool,
integer): integer columns are int64_t, ordered as integers as keys
(found without guide keys for now), stored in 8 bytes and logged as
zigzag varints of any size. Records name their fields' type in one
array (record_field_type) in place of text and bool flags.
- A number, a bool and an i64 carry over each other by one rule,
pure_cell_as, wherever one becomes another: a module replaced, a stored
value or item read by another module, a waiting send recast. A u32
field delivers to an i64 one.
- The host reads i64 fields from JSON, rows and queries as integer digits
in range and nothing else, and shows every digit in views, tables,
deliveries and pure-run. Faults are named by one function,
pure_fault_name, in place of three tables; overflow is new.
test-module: 28 arithmetic cases at the range's ends, relations,
conversions both ways, 14 refusals, a keyed i64 list app (order, a sum
that overflows, a failed add undone, messages and sends of 8 bytes, every
digit as text), recasts, replacement between numbers, bools and i64s, and
three stored runs against memory with restarts (46 images written).
|
| |
|
|
|
|
|
|
|
| |
Before i64 (016 stage 2): a state's value, a record field's default, a
list cell's number, the interpreter's stack slot and an instruction's
constant become unions of a double and an int64_t (PureValue, anonymous
unions in PureCell, Slot and PureOp). LOAD and STORE, and a list cell
made from the stack, copy the bits; everything else reads .num as before.
No behaviour changes: every check passes as it did.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Before building i64: a value slot holds a number or an i64 in the same 8
bytes (the stack, states, list cells, locals, constants), moves copy the
bits and only computing opcodes look at the type; eleven opcodes are
appended (arithmetic that fails as overflow, floor division and modulo,
compare, min, max, the two conversions), CONST and NUM_TEXT take
argument 2, and type codes are added, so v6 stays. i64-keyed lists compare
as integers without guide keys for now; folds over i64 run item by item.
JSON and queries take plain integer digits in range; views, tables and
deliveries show every digit. u32 widens to i64; i64, numbers and bools do
not deliver to each other; replacing converts when the value fits.
It is in when every check is green and the number paths keep their
speed: bank queries and TPC-C (RAM) no more than 3% slower at the median
of six interleaved pairs.
|
| |
|
|
|
|
|
| |
016's first stage is built and checked: no opcode was new, a stored
send's bool was refused on the way back in (found by server_meaning's
restart, fixed), -0 becomes false as 015 says zero has no sign, and
True in JSON is no JSON at all. i64 is what is left of 016.
|
| |
|
|
|
|
|
| |
server_meaning's origin gains a bool view and a view with a bool
argument: GET /views/any_ok answers true as text, with_ok?want=true and
?want=false count the kept values each way, and 1, True or nothing for
the argument is 400 invalid.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
server_meaning gives each of its 67 values a bool field, true or false:
in as JSON, a query, a message's rows; over the link to B (taking it by
name among reordered fields) and over HTTP to C; out through A's view and
query, B's and C's views, a restart, a restart after a checkpoint and B
with its module replaced. A bool must come out as true or false, never 1
or 0. Refused on every way in, with the reason checked: 1, 0, "true",
null or an array as json_type, True as json (no JSON at all), a query's
1, 0, True, TRUE, "true " or nothing as query, a row's 1 as json_type:
52 refusals, 39 before. 015 says what a bool is at each boundary.
This check found the stored outbox refusing bool fields (a node could not
start again with a bool send waiting), fixed in the commit before.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A condition was a number, non-zero being true: a flag was a number field,
a view showed 1 for true, and nothing kept a number from standing for a
bool or a bool from being added to. Now the verifier has a bool among its
stack types, and a verified module never mixes the two. No opcode is new:
- CONST with argument 1 is a bool (+0 or 1), NUM_TEXT with argument 1
writes one as true or false; LOAD and STORE take a number or a bool
state, locals may hold a bool (2).
- Comparisons (LT to NE, TEXT_COMPARE) and LIST_HAS give a bool; AND, OR
and NOT take and give them; JZ, REQUIRE, filters and folds' tests take
one (count and first leave a bool, sum, max, min and distinct a number
and a bool); EQ and NE compare two bools too.
- Type codes: state 5, record field 2, message field 6 (one byte in the
prefix, 0 or 1, or the message fails as limit), view 3. A bool's
constant, initial value and default are +0 or 1; a list is not keyed by
one. PUREMOD6: v5 modules are built again.
At run time a bool is a number's 0 or 1, so leaf-at-a-time folds run as
before. A number that becomes a bool (a module replaced, a stored state
or item read again, a waiting send recast) keeps 0 (either sign) and 1
and takes the default otherwise; a bool that becomes a number is 0 or 1.
The host carries bools on every boundary: JSON true and false only (a
row's field too), a query's true and false, true and false in views,
table JSON and HTTP deliveries; a link takes a bool only into a bool.
A stored outbox's field types were read back with a check that refused
bool (at or past rows): a node whose output had a bool field and a send
waiting could not start again. test-module now stores such a send and
opens it (it fails on the old check), beside 28 bool refusals of the
verifier and bool states, fields, views, sends, links and migration.
The lane generator makes numbers and bools apart.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Fetching a found item's cells, a leaf's keys and each level's keys ahead
made finding and reading items 20% faster on 100,000 items in a micro
benchmark, but TPC-C's clean pairs moved 1% at most and finding alone got
48% slower, so it is not in. The owner's finding samples were the descent,
not the fields: a write right after a read copies the leaf whole anyway,
and what costs is walking structures larger than the caches at random.
|
| | |
|
| |
|
|
| |
(015, 108)
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Values at the edges of what JSON, rows, a query and a plain body can say
go in to one node, over a link to a second that takes them by name in
another order and wider, and over HTTP to a third; each must come out
the same from views, a query, an answer, a restart, a restart after a
checkpoint and a replaced module: a number's bits (zero without its
sign), a text's bytes. What must be refused is refused on every way in
and leaves nothing behind.
|
| |
|
|
|
|
|
| |
A view's arguments were decoded and an input's fixed fields were not,
so a client that sends an exponent's + as %2B, as Python's urlencode and
browsers do, had ?n=1e%2B21 refused. show_given's comment is back above
show_given.
|
| |
|
|
| |
check (108)
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Most of it was the allocator: leaves past glibc's thread cache, freed by
whichever thread let the last snapshot go. jemalloc was 11-14% faster
at more memory; reusing the nodes in Pure gave a median +9.4% over six
pairs, 4-8% less CPU a transaction in all six, and 1.97x the state bytes
resident where it was 2.18x. TPC-C in RAM is now 4.0x a tuned SQLite at
W=1 and 3.5x at W=4; finding items is the owner's largest part now.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Each node a list copies once a snapshot shares it went, once the last
snapshot that could read it was released, back to the system: a leaf of
TPC-C's (1.3 KB) past glibc's thread cache, often freed by a connection
thread into the owner's arena, while the owner allocated the next. That
was 70% of what snapshots cost the owner. Now what no snapshot reads
comes back to the app under the register's lock, and each snapshot the
owner makes hands it to the lists of its size as spares, as many as each
made lately; retired entries carry what the pass and the reuse ask of a
node, so neither reads it again. TPC-C in RAM, W=1, 64 terminals: 10-14%
more a second at 7% less CPU a transaction, resident 1.97x the state
bytes where it was 2.18x (docs/research/107).
|
| |
|
|
|
|
|
|
| |
transaction
The 90x of 092 was against SQLite's defaults committing each
transaction; tuned for the card and grouped as Pure groups, SQLite is
4-11x behind, the bytes it writes a transaction being the difference.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
warehouses (106)
SQLite tuned first, by the criteria committed before (and, beyond them,
checkpoints each 40 MB of WAL, in its favour). On the SD card Pure stays
4.2-10.8x ahead at equal terminals, SQLite writing 8-18x the bytes a
transaction; in RAM 3.2-3.5x. All tables equal at W=1 and W=4, and
every run consistent. In RAM the one owner is the limit (0.94 of a core
at W=4, no gain from 16 to 64 terminals), a third of its busy time
copying nodes the snapshots share; checkpoints and memory are within
their criteria.
|
| |
|
|
|
|
|
| |
With four warehouses the order lines answered at once (about 140 MB of
JSON) outgrew pure-serve's 64 MiB budget, and the check stopped with
503; a larger budget would have left the answer's memory in the
resident size measured after.
|
| |
|
|
|
|
| |
Its default, each 1,000 pages, is 1 MB of WAL with the 1 KiB pages the
SD card favours: grouped commits there went from 203 to 694 a second,
and in RAM a thread's p99 from 1.2 to 0.2 ms (docs/research/106).
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
run costs
The terminals hand their writing transactions to one writer, which runs
those waiting in one transaction, each in a savepoint, and commits them
with one sync; reads stay on the terminals' connections, as pure-serve
shows views from the last save. An undone New-Order is rolled back by
its caller now (it rolled back the whole group before, and counted as
failed after the split). Each run reports its CPU a transaction, the
owner thread's share of a core, its memory, and the bytes a transaction
had written, by the process and to the device. SQLite runs with its
database mapped, a 64 MiB cache and savepoint journals in memory unless
told otherwise.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
The board is Vega's examples/shared-board.aup, and Vega's shared_board
check runs its two nodes: changes on either agree, and a partition
healed agrees on the later change. Reading a named field list once a
delivery would save part of 310 ns a record during upgrades alone: not
worth an API. What review 004 asked that is left is 103's third item,
broader performance comparisons under conditions stated first.
|
| |
|
|
|
|
|
|
|
| |
aup spreads an item or a message into a send or a record, and sets an
item whole (c69682e): the board sends { ..at } and applies cards[k] =
Card { ..remote }, and a new field is written in two places, the record
and the output. v6 on both nodes, cut apart and joined again, agrees on
the later change; its module is byte for byte the one with every field
written.
|
| |
|
|
|
|
|
| |
A board record of five fields (38 B) sent with v3's output fields took
310 ns to lay out for v4's input, reading the field list each time.
Only streams of another shape pay it, and only while nodes are upgraded
one after the other: less than a delivery's own work.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
103's next step: grow a real app, adding and reordering fields, cutting
the connection, starting again, and write down what gets in the way.
Two nodes share a board (from Vega's examples/board.aup), each sending
its changes to the other, the later change winning.
Upgrading them one after the other deadlocked: fixed by recasting
waiting sends by name (9aa123e) and taking fields by name (014,
5ce7655), and under it a link that stopped for good when a refused
stream's try was lost (baaf955). Now the older node's changes reach the
newer one during the upgrade, the newer one's new field waits for the
older, and the boards agree a second after. Fields in another order
agree at once; a partition healed agrees on the later change; both
nodes started again go on. Two rough edges eased on the way (the node's
name, message like in aup); sends and applies are still written field
by field. CHECKPOINT's row for links says so.
|
| |
|
|
|
|
|
|
|
| |
A node's sends are named NAME.OUTPUT, and a receiver's --sender must say
that name. Without --name it is the module file's name without .pmod,
which nothing said: two board nodes built from board2.aup sent as
board2.changed and were refused by receivers expecting a.changed. The
first line out now carries the name beside the port and the key, and
the usage says where it comes from.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A message from another node was taken only in its input's exact shape
(the link's STREAM shape, fields and order alike; over HTTP, every
field). So a board whose nodes are upgraded one after the other stopped
both ways until both were, and an output and its input whose fields
differ only in order were refused for good (docs/design/014).
An input now takes a message when it has every field the sender sends,
by name, of the same type or one that holds its values (a u32 in a
number, text in bytes); a field only the input has is 0 or empty. One
that sends a field the input lacks is refused: taking it would drop the
value, and with its number heard the value would never come again (two
boards apart for good). So add fields to receivers first and take them
from senders first, and nothing stops or is lost.
The link carries a sender's fields in a new frame, FIELDS (0x0d: names,
shape, u8 count, each {type, name length, name}): after a REFUSED for
another shape the sender puts it in its packets (after OFFER, before
the records, so a receiver that knows no FIELDS reads what it read
before) until the stream is taken, the first time at once. A receiver
seeing another shape waits for the packet's FIELDS before refusing, and
asks its host with them (PureLinkHost.input takes them); REFUSED says 4
when the input lacks a field. Deliveries carry the fields and the host
lays each record out anew (pure_module_takes, pure_module_recast,
pure_module_fields, over pure_send_recast's rule). A stream whose sender
starts again in another shape lets go what it held; the sender sends it
again laid out anew. A refusal already queued for a stream now says the
last reason judged, not the first. Over HTTP a delivery (X-Pure-Source)
may lack fields; an unknown member is still refused.
test-link named: a stream of another shape naming its fields is taken,
200 stored in 47 ms with every delivery naming them; an input that does
not take them refuses (4) and the sender backs off. check.py
server_fields: five outputs, three linked and two over HTTP, fields in
another order taken, an input with a field more taking messages without
it, one lacking a field refusing until it starts again with it, then
taking all. server_upgrade's new receiver declares its fields in another
order. 012 names the frame and the rules.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Under AddressSanitizer with leaks counted, the module checks lost 672
KB in 10,185 allocations. Two causes, both from the snapshot rework
(d1e8f38):
- A snapshot's register counted its snapshots with ++ under its spin,
but a snapshot's last release took one off atomically outside it. A
release on a reader's thread racing the owner's next snapshot could
lose either change: the register then outlived every snapshot (the
leak seen), or went while one still used it. The count is atomic now.
- pure_app_stop handed what its lists let go to the newest snapshot (or
freed it) and only then freed the outbox and the numbers heard, whose
shared nodes and texts went to that hand-over's list again, for good.
They are freed with the other lists now.
make check builds the module checks a second time under AddressSanitizer
and UndefinedBehaviorSanitizer (build/test-module-sanitized, 25 s) and
fails on a leak. Before d1e8f38 the same build stops at a
heap-use-after-free in retire; now it is clean.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 004's F3 made a module refuse to start while sends waited on an
output whose fields it had changed: their bytes would have meant
something else. Safe, but it deadlocks a rolling upgrade. Growing a
board shared by two nodes: one upgraded to a card with a new field
refuses the other's old sends (another shape), and the other, its old
sends waiting, cannot start the new module at all ("Cannot start the
module on the store", saying nothing more).
Waiting sends now move to the new fields by name, as a list's items do:
a value goes over when the new type can hold it (u32 and number are
numbers, bytes and text are bytes; a number that is not a u32, or bytes
that are not UTF-8 for a text, cannot), the rest start at 0 or empty
(pure_send_recast, with the payload layout load uses, pure_fields_lay_out).
A start recasts them and a replace does into a new outbox.
Their meaning must survive a crash between the recast and the save that
names the new fields in the links. So the outbox's image now names the
fields its sends were laid out with ("PLO1": the module's outputs after
the save it holds), and is written again before any send is logged with
other fields; a start reads those names over the links'. Checkpoints of
the outbox name them too.
A module without an output that sends wait on is still refused, now
saying so in PureApp.why, which pure-serve prints ("sends wait on the
output 'ordered', which this module does not have"); a failed start
keeps it.
test-module send_recast (widened, narrowed, UTF-8, a payload not laid
out, there and back) and output_meaning (the swapped fields recast on a
replace, read right after a restart that never saved, back again, sends
logged after the image, the refusal and its reason). check.py
server_upgrade: five sends wait on the old output, the sender starts
with the new one, the old receiver refuses the new shape, the new one
takes all eight.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A sender whose stream the receiver refuses sends one packet of records
a wait, and the refusal is over when that packet is acknowledged with
no REFUSED. When the try itself went to no one (the receiver started
again, as a node upgraded one after the other does, or the path lost
it) nothing acknowledged it: the sender took its epoch for lost, began
another, and then sent nothing more, still waiting on the try, with no
record in flight to probe for. Two nodes of a shared board stayed apart
for good after the second one's upgrade.
A try declared lost, or dropped with the epoch's flights, now lets the
next packet try again. test-link's refused_restarted case: the receiver
refuses a stream, is killed while the next try is due and comes back 8
s later with the input; before, nothing was stored in 120 s, now all
200 are by 18.2 s. Streams refused for good send as few packets as
before (9 in 90 s).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
013's second step is in (d9e0b68): each transport holds a window of its
output's sends. 104 measures it with the receivers down: once the window
is full a node grows by the outbox alone, 219 and 218 B a send over the
link and HTTP (399 and 534 B before).
The third step is a measurement. A checkpoint holds one list's snapshot
while it writes the image, so what it adds is what the app changed
meanwhile: nothing visible on tmpfs or for a list that fits the card's
fast writes; on the SD card, a 22.4 MB list written for about 8 s while
1,400 items a second were set left +3.0 and +4.4 MiB. The highest
resident memory was twice the state bytes, inside 013's bound (the state
and one snapshot) and its stop criterion (3x).
021's run.py takes --items, --stores (a directory on the device to
measure) and --label, and reports the peak (VmHWM).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A link's ring and a wire's queue took every saved send of their output
and kept it until delivered: with the receiver down they held the
outbox a second time, 178 B and 313 B more a send (104). Each outlet now
holds a window, --window bytes (4 MiB) and 65,536 sends past what its
receiver has, at least one; the outbox keeps the rest.
The owner remembers each outlet's place in the outbox and looks on from
there after a save and after its transports deliver. Places move back by
the sends that left the outbox since (PureApp.sends_gone), so a trim
before a place skips nothing, and sends queued before are not looked at
again however many wait. pure_app_sent reads a send by its place (the
newest-first visit it replaces is gone); pure_link_held and
pure_wire_held say what a transport holds.
With the receiver down a node now grows by the outbox alone once the
window is full: 219 and 218 B a send over the link and HTTP (399 and
534 B before). check.py server_windows keeps the receivers of a sender's
two outputs, one linked and one over HTTP, down through 300 buys with a
2,000-byte window, then starts them: each applies all 300 once. Without
the refill after deliveries the link stops at 117.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
104 measures a node's memory under a receiver that is down and under a
snapshot held for long (021's run.py: resident memory against what was
posted, 30 s a case). A held snapshot grew the node without end (6.4 to
678 MiB, fixed in d1e8f38); a slow receiver grows it by what was sent,
221 B a send, which is state until delivered, so the budget bounds state
and refuses past it.
013 is that budget's design: what it counts (state bytes, deterministic),
what an event over it does, why an answer takes its send's room, how
links and HTTP wait, what it does not count and those bounds, and the
checks. With --memory 4000000 the same load fills it in about 4 s and
the node stays at 9.6-9.8 MiB (45.9 MiB and growing without).
012 records the hole the budget found in the link (the full flag), and
103 that its clock fix fell short (385754c). CHECKPOINT gains the row.
|
