| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
and pushed (020)
- pure-serve --view-workers N (a CPU each; 0: on the connection threads):
a GET of a view with a body (or a table with such a column) goes to the
workers, which show it from the app's newest snapshot on readers of
their own and give the answer back as an owner does (eventfd). The
connection thread reads and writes its other connections meanwhile.
- A snapshot of another module than the request was resolved with, or
none (the app stopped), answers 503 stopped. Past an app's queue=N of
them, 503 busy.
- A push group of such a view hands one evaluation at a time; a newer
snapshot meanwhile is evaluated after it; a group whose subscribers all
left stays until its evaluation is back.
- check.py server_view_workers: 8 askers of a long view while its app
changes module (another view in its place) and then goes: every answer
the old module's, the new one's, 503 or 404; streams closed at once
leave nothing; --view-workers 0 as before.
|
| |
|
|
| |
show a subscriber costs writes nothing (019)
|
| |
|
|
| |
(019)
|
| | |
|
| |
|
|
| |
bounded (018 stage 2)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An app given a tokens file (tokens=FILE in --apps, --tokens FILE for the
one app of a host) answers a request only when it bears a token
(Authorization: Bearer) whose SHA-256 the file lists with a grant: read
for its views, queries and documents, write for all, input=NAME for
those inputs alone; 401 unauthorized (WWW-Authenticate: Bearer) without
one, 403 forbidden past its grant. A delivery sealed by a key --sender
names needs none. The digests are compared in constant time; a
connection keeps its last token's digest, so the same token is hashed
once. SHA-256 is the host's own (serve/sha256.c), no OpenSSL loaded. A
reading puts a changed tokens file in place with the app running on; a
file that is not one stops the app. pure-serve --make-token GRANT makes
a token and its line. load takes PURE_LOAD_TOKEN.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Stopping an app and starting it again reads its whole store again: a
million items took 142 ms, as long as restarting the host (030). A
running app whose line changed but not its store now goes quiet (what it
took applied, saved and answered, its batches back, its outlets closed,
its checkpoint finished), has its module replaced in memory
(pure_app_replace carries its states, and the sends in its outbox, by
name), its budgets set and its outlets made anew, and resumes; a module
its states do not go into stops it. An app whose store changed, one a
save of which is uncertain, and one that does not run still start from
their stores. A reading says changed for the first. A snapshot is
published again when the module changed though the states did not.
server_apps_lifecycle reads the changing apps' views (lists among them)
as they change, and moves an app's store; it passed 10 times alone, 5
under load and 6 with an ASan pure-serve.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A host of apps reads its --apps file again on SIGHUP, on a thread of its
own (pure-apps, the one that takes the signal): a file with a wrong line
changes nothing; the apps of lines gone stop and their routes go (404
no_app); those of changed lines, and those not running (they could not
start, or a save of theirs is uncertain), start again from their stores,
states carried by name; new ones take places of their own; a line out
says what was done. An app stops by taking nothing more (503 stopped),
applying, saving and answering what it took, waiting for its batches at
other apps, then closing its outlets (what they did not deliver stays in
the outbox) and its store. Its link streams find their inputs again as it
starts. Owners' locks are made with their places; counts are kept over
an app's runs; workers grow with the apps that run, a CPU each at most.
server_apps_lifecycle adds, removes and changes apps, and changes the
receivers (a local outlet's and a link's, their inputs at other numbers,
taken by name) and the senders while 600 sends each and writes go: every
send arrives once and in order, a write answered 200 is applied once and
one answered 503 stopped not at all, the app left alone fails no request;
an app whose store cannot be made starts at the next reading once it
can; kill -9 and the host starts as the file says. It passed 10 times
alone, 5 under load, 6 with an ASan pure-serve.
|
| |
|
|
|
|
|
|
|
| |
server_timers compares the times an app saw with the wall clock read
around them. WSL2's clock is set back about 2 s every 32 s (research
029): a window holding such a step says nothing of the times seen, and
failed the check there 4 times in 20. It is taken again (up to three
times) when the wall clock and the monotonic one moved more than 50 ms
apart over it; the report counts the windows taken again.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
send=OUTPUT=APP/INPUT in --apps wires an output to another app of the
same host through a local outlet: no socket, seal or packet. The
sender's turn queues its saved sends there, under the window links
and wires have, and hands them, one batch at a time, to the receiving
app's queue laid out as a link delivers records (first number, count,
source APP.OUTPUT, the records as stored, and the sender's fields when
the input takes them by name, which pure_module_takes checks at start).
The receiver applies them after its last number heard; its save over,
it tells the outlet what it heard and the sender's next turn trims its
outbox so far. What it did not store is queued again from the outbox,
after 100 ms when it had no room. An output now goes one way only: two
outlets would trim each other's sends.
server_apps_links: an app sends each of 400 messages to an app of
another host over a link and to three apps of its own host (one of
the same shape, one taking the fields by name, one with a memory
budget that fills), with a kill -9 of the sending host between two
requests; all arrive once and in order, and the full app's sends wait
in the sender's outbox until it lets go of what it holds.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An app of --apps takes send=OUTPUT=URL, seal=OUTPUT=KEY and
reply=OUTPUT=INPUT, as --send, --seal and --reply give the one app of a
host; its sends' source is APP.OUTPUT. An app wired wrong stops alone,
its wires closed. The host's --key, --sender, --senders, --sealed-only
and --link-port now go with --apps: the one listener resolves a
stream's input name APP/INPUT to that app's input (a stopped app or an
unknown name is no input, REFUSED 1), its deliveries queued at that
app's owner. 012 is unchanged.
server_apps_links: an app of one host sends 300 messages, sealed, over
a link to an app of another host, which takes them all once and in
order.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With several connection threads, a host of apps had each app's
connections spread over all of them: every thread's round handed each
owner a share, and every turn handed answers back to every thread. The
apps took a third more CPU a request than as processes, with five times
the voluntary switches, and when two apps were busy each waited a
kernel tick at its 99th percentile (4.1 ms, against 1.5 ms as two
processes).
Each app now has a home thread (its place in --apps, modulo the
threads). A connection whose first request names an app at home
elsewhere moves there once, its head read and no ticket taken yet: the
home thread adopts its descriptor and the bytes read, and reads the
request again. Each app's requests then meet its owner from one
thread, as from a process of its own. Two busy apps: 99th percentile
1.4 ms, the same 145k writes a second.
server_apps restarts its apps on three connection threads, asks each
app from fresh connections, and checks that some moved and all were
answered right.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve --apps FILE hosts an app a line: NAME MODULE STORE and its
budgets (memory=, work=, queue=). Each app's routes start with its name
and reach its own module, states, snapshot and queries; a view is shown
on the connection thread from its own app's snapshot, read once a round.
No app has a thread of its own. Its owner is what used to be a thread's
loop, now a turn: its due timers, what its wires delivered, and the
requests waiting for it applied and saved once, its answers handed back.
Workers (--workers N; one for a host of one app, else one a CPU at most)
take apps from a queue in turn; a worker sleeps until the soonest timer.
A turn takes every waiting request when no other app waits, else 256;
it gathers the clients its last save answered only when alone.
An app's queue budget refuses what would wait past it (503 busy) on the
connection thread, before its owner sees it. An app whose module does
not load, whose store does not open, or whose store is another app's
stops alone: the start line says why, and its routes answer 503
stopped. The stop line counts each app's turns, events and refusals.
A host of one app (--module) runs as before, its owner on one worker
named pure-owner: every check passes unchanged. server_apps runs six
apps of which two stop, floods one past its queue while another
answers, refuses a walk past one app's work budget that another runs,
holds an app's store from another process (that app answers 503
store_busy and applies nothing, the others save), and restarts.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An event's run was bounded only by its lists: a walk or a fold is its
block's length for each item, and a list holds up to 16 million. One
event could hold the owner for seconds, one view a connection thread.
pure_app_work(app, n) budgets one unit of work: an event, or one
evaluation of a view, a cell or a table's filter. It is counted as
written, as each part begins: a block's length, and a fold's or walk's
block length for every item of its list (or slice), whatever the plan
runs (lanes, a group or rank pass, a range). Nothing is counted inside
the interpreter's loop. Past the budget an event fails with the new
fault work and changes nothing (PURE_LIMIT, so 422 work over HTTP); a
view, a cell or a table's rows have no value and pure_app_view_fault
says work, which pure-serve answers 422 work, for a whole table when
a cell of it ran past. pure-serve --work N sets it (0: none).
test-module counts exact amounts on the shop's walk, the tasks' fold
view and a filtered table, and lane_folds now checks that a planned
fold counts what the same fold run item by item counts (3,600 folds,
every view, cell, filter and handler). server_work grows a list over
HTTP until the walk, the table and the fold each pass a budget of 400.
|
| |
|
|
|
|
|
|
|
| |
server_meaning gives every value an i64 field, 13 values from 0 and -0
to both ends of the range and just inside them, in over JSON, rows and
a query, over the link (where B takes id, a u32, as an i64) and HTTP,
and out at all seven exits with every digit. 32 more refusals, each
with its reason on JSON, a query and rows; an i64 view's text at both
ends and an i64 argument that counts exactly and refuses the rest.
|
| |
|
|
|
|
|
|
|
| |
Before i64 (016 stage 2): a state's value, a record field's default, a
list cell's number, the interpreter's stack slot and an instruction's
constant become unions of a double and an int64_t (PureValue, anonymous
unions in PureCell, Slot and PureOp). LOAD and STORE, and a list cell
made from the stack, copy the bits; everything else reads .num as before.
No behaviour changes: every check passes as it did.
|
| |
|
|
|
|
|
| |
server_meaning's origin gains a bool view and a view with a bool
argument: GET /views/any_ok answers true as text, with_ok?want=true and
?want=false count the kept values each way, and 1, True or nothing for
the argument is 400 invalid.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
server_meaning gives each of its 67 values a bool field, true or false:
in as JSON, a query, a message's rows; over the link to B (taking it by
name among reordered fields) and over HTTP to C; out through A's view and
query, B's and C's views, a restart, a restart after a checkpoint and B
with its module replaced. A bool must come out as true or false, never 1
or 0. Refused on every way in, with the reason checked: 1, 0, "true",
null or an array as json_type, True as json (no JSON at all), a query's
1, 0, True, TRUE, "true " or nothing as query, a row's 1 as json_type:
52 refusals, 39 before. 015 says what a bool is at each boundary.
This check found the stored outbox refusing bool fields (a node could not
start again with a bool send waiting), fixed in the commit before.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A condition was a number, non-zero being true: a flag was a number field,
a view showed 1 for true, and nothing kept a number from standing for a
bool or a bool from being added to. Now the verifier has a bool among its
stack types, and a verified module never mixes the two. No opcode is new:
- CONST with argument 1 is a bool (+0 or 1), NUM_TEXT with argument 1
writes one as true or false; LOAD and STORE take a number or a bool
state, locals may hold a bool (2).
- Comparisons (LT to NE, TEXT_COMPARE) and LIST_HAS give a bool; AND, OR
and NOT take and give them; JZ, REQUIRE, filters and folds' tests take
one (count and first leave a bool, sum, max, min and distinct a number
and a bool); EQ and NE compare two bools too.
- Type codes: state 5, record field 2, message field 6 (one byte in the
prefix, 0 or 1, or the message fails as limit), view 3. A bool's
constant, initial value and default are +0 or 1; a list is not keyed by
one. PUREMOD6: v5 modules are built again.
At run time a bool is a number's 0 or 1, so leaf-at-a-time folds run as
before. A number that becomes a bool (a module replaced, a stored state
or item read again, a waiting send recast) keeps 0 (either sign) and 1
and takes the default otherwise; a bool that becomes a number is 0 or 1.
The host carries bools on every boundary: JSON true and false only (a
row's field too), a query's true and false, true and false in views,
table JSON and HTTP deliveries; a link takes a bool only into a bool.
A stored outbox's field types were read back with a check that refused
bool (at or past rows): a node whose output had a bool field and a send
waiting could not start again. test-module now stores such a send and
opens it (it fails on the old check), beside 28 bool refusals of the
verifier and bool states, fields, views, sends, links and migration.
The lane generator makes numbers and bools apart.
|
| |
|
|
|
|
|
|
|
|
| |
Values at the edges of what JSON, rows, a query and a plain body can say
go in to one node, over a link to a second that takes them by name in
another order and wider, and over HTTP to a third; each must come out
the same from views, a query, an answer, a restart, a restart after a
checkpoint and a replaced module: a number's bits (zero without its
sign), a text's bytes. What must be refused is refused on every way in
and leaves nothing behind.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A message from another node was taken only in its input's exact shape
(the link's STREAM shape, fields and order alike; over HTTP, every
field). So a board whose nodes are upgraded one after the other stopped
both ways until both were, and an output and its input whose fields
differ only in order were refused for good (docs/design/014).
An input now takes a message when it has every field the sender sends,
by name, of the same type or one that holds its values (a u32 in a
number, text in bytes); a field only the input has is 0 or empty. One
that sends a field the input lacks is refused: taking it would drop the
value, and with its number heard the value would never come again (two
boards apart for good). So add fields to receivers first and take them
from senders first, and nothing stops or is lost.
The link carries a sender's fields in a new frame, FIELDS (0x0d: names,
shape, u8 count, each {type, name length, name}): after a REFUSED for
another shape the sender puts it in its packets (after OFFER, before
the records, so a receiver that knows no FIELDS reads what it read
before) until the stream is taken, the first time at once. A receiver
seeing another shape waits for the packet's FIELDS before refusing, and
asks its host with them (PureLinkHost.input takes them); REFUSED says 4
when the input lacks a field. Deliveries carry the fields and the host
lays each record out anew (pure_module_takes, pure_module_recast,
pure_module_fields, over pure_send_recast's rule). A stream whose sender
starts again in another shape lets go what it held; the sender sends it
again laid out anew. A refusal already queued for a stream now says the
last reason judged, not the first. Over HTTP a delivery (X-Pure-Source)
may lack fields; an unknown member is still refused.
test-link named: a stream of another shape naming its fields is taken,
200 stored in 47 ms with every delivery naming them; an input that does
not take them refuses (4) and the sender backs off. check.py
server_fields: five outputs, three linked and two over HTTP, fields in
another order taken, an input with a field more taking messages without
it, one lacking a field refusing until it starts again with it, then
taking all. server_upgrade's new receiver declares its fields in another
order. 012 names the frame and the rules.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 004's F3 made a module refuse to start while sends waited on an
output whose fields it had changed: their bytes would have meant
something else. Safe, but it deadlocks a rolling upgrade. Growing a
board shared by two nodes: one upgraded to a card with a new field
refuses the other's old sends (another shape), and the other, its old
sends waiting, cannot start the new module at all ("Cannot start the
module on the store", saying nothing more).
Waiting sends now move to the new fields by name, as a list's items do:
a value goes over when the new type can hold it (u32 and number are
numbers, bytes and text are bytes; a number that is not a u32, or bytes
that are not UTF-8 for a text, cannot), the rest start at 0 or empty
(pure_send_recast, with the payload layout load uses, pure_fields_lay_out).
A start recasts them and a replace does into a new outbox.
Their meaning must survive a crash between the recast and the save that
names the new fields in the links. So the outbox's image now names the
fields its sends were laid out with ("PLO1": the module's outputs after
the save it holds), and is written again before any send is logged with
other fields; a start reads those names over the links'. Checkpoints of
the outbox name them too.
A module without an output that sends wait on is still refused, now
saying so in PureApp.why, which pure-serve prints ("sends wait on the
output 'ordered', which this module does not have"); a failed start
keeps it.
test-module send_recast (widened, narrowed, UTF-8, a payload not laid
out, there and back) and output_meaning (the swapped fields recast on a
replace, read right after a restart that never saved, back again, sends
logged after the image, the refusal and its reason). check.py
server_upgrade: five sends wait on the old output, the sender starts
with the new one, the old receiver refuses the new shape, the new one
takes all eight.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A link's ring and a wire's queue took every saved send of their output
and kept it until delivered: with the receiver down they held the
outbox a second time, 178 B and 313 B more a send (104). Each outlet now
holds a window, --window bytes (4 MiB) and 65,536 sends past what its
receiver has, at least one; the outbox keeps the rest.
The owner remembers each outlet's place in the outbox and looks on from
there after a save and after its transports deliver. Places move back by
the sends that left the outbox since (PureApp.sends_gone), so a trim
before a place skips nothing, and sends queued before are not looked at
again however many wait. pure_app_sent reads a send by its place (the
newest-first visit it replaces is gone); pure_link_held and
pure_wire_held say what a transport holds.
With the receiver down a node now grows by the outbox alone once the
window is full: 219 and 218 B a send over the link and HTTP (399 and
534 B before). check.py server_windows keeps the receivers of a sender's
two outputs, one linked and one over HTTP, down through 300 buys with a
2,000-byte window, then starts them: each applies all 300 once. Without
the refill after deliveries the link stops at 117.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 004's timer fix (103) counted the deadlines on the monotonic
clock, but the owner still slept until a wall-clock time: with the
clock set back an hour a node's timers would stop for that hour. So
would a wire's pause between tries, and a lone commit after a group
would wait an hour for the others to join it. The test's LD_PRELOAD
moves only what the process reads, so there each wait ended at once
instead, and the timers went on ticking while the owner spun a core.
core/wait.h makes conditions that wait on the monotonic clock
(pthread_condattr_setclock; macOS keeps the wall clock) and deadlines
on it. The owner's waits, the wire's pauses and the group commit's
gathering use them.
clock-back.so can now move the clock forward too (CLOCK_SHIFT_S), and
check.py server_clocks moves it an hour each way under a node with a
timer and a wire to a port no one listens on, and under a document
server whose three threads commit together, then one alone. Before:
back, 159 % CPU and 50,886 tries in 1.2 s; forward, 1 tick, 1 try, and
a lone commit that never ended. After: 59 ticks, 0 % CPU, 4 tries and a
lone commit of 2.6 ms, both ways.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve --memory BYTES gives the app a budget of state bytes: its
lists' items as they are stored, its texts, the outbox and the numbers
heard (docs/design/013-memory-budget.md). The count is deterministic,
the same on any device. An event after which the states hold more than
the budget, and more than before it, fails with the fault memory and
changes nothing; HTTP answers 503 memory. One that frees or keeps them
goes on, so an app over its budget can always shrink.
Sends waiting for their receivers are state: a full outbox refuses the
next send until some are stored, which is the backpressure a slow
receiver puts on clients. A delivery refused for memory is not heard:
the link keeps it (c9c0edf) and HTTP senders try again after 503.
An answer (--reply) now takes its send out of the outbox in its own
event (pure_app_answer), so it has the room its send held. Apart, an app
whose outbox filled the budget could never apply an answer that keeps
anything, and the send leaves only after its answer: both would stay for
good. An answer that still does not fit waits, with the wire's after it,
and is tried again every 100 ms: pure_wire_answer leaves it in place
until pure_wire_answered.
Checks: test-module memory_budget (refused and undone, freeing over the
budget, a full outbox, a delivery not heard, answers that fit or wait,
across a restart); check.py server_memory (a sender refused 503 memory
with its receiver down and taking buys once it is up; a receiver
restarted below what it keeps, refusing 7 times in 2 s, then taking all)
and server_answers_memory (answers wait while receipts fill the budget,
then apply once each; the API called once per order). The stats report
state_bytes and memory_refused.
|
| |
|
|
|
|
|
|
|
| |
The usage showed --send OUTPUT=http[s]:// first; links are now the way
between nodes (012), so it shows pure:// for a Pure node and HTTP for
anything else. The statistics at exit count the streams a node's links
were refused (link_refused) and the ones its listener refused
(listener_refused), so a wiring mistake is in the numbers too, not
only the line on stderr; server_links_refused checks both.
|
| |
|
|
|
|
|
|
|
|
| |
pure-serve took a module with an input named show or edit for a
document app and refused it unless both were keyed messages: an app with
a plain message named edit compiled and then would not start (review
004, an earlier P2). Documents are what a module keeps in a keyed
collection; now the host serves them when the module declares one, and
inputs named show and edit are otherwise messages as any other.
check.py's server_plain_edit starts such an app and posts to edit.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
run_timers ticked nothing in an app without states, taking a timer to
be able to change nothing else; but a tick may send, and the outbox is
the app's state (the host gives such an app an owner for it). A timer
that only sends never ticked (review 004, F4). Now an app ticks when it
has states or outputs; one with neither runs on every connection thread
and has nothing a tick could do.
When a timer is due was reckoned on the wall clock: set back, a timer
waited for the time it had reached again (review 004, an earlier P2).
The next tick is now reckoned on the monotonic clock; events still see
the wall clock (now).
check.py's server_timers runs the review's heartbeat (a timer, an
output, no state) and sees it tick, then again with the wall clock set
back an hour 0.2 s in (build/clock-back.so, an LD_PRELOAD of
clock_gettime): 29 ticks in 0.6 s, where the server before this stops at
9.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A walk whose where bounds a keyed list's key finds the bound once, when
the walk begins, if its body writes nothing the bound reads. The body's
writes counted only states stored and lists changed, not what those
changes move with them: an insert or remove moves the indices into the
list, a text's change the cursors in it. A walk bounded by an index
into a list its body inserts into saw the bound it began with: of five
rows it took one (review 004, F1). Those states now count as written,
and a bound that moves an index (whose list is whichever the index's
is) is kept only when the body changes no list.
build/module-probe runs a module twice from the same keys, as it loads
and with every range let go, and says which state first differs.
check.py's module_ranges runs the review's case (all rows twice: done
10, at 10) and a walk whose bound its body cannot move, still ranged.
The probe finds the difference in the runtime before this.
|
| |
|
|
|
|
|
|
|
|
| |
check.py's server_links_independent is a receiver in Python (the
cryptography package, no code of link/) written from the link protocol's
description: the static secret, HKDF's keys, the nonces, the frames, a
CHALLENGE, ACK and STORED, and ACCEPT for epoch 1. A pure-serve node
links to it: every one of 200 orders is stored as the node knows it,
the first packet opened in epoch 0 and the other 199 in the epoch the
two made apart. None where the package is not there.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A receiver that took no records of a stream (a source the key may not
send as, no input of that name, an input of another shape) said nothing:
its sender heard them acknowledged, never stored, and sent them all
again every second, for ever, and its operator saw nothing. Now the
receiver answers REFUSED (0x0c: the names, and why), and the sender
sends no records for a second, then a packet of them to learn whether
they are taken now, the wait twice as long each time (64 s at most). A
refusal ends when that packet is acknowledged with no REFUSED beside, or
a record is stored. link.c says it once on stderr, as a wire says a
receiver does not take its sends; serve's input says which of the two
the app lacks.
test-link: each of the three is refused seven times in 90 s with nine
packets, and a stream refused until the app has its input goes on then,
all 200 records stored. check.py links a node to an input its receiver
lacks: said once, two packets in 2.5 s.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A link resolved its receiver's name once, at its start: a receiver whose
store moved to another machine (another address under the same name)
was never found, and a sender whose own network changed kept a socket
that no longer went anywhere. Now, when three probes in a row go
unanswered, and at each probe after (a second apart at most), the link
looks the name up again and sends from a new socket connected there.
It looks before the probe goes, so the probe's answer comes to the new
socket (looking after it lost every answer, one socket after another).
A sender at rest does not look: only probes unanswered count.
make check moves a receiver (H3 of docs/design/011-pure-transport.md):
in namespaces of its own, the sender links to pure://receiver.test at
127.0.0.2; the receiver is killed, its store, key and memo copied, the
old address made unreachable, the name pointed at 127.0.0.3 and the
copy started there. Every order is applied once, and the moved receiver
makes no X25519.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
A listener lets a quiet sender's secret go with its active state, so a
node started again made one X25519 for every sender that came back (169
us on this Pi, most of a wake's CPU): asymmetric work in proportion to the
links, which design 011 counts as a reason to design again. The secret is
now remembered in a file beside the key (KEY.links, 0600, begun again for
another node's key): a place of 64 bytes a sender in the order allowed,
mapped, so a sender at rest takes no memory for it. A place is the first
28 bytes of the sender's key, a check (FNV-1a of the key and the secret)
and the secret, written secret first and key last; a place whose check
fails is made again. The UDP link checks see a receiver started again
make no X25519.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A sender allowed is now a peer of 48 bytes (its key, its key's id, its
sources), found through an index by key id; ids are 32 bits, so keys that
share one are each tried. What a sending peer needs besides (the secret
its key shares with the node's, its runs' keys and addresses, its
streams) is made at its first packet and let go once it has been quiet
for the idle period (--link-idle-ms, 30 s at first) with nothing of it
waiting: the next packet makes it again, one X25519, no handshake. The
listener hands over and answers only the streams and runs that changed,
not every one at each turn. --senders FILE allows senders a line each.
The UDP link checks take their sender from such a file among a thousand
others, let it go after 400 ms of quiet, and see an order after a longer
pause arrive all the same.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve --send OUTPUT=pure://host:port/INPUT sends an output's saved
sends to another node's --link-port with no handshake: the first packet
carries records. Keys come from the two nodes' keys (X25519, HKDF-SHA256,
AES-128-GCM), with a random run in each packet's header so no nonce
repeats across restarts; nothing is stored for them. What lasts is the
apps' own: the sender's outbox and the receiver's heard numbers
(pure_app_heard, new). The sender detects losses as RFC 9002 says, with
NewReno, and trims a send once the receiver says it stored it; the
receiver hands the owner whole records in order, which it applies from
the one after the source's last heard number, stopping at a gap, and
answers how far it stored with every acknowledgment. Stored bytes are
the bytes on the wire: an output and an input of one shape need no JSON.
server_links_udp kills the receiver, the sender, then both (-9) while
four clients buy: every saved order is applied once, with no handshake
(five runs, 10k-11k orders each). server_links_udp_lossy does it again in
a namespace of its own with the link's packets delayed, lost (3 %) and
reordered (10 %) by netem (three runs, 11k-12k orders).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
LOCAL_LOAD and LOCAL_STORE name one of an event's 64 slots, what it holds
(a number, or an index of a list) and that list: slots start at 0 when an
event begins, are shared by its handlers, the walks in them and the folds
in those, and are never saved nor in the states' namespace. A handler uses
a slot for one thing, so its type is verified as a state's is; views have
no locals (IN_LOCALS: a handler's block, a walk in it, a fold in either;
stores only in a handler and its walks). A fold reading a local still runs
a leaf at a time (one value for every item) and a shaped fold takes one
only on its key's side. The signature table's where widens to 16 bits.
locals checks a handler that sums items into a local in a walk, counts in
a fold the items below it, keeps an index in a local and reads it back,
over 80 events against a model, and five refusals. server_rows' shop now
takes stock through an index local and counts in one.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
GET /views/name?argument=value gives a view that takes arguments its
arguments: the query as it came is kept on the request's work slot (it
outlives the connection, which may go before the view is shown), and when
the view is shown, on the connection thread from a snapshot or by the
owner, its pairs (values %XX decoded) are laid out as a payload of the
arguments' shape and given before the view is evaluated. from and count
still page a table. An argument missing, unknown, given twice, not of its
type or badly escaped answers 400 invalid. A registered query takes its
view's arguments the same way (GET /queries/name?k=4). An arguments input
is not an input to post to.
server_arguments opens 200 accounts, then asks 83 times against a model:
balances by key (404 no_value for a key not there), counts above an
argument, texts with &, =, % and non-ASCII shown back, a table filtered by
two arguments whose column folds with one, paged, counts by an owner's
name, a registered query with an argument, and six refusals. The format's
document (008) describes arguments.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A rows field's JSON value is an array of objects naming the record's
fields (numbers, strings; a field left out takes its default). The
connection thread encodes them as the list's stored form, which may be
longer than the JSON, grows the request's input by that much
(pure_runtime_grow: charged to the payload budget as admission charges an
input, 503 payload_budget when it does not fit) and places them after the
body, so the owner still applies a message it need not parse. A message
whose only variable field is rows takes a body that is not JSON as those
rows in their stored form: a bulk insert without JSON. An API's answer
(--reply) gives rows the same way. A delivered batch holds no rows.
server_rows loads 30,000 stock rows in one JSON request (some 250,000
rows a second, against 6,500 with a request a row) and 20,000 more as
bytes, refuses a load with a key already there whole (409 exists), then
checks 400 random orders against a model (294 applied, 106 refused with
none of their rows applied), texts with escapes and Korean, a walk that
removes, and a restart that keeps every state and no rows.
The format's document (008) describes walks, rows fields and lists of
rows, and OUTER_ROW.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
PUT /queries/NAME takes a module compiled with aup query: the node loads
and verifies it, checks that its states are the app's (names, types,
records) and that it has a view NAME, and keeps it (64 at most; 409 for
another app's module, 400 for bytes that are no module, 404 without the
view). GET /queries/NAME?from=&count= shows that view from the last
save's snapshot with a reader of the query's module, as a declared view
is shown, so a query runs beside writes and sees one save whole. A query
registered again answers anew.
server_queries registers a per-branch total (a correlated fold), a join
of history rows with their accounts and tellers, each rich account's
rank and a filtered count, checks them against a model of the same
transfers, asks while transfers go on, pages, and checks the refusals.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve --surface LIBRARY [-- ARGUMENTS] maps a library and runs its
pure_surface_run in a thread of its own (serve/surface.h). The surface
reads the app as of the last save where it is, a published snapshot in a
reader app, as a connection thread reads views; an eventfd tells it of
each new snapshot; and its events (a key pressed, text typed) are
requests without a host in the owner's queue, applied and saved with the
round, their results handed back to the surface's waiter. So one node
serves programs over HTTP and people on its display with one owner of
its states, and nothing between the states and the screen. pure-serve
exports its pure_* functions for the surface to call, one copy of the
app model in the process; the table carries sizeof(PureApp) and
sizeof(PureModule) so a surface built for another layout refuses.
The check builds a probe surface that types, presses and reads: its
events show in HTTP views, an HTTP input shows in it, wrong inputs are
refused, it stops with the server, and a module without states cannot
have one. Without a surface the request path measured the same (small
GETs 140.9k vs 139.9k, app totals 126.3k vs 126.3k over eight rounds);
the exported symbols cost 11 KB of the file and 24 KiB idle.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A sealed request cost three X25519 operations (about 0.5 ms), so sealed
sends drained at half the plain rate. A wire now keeps one HPKE context,
encapsulated at its first seal, and seals each request as the context's
next number (a failed seal or a restart starts a new session); the
header carries the session's encapsulated key and the number, so a
receiver opens a session once per connection thread (16 remembered, the
least used replaced) and each seal by its number, also after it restarts,
unasked. A sender with a key of its own seals in HPKE auth mode and names
that key in the header; a receiver started with --sender NAME=KEY takes
deliveries only from the senders it names, each sealed with its key
(403 sender otherwise). The body is now the ciphertext and its tag.
Sealed sends, sender authenticated, drain at 461-464k a second against
462-470k plain and 207-234k sealed a request at a time (Raspberry Pi 5).
The check adds: a wire's requests share one session numbered from 0; a
changed number or sender key does not open; a receiver restarted naming
its sender opens the same session again; another node claiming that
sender's name with its own key is refused and applies nothing.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
--seal OUTPUT=KEY seals each request's body whole to the receiving node's
public key with HPKE (RFC 9180, base mode: X25519, HKDF-SHA256, AES-128-GCM
where the CPU has AES, else ChaCha20-Poly1305), bound to the request's
source, first number and count; X-Pure-Seal names the suite. A receiver's
--key is a raw 32-byte file, made 0600 when missing, held in OpenSSL's
secure heap; it opens the body where it lies (400 seal when it does not
open) and zeroes the plain text when the request's ticket goes.
--sealed-only refuses unsealed deliveries (403 unsealed).
The check puts a recording relay between two apps: every order applies
once, no body shows an item or a field name, a flipped bit is refused, a
replayed request applies nothing twice, and a seal to another key does not
open. Its hosts choose their own ports (--port 0). Sealed sends drain at
233-235k a second against 469-470k plain (Raspberry Pi 5): three X25519
operations a request, 164 us each. The plain request path measured the
same as before.
|
|
|
The HTTP host's end-to-end checks (documents against the earlier server,
lending, kill -9 durability, JSON, timers, links, webhooks and answers) sat
in aup's tests, so a change to the host was verified from the toolchain.
They now live in tests/check.py; make check builds their apps with the
sibling aup and runs them after the unit tests, which stay beside their
parts. Design 009 states the rule for all three repositories.
|
