| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
| |
Changing an app in place keeps a million items' app answering but for
1.12 ms (a host restart: 150 ms; stopping and starting it again from its
store, as first built, 150.4 ms), 171.5 ms when its records change shape
(332 ms). Another app's p99 stays 0.97 times as it was while one changes
every 0.2 s; a host of one app is as it was (TPC-C +0.7%, instructions
at most +0.02%); each module change leaves 19 KB. 030 holds the windows
of both builds, the isolation, the pairs, the counts and the memory.
|
| |
|
|
|
|
| |
The same events make byte-identical stores on the x86_64 WSL2 (4 KiB
pages) as on the Pi and the Mac, and a store made on either of the Pi
and the WSL2 opens on the other and goes on the same.
|
| |
|
|
|
|
|
|
|
| |
The same modules and inputs give the same outputs, failures and store
bytes on the Pi, an x86_64 WSL2 and an M3 Mac; other compilers found an
argument read before it was set, and the core and store build on macOS
again. WSL2's wall clock steps back about 2 s every 32 s. The Pi froze
twice under a measuring tool that held its output: it now streams it,
and experiments run under research/guard.py.
|
| |
|
|
|
|
|
| |
The loop alone slowed four links to half: woken by every record queued,
it chased the owner a record at a time. Pushing once a turn made links
1.6 times as fast, with threads or the loop alike; the loop's own cost
is within the noise. Twenty outputs now add two threads, not 21.
|
| |
|
|
|
|
|
|
| |
Stage 2 is built and measured: what was learned (the receiving side
needed nothing new, an output goes one way only) and the numbers. The
checkpoint adds sending to the many-apps row, and keeps 2b (one loop
for the send threads) and stage 3 (apps added and replaced while the
host runs) among what is left.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
017's first stage is built and measured: the shape now says what was
built where it differs from the design (work counted as written,
nothing counted without a budget, every app of --apps run by workers,
a busy app kept on its worker, a home thread for each app's
connections), and what was learned on the way. The checkpoint lists
many apps on one host among what Pure shows, and stages 2 and 3 among
what is left.
|
| |
|
|
|
|
|
|
|
|
|
| |
016 is built: bool and i64. The second stage's findings: an integer
literal takes the other side's type, found by compiling it on trial
(linear once the wanted type is passed down); a literal with no context
stays a number, so no program changes meaning and modules without i64
are the same bytes; one rule carries values between numbers, bools and
i64s; and the number paths passed the timing criterion, with what i64
had added to them counted and halved (110). The checkpoint lists bool
and i64 in the app model and drops i64 from what is left.
|
| |
|
|
|
|
|
| |
016's first stage is built and checked: no opcode was new, a stored
send's bool was refused on the way back in (found by server_meaning's
restart, fixed), -0 becomes false as 015 says zero has no sign, and
True in JSON is no JSON at all. i64 is what is left of 016.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A condition was a number, non-zero being true: a flag was a number field,
a view showed 1 for true, and nothing kept a number from standing for a
bool or a bool from being added to. Now the verifier has a bool among its
stack types, and a verified module never mixes the two. No opcode is new:
- CONST with argument 1 is a bool (+0 or 1), NUM_TEXT with argument 1
writes one as true or false; LOAD and STORE take a number or a bool
state, locals may hold a bool (2).
- Comparisons (LT to NE, TEXT_COMPARE) and LIST_HAS give a bool; AND, OR
and NOT take and give them; JZ, REQUIRE, filters and folds' tests take
one (count and first leave a bool, sum, max, min and distinct a number
and a bool); EQ and NE compare two bools too.
- Type codes: state 5, record field 2, message field 6 (one byte in the
prefix, 0 or 1, or the message fails as limit), view 3. A bool's
constant, initial value and default are +0 or 1; a list is not keyed by
one. PUREMOD6: v5 modules are built again.
At run time a bool is a number's 0 or 1, so leaf-at-a-time folds run as
before. A number that becomes a bool (a module replaced, a stored state
or item read again, a waiting send recast) keeps 0 (either sign) and 1
and takes the default otherwise; a bool that becomes a number is 0 or 1.
The host carries bools on every boundary: JSON true and false only (a
row's field too), a query's true and false, true and false in views,
table JSON and HTTP deliveries; a link takes a bool only into a bool.
A stored outbox's field types were read back with a check that refused
bool (at or past rows): a node whose output had a bool field and a send
waiting could not start again. test-module now stores such a send and
opens it (it fails on the old check), beside 28 bool refusals of the
verifier and bool states, fields, views, sends, links and migration.
The lane generator makes numbers and bools apart.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
transaction
The 90x of 092 was against SQLite's defaults committing each
transaction; tuned for the card and grouped as Pure groups, SQLite is
4-11x behind, the bytes it writes a transaction being the difference.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
103's next step: grow a real app, adding and reordering fields, cutting
the connection, starting again, and write down what gets in the way.
Two nodes share a board (from Vega's examples/board.aup), each sending
its changes to the other, the later change winning.
Upgrading them one after the other deadlocked: fixed by recasting
waiting sends by name (9aa123e) and taking fields by name (014,
5ce7655), and under it a link that stopped for good when a refused
stream's try was lost (baaf955). Now the older node's changes reach the
newer one during the upgrade, the newer one's new field waits for the
older, and the boards agree a second after. Fields in another order
agree at once; a partition healed agrees on the later change; both
nodes started again go on. Two rough edges eased on the way (the node's
name, message like in aup); sends and applies are still written field
by field. CHECKPOINT's row for links says so.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
104 measures a node's memory under a receiver that is down and under a
snapshot held for long (021's run.py: resident memory against what was
posted, 30 s a case). A held snapshot grew the node without end (6.4 to
678 MiB, fixed in d1e8f38); a slow receiver grows it by what was sent,
221 B a send, which is state until delivered, so the budget bounds state
and refuses past it.
013 is that budget's design: what it counts (state bytes, deterministic),
what an event over it does, why an answer takes its send's room, how
links and HTTP wait, what it does not count and those bounds, and the
checks. With --memory 4000000 the same load fills it in about 4 s and
the node stays at 9.6-9.8 MiB (45.9 MiB and growing without).
012 records the hole the budget found in the link (the full flag), and
103 that its clock fix fell short (385754c). CHECKPOINT gains the row.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
102: the spec (012) and a receiver written from it alone; the four holes
writing it found (REFUSED, the record's size, a peer's run places, the
memo's places beside) and the credit M6 showed fixed at 1,024 records;
M6 with ngtcp2 as a third, two rounds. Under reordering Pure carries the
most (45 Mbit/s; ngtcp2 29-47, quinn 1-2), under random loss the three
are the same, on a clean bottleneck Pure carries 3-5 % less (its framing,
some 112 B a packet to QUIC's 41), and where loss and reordering meet
ngtcp2 carries 1.3-3.6 times as much: undoing a loss it took for
reordering, it restores the window a real loss had cut too (its log: 22
of 34 undos past the window before the last cut; none without
reordering). Shares beside TCP stay within 25 %.
011 and the status table say so; 012 also says that a receiver answers
within 25 ms (a sender's PTO counts on it), that a shape and a record's
bytes are the app's, and why a packet of the past does no harm.
|
| | |
|
| |
|
|
| |
and an SD card
|
| |
|
|
|
|
|
|
|
|
|
|
| |
091 measures four queries on pgbench's tables of 100,000 accounts, the
same rows and answers on Pure (registered queries, HTTP and JSON
included) and SQLite 3.46.1, through each step: items field by field in
leaves, folds and filters a leaf at a time, and one pass answering a fold
in a column for every row. Per-branch totals went from 398 ms to 4.9 ms
(SQLite 45, 14.3 with a DBA's indexes), a three-table join from 3.3 to
0.48 ms (1.47), and each rich account's rank from 9.2 s to 6.8 ms
(2.84 s, 7.6 ms with an index); counting accounts in credit takes 1.05 ms
where an index lets SQLite count 0.16 ms. Pure declares no index.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
benchmarks/bank/bank.aup keeps accounts, tellers and branches as keyed
lists and a history list; a transfer changes a row of each and appends a
history row in one event, as pgbench's TPC-B-like transaction does in
SQL. run.py makes a scale's rows and drives transfers through the load
generator's new transfer kind; sqlite-bank.c runs the same tables and
SQL on SQLite 3.46.1, a connection a thread, WAL.
With lists as trees, transfers run at 30.5k a second on 1,000 accounts
and 24.5k on 100,000 (RAM store, one connection thread, HTTP and JSON
included), where they ran at 3.2k on 30,000 before. Durable on the SD
card, Pure makes 1,339 a second and SQLite with synchronous=FULL 84 to
153. SQLite in process with no HTTP makes about 40k in RAM: recorded as
it is, with the write amplification of 100,000-row tables on the card
and ad hoc queries as what comes next.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The roadmap's E2EE messenger made of the runtime's general parts, with
no messenger code in it: sealed, authenticated sessions between nodes
(086, 088), Vega on each node's display as a surface (design 010), the
app in Vega's examples. Vega's check runs two nodes through a recording
relay: a line typed on one display is drawn on the other, frames match
the vega command's, and the relay sees no text.
A line takes 0.79 ms at p50 and 1.1 ms at p99 from one node's program
to the other node's views with stores in RAM, both commits, the HTTP
delivery and the seal included (sealing adds nothing measurable); on the
Pi's SD card 3.5 ms at p50, the long tail being the card's syncs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A sealed request cost three X25519 operations (about 0.5 ms), so sealed
sends drained at half the plain rate. A wire now keeps one HPKE context,
encapsulated at its first seal, and seals each request as the context's
next number (a failed seal or a restart starts a new session); the
header carries the session's encapsulated key and the number, so a
receiver opens a session once per connection thread (16 remembered, the
least used replaced) and each seal by its number, also after it restarts,
unasked. A sender with a key of its own seals in HPKE auth mode and names
that key in the header; a receiver started with --sender NAME=KEY takes
deliveries only from the senders it names, each sealed with its key
(403 sender otherwise). The body is now the ciphertext and its tag.
Sealed sends, sender authenticated, drain at 461-464k a second against
462-470k plain and 207-234k sealed a request at a time (Raspberry Pi 5).
The check adds: a wire's requests share one session numbered from 0; a
changed number or sender key does not open; a receiver restarted naming
its sender opens the same session again; another node claiming that
sender's name with its own key is refused and applies nothing.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
What the app model, snapshot reads and delivery had cost small reads
(126.7k -> 116.1k GETs a second) and idle memory (2.0 -> 4.5 MiB), how
the sampler found it, and the four steps and the crypto split that took
it back: 139.9k and 2.3 MiB. Measured again today beside axum 0.8.9 and
drogon 1.9.0 on the same Pi: a one-thread pure-serve answers small GETs
at 139k/s (2.2x axum and drogon a CPU second), 1 MiB GETs at 2,362/s
(axum 1,304, drogon 754), durable app writes at 21.3k/s (both about
200), with the least idle memory of the three.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
--seal OUTPUT=KEY seals each request's body whole to the receiving node's
public key with HPKE (RFC 9180, base mode: X25519, HKDF-SHA256, AES-128-GCM
where the CPU has AES, else ChaCha20-Poly1305), bound to the request's
source, first number and count; X-Pure-Seal names the suite. A receiver's
--key is a raw 32-byte file, made 0600 when missing, held in OpenSSL's
secure heap; it opens the body where it lies (400 seal when it does not
open) and zeroes the plain text when the request's ticket goes.
--sealed-only refuses unsealed deliveries (403 unsealed).
The check puts a recording relay between two apps: every order applies
once, no body shows an item or a field name, a flipped bit is refused, a
replayed request applies nothing twice, and a seal to another key does not
open. Its hosts choose their own ports (--port 0). Sealed sends drain at
233-235k a second against 469-470k plain (Raspberry Pi 5): three X25519
operations a request, 164 us each. The plain request path measured the
same as before.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
After each save the owner walked the whole outbox to find the sends
not yet queued, so a receiver long down made every save walk the
backlog. pure_app_outbox_newest visits saved sends newest first, and
the owner stops once every wired output reaches the number it queued
last, then queues what it found in order.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The final answer to each send of a wired output (a 2xx, or a 4xx but
408 and 429) becomes a message event of the given input: fields from
the answer's JSON object by name, other members skipped; a field the
answer lacks from the send's field of that name and type, so the
request's own id is there; a u32 http_status field is the status. It is
applied with the send's number from the wire's source, before the send
is trimmed and in the same save, so an answer that comes again after a
restart (the API repeating itself for the Idempotency-Key) applies once.
Sends that wait for answers go one per request; answers are saved with
the next round, or at once when none waits.
A server that writes an answer in pieces holds the second until the
first is acknowledged, and a delayed acknowledgement stalled every
answer 40 ms: the client now acknowledges at once (TCP_QUICKACK) after
each read. Against a payments API that answers 200 or 402, with the
sender killed (-9) midway, paid and declined orders matched the API's
answers one to one.
|
| |
|
|
|
|
|
|
|
|
| |
A wire to an https:// URL speaks TLS 1.2 or later through OpenSSL: the
receiver's certificate must chain to the system's CAs (or to the file
--send-ca names) and name its host (SNI and SSL_set1_host, or the IP
address). An untrusted receiver gets nothing: the send stays in the
outbox, the wire logs it and tries again every 10 s. pure-serve now
links libssl and libcrypto.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve --send OUTPUT=http://host:port/path wires an output: after
every save the owner queues the new sends as JSON (fields and groups)
and a thread per output posts them in order with X-Pure-Source,
X-Pure-Sequence and Idempotency-Key. A 2xx, or a Pure receiver's
refusal, delivers; the owner then trims. 408, 429, 5xx and no answer
are sent again after a pause growing to 10 s, other answers every 10 s:
nothing is dropped. A Pure receiver applies a delivered message with
pure_app_deliver (now taking placed fields) and advertises
X-Pure-Batch: 256; the sender then posts up to that many numbers in a
row as one JSON array (X-Pure-Count), applied in order in one burst
with one save.
Delivery kept up with orders once batched (391 deliveries/s one at a
time; a backlog of 20,000 went at 60k/s). With four clients ordering,
killing the receiver and then the sender (-9) left the receiver's count
and total equal to what the sender saved, five times and under ASan.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve no longer copies a JSON body's strings into the message's
payload. The ingress is the prefix, a table of each field's start and
size, and the body as it came; the connection thread scans the body
once, puts numbers into the prefix and records where each string is.
Only a string with escapes is unescaped, over itself (it never grows).
A group of fields comes as an object inside the body and, for a fixed
field, as its path in the query (?to.zip=7). A group where a field
belongs, a value where a group does, an array or true/false/null is
json_type; an unknown member, inside a group too, json_field.
The string scan and UTF-8 checks now pass over eight bytes at a time
while none needs a look (pure_utf8_step, so every text check gains).
On the Pi, posting {"name":"k","text":"x..."} to a stateless app went
from 652-655 to 3,737-3,953 requests/s with 256 KiB of text (server
CPU 1.52 to 0.25 ms a request, peak RSS 5.2 to 3.2-3.4 MiB) and from
29.1k to 76.3-76.7k/s with 4 KiB; the load generator gains a json
method and benchmarks/http/echo.aup is the app.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A tree store commit wrote its bytes over the written zeros of the tail
and its head into a slot at the start of the file: two ranges, and on
the Pi's SD card a sync of two ranges costs about twice one (2.4 ms
against 1.2 ms). A commit now writes a 128-byte record (its head) at
the published end and its bytes right after it, then syncs once. Every
16 commits the head also goes into the older slot. The newest commit
is found from the newer slot's head, or from the newest head this
process has seen in the file while its record is still there, along
the records whose bytes are whole; a torn record or bytes end the walk
at the commit before, which the next writer overwrites. Slots of this
format are PURETRE3; files of the earlier one are read and continued.
Store commits went from 220-233/s to 371-396/s (p50 2.5 to 1.3 ms),
and the scoreboard's durable posts from 13-16k/s to 21-26k/s with 64
connections and from 2.2-2.4k/s to 2.9-3.9k/s with 8, measured
alternately with the previous build. The torn head check now tears a
slot (the walk goes on from the other) and the newest record (a new
process sees the commit before); a new check walks 42 commits from the
slots, in a fresh process too, and stops at a damaged record.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The owner publishes a snapshot of the saved states after every save,
before it hands back the answers; connection threads take the newest
one each round and show views from it with a reader app of their own.
A read made after a write's answer sees the write, nothing unsaved or
undone is ever shown, and reads no longer wait while the owner applies
events or syncs. Apps with states always get an owner thread, even
with one connection thread. A round's inputs go to the owner first;
its views are shown and then answered together (answering each one as
it was shown made one thread's 20-row page 8% slower).
On the Pi 5, with 8 connections posting durable scores and 56 reading
the total, reads went from 10-16k/s at p99 13-29 ms to 107-149k/s at
p99 0.6-0.7 ms, and posts stayed in range; axum and drogon, which sync
under their lock, read 1.1-1.3k/s at p99 111-156 ms. A snapshot of a
changed 1,024-item list costs 1.2 us (65,536 items: 87 us) against a
3-4 ms save.
|
| |
|
|
|
|
|
|
|
|
| |
The checkpoint's table still said the document and scalar apps were not
one authorable model, and that equivalent-server latency was unproven.
A new section records what has been checked since: one app model (aup to
module format v5), persistence, Vega's surfaces and hot swap, apps as
HTTP services, and the measurements against axum and drogon, with what
remains. The README opens with the same in brief.
|
| |
|
|
|
|
|
| |
The recorded Raspberry Pi 5 (Linux AArch64, 16 KiB pages, ext4) replaces
Apple Silicon macOS as the device whose results decide completion. The macOS
CI job stays as an optional portability check; no macOS support is claimed.
|
| | |
|
| | |
|
| |
|
