| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
| |
libpure-link.a holds them; pure-serve and the M5 harness link it instead
of naming its source files.
|
| | |
|
| |
|
|
| |
and an SD card
|
| |
|
|
|
|
|
|
|
|
|
|
| |
091 measures four queries on pgbench's tables of 100,000 accounts, the
same rows and answers on Pure (registered queries, HTTP and JSON
included) and SQLite 3.46.1, through each step: items field by field in
leaves, folds and filters a leaf at a time, and one pass answering a fold
in a column for every row. Per-branch totals went from 398 ms to 4.9 ms
(SQLite 45, 14.3 with a DBA's indexes), a three-table join from 3.3 to
0.48 ms (1.47), and each rich account's rank from 9.2 s to 6.8 ms
(2.84 s, 7.6 ms with an index); counting accounts in credit takes 1.05 ms
where an index lets SQLite count 0.16 ms. Pure declares no index.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
benchmarks/bank/bank.aup keeps accounts, tellers and branches as keyed
lists and a history list; a transfer changes a row of each and appends a
history row in one event, as pgbench's TPC-B-like transaction does in
SQL. run.py makes a scale's rows and drives transfers through the load
generator's new transfer kind; sqlite-bank.c runs the same tables and
SQL on SQLite 3.46.1, a connection a thread, WAL.
With lists as trees, transfers run at 30.5k a second on 1,000 accounts
and 24.5k on 100,000 (RAM store, one connection thread, HTTP and JSON
included), where they ran at 3.2k on 30,000 before. Durable on the SD
card, Pure makes 1,339 a second and SQLite with synchronous=FULL 84 to
153. SQLite in process with no HTTP makes about 40k in RAM: recorded as
it is, with the write amplification of 100,000-row tables on the card
and ad hoc queries as what comes next.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The roadmap's E2EE messenger made of the runtime's general parts, with
no messenger code in it: sealed, authenticated sessions between nodes
(086, 088), Vega on each node's display as a surface (design 010), the
app in Vega's examples. Vega's check runs two nodes through a recording
relay: a line typed on one display is drawn on the other, frames match
the vega command's, and the relay sees no text.
A line takes 0.79 ms at p50 and 1.1 ms at p99 from one node's program
to the other node's views with stores in RAM, both commits, the HTTP
delivery and the seal included (sealing adds nothing measurable); on the
Pi's SD card 3.5 ms at p50, the long tail being the card's syncs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve --surface LIBRARY [-- ARGUMENTS] maps a library and runs its
pure_surface_run in a thread of its own (serve/surface.h). The surface
reads the app as of the last save where it is, a published snapshot in a
reader app, as a connection thread reads views; an eventfd tells it of
each new snapshot; and its events (a key pressed, text typed) are
requests without a host in the owner's queue, applied and saved with the
round, their results handed back to the surface's waiter. So one node
serves programs over HTTP and people on its display with one owner of
its states, and nothing between the states and the screen. pure-serve
exports its pure_* functions for the surface to call, one copy of the
app model in the process; the table carries sizeof(PureApp) and
sizeof(PureModule) so a surface built for another layout refuses.
The check builds a probe surface that types, presses and reads: its
events show in HTTP views, an HTTP input shows in it, wrong inputs are
refused, it stops with the server, and a module without states cannot
have one. Without a surface the request path measured the same (small
GETs 140.9k vs 139.9k, app totals 126.3k vs 126.3k over eight rounds);
the exported symbols cost 11 KB of the file and 24 KiB idle.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A sealed request cost three X25519 operations (about 0.5 ms), so sealed
sends drained at half the plain rate. A wire now keeps one HPKE context,
encapsulated at its first seal, and seals each request as the context's
next number (a failed seal or a restart starts a new session); the
header carries the session's encapsulated key and the number, so a
receiver opens a session once per connection thread (16 remembered, the
least used replaced) and each seal by its number, also after it restarts,
unasked. A sender with a key of its own seals in HPKE auth mode and names
that key in the header; a receiver started with --sender NAME=KEY takes
deliveries only from the senders it names, each sealed with its key
(403 sender otherwise). The body is now the ciphertext and its tag.
Sealed sends, sender authenticated, drain at 461-464k a second against
462-470k plain and 207-234k sealed a request at a time (Raspberry Pi 5).
The check adds: a wire's requests share one session numbered from 0; a
changed number or sender key does not open; a receiver restarted naming
its sender opens the same session again; another node claiming that
sender's name with its own key is refused and applies nothing.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
What the app model, snapshot reads and delivery had cost small reads
(126.7k -> 116.1k GETs a second) and idle memory (2.0 -> 4.5 MiB), how
the sampler found it, and the four steps and the crypto split that took
it back: 139.9k and 2.3 MiB. Measured again today beside axum 0.8.9 and
drogon 1.9.0 on the same Pi: a one-thread pure-serve answers small GETs
at 139k/s (2.2x axum and drogon a CPU second), 1 MiB GETs at 2,362/s
(axum 1,304, drogon 754), durable app writes at 21.3k/s (both about
200), with the least idle memory of the three.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Linking OpenSSL cost every pure-serve 2.3 MiB of resident memory, 0.66
MiB of it the process's own (relocated data), though most servers never
reach an https receiver or seal a send. The code that uses OpenSSL (TLS
to https receivers, sealing) is now built as pure-crypto.so beside
pure-serve and exports one table; pure-serve maps it the first time a
wire to an https URL opens or a key is read or a seal wired
(serve/crypto.c), and a missing library is reported at startup. What
needs no OpenSSL, a key's hex and the suite this CPU seals with, stays in
pure-serve.
An idle server: RSS 4,496 -> 2,320 kB, PSS 1,639 -> 881 kB, private
dirty 1,296 -> 864 kB. A static OpenSSL was measured too and is worse
(RSS 3.8 MiB, but 2.3 MiB private: all of its tables relocated). The
checks, https webhooks and sealed sends included, pass unchanged.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
--seal OUTPUT=KEY seals each request's body whole to the receiving node's
public key with HPKE (RFC 9180, base mode: X25519, HKDF-SHA256, AES-128-GCM
where the CPU has AES, else ChaCha20-Poly1305), bound to the request's
source, first number and count; X-Pure-Seal names the suite. A receiver's
--key is a raw 32-byte file, made 0600 when missing, held in OpenSSL's
secure heap; it opens the body where it lies (400 seal when it does not
open) and zeroes the plain text when the request's ticket goes.
--sealed-only refuses unsealed deliveries (403 unsealed).
The check puts a recording relay between two apps: every order applies
once, no body shows an item or a field name, a flipped bit is refused, a
replayed request applies nothing twice, and a seal to another key does not
open. Its hosts choose their own ports (--port 0). Sealed sends drain at
233-235k a second against 469-470k plain (Raspberry Pi 5): three X25519
operations a request, 164 us each. The plain request path measured the
same as before.
|
| |
|
|
|
|
|
|
|
|
| |
The HTTP host's end-to-end checks (documents against the earlier server,
lending, kill -9 durability, JSON, timers, links, webhooks and answers) sat
in aup's tests, so a change to the host was verified from the toolchain.
They now live in tests/check.py; make check builds their apps with the
sibling aup and runs them after the unit tests, which stay beside their
parts. Design 009 states the rule for all three repositories.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The app model, the tree store and the HTTP host lived in
prototypes/connections among the earlier document runtime, the store was
known only by its format's name, and Vega compiled six Pure sources by path.
Now, as the README's principle has it:
core/ runtime, flow and the app model (module*.c), with what the core
asks of storage (state.h, store.h)
store/ the tree store (PURETRE3, PUREREC1)
serve/ pure-serve: HTTP host, outbound delivery, JSON in place
cli/ pure-run, pure-edit-module
A top-level make builds libpure.a (app model and store, for hosts to link)
and the hosts; make check runs the store's and the app model's tests. The
earlier runtime still builds from prototypes/connections against core/ and
store/ (check, check-tree and check-core pass). Design 009 records the rule:
parts by role behind libraries; git repositories split by coupling, so the
store stays here while its contract moves with the core, and Vega stays apart.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve --send OUTPUT=http://host:port/path wires an output: after
every save the owner queues the new sends as JSON (fields and groups)
and a thread per output posts them in order with X-Pure-Source,
X-Pure-Sequence and Idempotency-Key. A 2xx, or a Pure receiver's
refusal, delivers; the owner then trims. 408, 429, 5xx and no answer
are sent again after a pause growing to 10 s, other answers every 10 s:
nothing is dropped. A Pure receiver applies a delivered message with
pure_app_deliver (now taking placed fields) and advertises
X-Pure-Batch: 256; the sender then posts up to that many numbers in a
row as one JSON array (X-Pure-Count), applied in order in one burst
with one save.
Delivery kept up with orders once batched (391 deliveries/s one at a
time; a backlog of 20,000 went at 60k/s). With four clients ordering,
killing the receiver and then the sender (-9) left the receiver's count
and total equal to what the sender saved, five times and under ASan.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve no longer copies a JSON body's strings into the message's
payload. The ingress is the prefix, a table of each field's start and
size, and the body as it came; the connection thread scans the body
once, puts numbers into the prefix and records where each string is.
Only a string with escapes is unescaped, over itself (it never grows).
A group of fields comes as an object inside the body and, for a fixed
field, as its path in the query (?to.zip=7). A group where a field
belongs, a value where a group does, an array or true/false/null is
json_type; an unknown member, inside a group too, json_field.
The string scan and UTF-8 checks now pass over eight bytes at a time
while none needs a look (pure_utf8_step, so every text check gains).
On the Pi, posting {"name":"k","text":"x..."} to a stateless app went
from 652-655 to 3,737-3,953 requests/s with 256 KiB of text (server
CPU 1.52 to 0.25 ms a request, peak RSS 5.2 to 3.2-3.4 MiB) and from
29.1k to 76.3-76.7k/s with 4 KiB; the load generator gains a json
method and benchmarks/http/echo.aup is the app.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A tree store commit wrote its bytes over the written zeros of the tail
and its head into a slot at the start of the file: two ranges, and on
the Pi's SD card a sync of two ranges costs about twice one (2.4 ms
against 1.2 ms). A commit now writes a 128-byte record (its head) at
the published end and its bytes right after it, then syncs once. Every
16 commits the head also goes into the older slot. The newest commit
is found from the newer slot's head, or from the newest head this
process has seen in the file while its record is still there, along
the records whose bytes are whole; a torn record or bytes end the walk
at the commit before, which the next writer overwrites. Slots of this
format are PURETRE3; files of the earlier one are read and continued.
Store commits went from 220-233/s to 371-396/s (p50 2.5 to 1.3 ms),
and the scoreboard's durable posts from 13-16k/s to 21-26k/s with 64
connections and from 2.2-2.4k/s to 2.9-3.9k/s with 8, measured
alternately with the previous build. The torn head check now tears a
slot (the walk goes on from the other) and the newest record (a new
process sees the commit before); a new check walks 42 commits from the
slots, in a fresh process too, and stops at a damaged record.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The owner publishes a snapshot of the saved states after every save,
before it hands back the answers; connection threads take the newest
one each round and show views from it with a reader app of their own.
A read made after a write's answer sees the write, nothing unsaved or
undone is ever shown, and reads no longer wait while the owner applies
events or syncs. Apps with states always get an owner thread, even
with one connection thread. A round's inputs go to the owner first;
its views are shown and then answered together (answering each one as
it was shown made one thread's 20-row page 8% slower).
On the Pi 5, with 8 connections posting durable scores and 56 reading
the total, reads went from 10-16k/s at p99 13-29 ms to 107-149k/s at
p99 0.6-0.7 ms, and posts stayed in range; axum and drogon, which sync
under their lock, read 1.1-1.3k/s at p99 111-156 ms. A snapshot of a
changed 1,024-item list costs 1.2 us (65,536 items: 87 us) against a
3-4 ms save.
|
| |
|
|
|
|
|
|
|
|
| |
The checkpoint's table still said the document and scalar apps were not
one authorable model, and that equivalent-server latency was unproven.
A new section records what has been checked since: one app model (aup to
module format v5), persistence, Vega's surfaces and hot swap, apps as
HTTP services, and the measurements against axum and drogon, with what
remains. The README opens with the same in brief.
|
| |
|
|
|
|
|
| |
The recorded Raspberry Pi 5 (Linux AArch64, 16 KiB pages, ext4) replaces
Apple Silicon macOS as the device whose results decide completion. The macOS
CI job stays as an optional portability check; no macOS support is claimed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
state.h gives any app a runtime-owned working version of a stored value on
the tree store. A draft shares unchanged extents with the stored version,
edits in memory by path copying, lends ordered extents without copying, and
commits only its new nodes when the key is still at its base revision. A
failed commit changes neither the draft nor the store; a draft whose file was
replaced by compaction copies its stored extents on commit. Unreachable
arena nodes are reclaimed once they outweigh max(reachable, 32 KiB).
Typing into the middle of a 1 MiB document and saving each character peaked
at about 1,660 KiB RSS with a draft against about 4,480 KiB for an app copy
compared with storage on save; saves were only 1-2 ms faster. This corrects
073, which had attributed the TUI's remaining 8 ms to that comparison.
Drafts are not yet charged to the runtime budget or visible to .pure
definitions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
store-tree.c keeps every document in one append-only file as a persistent
B-tree of byte extents, with the key catalog in the same structure. A splice
writes only the changed leaf and path; unchanged extents are shared between
versions, readers take no lock, and two alternating head slots publish a
commit after its nodes are durable. Compaction rewrites reachable nodes.
On the recorded ext4 host a durable 1-byte middle insert into a 1 MiB
document took 8.58 ms against 78.51 ms for the default store, and the real
TUI's Ctrl+S after typing took 16.60 ms against 84.59 ms. check-tree runs the
store's crash/oracle tests and the unchanged runtime, splice, async and
real-PTY editor checks on it. It stays opt-in: linked code is 12.7 KB larger
and contiguous multi-leaf reads copy.
|
| | |
|
| | |
|
| | |
|
| |
|
