| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
States have one owner, so a module with them ran on a single thread,
which spent most of its time in the kernel sending and receiving. With
--threads N such a module now gets N connection threads and one owner
thread: connection threads accept, parse and answer; the owner takes the
ready requests of every thread as one round, applies them in bursts with
one save each and hands each thread back its applied requests (eventfd),
which it completes and answers. A request's route and payload stay held
by its thread's runtime core until completion, so an accepted event runs
even when its connection closes first.
After a burst that saved, the owner gathers requests until none has come
for 100 µs (1 ms at most), so the requests its answers bring back share
the next save as they would on one thread: 30 events per save became 50
and the median latency of a durable post fell from 5.4 to 3.5 ms. A page
of 20 rows went from 87,600 to 141,900 answers a second on the Pi.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve answers GET /views/NAME with a view's value (a number or text
as text/plain, a table as JSON rows, a page of them with ?from=&count=,
each cell computed only for the rows asked) and POST /inputs/NAME with an
event: a text input's text is the body, a key input is pressed, a message
takes its u32 fields from the query and its bytes field from the body and
answers its handler's reply. A failed event answers with its fault (404
not_found, 409 exists, 422 the others). No routes are written: the app's
views and inputs are its interface. Documents stay served when the module
has show and edit.
The route of a request is kept per work slot, so an accepted event runs
even if its connection closes before the burst; views and app events join
any burst, whose one save comes before its answers.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A list state may name a key field of its record (the flags word of its
entry: field + 1). A keyed list keeps its items in key order, numbers by
value and texts by their bytes, each key once: LIST_APPEND places an item
by halving and fails with the new fault exists (PURE_CONFLICT) when the
key is there; LIST_FIND gives the index of a key's item and LIST_HAS
whether it is there. The verifier refuses inserting by position and
changing a key. Maps, sets and sorted maps are one list with an order
rule, so tables, filters, folds, indices, storage and replacement stay
as they are.
A module that opens over a list not in its key order (one that was not
keyed, keyed by another field, or stored so) puts it in order, moves its
indices with their items and stores it again; two items with one key
refuse the module rather than drop one.
TEXT_COMPARE compares two texts by their bytes under one of the six
relations.
|
| |
|
|
|
|
|
|
| |
A fifth fold kind, first, runs its block over the items until one leaves
a non-zero number and pushes that item's index, typed as an index of the
list, or fails not_found when none does. Handlers can so find the item to
change by a condition, as a message naming a task by its title would.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A table view can now carry a filter: its list goes in the byte after the
view's type (list + 1) and its code range is a block run as a column of
the list, leaving one number; the table has a row for each item the block
keeps, in the list's order. Columns read the item (ROW is the item's
position, not the row's). The filter runs once per item after a change of
the app's states, counted by a new number that never goes back (events,
points, a failed save's undo, a module replacement), and its rows serve
every cell until the next change.
pure_app_row_item and pure_app_item_row turn a table's rows into items
and back, so a host shows a focus (an index names an item) on the row it
is in and points at the item a clicked row shows. pure_app_rows now takes
a mutable app.
The checks cover rows, cells, the mapping both ways as items leave and
come back, six refusals, and 20,000 damaged filter modules.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
FOLD (opcode 65) runs the block that follows it once per item of a list,
as a column of that list, and pushes the count of items whose block left
a non-zero number, or the sum, maximum or minimum of the values the block
left with an include flag; an empty maximum or minimum is not_found. The
verifier checks the block as a column (reads only, one or two numbers
left, no fold inside, so the work is bounded by block length times items)
and treats it from outside as one instruction that nothing branches into.
Views and handlers can now say how many tasks are open or what the scores
add up to without keeping a second copy of it in state.
The checks cover values, empty lists, filters, a fold inside arithmetic,
six refusals and 20,000 damaged fold modules run under ASan and UBSan.
|
| |
|
|
|
|
|
|
|
|
|
| |
The zero tail wrote as many bytes as the commits under 64 writers, yet
turning it off changed nothing beyond the SD card's own noise and made
8 writers wait longer, so it stays until faster storage is measured.
Overlapping a sync with the next burst would win a few percent at most.
The second sync of a new process's first commit is rare with one owning
server. The larger lever is write amplification: a 16-byte edit rewrites
its whole leaf and catalog path.
|
| |
|
|
|
|
|
|
|
|
|
| |
Text inserted at a cursor's position pushed every cursor there behind
it. That suits a caret, not a remembered position: a mark, a selection's
start or a bookmark should stay before the new text. A cursor state's
reserved word in the module now carries a flag: 1 makes it a mark, which
stays where text is inserted at it and otherwise follows edits as any
cursor does. The flag is refused on other states and other values;
existing modules, with 0 there, keep their meaning.
|
| |
|
|
|
|
|
|
|
|
|
| |
pure_app_point moves an index state to an item, as a user pointing at a
row would: an event with no handler, saved like a key event, joining the
burst when saving is deferred; past the last item it is not_found. A host
learns which index to move from the view that shows it:
pure_module_position gives the state behind a number view that is just
`at.position`. An interface can so turn a click on a list row into its
focus without any code in the app.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
A read answered with extents of the store mapping was charged nothing,
so clients that stopped reading could keep any number of old file
versions mapped. The runtime core now takes resources a host sends in
pieces: body.data is NULL, body.size is charged like any other resource,
the outcome has no body to read and cannot be forwarded. pure-serve hands
the mapping reference over as such a resource, so a lent answer holds its
bytes of budget until it is in the socket and its ticket releases the
mapping; past the budget a read gets 503 payload_budget, as a copied one
would. The async core grows 52 bytes (5,776 of its 6,144).
|
| |
|
|
|
|
|
|
|
| |
pure-serve renewed a connection's deadline when a request arrived or an
answer finished, not while an answer was being sent, so a client that
read a large answer steadily but slowly was cut off after ten seconds
(a 1 MiB document over a 50 KB/s link never arrived). Every write that
makes progress now renews it; only a stalled connection expires.
|
| |
|
|
|
|
|
|
|
|
|
| |
A burst ended once 16 keys had joined it, so 64 writers needed four
commits a round and some waited through several (p50 17.4 ms). The
pending documents now grow on demand up to 256 keys, which also makes
PureApp 1.3 KB smaller (3,128 bytes). With 64 connections a commit now
carries 26 events on average instead of 12 and the median PUT waits
9.2 ms; throughput moves less, since each commit writes more and the SD
card's syncs take most of the time.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A drogon 1.9 service with the axum baseline's contract joins the benchmark
(memory, fdatasync and crash-safe modes, three IO threads), built from
Debian's packages unpacked under .deps without installing them. drogon
listens once per event loop, so the service picks a free port itself:
with port 0 every loop got a port of its own and the load reached one.
The load generator has one core and saturates near 150k answers a second,
so the harness now also reports answers per second of server CPU time
(all threads). Over three rotated rounds: small GETs 151.6k/s on three
shards (drogon 145.3k, axum 120.9k) and 128k per CPU-second on one thread
(drogon 65k, axum 62k); 1 MiB GETs 2,794/s (axum 1,372, drogon 708);
crash-safe PUTs 5.1-5.8 times axum and 13-30 times drogon; peak RSS
5.0 MiB against 14.4 and 76.5 MiB. --workloads is now a regular expression.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The executor shrinks from 49.8 to 42.2 KB of -O2 text (module.c 34.0 to
27.0 KB) with the module format and API unchanged:
- The verifier reads its rules from the signature table: what each
opcode's argument names, and which text or list it works on. Cursor and
index operands must belong to that text or list and so does a pushed one;
other values belong to nothing, so a value that merely came from a text no
longer blocks a join. The loader and verifier fail through one path.
- The interpreter advances as it fetches and leaves through two exits: a
stated fault, or a change the data refused.
- Key, text and message events take one path; a message brings its base
revision and key. Starting an app opens its module over none, the path
replacing takes, so every new state starts from its stored value.
- Paths that run once per module or app are marked cold and built for size;
event paths stay built for speed (small GETs: 128k/s, as before).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A commit used to sync its nodes, then write the head and sync again: the
order of writes was forced by syncs. Now the head carries where the
commit's bytes start and their digest, and one fdatasync makes the bytes
and the head durable together. A crash during it can leave a head whose
bytes did not all reach the device; its digest then fails and the older
slot stands, which is safe because a writer only builds on a durable head.
Commits overwrite written zeros kept after the published end, so the sync
changes neither the size nor the block map and needs no journal commit.
On the SD card this is 2.4 ms per commit against 9.1 ms for two syncs;
the store commits 289-304 times a second instead of 94-109, and pure-serve
answers 2,012-2,611 durable PUTs a second against 583 for crash-safe axum.
A process remembers the heads it published or checked, so it reads a
commit's bytes at most once and never its own. Its readers pass over the
head it is still making durable and so never answer with a commit a crash
could take back. Head slots get a block each and the format becomes
PURETRE2; a handle's refresh reads the slots through its mapping.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
A read of a stored document now answers with the store mapping's extents
through writev, without copying the document. pure_draft_mapping takes a
reference to the mapping a draft opened through a handle reads from; the
answer keeps it until the last byte is sent, so the version read goes out
intact even if the document changes or a compaction replaces the file
meanwhile. Replies the mapping does not hold are copied as before.
pure-serve's output is now a list of extents (the head first). One thread
serves 2,761 1 MiB reads per second (axum: 1,380) and 128k small ones.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A burst now looks at the store once (pure_store_pin) instead of once per
request; another shard's commit ends before its answer is sent, so a
request made after that answer still reads it. With this alone one thread
serves 125k small keep-alive reads per second, as many as axum on three
cores, with a shorter p99.
--threads N runs a shard per thread: its own SO_REUSEPORT listener, epoll
set, runtime domain, app and store handle. Writer sessions of the tree
store now take turns inside a process with a mutex (the file lock belongs
to the process and does not keep threads apart), and revisions settle
concurrent edits of a key. A module with states runs on one thread. Three
shards serve 151k reads per second; they still commit one by one, so
durable writes and 1 MiB reads are better on one thread for now, which
stays the default. Note 075 records the numbers.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve replaces the module mode of server.c, which goes back to what
it was. It keeps the document contract, the runtime core (a work slot per
connection, the payload budget, core-owned ingress, borrowed PUT replies)
and the group commits, and adds what server.c lacked: HTTP/1.1 persistent
connections with in-order pipelining, an edge-triggered epoll owner
thread, up to 1,024 connections allocated as they arrive and reused, one
writev per answer, and documents opened through a store handle.
On the Pi 5 with one thread it serves 106k small keep-alive reads per
second (axum on three cores: 126k) at the same p99, 1,741 1 MiB reads
(axum: 1,370), and 1,126 durable writes from 64 connections (crash-safe
axum: 372), in 7.3 MiB against axum's 17-20 MiB. Note 075 has the table.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
A C load generator, an axum service with the same document contract (in
memory, fdatasync, and crash-safe replacement), and a runner that pins the
servers and the load to separate CPUs. On the Pi 5, Pure uses about 40%
less memory, wins with a new connection per request and with eight durable
writers (3.2x the crash-safe axum), but loses 5x on keep-alive reads and
1.5x on 64 durable writers: no persistent connections, one thread, eight
connection slots and two fsyncs per commit. Note 075 records the numbers
and the targets.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-connections-module offers the document server's HTTP contract with an
aup module: GET is its show message and PUT its edit message. Requests go
through the runtime core as before (work slots, payload budget, ingress,
replies borrowed from the request), and effects are taken on the owner
thread with the core's asynchronous interface.
Each poll round is one burst: every ready request is applied to the app
with deferred saving, one save stores them, and only then are the tickets
completed and answered. A failed save fails the whole burst, reads too. A
key seen again ends the burst first, and the module server accepts every
waiting connection at once so concurrent requests share a burst. On the
Pi's SD card eight clients' 240 edits take 0.59 s instead of 2.14 s.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A save now publishes everything since the last save in one commit: the
documents messages edited and the states that changed. If it fails, every
event since the last save is undone (values, text and list drafts, the
list undo log, event and tick counts), so an event reported as successful
is stored and a failed one leaves no trace.
By default each key, text and message event saves before it returns, as
before. With pure_app_defer events apply at once and a host saves a burst
with one commit; a key's document joins a burst once, so revisions move
one per save. On the Pi's SD card 600 events take 6.3 s saved one by one,
0.97 s in bursts of 8 and 0.34 s in bursts of 64. pure-run takes --batch N.
Tests compare a deferred stored app that saves at random and sometimes
stops unsaved with an app that replays only the saved events, undo a whole
burst on a failed save, and kill a process while it saves a burst.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When the data host keeps states (PureData.state and .save, implemented by
pure_tree_data), every state of an app lives in the store: a text state's
draft is its stored value, a list mirrors each change into a stored draft
(record header, then items), and numbers, cursors and indices are stored
together by name. An app starts from what is stored, moves list fields by
name when a record changed, resets cursors and indices that no longer fit,
and refuses damaged stored values.
Each key, text and message event publishes the states it changed together
with the message's document in one store commit, or fails and leaves
memory and store as they were. Frame changes go with the next event or
pure_app_save. pure-run takes --store PATH.
Tests run the same random events on an app in memory and on a stored app
that restarts every few events and require equal states after every step;
kill a process at every commit stage of an event that changes a number, a
text and a list; and check frames, message atomicity and migration by
restart and by replacement. A stored event costs about 10.6 ms more on the
Pi's SD card, mostly two fsyncs.
|
| |
|
|
|
|
|
|
|
| |
PICK copies the value n below the top, with its type and owner; SLIDE
drops n values under the top. A toolchain can compute a function's
arguments once and read each as often as the inlined body needs, instead
of repeating the argument's code. The format stays v5: no producer has
shipped modules without them.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Module format v5 lets a value cross a join: the verifier records the
stack each branch brings to its target and requires every path to agree
on height, types and owners, with at most 8 such joins ahead. Branches
are now allowed in views, so a view can be a conditional expression.
A table view has one row per item of a list. Its columns are number or
text views evaluated for one row at a time and may read that row (ROW).
Hosts compute only the cells they show through pure_app_rows,
pure_app_cell and pure_app_cell_text. Column names are scoped to their
table. The view limit rises to 64, columns included.
pure-run prints a table view as an array of row objects.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Module format v4 declares record types and adds list states (items of a
record) and index states (a position in one list). The verifier types
record operands field by field and rejects an index used on another list.
Indices follow their item across inserts and removes. List changes are
logged per event and undone when the event fails. Replacing a module keeps
a list by name and moves its fields by name when the record changed.
pure-run prints lists as arrays and a failed view as null.
|
| | |
|
| |
|
|
|
|
|
| |
A text editor needs more keys than the workbench-era limit of 8: typing,
four arrows, Backspace, Delete, Home, End and a mark already make 10. Event
numbers stay 1 + 2i and 2 + 2i; the module grows by the input name tables.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Module format v3 adds text states, cursors, UTF-8 constants, text inputs
and text views. A text state is a runtime-owned draft without a store, so a
large text edits as an extent tree. Cursors are byte positions the runtime
keeps on character boundaries: insertion pushes cursors at or after the
point, removal gathers cursors inside the range, and a failed event returns
every text draft and cursor to where it was. The verifier types a cursor by
the text it belongs to and rejects editing one text with another's cursor.
Operations cover constants, concatenation, shortest round-trip number text,
character, line and byte counts, character and column-keeping line moves,
insert, remove, line views, whole-text replacement and UTF-8 checked
conversion from bytes. Replies of edited documents or texts are copied before
they are finished. pure-run gains text input and text output.
test-module covers typing Korean and emoji, moves that stop at the ends,
line moves, views, rollback, invalid UTF-8 and replacement, text-specific
verifier rules, and 20,000 damaged text modules whose cursors always stay
on character boundaries under ASan/UBSan.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Module format v2 adds byte values, message inputs with u32 and bytes-tail
fields, one keyed byte collection, replies and require. The verifier checks
every operand's type and the context each operation may run in, and allows
branches and joins only at an empty stack, so typed values never merge.
A message event opens its key's working version through a PureData host,
conflicts early on a stale base revision, runs its handlers, and commits
only if they succeed; a failed event leaves documents and states unchanged.
Replies lend payload or document extents, and a document reply is copied
before commit. pure_tree_data implements the host on tree store drafts, and
pure-edit-module offers the document CLI contract of pure-edit from a module.
test-module adds document semantics on an in-memory host (not found, create,
conflict before bounds, limits, short payloads, busy commits, stable replies,
require, state rollback), typed verifier rules, and 20,000 damaged message
modules run safely under ASan/UBSan.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Pure now owns the format its toolchains target, as the JVM owns class files.
A v1 module carries f64 states, key inputs, atomic event handlers and pure
views. pure_module_load checks sizes, names, keys, operands, forward-only
branches, reachability and stack heights at every join before anything runs,
so execution needs no stack checks and each event is bounded by its handler
length. PureApp runs events on a pending copy and replaces modules keeping
states by name, without allocation. Semantics match the workbench engine,
including IEEE minimum/maximum on signed zeros.
test-module covers behaviour, atomic faults, 26 rejection checks and 20,000
damaged modules under ASan/UBSan; pure-run is a JSON test host.
|
| |
|
|
|
|
|
| |
The recorded Raspberry Pi 5 (Linux AArch64, 16 KiB pages, ext4) replaces
Apple Silicon macOS as the device whose results decide completion. The macOS
CI job stays as an optional portability check; no macOS support is claimed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
state.h gives any app a runtime-owned working version of a stored value on
the tree store. A draft shares unchanged extents with the stored version,
edits in memory by path copying, lends ordered extents without copying, and
commits only its new nodes when the key is still at its base revision. A
failed commit changes neither the draft nor the store; a draft whose file was
replaced by compaction copies its stored extents on commit. Unreachable
arena nodes are reclaimed once they outweigh max(reachable, 32 KiB).
Typing into the middle of a 1 MiB document and saving each character peaked
at about 1,660 KiB RSS with a draft against about 4,480 KiB for an app copy
compared with storage on save; saves were only 1-2 ms faster. This corrects
073, which had attributed the TUI's remaining 8 ms to that comparison.
Drafts are not yet charged to the runtime budget or visible to .pure
definitions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
store-tree.c keeps every document in one append-only file as a persistent
B-tree of byte extents, with the key catalog in the same structure. A splice
writes only the changed leaf and path; unchanged extents are shared between
versions, readers take no lock, and two alternating head slots publish a
commit after its nodes are durable. Compaction rewrites reachable nodes.
On the recorded ext4 host a durable 1-byte middle insert into a 1 MiB
document took 8.58 ms against 78.51 ms for the default store, and the real
TUI's Ctrl+S after typing took 16.60 ms against 84.59 ms. check-tree runs the
store's crash/oracle tests and the unchanged runtime, splice, async and
real-PTY editor checks on it. It stays opt-in: linked code is 12.7 KB larger
and contiguous multi-leaf reads copy.
|
| | |
|
| | |
|
| |
|
