| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
| |
The in-place JSON reader of pure-serve's request bodies moves into its
own file with a small interface (pure_json_message, pure_json_fixed,
pure_json_space), so the delivery client can read an API's answers
with it. It gains a lenient mode for answers: a member that names no
field is skipped, value and all. Requests read as before.
|
| |
|
|
|
|
|
|
|
|
| |
A wire to an https:// URL speaks TLS 1.2 or later through OpenSSL: the
receiver's certificate must chain to the system's CAs (or to the file
--send-ca names) and name its host (SNI and SSL_set1_host, or the IP
address). An untrusted receiver gets nothing: the send stays in the
outbox, the wire logs it and tries again every 10 s. pure-serve now
links libssl and libcrypto.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve --send OUTPUT=http://host:port/path wires an output: after
every save the owner queues the new sends as JSON (fields and groups)
and a thread per output posts them in order with X-Pure-Source,
X-Pure-Sequence and Idempotency-Key. A 2xx, or a Pure receiver's
refusal, delivers; the owner then trims. 408, 429, 5xx and no answer
are sent again after a pause growing to 10 s, other answers every 10 s:
nothing is dropped. A Pure receiver applies a delivered message with
pure_app_deliver (now taking placed fields) and advertises
X-Pure-Batch: 256; the sender then posts up to that many numbers in a
row as one JSON array (X-Pure-Count), applied in order in one burst
with one save.
Delivery kept up with orders once batched (391 deliveries/s one at a
time; a backlog of 20,000 went at 60k/s). With four clients ordering,
killing the receiver and then the sender (-9) left the receiver's count
and total equal to what the sender saved, five times and under ASan.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An output (input kind 5) is a message the app sends; SEND (72) takes
an output's field values in order in a handler, lays them out as a
message's payload and keeps it, numbered per output from 1, with the
event's changes: a failed event sent nothing, and a failed save takes
back every send since the last save with its numbers. A host sees only
saved sends (pure_app_outbox) and trims what its receiver has saved
(pure_app_trim, gone with the next save). pure_app_deliver applies a
message from a named source once: a number heard before succeeds
without applying, a refused message counts as heard, a failed save
forgets what it heard.
Sends, numbers and heard numbers are the app's links state, stored
whole in one runtime document (PLK1) with the states, in the same
commit. Outputs follow their names across module changes; a module
that drops or retypes an output with sends left is refused, and a
dropped output's number is kept for when it returns. Killing the
process at every commit stage leaves an event and its send, or a
delivery and its effect, together or neither. pure-serve answers a
post to an output or a timer with 404 no_input.
|
| |
|
|
|
|
|
|
| |
One connection thread for each CPU the process may run on (its
affinity, or the online CPUs), less the one the owner of an app with
states takes: three CPUs give two connection threads and the owner.
The default stays one thread.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
After a round that saved, the owner waited for 100 us without a new
request (1 ms at most) before the next round. Requests that queued up
during the save made that quiet start at once, so the clients just
answered missed the next round; the clients then settled into two
halves taking turns, each save carrying half of them. It showed with
two connection threads: 34 events a round and 15.8-16.0k posts/s with
64 connections, against about 61 and 22-24k/s with one or three. The
quiet now counts only once a request has come after the answers were
handed back: two threads carry 61.6 events a round, 22.9-23.9k/s.
|
| |
|
|
|
|
|
| |
Measured with a build that grows its tail by 64 MiB once and never
compacts: durable posts +5-8% (64 connections), p99 about halved; the
slow syncs that remain come from the device.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve no longer copies a JSON body's strings into the message's
payload. The ingress is the prefix, a table of each field's start and
size, and the body as it came; the connection thread scans the body
once, puts numbers into the prefix and records where each string is.
Only a string with escapes is unescaped, over itself (it never grows).
A group of fields comes as an object inside the body and, for a fixed
field, as its path in the query (?to.zip=7). A group where a field
belongs, a value where a group does, an array or true/false/null is
json_type; an unknown member, inside a group too, json_field.
The string scan and UTF-8 checks now pass over eight bytes at a time
while none needs a look (pure_utf8_step, so every text check gains).
On the Pi, posting {"name":"k","text":"x..."} to a stateless app went
from 652-655 to 3,737-3,953 requests/s with 256 KiB of text (server
CPU 1.52 to 0.25 ms a request, peak RSS 5.2 to 3.2-3.4 MiB) and from
29.1k to 76.3-76.7k/s with 4 KiB; the load generator gains a json
method and benchmarks/http/echo.aup is the app.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A message field's name may now be a path of identifiers joined by dots
(user.name, a field inside the group user); a module that names both a
field and a group (user and user.name) is refused, as are empty,
leading, trailing or digit-first segments. A module may have 64
message fields instead of 32 (PureModule 11.9 to 13.2 KiB).
pure_app_message_fields takes each variable field's start and size in
the payload, so a host that parsed a request in place passes the
fields' bytes where they are instead of laying them out after the
prefix. The runtime checks them as before: a field past the payload
fails with limit, a text that is not UTF-8 with encoding.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A tree store commit wrote its bytes over the written zeros of the tail
and its head into a slot at the start of the file: two ranges, and on
the Pi's SD card a sync of two ranges costs about twice one (2.4 ms
against 1.2 ms). A commit now writes a 128-byte record (its head) at
the published end and its bytes right after it, then syncs once. Every
16 commits the head also goes into the older slot. The newest commit
is found from the newer slot's head, or from the newest head this
process has seen in the file while its record is still there, along
the records whose bytes are whole; a torn record or bytes end the walk
at the commit before, which the next writer overwrites. Slots of this
format are PURETRE3; files of the earlier one are read and continued.
Store commits went from 220-233/s to 371-396/s (p50 2.5 to 1.3 ms),
and the scoreboard's durable posts from 13-16k/s to 21-26k/s with 64
connections and from 2.2-2.4k/s to 2.9-3.9k/s with 8, measured
alternately with the previous build. The torn head check now tears a
slot (the walk goes on from the other) and the newest record (a new
process sees the commit before); a new check walks 42 commits from the
slots, in a fresh process too, and stops at a damaged record.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The owner publishes a snapshot of the saved states after every save,
before it hands back the answers; connection threads take the newest
one each round and show views from it with a reader app of their own.
A read made after a write's answer sees the write, nothing unsaved or
undone is ever shown, and reads no longer wait while the owner applies
events or syncs. Apps with states always get an owner thread, even
with one connection thread. A round's inputs go to the owner first;
its views are shown and then answered together (answering each one as
it was shown made one thread's 20-row page 8% slower).
On the Pi 5, with 8 connections posting durable scores and 56 reading
the total, reads went from 10-16k/s at p99 13-29 ms to 107-149k/s at
p99 0.6-0.7 ms, and posts stayed in range; axum and drogon, which sync
under their lock, read 1.1-1.3k/s at p99 111-156 ms. A snapshot of a
changed 1,024-item list costs 1.2 us (65,536 items: 87 us) against a
3-4 ms save.
|
| |
|
|
|
|
|
|
| |
The app suite's last workload, mixed, runs two load generators at once:
8 connections post durable scores while 56 read the total. It shows
whether a server's reads wait while its writes sync. It is reported as
two rows, posts and reads; answers per CPU second count both.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Right after a save, pure_app_snapshot makes a read-only snapshot of the
saved states: numbers are copied, text states are frozen drafts that
share the store mapping (pure_draft_freeze, no copy), and lists are
copies of their item pointers that share the items. An unchanged list
shares the last snapshot's copy. pure_app_read binds a reader app on
another thread to a snapshot so the existing view functions read it.
A shared item never changes again: a list stamps each item with its
share count, copies a shared item it changes, and retires shared items
it lets go instead of freeing them. Snapshots form a chain that goes
oldest first, and items retired while a snapshot was the newest go
with it. References are atomic and may be released on any thread.
test-module checks that each snapshot equals the app, that kept
snapshots stay as made through changes, removals, a module switch, the
app stopping and the store closing, that a failed save's snapshot
equals the one before, and that two threads read the newest snapshots
while the app goes on (also under ASan, UBSan and LeakSanitizer).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
With durable writes going on, a read waited for the save of the burst it
landed in. A round's requests come from different connections, one
request each at a time, so nothing orders them: the owner now applies a
round's reads first and answers them at once, seeing only saved states,
then its writes in bursts. 8 writers and 56 readers of the scoreboard,
owner mode: reads 7,100-7,700 to 16,100-16,600 a second, their p99 55 to
13 ms, writes 1,000-1,100 to 2,300 a second. A single thread keeps the
order: it waits out each save itself, and reads answered first come back
first and split its next writes over more saves (the owner gathers after
a save instead); there reads are 12,800-13,400 a second as before.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
A view that shows a text state as it is (pure_module_text_state) now goes
out as the state's stored extents, as a document read does: the answer
holds the store mapping, charged to the budget, until it is sent and its
ticket released; a state with edits not yet stored is copied as before.
Documents and views share one loan path, and the extent array grows as
needed. A 1 MiB note read by 8 connections: 776-817 answers a second
copied, 1,875-1,901 lent, p50 9.2 to 3.5 ms, peak RSS 10.3 to 5.7 MiB.
The load generator learns GET /views/page.
|
| |
|
|
|
|
|
|
|
|
| |
A sent message belongs to its event's transaction in a runtime-owned
outbox; hosts deliver committed messages in order, and a receiver keeps
the last sequence per source in the same commit, so the effect is exactly
once. The note records the design and its order of work; it waits behind
the roadmap's short-term items and goes with outbound calls, which stand
on the same outbox.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
A real app (a board, a stopwatch) already used a quarter of the 512
instructions, since a fn is inlined where it is called, and every text
it shows is one of 64 constants. The code and the constants now live in
one block sized to the module, which load takes and pure_module_release
gives back; the limits rise to 65,535 instructions and 4,096 constants
of up to 1 MiB, while PureModule shrinks from 28,664 to 11,936 bytes.
The verifier's branch marks are sized to the block instead of 64 KiB on
the stack. pure_module_read reads a module file whole, and every host
uses it instead of a 128 KiB buffer.
|
| |
|
|
|
|
|
|
|
|
| |
The checkpoint's table still said the document and scalar apps were not
one authorable model, and that equivalent-server latency was unproven.
A new section records what has been checked since: one app model (aup to
module format v5), persistence, Vega's surfaces and hot swap, apps as
HTTP services, and the measurements against axum and drogon, with what
remains. The README opens with the same in brief.
|
| |
|
|
|
|
|
|
|
|
| |
pure-serve sets the time an event sees to the wall clock when the event
is applied, and the thread that runs an app with states (the connection
thread alone, or the owner) ticks its timers on time: epoll waits no
longer than the next tick, the owner's condition wait is timed to it, and
the ticks due at once share one save, so a restarted server goes on from
the stored ticks. The statistics count ticks.
|
| |
|
|
|
|
|
|
|
|
| |
An input may be a timer (kind 4, its period of 1 ms to a day in place of
a key name): pure_app_tick delivers a tick, an event saved like a key's.
NOW gives handlers the time the host set in PureApp.now (milliseconds
since 1970, UTC), so a script replays any time and views stay functions
of state; FLOOR and MOD take times apart, MOD taking the divisor's sign as
floor division leaves it, a zero too. pure-run gains time MS and tick NAME.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A message field is now a u32, a number (binary64), bytes or text, and
variable fields may be several and anywhere: the payload's prefix holds
the fixed fields and each variable field's length but the last one's,
which takes the rest; a module whose one bytes field came last is laid
out as before. On arrival a length past the payload, a text that is not
UTF-8 or a number that is not finite fails the message.
pure-serve fills an input's message from a JSON object body
(Content-Type: application/json) on the connection thread before the
request is ready, so the owner never parses and replies are still
borrowed: the ingress is [u32 payload size][payload][the JSON as it
came]. A missing, unknown, doubled or mistyped field, or text that is not
JSON, answers 400 and the request never runs. Numbers may come in the
query too.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The scoreboard, 22 lines of aup served by its views and inputs, against
the same service written by hand in axum and drogon on the Pi: durable
posts about 76 times as many, reads level with drogon and ahead of axum
with the least CPU and memory. The note keeps what the profile showed on
the way (the kernel's share of a single thread, integers through printf,
an owner that saved too early) and what comes next. run.py gains
pure-serve-2 and waits for a server that prints its port before it
listens.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
States have one owner, so a module with them ran on a single thread,
which spent most of its time in the kernel sending and receiving. With
--threads N such a module now gets N connection threads and one owner
thread: connection threads accept, parse and answer; the owner takes the
ready requests of every thread as one round, applies them in bursts with
one save each and hands each thread back its applied requests (eventfd),
which it completes and answers. A request's route and payload stay held
by its thread's runtime core until completion, so an accepted event runs
even when its connection closes first.
After a burst that saved, the owner gathers requests until none has come
for 100 µs (1 ms at most), so the requests its answers bring back share
the next save as they would on one thread: 30 events per save became 50
and the median latency of a durable post fell from 5.4 to 3.5 ms. A page
of 20 rows went from 87,600 to 141,900 answers a second on the Pi.
|
| |
|
|
|
|
|
| |
Each column's key is made once per request and a page's buffer is taken
at once, instead of three small appends per cell. A POST with no payload
(a key press) no longer copies nothing to a missing ingress.
|
| |
|
|
|
|
|
| |
A read's lent extents hold the store mapping until its answer is sent;
with an owner thread running the app and connection threads answering,
the last reference may drop on a thread other than the one that took it.
|
| |
|
|
|
|
|
|
|
| |
pure_format_number, behind every number a text shows (interpolation, the
views pure-serve answers, pure-run), printed integers with
snprintf("%.0f"): hundreds of nanoseconds each on the Pi, most of a page of
20 rows. Integers below 1e15 now take their digits directly (21 ns),
identical to printf on 2.6 million random integers.
|
| |
|
|
|
|
|
|
|
|
| |
run.py --suite app serves the scoreboard of scores.aup (22 lines of aup)
with pure-serve against the same scoreboard written by hand in axum and
drogon: POST /inputs/score?points=N with a name as the body, GET
/views/total and a page of 20 rows as JSON. Durable, the baselines append
each score to a log and fdatasync it before answering (replayed at
start); the load generator learns the three requests.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve answers GET /views/NAME with a view's value (a number or text
as text/plain, a table as JSON rows, a page of them with ?from=&count=,
each cell computed only for the rows asked) and POST /inputs/NAME with an
event: a text input's text is the body, a key input is pressed, a message
takes its u32 fields from the query and its bytes field from the body and
answers its handler's reply. A failed event answers with its fault (404
not_found, 409 exists, 422 the others). No routes are written: the app's
views and inputs are its interface. Documents stay served when the module
has show and edit.
The route of a request is kept per work slot, so an accepted event runs
even if its connection closes before the burst; views and app events join
any burst, whose one save comes before its answers.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A list state may name a key field of its record (the flags word of its
entry: field + 1). A keyed list keeps its items in key order, numbers by
value and texts by their bytes, each key once: LIST_APPEND places an item
by halving and fails with the new fault exists (PURE_CONFLICT) when the
key is there; LIST_FIND gives the index of a key's item and LIST_HAS
whether it is there. The verifier refuses inserting by position and
changing a key. Maps, sets and sorted maps are one list with an order
rule, so tables, filters, folds, indices, storage and replacement stay
as they are.
A module that opens over a list not in its key order (one that was not
keyed, keyed by another field, or stored so) puts it in order, moves its
indices with their items and stores it again; two items with one key
refuse the module rather than drop one.
TEXT_COMPARE compares two texts by their bytes under one of the six
relations.
|
| |
|
|
|
|
|
|
|
| |
Every insert, remove and set computed the item's offset in the draft,
linear in the items before it, even for a list without a draft: building
a list in memory (reading a stored one, migrating a record, or a list that
is not stored) was quadratic. The offset is now found where the draft is
spliced.
|
| |
|
|
|
|
|
|
| |
A fifth fold kind, first, runs its block over the items until one leaves
a non-zero number and pushes that item's index, typed as an index of the
list, or fails not_found when none does. Handlers can so find the item to
change by a condition, as a message naming a task by its title would.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A table view can now carry a filter: its list goes in the byte after the
view's type (list + 1) and its code range is a block run as a column of
the list, leaving one number; the table has a row for each item the block
keeps, in the list's order. Columns read the item (ROW is the item's
position, not the row's). The filter runs once per item after a change of
the app's states, counted by a new number that never goes back (events,
points, a failed save's undo, a module replacement), and its rows serve
every cell until the next change.
pure_app_row_item and pure_app_item_row turn a table's rows into items
and back, so a host shows a focus (an index names an item) on the row it
is in and points at the item a clicked row shows. pure_app_rows now takes
a mutable app.
The checks cover rows, cells, the mapping both ways as items leave and
come back, six refusals, and 20,000 damaged filter modules.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
FOLD (opcode 65) runs the block that follows it once per item of a list,
as a column of that list, and pushes the count of items whose block left
a non-zero number, or the sum, maximum or minimum of the values the block
left with an include flag; an empty maximum or minimum is not_found. The
verifier checks the block as a column (reads only, one or two numbers
left, no fold inside, so the work is bounded by block length times items)
and treats it from outside as one instruction that nothing branches into.
Views and handlers can now say how many tasks are open or what the scores
add up to without keeping a second copy of it in state.
The checks cover values, empty lists, filters, a fold inside arithmetic,
six refusals and 20,000 damaged fold modules run under ASan and UBSan.
|
| |
|
|
|
|
|
|
|
|
|
| |
The zero tail wrote as many bytes as the commits under 64 writers, yet
turning it off changed nothing beyond the SD card's own noise and made
8 writers wait longer, so it stays until faster storage is measured.
Overlapping a sync with the next burst would win a few percent at most.
The second sync of a new process's first commit is rare with one owning
server. The larger lever is write amplification: a 16-byte edit rewrites
its whole leaf and catalog path.
|
| |
|
|
|
|
|
|
|
|
|
| |
Text inserted at a cursor's position pushed every cursor there behind
it. That suits a caret, not a remembered position: a mark, a selection's
start or a bookmark should stay before the new text. A cursor state's
reserved word in the module now carries a flag: 1 makes it a mark, which
stays where text is inserted at it and otherwise follows edits as any
cursor does. The flag is refused on other states and other values;
existing modules, with 0 there, keep their meaning.
|
| |
|
|
|
|
|
|
|
|
|
| |
pure_app_point moves an index state to an item, as a user pointing at a
row would: an event with no handler, saved like a key event, joining the
burst when saving is deferred; past the last item it is not_found. A host
learns which index to move from the view that shows it:
pure_module_position gives the state behind a number view that is just
`at.position`. An interface can so turn a click on a list row into its
focus without any code in the app.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
A read answered with extents of the store mapping was charged nothing,
so clients that stopped reading could keep any number of old file
versions mapped. The runtime core now takes resources a host sends in
pieces: body.data is NULL, body.size is charged like any other resource,
the outcome has no body to read and cannot be forwarded. pure-serve hands
the mapping reference over as such a resource, so a lent answer holds its
bytes of budget until it is in the socket and its ticket releases the
mapping; past the budget a read gets 503 payload_budget, as a copied one
would. The async core grows 52 bytes (5,776 of its 6,144).
|
| |
|
|
|
|
|
|
|
| |
pure-serve renewed a connection's deadline when a request arrived or an
answer finished, not while an answer was being sent, so a client that
read a large answer steadily but slowly was cut off after ten seconds
(a 1 MiB document over a 50 KB/s link never arrived). Every write that
makes progress now renews it; only a stalled connection expires.
|
| |
|
|
|
|
|
|
|
|
|
| |
A burst ended once 16 keys had joined it, so 64 writers needed four
commits a round and some waited through several (p50 17.4 ms). The
pending documents now grow on demand up to 256 keys, which also makes
PureApp 1.3 KB smaller (3,128 bytes). With 64 connections a commit now
carries 26 events on average instead of 12 and the median PUT waits
9.2 ms; throughput moves less, since each commit writes more and the SD
card's syncs take most of the time.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A drogon 1.9 service with the axum baseline's contract joins the benchmark
(memory, fdatasync and crash-safe modes, three IO threads), built from
Debian's packages unpacked under .deps without installing them. drogon
listens once per event loop, so the service picks a free port itself:
with port 0 every loop got a port of its own and the load reached one.
The load generator has one core and saturates near 150k answers a second,
so the harness now also reports answers per second of server CPU time
(all threads). Over three rotated rounds: small GETs 151.6k/s on three
shards (drogon 145.3k, axum 120.9k) and 128k per CPU-second on one thread
(drogon 65k, axum 62k); 1 MiB GETs 2,794/s (axum 1,372, drogon 708);
crash-safe PUTs 5.1-5.8 times axum and 13-30 times drogon; peak RSS
5.0 MiB against 14.4 and 76.5 MiB. --workloads is now a regular expression.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The executor shrinks from 49.8 to 42.2 KB of -O2 text (module.c 34.0 to
27.0 KB) with the module format and API unchanged:
- The verifier reads its rules from the signature table: what each
opcode's argument names, and which text or list it works on. Cursor and
index operands must belong to that text or list and so does a pushed one;
other values belong to nothing, so a value that merely came from a text no
longer blocks a join. The loader and verifier fail through one path.
- The interpreter advances as it fetches and leaves through two exits: a
stated fault, or a change the data refused.
- Key, text and message events take one path; a message brings its base
revision and key. Starting an app opens its module over none, the path
replacing takes, so every new state starts from its stored value.
- Paths that run once per module or app are marked cold and built for size;
event paths stay built for speed (small GETs: 128k/s, as before).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A commit used to sync its nodes, then write the head and sync again: the
order of writes was forced by syncs. Now the head carries where the
commit's bytes start and their digest, and one fdatasync makes the bytes
and the head durable together. A crash during it can leave a head whose
bytes did not all reach the device; its digest then fails and the older
slot stands, which is safe because a writer only builds on a durable head.
Commits overwrite written zeros kept after the published end, so the sync
changes neither the size nor the block map and needs no journal commit.
On the SD card this is 2.4 ms per commit against 9.1 ms for two syncs;
the store commits 289-304 times a second instead of 94-109, and pure-serve
answers 2,012-2,611 durable PUTs a second against 583 for crash-safe axum.
A process remembers the heads it published or checked, so it reads a
commit's bytes at most once and never its own. Its readers pass over the
head it is still making durable and so never answer with a commit a crash
could take back. Head slots get a block each and the format becomes
PURETRE2; a handle's refresh reads the slots through its mapping.
|
| |
|
|
|
|
|
|
| |
MODULE and MODULE_HEADERS were defined below the pure-serve rule, and a
rule's prerequisites expand when make reads it, so pure-serve depended on
neither: changing the store or the executor left a stale server behind.
Define them before any rule names them.
|
| |
|
|
|
|
|
|
|
|
| |
An SD card slows down under sustained syncs, so the order of runs alone
changed write results several times over. The HTTP benchmark now repeats
rounds with the servers in a rotated order, gives every run a fresh store,
times the device (an in-place 4 KiB write + fdatasync) before each write
workload and reports medians over rounds with the probe range beside them.
--workloads picks workloads by label prefix.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
A read of a stored document now answers with the store mapping's extents
through writev, without copying the document. pure_draft_mapping takes a
reference to the mapping a draft opened through a handle reads from; the
answer keeps it until the last byte is sent, so the version read goes out
intact even if the document changes or a compaction replaces the file
meanwhile. Replies the mapping does not hold are copied as before.
pure-serve's output is now a list of extents (the head first). One thread
serves 2,761 1 MiB reads per second (axum: 1,380) and 128k small ones.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Threads that commit at the same time now share one commit: they queue,
the first becomes the leader and publishes every queued request of the
same store with one head, and the others wait for its result. Each
request stays all or nothing on its own: one with a stale draft is left
out, and so is one whose key an earlier request of the group changes.
When several threads have been committing together, a leader waits up to
2 ms for as many to join as groups held lately; a single thread never
waits.
A test pauses a publication and checks that five commits make exactly two
publications. Under eight writers on three pure-serve shards, 1,234
bursts went out in 421 publications. pure_store_group_counts reports the
groups, and pure-serve prints them.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A burst now looks at the store once (pure_store_pin) instead of once per
request; another shard's commit ends before its answer is sent, so a
request made after that answer still reads it. With this alone one thread
serves 125k small keep-alive reads per second, as many as axum on three
cores, with a shorter p99.
--threads N runs a shard per thread: its own SO_REUSEPORT listener, epoll
set, runtime domain, app and store handle. Writer sessions of the tree
store now take turns inside a process with a mutex (the file lock belongs
to the process and does not keep threads apart), and revisions settle
concurrent edits of a key. A module with states runs on one thread. Three
shards serve 151k reads per second; they still commit one by one, so
durable writes and 1 MiB reads are better on one thread for now, which
stays the default. Note 075 records the numbers.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
pure-serve replaces the module mode of server.c, which goes back to what
it was. It keeps the document contract, the runtime core (a work slot per
connection, the payload budget, core-owned ingress, borrowed PUT replies)
and the group commits, and adds what server.c lacked: HTTP/1.1 persistent
connections with in-order pipelining, an edge-triggered epoll owner
thread, up to 1,024 connections allocated as they arrive and reused, one
writev per answer, and documents opened through a store handle.
On the Pi 5 with one thread it serves 106k small keep-alive reads per
second (axum on three cores: 126k) at the same p99, 1,741 1 MiB reads
(axum: 1,370), and 1,126 durable writes from 64 connections (crash-safe
axum: 372), in 7.3 MiB against axum's 17-20 MiB. Note 075 has the table.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Opening a draft by path opened the directory and the file, mapped the
whole file, read the head, and undid all of it on close: nine system
calls for every GET. pure_store_open keeps the store open for one owner
thread; drafts opened through it share a reference-counted mapping of the
newest version, and opening one only checks whether the file changed (its
identity and its head: two system calls). A commit of its drafts maps the
new version once and moves them all onto it. A compaction by another
writer is noticed and followed, while drafts of the old file keep reading
it and relocate their extents when they commit.
pure_store_data gives modules the same through a handle.
|
