| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Commits grew the tree store's tail 256 KiB at a time, each growth a sync
that also committed the ext4 journal for the new size and blocks. A file
installed by compaction now carries written zeros up to where the next
compaction will come (4 x live + slack, plus a commit's margin), capped at
the size of the file it replaces, so a store never takes more space than
before and shrinks when its live data does. The cycle's commits then only
overwrite blocks the file has.
Store commits: 317.5 -> 385.5 a second, ext4 journal commits 109 -> 45 for
4,000 commits; durable app posts +5.3%, their p99 the same within this SD
card's noise. The new cycles test keeps sizes still between compactions
once the live data is whole, and shows the file shrinking with it.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Nothing a thread can give: with writes saturated the device's one line of
syncs is the bottleneck, so zeroing the tail or compacting on another
thread only queues that device work behind the commits, and the owner's
idle gap is shorter than one journal commit. Built without compaction the
store is no faster; without compaction and without tail growth durable
posts gain 1.4% and the p99 drops 24% (17.4 to 13.3 ms). New blocks are the
tail latency, so the way to it is a store that rewrites its space in place,
a ring cleaned forward instead of compacted into new files.
|
| |
|
|
|
|
|
|
|
|
|
| |
It would not, on this device. One file's syncs or four at once, the SD card
completes the same 510-690 commits per second, the latency growing with the
number of files. The scoreboard app with two owners, each with a store of
its own, took 23% fewer durable posts than one owner (median 16,230 against
20,974 a second) with 2-3x the p99: syncs queue, and each carries half the
events. One owner's group commit stays; cores serve reads, and writes grow
by making the sync itself cheaper.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The app model, the tree store and the HTTP host lived in
prototypes/connections among the earlier document runtime, the store was
known only by its format's name, and Vega compiled six Pure sources by path.
Now, as the README's principle has it:
core/ runtime, flow and the app model (module*.c), with what the core
asks of storage (state.h, store.h)
store/ the tree store (PURETRE3, PUREREC1)
serve/ pure-serve: HTTP host, outbound delivery, JSON in place
cli/ pure-run, pure-edit-module
A top-level make builds libpure.a (app model and store, for hosts to link)
and the hosts; make check runs the store's and the app model's tests. The
earlier runtime still builds from prototypes/connections against core/ and
store/ (check, check-tree and check-core pass). Design 009 records the rule:
parts by role behind libraries; git repositories split by coupling, so the
store stays here while its contract moves with the core, and Vega stays apart.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A tree store commit wrote its bytes over the written zeros of the tail
and its head into a slot at the start of the file: two ranges, and on
the Pi's SD card a sync of two ranges costs about twice one (2.4 ms
against 1.2 ms). A commit now writes a 128-byte record (its head) at
the published end and its bytes right after it, then syncs once. Every
16 commits the head also goes into the older slot. The newest commit
is found from the newer slot's head, or from the newest head this
process has seen in the file while its record is still there, along
the records whose bytes are whole; a torn record or bytes end the walk
at the commit before, which the next writer overwrites. Slots of this
format are PURETRE3; files of the earlier one are read and continued.
Store commits went from 220-233/s to 371-396/s (p50 2.5 to 1.3 ms),
and the scoreboard's durable posts from 13-16k/s to 21-26k/s with 64
connections and from 2.2-2.4k/s to 2.9-3.9k/s with 8, measured
alternately with the previous build. The torn head check now tears a
slot (the walk goes on from the other) and the newest record (a new
process sees the commit before); a new check walks 42 commits from the
slots, in a fresh process too, and stops at a damaged record.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A drogon 1.9 service with the axum baseline's contract joins the benchmark
(memory, fdatasync and crash-safe modes, three IO threads), built from
Debian's packages unpacked under .deps without installing them. drogon
listens once per event loop, so the service picks a free port itself:
with port 0 every loop got a port of its own and the load reached one.
The load generator has one core and saturates near 150k answers a second,
so the harness now also reports answers per second of server CPU time
(all threads). Over three rotated rounds: small GETs 151.6k/s on three
shards (drogon 145.3k, axum 120.9k) and 128k per CPU-second on one thread
(drogon 65k, axum 62k); 1 MiB GETs 2,794/s (axum 1,372, drogon 708);
crash-safe PUTs 5.1-5.8 times axum and 13-30 times drogon; peak RSS
5.0 MiB against 14.4 and 76.5 MiB. --workloads is now a regular expression.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A commit used to sync its nodes, then write the head and sync again: the
order of writes was forced by syncs. Now the head carries where the
commit's bytes start and their digest, and one fdatasync makes the bytes
and the head durable together. A crash during it can leave a head whose
bytes did not all reach the device; its digest then fails and the older
slot stands, which is safe because a writer only builds on a durable head.
Commits overwrite written zeros kept after the published end, so the sync
changes neither the size nor the block map and needs no journal commit.
On the SD card this is 2.4 ms per commit against 9.1 ms for two syncs;
the store commits 289-304 times a second instead of 94-109, and pure-serve
answers 2,012-2,611 durable PUTs a second against 583 for crash-safe axum.
A process remembers the heads it published or checked, so it reads a
commit's bytes at most once and never its own. Its readers pass over the
head it is still making durable and so never answer with a commit a crash
could take back. Head slots get a block each and the format becomes
PURETRE2; a handle's refresh reads the slots through its mapping.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
state.h gives any app a runtime-owned working version of a stored value on
the tree store. A draft shares unchanged extents with the stored version,
edits in memory by path copying, lends ordered extents without copying, and
commits only its new nodes when the key is still at its base revision. A
failed commit changes neither the draft nor the store; a draft whose file was
replaced by compaction copies its stored extents on commit. Unreachable
arena nodes are reclaimed once they outweigh max(reachable, 32 KiB).
Typing into the middle of a 1 MiB document and saving each character peaked
at about 1,660 KiB RSS with a draft against about 4,480 KiB for an app copy
compared with storage on save; saves were only 1-2 ms faster. This corrects
073, which had attributed the TUI's remaining 8 ms to that comparison.
Drafts are not yet charged to the runtime budget or visible to .pure
definitions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
store-tree.c keeps every document in one append-only file as a persistent
B-tree of byte extents, with the key catalog in the same structure. A splice
writes only the changed leaf and path; unchanged extents are shared between
versions, readers take no lock, and two alternating head slots publish a
commit after its nodes are durable. Compaction rewrites reachable nodes.
On the recorded ext4 host a durable 1-byte middle insert into a 1 MiB
document took 8.58 ms against 78.51 ms for the default store, and the real
TUI's Ctrl+S after typing took 16.60 ms against 84.59 ms. check-tree runs the
store's crash/oracle tests and the unchanged runtime, splice, async and
real-PTY editor checks on it. It stays opt-in: linked code is 12.7 KB larger
and contiguous multi-leaf reads copy.
|
| | |
|
| |
|
