aboutsummaryrefslogtreecommitdiffstats
path: root/docs/research
Commit message (Collapse)AuthorAgeFilesLines
* Say what one sending loop showed (017 2b, research 113)eunhhu2026-10-011-0/+107
| | | | | | | The loop alone slowed four links to half: woken by every record queued, it chased the owner a record at a time. Pushing once a turn made links 1.6 times as fast, with threads or the loop alike; the loop's own cost is within the noise. Twenty outputs now add two threads, not 21.
* Measure sending between two apps of a host: local outlet or link (112)eunhhu2026-10-011-0/+56
| | | | | | | | | | Two apps of one host, src sending one message for each of 1,000 items in a burst event, 2,000 bursts a run, five rounds with the ways in turns: the local outlet stores 1,141,642 sends a second at 1.17 us of the host's CPU each, a loopback pure:// link to the host's own port 578,879 at 2.93 us (1.97 times the sends, 0.40 times the CPU). 017's second stage asked for at least the link's rate at half its CPU or less: it goes in. Every run checks dst's sum: each send came once.
* Measure one host of many apps against a process each (111)eunhhu2026-10-011-0/+109
| | | | | | | | | | | | | | | | | | | | 017's first stage met its four criteria on the Pi (RAM stores, five rounds, the order of the ways turned): app B's 99th percentile while app A is flooded by 64 connections is 168 us, half of two processes' 336 us; the two apps busy together write 146.1k a second against 146.2k, with the same 1.4 ms tail; an app costs a host 120 KiB of its own memory (a process: 1,311 KiB PSS) and 100 apps still run on three threads; a host of one app keeps its speed (TPC-C +1.2% at the median of six pairs, two of them interfered and kept; the bank's queries within 2.6%) and its instructions (+0.94% at most). On the way: apps hopped between workers each turn (fixed in db4ee9c), connections of one app spread over all connection threads made the host a third more expensive a request and its tail a kernel tick (fixed by home threads, d24881d), and the work budget's counting cost every table row 26 instructions (fixed in be297ea). The earlier runs are kept apart (db4ee9c's isolation, d24881d's pairs and counts, the interrupted first try).
* Measure 07047c3 again on a quiet Pi (110)eunhhu2026-09-301-3/+20
| | | | | | | | | | The six pairs rerun once the Pi had been quiet for a minute, on the same binaries. Two pairs were still interfered (the first's before at an owner share of 0.54, the sixth at 0.73 on both sides); the four clean ones are -1.7%, -0.4%, +1.8% and +1.1%, CPU per transaction 43.25 to 42.9 us. The bank's queries are as fast or faster after, though the interference fell more on the before runs; the instruction counts stay the comparison to trust.
* Count instructions where timing cannot tell 3% apart (110)eunhhu2026-09-301-0/+104
| | | | | | | | | | | | | | | | | 016's second stage passed its criterion: six interleaved pairs against ee21aec gave TPC-C (RAM, W=1) -0.6% at the median and the bank's queries 0.0% to +2.2%, no run interfered. The ranks query was slower all six times, and chasing it showed that on this Pi timing cannot separate 1-3%: two servers asked in turns disagree with themselves (ee21aec to e02bfc9 +1.6%, e02bfc9 to 481642f -1.2%, ee21aec to 481642f +2.8%), and a query whose instructions grew 0.6% ran 3.5% faster. Instructions do not move so: pcount.c counts the server's user-mode instructions with perf_event_open (no perf here), and psample.c samples where they go. ranks had grown 0.4% in instructions (the rest was where code and data landed), per_branch 3.6% and tables 20 instructions a row, which 07047c3 halved. The six pairs of 07047c3 were interfered (another session's build, load 4-12) and are kept apart, unused.
* Say what 109 inferred and what it did not measureeunhhu2026-09-301-1/+3
|
* Say that prefetching items did not pay, and why (109)eunhhu2026-09-301-0/+57
| | | | | | | | | Fetching a found item's cells, a leaf's keys and each level's keys ahead made finding and reading items 20% faster on 100,000 items in a micro benchmark, but TPC-C's clean pairs moved 1% at most and finding alone got 48% slower, so it is not in. The owner's finding samples were the descent, not the fields: a write right after a read copies the leaf whole anyway, and what costs is walking structures larger than the caches at random.
* Say what finding items should gain, before measuring it (109)eunhhu2026-09-301-0/+34
|
* Say what the screen showed of a value, and what Vega now keeps from acting ↵eunhhu2026-09-301-2/+21
| | | | (015, 108)
* Say what a value means at each boundary, and what checking it found (015, 108)eunhhu2026-09-301-0/+45
|
* Say what pinning a value across every boundary checks, before writing the ↵eunhhu2026-09-301-0/+37
| | | | check (108)
* Say TPC-C's RAM numbers since snapshot nodes are reusedeunhhu2026-09-301-0/+1
|
* Say what snapshots cost the owner, and what reusing their nodes gave (107)eunhhu2026-09-301-0/+141
| | | | | | | | | Most of it was the allocator: leaves past glibc's thread cache, freed by whichever thread let the last snapshot go. jemalloc was 11-14% faster at more memory; reusing the nodes in Pure gave a median +9.4% over six pairs, 4-8% less CPU a transaction in all six, and 1.97x the state bytes resident where it was 2.18x. TPC-C in RAM is now 4.0x a tuned SQLite at W=1 and 3.5x at W=4; finding items is the owner's largest part now.
* Compare TPC-C on the SD card with SQLite committing in groups, not each ↵eunhhu2026-09-301-0/+2
| | | | | | | | transaction The 90x of 092 was against SQLite's defaults committing each transaction; tuned for the card and grouped as Pure groups, SQLite is 4-11x behind, the bytes it writes a transaction being the difference.
* Say what TPC-C costs when SQLite commits in groups too, and with four ↵eunhhu2026-09-301-2/+175
| | | | | | | | | | | | | warehouses (106) SQLite tuned first, by the criteria committed before (and, beyond them, checkpoints each 40 MB of WAL, in its favour). On the SD card Pure stays 4.2-10.8x ahead at equal terminals, SQLite writing 8-18x the bytes a transaction; in RAM 3.2-3.5x. All tables equal at W=1 and W=4, and every run consistent. In RAM the one owner is the limit (0.94 of a core at W=4, no gain from 16 to 64 terminals), a third of its busy time copying nodes the snapshots share; checkpoints and memory are within their criteria.
* Say what the TPC-C comparison decides, before running it (106)eunhhu2026-09-301-0/+44
|
* Say where the shared board now lives, and what is left (105)eunhhu2026-09-301-1/+9
| | | | | | | | | The board is Vega's examples/shared-board.aup, and Vega's shared_board check runs its two nodes: changes on either agree, and a partition healed agrees on the later change. Reading a named field list once a delivery would save part of 310 ns a record during upgrades alone: not worth an API. What review 004 asked that is left is 103's third item, broader performance comparisons under conditions stated first.
* Say that sends and applies now spread their fields (105)eunhhu2026-09-301-4/+8
| | | | | | | | | aup spreads an item or a message into a send or a record, and sets an item whole (c69682e): the board sends { ..at } and applies cards[k] = Card { ..remote }, and a new field is written in two places, the record and the output. v6 on both nodes, cut apart and joined again, agrees on the later change; its module is byte for byte the one with every field written.
* Say what laying a record out anew costs (014)eunhhu2026-09-301-1/+1
| | | | | | | A board record of five fields (38 B) sent with v3's output fields took 310 ns to lay out for v4's input, reading the field list each time. Only streams of another shape pay it, and only while nodes are upgraded one after the other: less than a delivery's own work.
* Say how growing a board two nodes share went (105)eunhhu2026-09-301-0/+119
| | | | | | | | | | | | | | | | | | 103's next step: grow a real app, adding and reordering fields, cutting the connection, starting again, and write down what gets in the way. Two nodes share a board (from Vega's examples/board.aup), each sending its changes to the other, the later change winning. Upgrading them one after the other deadlocked: fixed by recasting waiting sends by name (9aa123e) and taking fields by name (014, 5ce7655), and under it a link that stopped for good when a refused stream's try was lost (baaf955). Now the older node's changes reach the newer one during the upgrade, the newer one's new field waits for the older, and the boards agree a second after. Fields in another order agree at once; a partition healed agrees on the later change; both nodes started again go on. Two rough edges eased on the way (the node's name, message like in aup); sends and applies are still written field by field. CHECKPOINT's row for links says so.
* Say what the windows and checkpoints cost (013 steps 2 and 3, 104)eunhhu2026-09-301-1/+41
| | | | | | | | | | | | | | | | | | 013's second step is in (d9e0b68): each transport holds a window of its output's sends. 104 measures it with the receivers down: once the window is full a node grows by the outbox alone, 219 and 218 B a send over the link and HTTP (399 and 534 B before). The third step is a measurement. A checkpoint holds one list's snapshot while it writes the image, so what it adds is what the app changed meanwhile: nothing visible on tmpfs or for a list that fits the card's fast writes; on the SD card, a 22.4 MB list written for about 8 s while 1,400 items a second were set left +3.0 and +4.4 MiB. The highest resident memory was twice the state bytes, inside 013's bound (the state and one snapshot) and its stop criterion (3x). 021's run.py takes --items, --stores (a directory on the device to measure) and --label, and reports the peak (VmHWM).
* Say where a node's memory goes, and the budget it now keeps (104, 013)eunhhu2026-09-302-0/+111
| | | | | | | | | | | | | | | | | | 104 measures a node's memory under a receiver that is down and under a snapshot held for long (021's run.py: resident memory against what was posted, 30 s a case). A held snapshot grew the node without end (6.4 to 678 MiB, fixed in d1e8f38); a slow receiver grows it by what was sent, 221 B a send, which is state until delivered, so the budget bounds state and refuses past it. 013 is that budget's design: what it counts (state bytes, deterministic), what an event over it does, why an answer takes its send's room, how links and HTTP wait, what it does not count and those bounds, and the checks. With --memory 4000000 the same load fills it in about 4 s and the node stays at 9.6-9.8 MiB (45.9 MiB and growing without). 012 records the hole the budget found in the link (the full flag), and 103 that its clock fix fell short (385754c). CHECKPOINT gains the row.
* Say how review 004's seven defects went (103)eunhhu2026-09-291-0/+61
| | | | | | | | | | | | Three broke what data means (a walk's range missed the indices an insert moves; an uncertain save of numbers alone let the next save through; an output whose fields swapped names kept its waiting sends), four were errors (a stateless timer never ticked, Vega's terminal lost a key cut across reads, an input named edit forced documents, timers waited on the wall clock). Each is fixed with a check that fails on the code before and passes now, in Pure's and Vega's make check. The review's other advice (a whole app's memory budget, one real app grown, comparisons with their conditions set first) is what comes next.
* Say what adopting the link took, and how it fares against ngtcp2 (102)eunhhu2026-09-291-0/+253
| | | | | | | | | | | | | | | | | | 102: the spec (012) and a receiver written from it alone; the four holes writing it found (REFUSED, the record's size, a peer's run places, the memo's places beside) and the credit M6 showed fixed at 1,024 records; M6 with ngtcp2 as a third, two rounds. Under reordering Pure carries the most (45 Mbit/s; ngtcp2 29-47, quinn 1-2), under random loss the three are the same, on a clean bottleneck Pure carries 3-5 % less (its framing, some 112 B a packet to QUIC's 41), and where loss and reordering meet ngtcp2 carries 1.3-3.6 times as much: undoing a loss it took for reordering, it restores the window a real loss had cut too (its log: 22 of 34 undos past the window before the last cut; none without reordering). Shares beside TCP stay within 25 %. 011 and the status table say so; 012 also says that a receiver answers within 25 ms (a sender's PTO counts on it), that a shape and a record's bytes are the app's, and why a packet of the past does no harm.
* Say how the four limits left went: three solved, one measured away (101)eunhhu2026-09-292-1/+146
| | | | | | | | | A path MTU black hole is seen and the link goes back to 1,200 bytes; a receiver that lost its memo says LOST and M2 without a memo catches up in 0.4-0.5 s, not 7.7; M5 stands beside ngtcp2 too (Pure 25 % lighter than both QUIC from 1 KiB, 1.04 of ngtcp2 and 0.90 of quinn at 61 B); and the stream's name in every packet, 096's guess for 61 B's gap, cost nothing measurable, so it stays. 098 notes the one line LOST changes.
* Say what a small record costs a link now, beside quinn in the same rounds ↵eunhhu2026-09-292-2/+10
| | | | | | | | | (096, 100) 61-byte records: 0.287 us a record against quinn's 0.31 in the same five rounds (0.93 of it); 096's 1.06-1.12 was against quinn at 0.27, which moves between 0.27 and 0.32 from day to day. H4 is no longer at the edge.
* Say that a receiver moved to another machine is followed, and how (100)eunhhu2026-09-291-0/+125
| | | | | | | | | | | | | H3 of design 011 stands: with ten thousand devices sending 2,000 records a second, the node's store, key and memo carried to another address, nothing stopped for a 100 ms step, the moved node made no X25519, and every record was applied once. Each device's first record after the move waits one probe (30-45 ms here), as its first packet finds no way there. The search found the listener answering from an address other than the one a packet came to, and a link that looked for its receiver after its probe went, losing every answer. H1-H4 are all measured now; 61 B's CPU a delivery is at the edge of the criterion, to be looked at before step 5 (adoption and a specification).
* Say what epochs cost a link, and what the search for it found (099)eunhhu2026-09-292-2/+220
| | | | | | | | | | | | | | Forward secrecy, 098's first task left: keys that leak later open only epoch 0. M2 after the epoch began and after a restart is as before it (p50 0.4 ms, p99 0.8 ms, the same CPU, no X25519 on restart). It costs a p50 of 0.7-1.0 ms more while ten thousand devices first meet and for about 40 s of accepts after, and a receiver that lost its memo now stalls 0.8 s and catches up in 7.7 s, not 0.2-0.7: it lost the epochs, and each sender opens one again. The search also found a packet more a device from each of M6's path probe and the security checks' challenge, and a load generator whose own pass over its devices was the floor of the latency it measured. 098's note that epochs narrow KCI to the first epoch was wrong; corrected.
* Say what an attacker on the path could do to a link, and what now stops it (098)eunhhu2026-09-291-0/+106
|
* Measure a link on paths of loss, reordering and a bottleneck, against quinn ↵eunhhu2026-09-292-0/+212
| | | | | | | | | | (097) M6 of design 011: with a 50 Mbit/s bottleneck, a link carries 21-41 times quinn's goodput when packets reorder, the same under random loss, and 1.3-3.5 times with both; its p99 at 50 records a second is the same or shorter; beside TCP CUBIC it takes 0.48-0.55 of the path, inside the guardrail. Only on a clean path is it 3-5% below quinn.
* Put the links and the crypto they use in link/, a part of their owneunhhu2026-09-292-2/+2
| | | | | libpure-link.a holds them; pure-serve and the M5 harness link it instead of naming its source files.
* Measure the drains again with the outbox as a list (096)eunhhu2026-09-291-17/+30
| | | | | | | Between two pure-serve nodes, 16 KiB records drain at 33,600 a second (23,000 before), the links spending 58 us a record (76), under quinn's 60 while the receiver also stores each one. 1,013 bytes: 406,000 a second (347,000). Saving sends one by one: 5,000 of 16 KiB in 0.91 s (84.8).
* Measure the CPU a record costs a link, against quinn (096)eunhhu2026-09-291-0/+122
| | | | | | | | | | | | | | | | | | | | | M5 of design 011. At first a link spent eight times quinn's CPU on a small record: scans of the backlog, a system call and allocations a record, 19 bytes of framing. After the seven changes before this one, the transport alone (transport.c; a backlog drained, every record checked) spends per record 0.30 us for 61 bytes (quinn 0.27: 1.06 to 1.12 times over three campaigns, at the design's 10 % line), 2.91 us for 1,013 bytes (quinn 3.88) and 45.2 us for 16,013 bytes (quinn 60.9): 25 % less for the larger ones, and 30 % more records a second. What is left for small records is per packet: the stream's names and the RECORDS head. quinn's numbers carry no confirmation of storing and no exactly once. Between two pure-serve nodes the links spend more per record, woken at the receiving app's pace. Filling the sender showed the outbox is written whole at every save, as the numbers heard were (094): the next thing to change. run.py gains --transport and --drain; the quinn service reads a drain a chunk at a time and its client writes 64 KiB at a time.
* Measure a restart under load and a first contact (095)eunhhu2026-09-282-2/+133
| | | | | | | | | | | | | | | | | | M2 and M4 of design 011. Ten thousand devices send an order every five seconds (2,000 a second); the receiver is killed (-9) at about 21 s and started again at once. With its secrets remembered the node shows no 100 ms step without an order stored, catches up in 0.2 s, spends no more CPU than before and makes no X25519; 34 orders were sent again, and every order made was applied once (three rounds; on the SD card the same, with 0.2 s silent). Without the memo it makes 10,000 X25519, 1.6 s of CPU. quinn under the same load stops for 1.0 s and catches up after 10.9 s: 10,115 new connections, about 7 s more of the service's CPU and 8 s of the clients'. A first contact takes Pure one round trip and QUIC two, at 10, 50 and 100 ms of RTT. run.py --restart and --first-contact make them; the quinn harness gains a fixed port with a reset key from its key, clients that connect again, and a drain for M5.
* Measure a link at rest: 132 bytes against a QUIC connection's 38,106 (094)eunhhu2026-09-281-0/+142
| | | | | | | | | | | | | | | | | M1 of design 011. A Pure node allows 1,000 to 1,000,000 senders; each sends one order and, once told it is stored, rests. At rest a link costs the node 132 bytes (the sender allowed, 73; its number heard, about 60) and no transport state; with a million the node runs in 139 MB. A quinn 0.11 service holding the same exchange's clients connected at rest costs 38,106 bytes a connection: 289 times as much, past the design's 1/100. A link at rest takes its next record in 0.56 ms to stored, one X25519 on the node included; a new QUIC connection takes 1.06 ms and 2.8 times the receiver's CPU, without storing. research/experiments/019-idle-links: links.c, many senders written from serve/link.h's packet format alone (a second implementation), and the quinn service and clients (quic/, built offline); run.py makes both measurements.
* Link two nodes without a connection over UDP (093, design 011's first stage)eunhhu2026-09-281-0/+84
| | | | | | | | | | | | | | | | | | | | | pure-serve --send OUTPUT=pure://host:port/INPUT sends an output's saved sends to another node's --link-port with no handshake: the first packet carries records. Keys come from the two nodes' keys (X25519, HKDF-SHA256, AES-128-GCM), with a random run in each packet's header so no nonce repeats across restarts; nothing is stored for them. What lasts is the apps' own: the sender's outbox and the receiver's heard numbers (pure_app_heard, new). The sender detects losses as RFC 9002 says, with NewReno, and trims a send once the receiver says it stored it; the receiver hands the owner whole records in order, which it applies from the one after the source's last heard number, stopping at a gap, and answers how far it stored with every acknowledgment. Stored bytes are the bytes on the wire: an output and an input of one shape need no JSON. server_links_udp kills the receiver, the sender, then both (-9) while four clients buy: every saved order is applied once, with no handshake (five runs, 10k-11k orders each). server_links_udp_lossy does it again in a namespace of its own with the link's packets delayed, lost (3 %) and reordered (10 %) by netem (three runs, 11k-12k orders).
* Measure TPC-C again with compact log recordseunhhu2026-09-281-19/+24
|
* Check TPC-C's consistency from one snapshot a side, and measure TPC-C on RAM ↵eunhhu2026-09-281-0/+187
| | | | and an SD card
* Record queries on the bank beside SQLite (091)eunhhu2026-09-281-0/+110
| | | | | | | | | | | | 091 measures four queries on pgbench's tables of 100,000 accounts, the same rows and answers on Pure (registered queries, HTTP and JSON included) and SQLite 3.46.1, through each step: items field by field in leaves, folds and filters a leaf at a time, and one pass answering a fold in a column for every row. Per-branch totals went from 398 ms to 4.9 ms (SQLite 45, 14.3 with a DBA's indexes), a three-table join from 3.3 to 0.48 ms (1.47), and each rich account's rank from 9.2 s to 6.8 ms (2.84 s, 7.6 ms with an index); counting accounts in credit takes 1.05 ms where an index lets SQLite count 0.16 ms. Pure declares no index.
* Run pgbench's transaction on a bank of 100,000 accounts, beside SQLite (090)eunhhu2026-09-271-0/+99
| | | | | | | | | | | | | | | | | benchmarks/bank/bank.aup keeps accounts, tellers and branches as keyed lists and a history list; a transfer changes a row of each and appends a history row in one event, as pgbench's TPC-B-like transaction does in SQL. run.py makes a scale's rows and drives transfers through the load generator's new transfer kind; sqlite-bank.c runs the same tables and SQL on SQLite 3.46.1, a connection a thread, WAL. With lists as trees, transfers run at 30.5k a second on 1,000 accounts and 24.5k on 100,000 (RAM store, one connection thread, HTTP and JSON included), where they ran at 3.2k on 30,000 before. Durable on the SD card, Pure makes 1,339 a second and SQLite with synchronous=FULL 84 to 153. SQLite in process with no HTTP makes about 40k in RAM: recorded as it is, with the write amplification of 100,000-row tables on the card and ad hoc queries as what comes next.
* Record the messenger between two nodes and its latency (089)eunhhu2026-09-271-0/+80
| | | | | | | | | | | | | | The roadmap's E2EE messenger made of the runtime's general parts, with no messenger code in it: sealed, authenticated sessions between nodes (086, 088), Vega on each node's display as a surface (design 010), the app in Vega's examples. Vega's check runs two nodes through a recording relay: a line typed on one display is drawn on the other, frames match the vega command's, and the relay sees no text. A line takes 0.79 ms at p50 and 1.1 ms at p99 from one node's program to the other node's views with stores in RAM, both commits, the HTTP delivery and the seal included (sealing adds nothing measurable); on the Pi's SD card 3.5 ms at p50, the long tail being the card's syncs.
* Seal in one session a wire, authenticated by the sender's key (088)eunhhu2026-09-271-0/+83
| | | | | | | | | | | | | | | | | | | | | | A sealed request cost three X25519 operations (about 0.5 ms), so sealed sends drained at half the plain rate. A wire now keeps one HPKE context, encapsulated at its first seal, and seals each request as the context's next number (a failed seal or a restart starts a new session); the header carries the session's encapsulated key and the number, so a receiver opens a session once per connection thread (16 remembered, the least used replaced) and each seal by its number, also after it restarts, unasked. A sender with a key of its own seals in HPKE auth mode and names that key in the header; a receiver started with --sender NAME=KEY takes deliveries only from the senders it names, each sealed with its key (403 sender otherwise). The body is now the ciphertext and its tag. Sealed sends, sender authenticated, drain at 461-464k a second against 462-470k plain and 207-234k sealed a request at a time (Raspberry Pi 5). The check adds: a wire's requests share one session numbered from 0; a changed number or sender key does not open; a receiver restarted naming its sender opens the same session again; another node claiming that sender's name with its own key is refused and applies nothing.
* Record the small reads and idle memory work, next to axum and drogon (087)eunhhu2026-09-271-0/+120
| | | | | | | | | | | | What the app model, snapshot reads and delivery had cost small reads (126.7k -> 116.1k GETs a second) and idle memory (2.0 -> 4.5 MiB), how the sampler found it, and the four steps and the crypto split that took it back: 139.9k and 2.3 MiB. Measured again today beside axum 0.8.9 and drogon 1.9.0 on the same Pi: a one-thread pure-serve answers small GETs at 139k/s (2.2x axum and drogon a CPU second), 1 MiB GETs at 2,362/s (axum 1,304, drogon 754), durable app writes at 21.3k/s (both about 200), with the least idle memory of the three.
* Seal delivered sends to the receiver's key (086)eunhhu2026-09-271-0/+96
| | | | | | | | | | | | | | | | | | | | --seal OUTPUT=KEY seals each request's body whole to the receiving node's public key with HPKE (RFC 9180, base mode: X25519, HKDF-SHA256, AES-128-GCM where the CPU has AES, else ChaCha20-Poly1305), bound to the request's source, first number and count; X-Pure-Seal names the suite. A receiver's --key is a raw 32-byte file, made 0600 when missing, held in OpenSSL's secure heap; it opens the body where it lies (400 seal when it does not open) and zeroes the plain text when the request's ticket goes. --sealed-only refuses unsealed deliveries (403 unsealed). The check puts a recording relay between two apps: every order applies once, no body shows an item or a field name, a flipped bit is refused, a replayed request applies nothing twice, and a seal to another key does not open. Its hosts choose their own ports (--port 0). Sealed sends drain at 233-235k a second against 469-470k plain (Raspberry Pi 5): three X25519 operations a request, 164 us each. The plain request path measured the same as before.
* Give a compacted file the zeros of its whole cycle (085)eunhhu2026-09-271-0/+62
| | | | | | | | | | | | | | | | Commits grew the tree store's tail 256 KiB at a time, each growth a sync that also committed the ext4 journal for the new size and blocks. A file installed by compaction now carries written zeros up to where the next compaction will come (4 x live + slack, plus a commit's margin), capped at the size of the file it replaces, so a store never takes more space than before and shrinks when its live data does. The cycle's commits then only overwrite blocks the file has. Store commits: 317.5 -> 385.5 a second, ext4 journal commits 109 -> 45 for 4,000 commits; durable app posts +5.3%, their p99 the same within this SD card's noise. The new cycles test keeps sizes still between compactions once the live data is whole, and shows the file shrinking with it.
* Measure what moving saves off the owner would buy (084)eunhhu2026-09-271-2/+30
| | | | | | | | | | | | Nothing a thread can give: with writes saturated the device's one line of syncs is the bottleneck, so zeroing the tail or compacting on another thread only queues that device work behind the commits, and the owner's idle gap is shorter than one journal commit. Built without compaction the store is no faster; without compaction and without tail growth durable posts gain 1.4% and the p99 drops 24% (17.4 to 13.3 ms). New blocks are the tail latency, so the way to it is a store that rewrites its space in place, a ring cleaned forward instead of compacted into new files.
* Measure whether sharding owners by key would add writes (084)eunhhu2026-09-271-0/+47
| | | | | | | | | | | It would not, on this device. One file's syncs or four at once, the SD card completes the same 510-690 commits per second, the latency growing with the number of files. The scoreboard app with two owners, each with a store of its own, took 23% fewer durable posts than one owner (median 16,230 against 20,974 a second) with 2-3x the p99: syncs queue, and each carries half the events. One owner's group commit stays; cores serve reads, and writes grow by making the sync itself cheaper.
* Check pure-serve in Pure, not in aupeunhhu2026-09-275-7/+7
| | | | | | | | | | The HTTP host's end-to-end checks (documents against the earlier server, lending, kill -9 durability, JSON, timers, links, webhooks and answers) sat in aup's tests, so a change to the host was verified from the toolchain. They now live in tests/check.py; make check builds their apps with the sibling aup and runs them after the unit tests, which stay beside their parts. Design 009 states the rule for all three repositories.
* Lay Pure out as a core with the parts selected around iteunhhu2026-09-279-10/+10
| | | | | | | | | | | | | | | | | | | | | The app model, the tree store and the HTTP host lived in prototypes/connections among the earlier document runtime, the store was known only by its format's name, and Vega compiled six Pure sources by path. Now, as the README's principle has it: core/ runtime, flow and the app model (module*.c), with what the core asks of storage (state.h, store.h) store/ the tree store (PURETRE3, PUREREC1) serve/ pure-serve: HTTP host, outbound delivery, JSON in place cli/ pure-run, pure-edit-module A top-level make builds libpure.a (app model and store, for hosts to link) and the hosts; make check runs the store's and the app model's tests. The earlier runtime still builds from prototypes/connections against core/ and store/ (check, check-tree and check-core pass). Design 009 records the rule: parts by role behind libraries; git repositories split by coupling, so the store stays here while its contract moves with the core, and Vega stays apart.
* Look for new sends newest first, only as far back as they goeunhhu2026-09-271-2/+2
| | | | | | | | | After each save the owner walked the whole outbox to find the sends not yet queued, so a receiver long down made every save walk the backlog. pure_app_outbox_newest visits saved sends newest first, and the owner stops once every wired output reaches the number it queued last, then queues what it found in order.
generated by cgit v1.2.3 (git 2.46.0) at 2026-10-20 18:07:15 +0000