<feed xmlns='http://www.w3.org/2005/Atom'>
<title>pure/prototypes/connections/store-tree.c, branch master</title>
<subtitle>Verified execution runtime unifying state, persistence, and replication.
</subtitle>
<id>http://git.yeowoo.org/pure/atom?h=master</id>
<link rel='self' href='http://git.yeowoo.org/pure/atom?h=master'/>
<link rel='alternate' type='text/html' href='http://git.yeowoo.org/pure/'/>
<updated>2026-09-27T12:14:38Z</updated>
<entry>
<title>Lay Pure out as a core with the parts selected around it</title>
<updated>2026-09-27T12:14:38Z</updated>
<author>
<name>eunhhu</name>
<email>necpition205@gmail.com</email>
</author>
<published>2026-09-27T12:14:38Z</published>
<link rel='alternate' type='text/html' href='http://git.yeowoo.org/pure/commit/?id=37d3a1e152707d2da587440c4cea8601b9ef0fd2'/>
<id>urn:sha1:37d3a1e152707d2da587440c4cea8601b9ef0fd2</id>
<content type='text'>
The app model, the tree store and the HTTP host lived in
prototypes/connections among the earlier document runtime, the store was
known only by its format's name, and Vega compiled six Pure sources by path.
Now, as the README's principle has it:

  core/   runtime, flow and the app model (module*.c), with what the core
          asks of storage (state.h, store.h)
  store/  the tree store (PURETRE3, PUREREC1)
  serve/  pure-serve: HTTP host, outbound delivery, JSON in place
  cli/    pure-run, pure-edit-module

A top-level make builds libpure.a (app model and store, for hosts to link)
and the hosts; make check runs the store's and the app model's tests. The
earlier runtime still builds from prototypes/connections against core/ and
store/ (check, check-tree and check-core pass). Design 009 records the rule:
parts by role behind libraries; git repositories split by coupling, so the
store stays here while its contract moves with the core, and Vega stays apart.

</content>
</entry>
<entry>
<title>Write each commit's head just before its bytes: one range per sync (081)</title>
<updated>2026-09-26T22:24:31Z</updated>
<author>
<name>eunhhu</name>
<email>necpition205@gmail.com</email>
</author>
<published>2026-09-26T22:24:31Z</published>
<link rel='alternate' type='text/html' href='http://git.yeowoo.org/pure/commit/?id=2e7ad9f8488d4f02c688ec24d6d0b571b3fa61ec'/>
<id>urn:sha1:2e7ad9f8488d4f02c688ec24d6d0b571b3fa61ec</id>
<content type='text'>
A tree store commit wrote its bytes over the written zeros of the tail
and its head into a slot at the start of the file: two ranges, and on
the Pi's SD card a sync of two ranges costs about twice one (2.4 ms
against 1.2 ms). A commit now writes a 128-byte record (its head) at
the published end and its bytes right after it, then syncs once. Every
16 commits the head also goes into the older slot. The newest commit
is found from the newer slot's head, or from the newest head this
process has seen in the file while its record is still there, along
the records whose bytes are whole; a torn record or bytes end the walk
at the commit before, which the next writer overwrites. Slots of this
format are PURETRE3; files of the earlier one are read and continued.

Store commits went from 220-233/s to 371-396/s (p50 2.5 to 1.3 ms),
and the scoreboard's durable posts from 13-16k/s to 21-26k/s with 64
connections and from 2.2-2.4k/s to 2.9-3.9k/s with 8, measured
alternately with the previous build. The torn head check now tears a
slot (the walk goes on from the other) and the newest record (a new
process sees the commit before); a new check walks 42 commits from the
slots, in a fresh process too, and stops at a damaged record.

</content>
</entry>
<entry>
<title>Snapshot an app's saved states for readers on other threads</title>
<updated>2026-09-26T21:56:32Z</updated>
<author>
<name>eunhhu</name>
<email>necpition205@gmail.com</email>
</author>
<published>2026-09-26T21:56:32Z</published>
<link rel='alternate' type='text/html' href='http://git.yeowoo.org/pure/commit/?id=42021a500d19dbe091075338d1a621498a81f167'/>
<id>urn:sha1:42021a500d19dbe091075338d1a621498a81f167</id>
<content type='text'>
Right after a save, pure_app_snapshot makes a read-only snapshot of the
saved states: numbers are copied, text states are frozen drafts that
share the store mapping (pure_draft_freeze, no copy), and lists are
copies of their item pointers that share the items. An unchanged list
shares the last snapshot's copy. pure_app_read binds a reader app on
another thread to a snapshot so the existing view functions read it.

A shared item never changes again: a list stamps each item with its
share count, copies a shared item it changes, and retires shared items
it lets go instead of freeing them. Snapshots form a chain that goes
oldest first, and items retired while a snapshot was the newest go
with it. References are atomic and may be released on any thread.

test-module checks that each snapshot equals the app, that kept
snapshots stay as made through changes, removals, a module switch, the
app stopping and the store closing, that a failed save's snapshot
equals the one before, and that two threads read the newest snapshots
while the app goes on (also under ASan, UBSan and LeakSanitizer).

</content>
</entry>
<entry>
<title>Count a store mapping's references atomically</title>
<updated>2026-09-26T16:29:16Z</updated>
<author>
<name>eunhhu</name>
<email>necpition205@gmail.com</email>
</author>
<published>2026-09-26T16:29:16Z</published>
<link rel='alternate' type='text/html' href='http://git.yeowoo.org/pure/commit/?id=f563f3da1de1e551b2f7e9fe825f9b19fa13d481'/>
<id>urn:sha1:f563f3da1de1e551b2f7e9fe825f9b19fa13d481</id>
<content type='text'>
A read's lent extents hold the store mapping until its answer is sent;
with an owner thread running the app and connection threads answering,
the last reference may drop on a thread other than the one that took it.

</content>
</entry>
<entry>
<title>Make a tree store commit durable with one sync</title>
<updated>2026-09-26T06:12:17Z</updated>
<author>
<name>eunhhu</name>
<email>necpition205@gmail.com</email>
</author>
<published>2026-09-26T06:12:17Z</published>
<link rel='alternate' type='text/html' href='http://git.yeowoo.org/pure/commit/?id=46fd0601c89a535a6501b475ff08ae11b3137ea9'/>
<id>urn:sha1:46fd0601c89a535a6501b475ff08ae11b3137ea9</id>
<content type='text'>
A commit used to sync its nodes, then write the head and sync again: the
order of writes was forced by syncs. Now the head carries where the
commit's bytes start and their digest, and one fdatasync makes the bytes
and the head durable together. A crash during it can leave a head whose
bytes did not all reach the device; its digest then fails and the older
slot stands, which is safe because a writer only builds on a durable head.

Commits overwrite written zeros kept after the published end, so the sync
changes neither the size nor the block map and needs no journal commit.
On the SD card this is 2.4 ms per commit against 9.1 ms for two syncs;
the store commits 289-304 times a second instead of 94-109, and pure-serve
answers 2,012-2,611 durable PUTs a second against 583 for crash-safe axum.

A process remembers the heads it published or checked, so it reads a
commit's bytes at most once and never its own. Its readers pass over the
head it is still making durable and so never answer with a commit a crash
could take back. Head slots get a block each and the format becomes
PURETRE2; a handle's refresh reads the slots through its mapping.

</content>
</entry>
<entry>
<title>Lend stored extents to reads instead of copying them</title>
<updated>2026-09-26T05:34:39Z</updated>
<author>
<name>eunhhu</name>
<email>necpition205@gmail.com</email>
</author>
<published>2026-09-26T05:34:39Z</published>
<link rel='alternate' type='text/html' href='http://git.yeowoo.org/pure/commit/?id=e7a2d8a6715ab48f887acfe1d14e32099e0ed7ff'/>
<id>urn:sha1:e7a2d8a6715ab48f887acfe1d14e32099e0ed7ff</id>
<content type='text'>
A read of a stored document now answers with the store mapping's extents
through writev, without copying the document. pure_draft_mapping takes a
reference to the mapping a draft opened through a handle reads from; the
answer keeps it until the last byte is sent, so the version read goes out
intact even if the document changes or a compaction replaces the file
meanwhile. Replies the mapping does not hold are copied as before.

pure-serve's output is now a list of extents (the head first). One thread
serves 2,761 1 MiB reads per second (axum: 1,380) and 128k small ones.

</content>
</entry>
<entry>
<title>Group the commits of concurrent threads</title>
<updated>2026-09-26T05:30:03Z</updated>
<author>
<name>eunhhu</name>
<email>necpition205@gmail.com</email>
</author>
<published>2026-09-26T05:30:03Z</published>
<link rel='alternate' type='text/html' href='http://git.yeowoo.org/pure/commit/?id=9408c57e0f2485fd7960135c931aa120dcb3fa8f'/>
<id>urn:sha1:9408c57e0f2485fd7960135c931aa120dcb3fa8f</id>
<content type='text'>
Threads that commit at the same time now share one commit: they queue,
the first becomes the leader and publishes every queued request of the
same store with one head, and the others wait for its result. Each
request stays all or nothing on its own: one with a stale draft is left
out, and so is one whose key an earlier request of the group changes.
When several threads have been committing together, a leader waits up to
2 ms for as many to join as groups held lately; a single thread never
waits.

A test pauses a publication and checks that five commits make exactly two
publications. Under eight writers on three pure-serve shards, 1,234
bursts went out in 421 publications. pure_store_group_counts reports the
groups, and pure-serve prints them.

</content>
</entry>
<entry>
<title>Shard pure-serve across cores</title>
<updated>2026-09-26T04:48:40Z</updated>
<author>
<name>eunhhu</name>
<email>necpition205@gmail.com</email>
</author>
<published>2026-09-26T04:48:40Z</published>
<link rel='alternate' type='text/html' href='http://git.yeowoo.org/pure/commit/?id=c7ae77cdf4623c3eb07935081510478be9063024'/>
<id>urn:sha1:c7ae77cdf4623c3eb07935081510478be9063024</id>
<content type='text'>
A burst now looks at the store once (pure_store_pin) instead of once per
request; another shard's commit ends before its answer is sent, so a
request made after that answer still reads it. With this alone one thread
serves 125k small keep-alive reads per second, as many as axum on three
cores, with a shorter p99.

--threads N runs a shard per thread: its own SO_REUSEPORT listener, epoll
set, runtime domain, app and store handle. Writer sessions of the tree
store now take turns inside a process with a mutex (the file lock belongs
to the process and does not keep threads apart), and revisions settle
concurrent edits of a key. A module with states runs on one thread. Three
shards serve 151k reads per second; they still commit one by one, so
durable writes and 1 MiB reads are better on one thread for now, which
stays the default. Note 075 records the numbers.

</content>
</entry>
<entry>
<title>Add a store handle that shares one mapping</title>
<updated>2026-09-26T04:39:47Z</updated>
<author>
<name>eunhhu</name>
<email>necpition205@gmail.com</email>
</author>
<published>2026-09-26T04:39:47Z</published>
<link rel='alternate' type='text/html' href='http://git.yeowoo.org/pure/commit/?id=e4bc782d09fe1c80098a38380b3959863dfe188f'/>
<id>urn:sha1:e4bc782d09fe1c80098a38380b3959863dfe188f</id>
<content type='text'>
Opening a draft by path opened the directory and the file, mapped the
whole file, read the head, and undid all of it on close: nine system
calls for every GET. pure_store_open keeps the store open for one owner
thread; drafts opened through it share a reference-counted mapping of the
newest version, and opening one only checks whether the file changed (its
identity and its head: two system calls). A commit of its drafts maps the
new version once and moves them all onto it. A compaction by another
writer is noticed and followed, while drafts of the old file keep reading
it and relocate their extents when they commit.

pure_store_data gives modules the same through a handle.

</content>
</entry>
<entry>
<title>Commit several drafts of the tree store atomically</title>
<updated>2026-09-25T14:37:53Z</updated>
<author>
<name>eunhhu</name>
<email>necpition205@gmail.com</email>
</author>
<published>2026-09-25T14:37:53Z</published>
<link rel='alternate' type='text/html' href='http://git.yeowoo.org/pure/commit/?id=49c29a1255e1ad3f7fde6c64910c7d2c38edad78'/>
<id>urn:sha1:49c29a1255e1ad3f7fde6c64910c7d2c38edad78</id>
<content type='text'>
pure_draft_commit_all publishes drafts of one store with a single head:
every key must still be at its draft's base revision, all new extents and
catalog records are written and synced once, and either every document
becomes visible or none does. A single draft commit is the one-draft case.

pure_state_open stores runtime-owned values (app states) under '.' plus a
key. Document keys never start with '.', listings skip these values, and
document operations refuse their names, so an app's state cannot collide
with a document a client names.

Tests kill a process at every commit stage of a three-draft batch and see
all or none, inject each fault, stop a batch on one stale draft, and
commit fifty keys in no order at once.

</content>
</entry>
</feed>
