aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authoreunhhu <[email protected]>2026-09-25 09:33:43 +0100
committereunhhu <[email protected]>2026-09-25 09:33:43 +0100
commit6a28879bd56354483fe940c0c80d511eccb89983 (patch)
treeed919d234ea2dbe8099e53fae51e495d8bee984e
parent8a65fd032e6f74938caf84d6f6fcda6314e76a01 (diff)
downloadpure-6a28879bd56354483fe940c0c80d511eccb89983.tar.gz
pure-6a28879bd56354483fe940c0c80d511eccb89983.zip
Add selected single-file persistent tree store
store-tree.c keeps every document in one append-only file as a persistent B-tree of byte extents, with the key catalog in the same structure. A splice writes only the changed leaf and path; unchanged extents are shared between versions, readers take no lock, and two alternating head slots publish a commit after its nodes are durable. Compaction rewrites reachable nodes. On the recorded ext4 host a durable 1-byte middle insert into a 1 MiB document took 8.58 ms against 78.51 ms for the default store, and the real TUI's Ctrl+S after typing took 16.60 ms against 84.59 ms. check-tree runs the store's crash/oracle tests and the unchanged runtime, splice, async and real-PTY editor checks on it. It stays opt-in: linked code is 12.7 KB larger and contiguous multi-leaf reads copy.
-rw-r--r--README.md12
-rw-r--r--docs/research/073-single-file-persistent-tree.md126
-rw-r--r--prototypes/connections/Makefile13
-rw-r--r--prototypes/connections/README.md10
-rw-r--r--prototypes/connections/check_tui.py14
-rw-r--r--prototypes/connections/store-tree.c971
-rw-r--r--prototypes/connections/test-splice.c3
-rw-r--r--prototypes/connections/test-tree-store.c394
-rw-r--r--research/experiments/008-tree-store-edit-cost/2026-09-25-ext4-linux-aarch64.json346
-rw-r--r--research/experiments/008-tree-store-edit-cost/2026-09-25-tui-ext4-linux-aarch64.json285
-rw-r--r--research/experiments/008-tree-store-edit-cost/bench.c104
-rwxr-xr-xresearch/experiments/008-tree-store-edit-cost/run.py49
-rwxr-xr-xresearch/experiments/008-tree-store-edit-cost/tui.py128
13 files changed, 2449 insertions, 6 deletions
diff --git a/README.md b/README.md
index 75698e6..2ae94db 100644
--- a/README.md
+++ b/README.md
@@ -67,6 +67,18 @@ page-store assertions plus a 1 MiB native CLI edit and stale-revision check.
Physical-disk durability, Linux x86_64 CI and macOS remain unverified; the
default core and storage path are unchanged.
+The [selected single-file persistent tree](docs/research/073-single-file-persistent-tree.md)
+questions the one-contiguous-file-per-document model instead. All documents
+share one append-only file of immutable extents. An insert, delete or append
+writes only the changed leaf and path, and unchanged extents are shared
+between versions. On the same ext4 host, a durable 1-byte middle insert into a
+1 MiB document took 8.58 ms against 78.51 ms for the default store. The real
+TUI's Ctrl+S after typing one character took 16.60 ms against 84.59 ms. The
+unchanged runtime, splice, async and real-PTY TUI checks pass on it
+(`make -C prototypes/connections check-tree`). Its linked CLI is 12,710 B
+larger, contiguous reads of multi-leaf documents copy, and space is reclaimed
+only by compaction, so it remains an opt-in profile.
+
The current execution profile has separate `core`, `fixed-editor`, `editor`, and
`server` build targets. `make all` produces the core and the parser-free CLI
from the build-time prepared `editor.pure`; it also builds a compiler tool that
diff --git a/docs/research/073-single-file-persistent-tree.md b/docs/research/073-single-file-persistent-tree.md
new file mode 100644
index 0000000..e2a2ee9
--- /dev/null
+++ b/docs/research/073-single-file-persistent-tree.md
@@ -0,0 +1,126 @@
+# 073 — 한 파일 안의 영속 트리: 편집 비용을 문서 크기에서 경로 길이로
+
+2026-09-25. 선택형 저장소 `store-tree.c`를 추가했다. 기본 빌드와 기본 저장 형식은 바뀌지 않았다.
+
+착수 기준:
+
+- 바꿀 결정: 문서를 "키 하나 = 연속 바이트 파일 하나"로 두는 관례를 계속 전제로 할지,
+ 모든 문서를 한 파일의 불변·구조 공유 트리로 둘지.
+- 남은 질문: 삽입·삭제에서도 문서 전체를 다시 쓰지 않고 내구성 있게 커밋할 수 있는가?
+ 기존 `store.h` 계약과 호스트·충돌·고장 검사를 그대로 통과하는가? 작은 문서·읽기·코드 비용은?
+- 최소 확인: 기존 계약을 그대로 구현한 저장소 하나를 만들어 기존 검사와 새 고장·오라클 검사를
+ 통과시키고, 같은 SD 카드에서 세 저장소를 같은 편집으로 비교한다.
+- 중단 기준: 기존 계약 검사를 통과하지 못하거나, 1 MiB 문서의 삽입 저장이 기본 대비 크게
+ 줄지 않거나, 작은 문서가 눈에 띄게 나빠지면 닫는다.
+
+## 의심한 관례
+
+| 관례 | 이 저장소에서 바꾼 것 |
+|---|---|
+| 값 = 연속 바이트 배열 | 값 = 불변 익스텐트(최대 4 KiB)의 순서열. 연속 배열은 필요한 소비자에게만 만든다 |
+| 새 버전 = 새 사본 | 새 버전 = 바뀐 leaf와 그 조상 경로만 새로 쓰고 나머지는 이전 버전과 공유 |
+| 저장 단위 = 키마다 파일, 목록 = 디렉터리 | 저장소 = 추가 전용 파일 하나. 문서 목록(카탈로그)도 같은 트리 구조의 레코드열 |
+| 읽기도 잠근다 | 읽기는 잠금 없이 공개된 불변 접두부를 매핑한다 |
+| 호스트가 저장 형식을 안다 | TUI가 `obj-` 파일명을 직접 읽던 코드를 `pure_store_keys`로 옮김 |
+
+## 구조
+
+- 파일: `[head 슬롯 0][head 슬롯 1][커밋된 노드…]`. head는 순번·공개 끝 위치·카탈로그 루트·
+ 압축 후 크기와 찢김 검출용 체크섬을 가진다.
+- leaf는 원시 문서 바이트다. 여러 leaf로 나눌 때는 균형 분할해 각 leaf가 2 KiB 이상이다.
+ branch는 `PTB1`, 자식 수·높이, 자식별 (참조, 하위 바이트 수) 최대 32개다. 모든 leaf의 깊이가 같다.
+- 편집 `splice(offset, remove, insert)`는 영향받은 leaf와 조상만 새로 만든다. 새로 만든 노드가
+ 기준보다 작으면 이웃과 병합하고, 루트의 단일 자식은 접는다.
+- 카탈로그는 키 순서의 96 B 레코드(키 64 B, 루트, 크기, 리비전)다. 같은 편집 코드로 갱신하고
+ 이진 탐색으로 찾는다. 문서 하나를 바꾸면 카탈로그도 한 경로만 다시 쓴다.
+- 커밋: 새 노드 추가 → `fsync` → 오래된 head 슬롯에 새 head → `fsync`. OK는 두 번째 동기화 뒤다.
+ 복구는 유효한 슬롯 중 순번이 큰 쪽을 쓰고, 그 끝 뒤의 바이트는 확인받지 않은 커밋이라
+ 다음 writer가 잘라낸다. 쓰기 중 고장 단계는 기존 저장소와 같은 결과(오류 또는 결과 불명)를 낸다.
+- 압축: 공개 크기의 증가가 `4 × 압축 후 크기 + 1 MiB`를 넘으면 도달 가능한 노드만 새 파일에 쓰고
+ 교체한다. 이미 읽은 쪽은 이전 파일의 매핑을 계속 사용한다.
+- 읽기: 한 leaf 안의 범위는 파일을 그대로 매핑한다(기본 저장소와 같은 무복사). 여러 leaf에
+ 걸친 범위는 기존 API가 연속 뷰를 요구하므로 익명 매핑에 복사한다.
+
+## 측정
+
+같은 Raspberry Pi 5(Linux 6.18.39 AArch64), ext4 `/dev/mmcblk0p2`, GCC 14 `-O2`.
+[저장소 API 직접 측정](../../research/experiments/008-tree-store-edit-cost/bench.c)은 시나리오마다 32회 편집의
+중앙값이다. [원시 결과](../../research/experiments/008-tree-store-edit-cost/2026-09-25-ext4-linux-aarch64.json).
+
+| 1 MiB 문서, 편집 1회 | 기본 파일 | 선택형 페이지 | 영속 트리 |
+|---|---:|---:|---:|
+| 가운데 1 B 삽입 | 78.51 ms | 86.57 ms | **8.58 ms** |
+| 가운데 1 B 삭제 | 78.76 ms | 85.29 ms | **8.51 ms** |
+| 마지막 1 B 교체 | 79.20 ms | 13.96 ms | **6.49 ms** |
+| 끝에 1 B 추가 | 77.89 ms | 86.04 ms | **8.47 ms** |
+| write 호출 바이트/편집 | 약 1,048,600 | 16,944–1,065,528 | **2,716–4,912** |
+| 장치 쓰기 바이트/편집 | 약 1,064,960 | 32,768–1,081,344 | **35,328–37,376** |
+
+페이지 저장소는 크기가 같은 교체에서만 부분 기록을 하고, 삽입·삭제·추가는 전체를 다시 쓴다.
+트리는 네 가지 편집 모두 경로만 쓴다.
+
+[실제 TUI 측정](../../research/experiments/008-tree-store-edit-cost/tui.py)은 065와 같은 방법으로
+Ctrl+S부터 저장 완료 프레임까지 잰다. 1 MiB 문서, 저장소별 3세션 × 8회 = 24회의 중앙값이다.
+[원시 결과](../../research/experiments/008-tree-store-edit-cost/2026-09-25-tui-ext4-linux-aarch64.json).
+
+| Ctrl+S → 완료 표시 | 기본 파일 | 선택형 페이지 | 영속 트리 |
+|---|---:|---:|---:|
+| 마지막 문자 교체 | 86.32 ms | 21.31 ms | **16.97 ms** |
+| 끝에 한 글자 입력 | 84.59 ms | 93.29 ms | **16.60 ms** |
+
+일반 입력처럼 문서가 커지는 저장에서 기본 대비 약 5.1배, 페이지 대비 약 5.6배 짧았다.
+
+## 검증
+
+[`test-tree-store.c`](../../prototypes/connections/test-tree-store.c)는 ASan·UBSan에서도 통과했다.
+
+- 4개 키에 6,000회 무작위 편집을 평면 배열 오라클과 매번 비교했다(최대 333,510 B, 높이 2).
+ 매 편집 뒤 저장된 모든 비루트 노드가 기준 이상으로 채워졌는지 확인했다.
+- 이전에 받은 뷰는 이후의 커밋과 압축을 지나도 바이트가 그대로다.
+- 1 MiB 문서의 1 B 편집은 최대 4,912 B만 추가한다(높이 2, leaf 258개).
+- 다섯 고장 단계, 각 단계에서의 프로세스 사망(공개 안 된 꼬리는 다음 writer가 잘라냄),
+ 찢긴 최신 head, 다른 프로세스의 400회 커밋·압축 중 잠금 없는 읽기의 격리,
+ 3,000회 1비트 손상(OK·저장 오류·없음만 반환), 1,024개 키, 리비전 상한, 키 목록.
+
+저장소만 바꾸고 기존 검사를 그대로 실행하는 `make check-tree`도 통과했다.
+`test-runtime`, `test-splice`(오라클·이전 스냅샷·경계·충돌·고장 단계), `test-async`, 고정·소스
+TUI의 실제 PTY 검사(1 MiB, UTF-8, 충돌, 1,024개 문서 목록)다. 기본 `check-release`와
+선택형 페이지 검사도 TUI의 목록 변경 뒤 다시 통과했다.
+
+## 비용과 한계
+
+- **코드 크기가 크다.** `store-tree.c` 오브젝트 `text`는 `-O2` 15,899 B, `-Os` 13,242 B다.
+ 기본 저장소는 4,453 B, 페이지 저장소는 9,198 B다. 고정 정의 CLI의 링크 `text`는
+ 19,268 → 31,978 B, 고정 TUI는 32,375 → 44,217 B다. 선택형이며 기본 빌드에는 들어가지 않는다.
+ 마이크로 목표에 맞추려면 줄여야 할 첫 항목이다.
+- **연속 읽기는 복사한다.** 여러 leaf에 걸친 1 MiB 전체 읽기의 중앙값은 0.23–0.37 ms로,
+ 기본 매핑의 0.06–0.10 ms보다 느리다. 기존 API가 연속 뷰를 요구하기 때문이다.
+- **작은 문서는 이득이 없다.** 256 B 편집 지연은 비슷하지만(6.47–9.14 ms 대 8.32–9.31 ms)
+ 동기화가 두 번이라 장치 쓰기가 16 KiB에서 32 KiB로 늘었다.
+- **공간이 늦게 회수된다.** 압축 전까지 파일이 자란다. 1 MiB 문서를 32회 편집한 뒤
+ 1.14–1.21 MB였다. 압축은 전체 재작성이며 그동안 writer 잠금을 잡는다.
+- 문서 하나를 파일 단위로 지우거나 복사하는 외부 조작은 더 이상 없다. TUI 검사의 외부 삭제
+ 시나리오는 이 저장소에서 생략한다. 기본 형식으로의 전환은 CLI `show`/`apply` 경로뿐이며
+ 전용 변환 도구는 아직 없다.
+- head 체크섬은 찢김 검출용이며 적대적 파일 방어가 아니다. 노드 데이터에는 체크섬이 없어
+ 비트 부패를 검출하지 않는다(기존 저장소와 같음). writer 잠금은 기존의 프로세스 단위
+ `fcntl` 잠금이다. 읽기는 잠그지 않으므로 페이지 저장소의 같은 프로세스 읽기·쓰기 잠금 문제는 없다.
+- 물리 전원 차단 내구성, macOS, x86_64는 검증하지 않았다.
+
+## 선행 기술
+
+경로 복사 B-tree와 번갈아 쓰는 두 메타 슬롯은 LMDB, 텍스트 편집용 익스텐트 트리는 로프·
+피스 테이블 계열 편집기, 추가 전용 기록과 압축은 로그 구조 저장소가 이미 쓰는 기법이다.
+이번 결과는 이 기법들을 Pure 계약 아래에 작게 결합했다는 것이며 신규성 주장이 아니다.
+차별성은 같은 구조가 저장 계층을 넘어 편집 초안·전송까지 공유될 때 판단한다.
+
+## 다음 결정
+
+TUI의 17 ms 중 저장소 자체는 약 8.5 ms다. 나머지는 저장본 전체를 연속 뷰로 다시 읽어
+초안과 비교하는 "초안 대 저장본 비교" 관례의 비용이다. 편집기는 무엇을 바꿨는지 이미 알고 있다.
+
+1. 편집 연산을 그대로 전달하고, 초안을 같은 트리의 커밋되지 않은 루트로 둔다. 그러면
+ 전체 비교, 전체 복사, 별도 gap buffer가 사라지는지 확인한다.
+2. 연속 뷰 대신 익스텐트 열을 소비자에게 준다(HTTP `writev`, 피어 전송, TUI 렌더).
+3. Pure 간 동기화에서 상대에게 없는 노드만 보낸다.
+4. 코드 크기를 줄인다. 기본 저장소 승격 여부는 1–4의 결과와 macOS·전원 차단 검증 뒤에 판단한다.
diff --git a/prototypes/connections/Makefile b/prototypes/connections/Makefile
index 47a6530..661fbcd 100644
--- a/prototypes/connections/Makefile
+++ b/prototypes/connections/Makefile
@@ -3,7 +3,7 @@ CFLAGS ?= -O2 -g
CFLAGS += -std=c17 -Wall -Wextra -Wpedantic -Werror
BUILD ?= build
-.PHONY: all core async-core editor fixed-editor fixed-server fixed-tui source-tui compiler mirror window server async-server document-peer document-server check check-policy check-policy-peer check-portable check-portable-tui check-paged-portable check-portable-peer check-native-checkpoint check-mirror check-core check-async check-async-server check-async-service-scale check-profile check-fixed-profile check-fixed-server check-service-scale check-tui-profile check-source-tui-profile check-tui check-source-tui check-release check-micro-memory check-hybrid check-flat-export check-copy-range check-peer-profile check-document-server-profile check-window-profile check-document-peer check-document-server FORCE
+.PHONY: all core async-core editor fixed-editor fixed-server fixed-tui source-tui compiler mirror window server async-server document-peer document-server check check-policy check-policy-peer check-portable check-portable-tui check-paged-portable check-tree check-portable-peer check-native-checkpoint check-mirror check-core check-async check-async-server check-async-service-scale check-profile check-fixed-profile check-fixed-server check-service-scale check-tui-profile check-source-tui-profile check-tui check-source-tui check-release check-micro-memory check-hybrid check-flat-export check-copy-range check-peer-profile check-document-server-profile check-window-profile check-document-peer check-document-server FORCE
CORE = runtime.c
STORE ?= store.c
FIXED_PROGRAM ?= examples/editor.pure
@@ -115,6 +115,9 @@ $(BUILD)/test-async: test-async.c program.c $(CORE) $(HOST) $(HEADERS) | $(BUILD
$(BUILD)/test-splice: test-splice.c program.c $(CORE) $(HOST) $(HEADERS) | $(BUILD)
$(CC) $(CPPFLAGS) $(CFLAGS) -DPURE_RUNTIME_METRICS -DPURE_STORE_TEST test-splice.c program.c $(CORE) $(HOST) $(LDFLAGS) $(LDLIBS) -o $@
+$(BUILD)/test-tree-store: test-tree-store.c store-tree.c $(HEADERS) | $(BUILD)
+ $(CC) $(CPPFLAGS) $(CFLAGS) -DPURE_STORE_TEST test-tree-store.c store-tree.c $(LDFLAGS) $(LDLIBS) -o $@
+
$(BUILD)/test-paged-store: test-paged-store.c store-paged.c $(HEADERS) | $(BUILD)
$(CC) $(CPPFLAGS) $(CFLAGS) -DPURE_STORE_TEST test-paged-store.c store-paged.c $(LDFLAGS) $(LDLIBS) -o $@
@@ -246,6 +249,14 @@ check-portable-tui: fixed-editor fixed-tui editor source-tui
python3 check_tui.py --build $(BUILD) --portable $(if $(PORTABLE_FIXED_TUI_EVIDENCE),--output "$(PORTABLE_FIXED_TUI_EVIDENCE)")
python3 check_tui.py --build $(BUILD) --source --portable $(if $(PORTABLE_SOURCE_TUI_EVIDENCE),--output "$(PORTABLE_SOURCE_TUI_EVIDENCE)")
+# Selected single-file persistent-tree store: its own crash/oracle checks, then
+# the unchanged runtime, splice, async and real-PTY editor checks on it.
+check-tree:
+ $(MAKE) BUILD=build/tree STORE=store-tree.c CPPFLAGS="$(CPPFLAGS) -DPURE_STORE_TREE" check-portable build/tree/test-tree-store fixed-editor fixed-tui editor source-tui
+ build/tree/test-tree-store
+ python3 check_tui.py --build build/tree --portable --single-file-store $(if $(TREE_TUI_EVIDENCE),--output "$(TREE_TUI_EVIDENCE)")
+ python3 check_tui.py --build build/tree --source --portable --single-file-store
+
# Exercise the selected logical-page store without Linux-only size/RSS gates.
# The CI runner separately asserts its real OS page size before this target.
check-paged-portable:
diff --git a/prototypes/connections/README.md b/prototypes/connections/README.md
index 7da7eab..f0b041e 100644
--- a/prototypes/connections/README.md
+++ b/prototypes/connections/README.md
@@ -144,6 +144,16 @@ python3 prototypes/connections/export_flat_store.py \
[실제 큰·작은 문서 쓰기량과 전체 제한](../../docs/research/034-hybrid-document-storage.md),
[기본 형식으로 내보내는 검증](../../docs/research/062-paged-store-rollback.md).
+선택형 영속 트리 저장소(`store-tree.c`)는 모든 문서를 추가 전용 파일 하나에 불변 익스텐트
+트리로 둔다. 편집은 바뀐 leaf와 경로만 기록하고, 읽기는 잠그지 않는다. 기존 런타임·부분 수정·
+비동기·실제 PTY 검사와 자체 고장·오라클 검사를 함께 실행한다. 형식이 달라 별도 `--store`
+디렉터리를 쓴다. [설계·측정·한계](../../docs/research/073-single-file-persistent-tree.md).
+
+```sh
+make -C prototypes/connections check-tree
+prototypes/connections/build/tree/pure-edit-tui-fixed --store /tmp/pure-tree-documents note
+```
+
같은 `.pure` 문서 편집기를 선택형 페이지 저장소와 함께 터미널에서 사용하려면
새 저장 디렉터리를 지정한다. 논리 페이지는 16 KiB이며 OS 페이지가 이를
나눌 때만 초기화한다. Linux 16 KiB 호스트에서 전체 선택형 검사가 통과했고,
diff --git a/prototypes/connections/check_tui.py b/prototypes/connections/check_tui.py
index 5328d02..adb0fef 100644
--- a/prototypes/connections/check_tui.py
+++ b/prototypes/connections/check_tui.py
@@ -128,7 +128,7 @@ def wait_stored(editor, store, key, expected, revision):
raise AssertionError((key, expected[:100], shown(editor, store, key)))
-def check(build, rss_limit_kib, source, program_path, portable=False):
+def check(build, rss_limit_kib, source, program_path, portable=False, object_files=True):
binary = (build / ('pure-edit-tui' if source else 'pure-edit-tui-fixed')).resolve()
program = (program_path or HERE / 'examples/editor.pure').resolve() if source else None
editor = [str((build / ('pure-edit' if source else 'pure-edit-fixed')).resolve())]
@@ -171,9 +171,11 @@ def check(build, rss_limit_kib, source, program_path, portable=False):
terminal.wait_for(b'Pure | note')
terminal.type(b'\x0c')
terminal.wait_for(b'Pure | documents')
- (store / 'obj-second').unlink()
- terminal.type(b'\x1b[B\r')
- terminal.wait_for(b'Document disappeared; refresh the list')
+ if object_files:
+ # Only per-object file stores can lose one document underneath the list.
+ (store / 'obj-second').unlink()
+ terminal.type(b'\x1b[B\r')
+ terminal.wait_for(b'Document disappeared; refresh the list')
terminal.type(b'\x1b')
terminal.wait_for(b'Pure | note')
terminal.type(b'!')
@@ -335,12 +337,14 @@ def main():
parser.add_argument('--portable', action='store_true',
help='run all PTY behavior without Linux /proc RSS measurement')
parser.add_argument('--program', type=Path, help='definition for --source (including relocated bundles)')
+ parser.add_argument('--single-file-store', action='store_true',
+ help='store keeps all documents in one file (skips external deletion)')
parser.add_argument('--output', type=Path)
args = parser.parse_args()
if args.output and args.output.exists():
parser.error('evidence file already exists')
result = check(args.build, 0 if args.portable else args.rss_limit_kib,
- args.source, args.program, args.portable)
+ args.source, args.program, args.portable, not args.single_file_store)
body = json.dumps(result, indent=2) + '\n'
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
diff --git a/prototypes/connections/store-tree.c b/prototypes/connections/store-tree.c
new file mode 100644
index 0000000..618028a
--- /dev/null
+++ b/prototypes/connections/store-tree.c
@@ -0,0 +1,971 @@
+/* Selected storage experiment: every document lives in one append-only file as
+ * a persistent B-tree of byte extents. An edit writes only the leaves and the
+ * path it changes, so inserting or deleting in a large document costs O(log n)
+ * nodes instead of a whole-document copy. Unchanged extents are shared by old
+ * and new versions. The document catalog is the same structure holding fixed
+ * records sorted by key. Two alternating head slots publish a commit only after
+ * its nodes are durable; readers take no lock and map one immutable prefix. */
+#define _POSIX_C_SOURCE 200809L
+#ifdef __APPLE__
+#define _DARWIN_C_SOURCE
+#endif
+#include "store.h"
+#include <errno.h>
+#include <fcntl.h>
+#include <inttypes.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+enum { SLOT = 64, DATA = 2 * SLOT, LEAF_MAX = 4096, LEAF_MIN = 1024,
+ FAN_MAX = 32, FAN_MIN = 8, RECORD = 96, KEY = 64, HEIGHT_MAX = 12,
+ BUFFER = 65536 };
+#ifdef PURE_STORE_TEST
+enum { COMPACT_SLACK = 65536 };
+#else
+enum { COMPACT_SLACK = 1048576 };
+#endif
+#define LEAF_BIT (UINT64_C(1) << 63)
+#define MEMORY_BIT (UINT64_C(1) << 62)
+#define LOCATION (MEMORY_BIT - 1)
+static const unsigned char head_magic[8] = {'P','U','R','E','T','R','E','1'};
+static const unsigned char branch_magic[4] = {'P','T','B','1'};
+static const char tree_name[] = "pure.tree";
+static uint64_t temporary_sequence;
+
+/* A reference is a file offset, or an index into this edit's node arena.
+ * bytes is the size of the whole subtree. Leaves hold raw document bytes. */
+typedef struct { uint64_t ref, bytes; } Entry;
+typedef struct { Entry *items; size_t count, capacity; } List;
+typedef struct {
+ unsigned char *bytes; /* leaf only */
+ Entry *children; /* branch only */
+ size_t size; /* leaf bytes or child count */
+ unsigned height;
+} Node;
+typedef struct {
+ const unsigned char *map;
+ uint64_t end;
+ Node *nodes;
+ size_t count, capacity;
+} Tree;
+typedef struct { uint64_t seq, end, catalog, catalog_bytes, live; } Head;
+typedef struct {
+ int fd;
+ uint64_t at; /* file offset of the next byte */
+ size_t used;
+ bool failed;
+ unsigned char buffer[BUFFER];
+} Writer;
+typedef struct { PureSpan part[3]; size_t index, offset; } Source;
+
+static bool fault(PureStoreStage stage) {
+#ifdef PURE_STORE_TEST
+ if (pure_store_test_fault(stage)) { errno = EIO; return true; }
+#else
+ (void)stage;
+#endif
+ return false;
+}
+
+static bool valid_key(const char *key) {
+ if (!key) return false;
+ size_t length = strlen(key);
+ if (!length || length > KEY) return false;
+ for (size_t i = 0; i < length; ++i) {
+ char b = key[i];
+ if (!((b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') ||
+ (b >= '0' && b <= '9') || b == '-' || b == '_')) return false;
+ }
+ return true;
+}
+
+static uint64_t decode64(const unsigned char *bytes) {
+ uint64_t value = 0;
+ for (unsigned i = 0; i < 8; ++i) value |= (uint64_t)bytes[i] << (8 * i);
+ return value;
+}
+
+static void encode64(unsigned char *bytes, uint64_t value) {
+ for (unsigned i = 0; i < 8; ++i) bytes[i] = (unsigned char)(value >> (8 * i));
+}
+
+/* Detects a torn head slot; it is not a defence against a hostile file. */
+static uint64_t checksum(const unsigned char *bytes, size_t length) {
+ uint64_t hash = UINT64_C(14695981039346656037);
+ for (size_t i = 0; i < length; ++i) hash = (hash ^ bytes[i]) * UINT64_C(1099511628211);
+ return hash;
+}
+
+static bool sync_fd(int fd) {
+ int rc;
+ do { rc = fsync(fd); } while (rc && errno == EINTR);
+ return rc == 0;
+}
+
+static bool write_at(int fd, const unsigned char *bytes, size_t length, uint64_t offset) {
+ while (length) {
+ ssize_t n = pwrite(fd, bytes, length, (off_t)offset);
+ if (n < 0 && errno == EINTR) continue;
+ if (n <= 0) return false;
+ bytes += (size_t)n;
+ length -= (size_t)n;
+ offset += (uint64_t)n;
+ }
+ return true;
+}
+
+static bool read_at(int fd, unsigned char *bytes, size_t length, uint64_t offset) {
+ while (length) {
+ ssize_t n = pread(fd, bytes, length, (off_t)offset);
+ if (n < 0 && errno == EINTR) continue;
+ if (n <= 0) return false;
+ bytes += (size_t)n;
+ length -= (size_t)n;
+ offset += (uint64_t)n;
+ }
+ return true;
+}
+
+static void encode_head(unsigned char out[SLOT], const Head *head) {
+ memset(out, 0, SLOT);
+ memcpy(out, head_magic, sizeof(head_magic));
+ encode64(out + 8, head->seq);
+ encode64(out + 16, head->end);
+ encode64(out + 24, head->catalog);
+ encode64(out + 32, head->catalog_bytes);
+ encode64(out + 40, head->live);
+ encode64(out + 56, checksum(out, 56));
+}
+
+static bool decode_head(const unsigned char in[SLOT], Head *head) {
+ if (memcmp(in, head_magic, sizeof(head_magic)) || decode64(in + 56) != checksum(in, 56))
+ return false;
+ *head = (Head){decode64(in + 8), decode64(in + 16), decode64(in + 24),
+ decode64(in + 32), decode64(in + 40)};
+ return head->seq && head->end >= DATA && head->end <= (uint64_t)INT64_MAX &&
+ head->end <= SIZE_MAX && head->catalog_bytes % RECORD == 0 &&
+ !head->catalog == !head->catalog_bytes;
+}
+
+/* The newer valid slot wins; a torn newer slot leaves the previous commit. */
+static bool read_head(int fd, Head *out) {
+ unsigned char slots[2 * SLOT];
+ Head a, b;
+ if (!read_at(fd, slots, sizeof(slots), 0)) return false;
+ bool first = decode_head(slots, &a), second = decode_head(slots + SLOT, &b);
+ if (!first && !second) return false;
+ *out = first && (!second || a.seq > b.seq) ? a : b;
+ return true;
+}
+
+static bool map_tree(int fd, const Head *head, Tree *tree) {
+ struct stat info;
+ *tree = (Tree){0};
+ if (fstat(fd, &info) || !S_ISREG(info.st_mode) || info.st_size < 0 ||
+ (uint64_t)info.st_size < head->end) return false;
+ void *map = mmap(NULL, (size_t)head->end, PROT_READ, MAP_SHARED, fd, 0);
+ if (map == MAP_FAILED) return false;
+ *tree = (Tree){.map = map, .end = head->end};
+ return true;
+}
+
+static void release_tree(Tree *tree) {
+ if (tree->map) munmap((void *)tree->map, (size_t)tree->end);
+ for (size_t i = 0; i < tree->count; ++i) {
+ free(tree->nodes[i].bytes);
+ free(tree->nodes[i].children);
+ }
+ free(tree->nodes);
+ *tree = (Tree){0};
+}
+
+static const unsigned char *leaf_bytes(const Tree *tree, Entry entry) {
+ if (!(entry.ref & LEAF_BIT) || !entry.bytes || entry.bytes > LEAF_MAX) return NULL;
+ uint64_t at = entry.ref & LOCATION;
+ if (entry.ref & MEMORY_BIT) {
+ if (at >= tree->count) return NULL;
+ const Node *node = &tree->nodes[at];
+ return node->bytes && node->size == entry.bytes ? node->bytes : NULL;
+ }
+ if (at < DATA || at > tree->end || entry.bytes > tree->end - at) return NULL;
+ return tree->map + at;
+}
+
+/* Returns the child count of a checked branch at height, or 0. */
+static size_t children(const Tree *tree, Entry entry, unsigned height, Entry out[FAN_MAX]) {
+ if ((entry.ref & LEAF_BIT) || !height) return 0;
+ uint64_t at = entry.ref & LOCATION;
+ size_t count;
+ if (entry.ref & MEMORY_BIT) {
+ if (at >= tree->count) return 0;
+ const Node *node = &tree->nodes[at];
+ if (node->bytes || node->height != height || !node->size || node->size > FAN_MAX) return 0;
+ count = node->size;
+ memcpy(out, node->children, count * sizeof(*out));
+ } else {
+ if (at < DATA || at % 8 || at > tree->end || tree->end - at < 8) return 0;
+ const unsigned char *p = tree->map + at;
+ count = (size_t)p[4] | (size_t)p[5] << 8;
+ unsigned stored = (unsigned)p[6] | (unsigned)p[7] << 8;
+ if (memcmp(p, branch_magic, sizeof(branch_magic)) || stored != height || !count ||
+ count > FAN_MAX || (tree->end - at - 8) / 16 < count) return 0;
+ for (size_t i = 0; i < count; ++i)
+ out[i] = (Entry){decode64(p + 8 + 16 * i), decode64(p + 16 + 16 * i)};
+ }
+ uint64_t sum = 0;
+ for (size_t i = 0; i < count; ++i) {
+ bool leaf = (out[i].ref & LEAF_BIT) != 0;
+ if (!out[i].bytes || out[i].bytes > UINT64_MAX - sum || leaf != (height == 1)) return 0;
+ sum += out[i].bytes;
+ }
+ return sum == entry.bytes ? count : 0;
+}
+
+static bool root_height(const Tree *tree, Entry root, unsigned *height) {
+ *height = 0;
+ if (!root.bytes) return !root.ref;
+ if (root.ref & LEAF_BIT) return true;
+ uint64_t at = root.ref & LOCATION;
+ if (root.ref & MEMORY_BIT) {
+ if (at >= tree->count || tree->nodes[at].bytes) return false;
+ *height = tree->nodes[at].height;
+ } else {
+ if (at < DATA || at > tree->end || tree->end - at < 8) return false;
+ *height = (unsigned)tree->map[at + 6] | (unsigned)tree->map[at + 7] << 8;
+ }
+ return *height >= 1 && *height <= HEIGHT_MAX;
+}
+
+/* Visit a range as borrowed extents: 1 done, 0 damaged, -1 stopped by visit. */
+static int visit_range(const Tree *tree, Entry entry, unsigned height, uint64_t offset,
+ uint64_t length, bool (*visit)(void *, PureSpan), void *context) {
+ if (!height) {
+ const unsigned char *bytes = leaf_bytes(tree, entry);
+ if (!bytes || offset > entry.bytes || length > entry.bytes - offset) return 0;
+ return visit(context, (PureSpan){bytes + offset, (size_t)length}) ? 1 : -1;
+ }
+ Entry kids[FAN_MAX];
+ size_t count = children(tree, entry, height, kids);
+ if (!count) return 0;
+ uint64_t start = 0;
+ for (size_t i = 0; i < count && length; ++i) {
+ uint64_t end = start + kids[i].bytes;
+ if (offset < end) {
+ uint64_t inside = offset - start, take = kids[i].bytes - inside;
+ if (take > length) take = length;
+ int result = visit_range(tree, kids[i], height - 1, inside, take, visit, context);
+ if (result != 1) return result;
+ offset += take;
+ length -= take;
+ }
+ start = end;
+ }
+ return length ? 0 : 1;
+}
+
+static bool copy_span(void *context, PureSpan span) {
+ unsigned char **out = context;
+ memcpy(*out, span.data, span.size);
+ *out += span.size;
+ return true;
+}
+
+static bool copy_range(const Tree *tree, Entry entry, unsigned height, uint64_t offset,
+ uint64_t length, unsigned char *out) {
+ return !length || visit_range(tree, entry, height, offset, length, copy_span, &out) == 1;
+}
+
+/* The bytes of a range held by one leaf, or NULL when it spans leaves. */
+static const unsigned char *single_leaf(const Tree *tree, Entry entry, unsigned height,
+ uint64_t offset, uint64_t length) {
+ while (height) {
+ Entry kids[FAN_MAX];
+ size_t count = children(tree, entry, height, kids), i = 0;
+ uint64_t start = 0;
+ if (!count) return NULL;
+ while (i + 1 < count && offset >= start + kids[i].bytes) start += kids[i++].bytes;
+ if (offset - start > kids[i].bytes || length > kids[i].bytes - (offset - start)) return NULL;
+ entry = kids[i];
+ offset -= start;
+ --height;
+ }
+ const unsigned char *bytes = leaf_bytes(tree, entry);
+ return bytes && offset <= entry.bytes && length <= entry.bytes - offset ? bytes + offset : NULL;
+}
+
+static bool push(List *list, Entry entry) {
+ if (list->count == list->capacity) {
+ size_t capacity = list->capacity ? 2 * list->capacity : 8;
+ Entry *items = realloc(list->items, capacity * sizeof(*items));
+ if (!items) return false;
+ list->items = items;
+ list->capacity = capacity;
+ }
+ list->items[list->count++] = entry;
+ return true;
+}
+
+static bool add_node(Tree *tree, Node node, uint64_t *index) {
+ if (tree->count == tree->capacity) {
+ size_t capacity = tree->capacity ? 2 * tree->capacity : 64;
+ Node *nodes = realloc(tree->nodes, capacity * sizeof(*nodes));
+ if (!nodes) return false;
+ tree->nodes = nodes;
+ tree->capacity = capacity;
+ }
+ tree->nodes[tree->count] = node;
+ *index = tree->count++;
+ return true;
+}
+
+static void take(Source *source, unsigned char *out, size_t length) {
+ while (length) {
+ PureSpan part = source->part[source->index];
+ size_t left = part.size - source->offset;
+ if (!left) { ++source->index; source->offset = 0; continue; }
+ size_t n = left < length ? left : length;
+ memcpy(out, part.data + source->offset, n);
+ out += n;
+ length -= n;
+ source->offset += n;
+ }
+}
+
+/* Balanced leaves: every leaf of a multi-leaf result holds >= LEAF_MAX / 2. */
+static bool chunk_bytes(Tree *tree, Source *source, uint64_t total, List *out) {
+ uint64_t pieces = (total + LEAF_MAX - 1) / LEAF_MAX;
+ for (uint64_t i = 0; i < pieces; ++i) {
+ size_t size = (size_t)(total / pieces + (i < total % pieces));
+ unsigned char *bytes = malloc(size);
+ uint64_t index;
+ if (!bytes) return false;
+ take(source, bytes, size);
+ if (!add_node(tree, (Node){.bytes = bytes, .size = size}, &index)) { free(bytes); return false; }
+ if (!push(out, (Entry){LEAF_BIT | MEMORY_BIT | index, size})) return false;
+ }
+ return true;
+}
+
+static bool chunk_entries(Tree *tree, const Entry *items, size_t count, unsigned height, List *out) {
+ size_t pieces = (count + FAN_MAX - 1) / FAN_MAX;
+ for (size_t i = 0, at = 0; i < pieces; ++i) {
+ size_t size = count / pieces + (i < count % pieces);
+ Entry *kids = malloc(size * sizeof(*kids));
+ uint64_t sum = 0, index;
+ if (!kids) return false;
+ memcpy(kids, items + at, size * sizeof(*kids));
+ for (size_t k = 0; k < size; ++k) sum += kids[k].bytes;
+ if (!add_node(tree, (Node){.children = kids, .size = size, .height = height}, &index)) {
+ free(kids);
+ return false;
+ }
+ if (!push(out, (Entry){MEMORY_BIT | index, sum})) return false;
+ at += size;
+ }
+ return true;
+}
+
+/* Only nodes created by this edit can be underfull; stored siblings are left as they are. */
+static bool underfull(const Tree *tree, Entry entry, unsigned height) {
+ if (!(entry.ref & MEMORY_BIT)) return false;
+ return height ? tree->nodes[entry.ref & LOCATION].size < FAN_MIN : entry.bytes < LEAF_MIN;
+}
+
+static bool replace(List *list, size_t at, size_t removed, const List *with) {
+ size_t count = list->count - removed + with->count;
+ if (count > list->capacity) {
+ Entry *items = realloc(list->items, count * sizeof(*items));
+ if (!items) return false;
+ list->items = items;
+ list->capacity = count;
+ }
+ memmove(list->items + at + with->count, list->items + at + removed,
+ (list->count - at - removed) * sizeof(*list->items));
+ memcpy(list->items + at, with->items, with->count * sizeof(*list->items));
+ list->count = count;
+ return true;
+}
+
+static bool merge(Tree *tree, Entry a, Entry b, unsigned height, List *out);
+
+/* Merge each new underfull node with a neighbour. A merge removes a node or
+ * yields two nodes of at least half capacity, so the loop terminates. */
+static bool repair(Tree *tree, List *all, unsigned height) {
+ bool ok = true;
+ for (size_t i = 0; ok && all->count > 1 && i < all->count;) {
+ if (!underfull(tree, all->items[i], height)) { ++i; continue; }
+ size_t at = i + 1 < all->count ? i : i - 1;
+ List merged = {0};
+ ok = merge(tree, all->items[at], all->items[at + 1], height, &merged) &&
+ replace(all, at, 2, &merged);
+ free(merged.items);
+ i = at;
+ }
+ return ok;
+}
+
+static bool merge(Tree *tree, Entry a, Entry b, unsigned height, List *out) {
+ if (!height) {
+ const unsigned char *x = leaf_bytes(tree, a), *y = leaf_bytes(tree, b);
+ if (!x || !y) return false;
+ Source source = {.part = {{x, (size_t)a.bytes}, {y, (size_t)b.bytes}}};
+ return chunk_bytes(tree, &source, a.bytes + b.bytes, out);
+ }
+ Entry kids[FAN_MAX];
+ List all = {0};
+ size_t count = children(tree, a, height, kids);
+ bool ok = count != 0;
+ for (size_t i = 0; ok && i < count; ++i) ok = push(&all, kids[i]);
+ count = ok ? children(tree, b, height, kids) : 0;
+ ok = count != 0;
+ for (size_t i = 0; ok && i < count; ++i) ok = push(&all, kids[i]);
+ /* A small child carried in from a one-child parent is repaired here too. */
+ ok = ok && repair(tree, &all, height - 1) && chunk_entries(tree, all.items, all.count, height, out);
+ free(all.items);
+ return ok;
+}
+
+/* Replace [offset, offset + remove) of entry with insert. Appends the
+ * replacement nodes, all at entry's height, to out (possibly none). */
+static bool edit(Tree *tree, Entry entry, unsigned height, uint64_t offset, uint64_t remove,
+ PureSpan insert, List *out) {
+ if (offset > entry.bytes || remove > entry.bytes - offset) return false;
+ if (!height) {
+ const unsigned char *bytes = leaf_bytes(tree, entry);
+ if (!bytes) return false;
+ Source source = {.part = {{bytes, (size_t)offset}, insert,
+ {bytes + offset + remove, (size_t)(entry.bytes - offset - remove)}}};
+ return chunk_bytes(tree, &source, entry.bytes - remove + insert.size, out);
+ }
+ Entry kids[FAN_MAX];
+ size_t count = children(tree, entry, height, kids), first = count, final = count;
+ if (!count) return false;
+ uint64_t last = remove ? offset + remove - 1 : offset, start = 0, first_start = 0, final_start = 0;
+ for (size_t i = 0; i < count; ++i) {
+ uint64_t end = start + kids[i].bytes;
+ bool tail = i + 1 == count;
+ if (first == count && (offset < end || tail)) { first = i; first_start = start; }
+ if (first != count && (last < end || tail)) { final = i; final_start = start; break; }
+ start = end;
+ }
+ List middle = {0}, all = {0};
+ bool ok = first == final ?
+ edit(tree, kids[first], height - 1, offset - first_start, remove, insert, &middle) :
+ edit(tree, kids[first], height - 1, offset - first_start,
+ first_start + kids[first].bytes - offset, insert, &middle) &&
+ edit(tree, kids[final], height - 1, 0, offset + remove - final_start, (PureSpan){0}, &middle);
+ for (size_t i = 0; ok && i < first; ++i) ok = push(&all, kids[i]);
+ for (size_t i = 0; ok && i < middle.count; ++i) ok = push(&all, middle.items[i]);
+ for (size_t i = final + 1; ok && i < count; ++i) ok = push(&all, kids[i]);
+ free(middle.items);
+ ok = ok && repair(tree, &all, height - 1) && chunk_entries(tree, all.items, all.count, height, out);
+ free(all.items);
+ return ok;
+}
+
+/* The new root of root with [offset, offset + remove) replaced by insert. */
+static bool rebuild(Tree *tree, Entry root, uint64_t offset, uint64_t remove,
+ PureSpan insert, Entry *out) {
+ unsigned height;
+ List list = {0};
+ bool ok = root_height(tree, root, &height);
+ if (ok && !root.bytes) {
+ Source source = {.part = {insert}};
+ ok = !offset && !remove && chunk_bytes(tree, &source, insert.size, &list);
+ } else if (ok) ok = edit(tree, root, height, offset, remove, insert, &list);
+ while (ok && list.count > 1) {
+ List up = {0};
+ ok = height < HEIGHT_MAX && chunk_entries(tree, list.items, list.count, ++height, &up);
+ free(list.items);
+ list = up;
+ }
+ Entry result = ok && list.count ? list.items[0] : (Entry){0};
+ free(list.items);
+ while (ok && height && result.bytes) {
+ Entry kids[FAN_MAX];
+ size_t count = children(tree, result, height, kids);
+ if (count != 1) { ok = count > 1; break; }
+ result = kids[0];
+ --height;
+ }
+ *out = result;
+ return ok;
+}
+
+static bool flush(Writer *writer) {
+ if (writer->used && !writer->failed &&
+ !write_at(writer->fd, writer->buffer, writer->used, writer->at - writer->used))
+ writer->failed = true;
+ writer->used = 0;
+ return !writer->failed;
+}
+
+static void put_bytes(Writer *writer, const unsigned char *bytes, size_t length) {
+ while (length && !writer->failed) {
+ if (writer->used == BUFFER && !flush(writer)) return;
+ size_t n = BUFFER - writer->used < length ? BUFFER - writer->used : length;
+ memcpy(writer->buffer + writer->used, bytes, n);
+ writer->used += n;
+ writer->at += n;
+ bytes += n;
+ length -= n;
+ }
+}
+
+static uint64_t put_branch(Writer *writer, const Entry *kids, size_t count, unsigned height) {
+ static const unsigned char zero[8];
+ put_bytes(writer, zero, (size_t)((8 - writer->at % 8) % 8));
+ uint64_t at = writer->at;
+ unsigned char header[8];
+ memcpy(header, branch_magic, sizeof(branch_magic));
+ header[4] = (unsigned char)count;
+ header[5] = (unsigned char)(count >> 8);
+ header[6] = (unsigned char)height;
+ header[7] = (unsigned char)(height >> 8);
+ put_bytes(writer, header, sizeof(header));
+ for (size_t i = 0; i < count; ++i) {
+ unsigned char pair[16];
+ encode64(pair, kids[i].ref);
+ encode64(pair + 8, kids[i].bytes);
+ put_bytes(writer, pair, sizeof(pair));
+ }
+ return at;
+}
+
+/* Append a subtree's arena nodes, children first, and set entry to the
+ * stored reference without changing the arena. With relocate, stored nodes
+ * are copied too; compaction uses this to rewrite reachable nodes. */
+static bool write_subtree(const Tree *tree, Writer *writer, Entry *entry, unsigned height,
+ bool relocate) {
+ if (!entry->bytes || (!(entry->ref & MEMORY_BIT) && !relocate)) return true;
+ if (!height) {
+ const unsigned char *bytes = leaf_bytes(tree, *entry);
+ if (!bytes) return false;
+ entry->ref = LEAF_BIT | writer->at;
+ put_bytes(writer, bytes, (size_t)entry->bytes);
+ return !writer->failed;
+ }
+ Entry kids[FAN_MAX];
+ size_t count = children(tree, *entry, height, kids);
+ if (!count) return false;
+ for (size_t i = 0; i < count; ++i)
+ if (!write_subtree(tree, writer, &kids[i], height - 1, relocate)) return false;
+ entry->ref = put_branch(writer, kids, count, height);
+ return !writer->failed;
+}
+
+static bool write_root(const Tree *tree, Writer *writer, Entry *root, bool relocate) {
+ unsigned height;
+ return root_height(tree, *root, &height) && write_subtree(tree, writer, root, height, relocate);
+}
+
+static bool read_record(const Tree *tree, Entry catalog, uint64_t index,
+ unsigned char record[RECORD], Entry *document, int64_t *revision) {
+ unsigned height;
+ if (!root_height(tree, catalog, &height) ||
+ !copy_range(tree, catalog, height, index * RECORD, RECORD, record)) return false;
+ *document = (Entry){decode64(record + KEY), decode64(record + KEY + 8)};
+ uint64_t stored = decode64(record + KEY + 16);
+ *revision = (int64_t)stored;
+ return stored && stored <= (uint64_t)INT64_MAX && !document->ref == !document->bytes &&
+ document->bytes <= PURE_OBJECT_LIMIT;
+}
+
+/* Binary search over records ordered by their zero-padded key bytes. */
+static bool find(const Tree *tree, Entry catalog, const char *key, uint64_t *index,
+ bool *found, Entry *document, int64_t *revision) {
+ unsigned char wanted[KEY] = {0}, record[RECORD];
+ memcpy(wanted, key, strlen(key));
+ uint64_t low = 0, high = catalog.bytes / RECORD;
+ *found = false;
+ *document = (Entry){0};
+ *revision = 0;
+ while (low < high) {
+ uint64_t middle = low + (high - low) / 2;
+ if (!read_record(tree, catalog, middle, record, document, revision)) return false;
+ int order = memcmp(wanted, record, KEY);
+ if (!order) { *index = middle; *found = true; return true; }
+ if (order < 0) high = middle; else low = middle + 1;
+ }
+ *document = (Entry){0};
+ *revision = 0;
+ *index = low;
+ return true;
+}
+
+static int create_temporary(int directory, char name[96]) {
+ for (unsigned attempt = 0; attempt < 64; ++attempt) {
+ if (temporary_sequence == UINT64_MAX) break;
+ snprintf(name, 96, ".pending-%ld-%" PRIu64, (long)getpid(), ++temporary_sequence);
+ int fd = openat(directory, name, O_RDWR | O_CREAT | O_EXCL | O_NOFOLLOW | O_CLOEXEC, 0600);
+ if (fd >= 0) return fd;
+ if (errno != EEXIST) break;
+ }
+ name[0] = 0; /* Never remove an existing file on a name collision. */
+ return -1;
+}
+
+/* Publish a fresh file with both slots pointing at head. */
+static bool install(int directory, char temporary[96], int fd, const Head *head) {
+ unsigned char slots[2 * SLOT];
+ encode_head(slots, head);
+ memcpy(slots + SLOT, slots, SLOT);
+ bool ok = write_at(fd, slots, sizeof(slots), 0) && sync_fd(fd);
+ if (close(fd)) ok = false;
+ ok = ok && !renameat(directory, temporary, directory, tree_name);
+ if (ok) temporary[0] = 0;
+ return ok && sync_fd(directory);
+}
+
+/* Rewrite only reachable nodes. Open readers keep their mapping of the
+ * previous file. A failure leaves the published file untouched. */
+static void compact(int directory, int source, const Head *head) {
+ Tree tree;
+ if (!map_tree(source, head, &tree)) return;
+ char temporary[96];
+ int fd = create_temporary(directory, temporary);
+ Writer *writer = fd >= 0 ? malloc(sizeof(*writer)) : NULL;
+ uint64_t count = head->catalog_bytes / RECORD;
+ unsigned char *records = count ? malloc((size_t)(count * RECORD)) : NULL;
+ Entry catalog = {head->catalog, head->catalog_bytes};
+ unsigned height;
+ bool ok = writer && (!count || records) && root_height(&tree, catalog, &height) &&
+ (!count || copy_range(&tree, catalog, height, 0, count * RECORD, records));
+ if (writer) *writer = (Writer){.fd = fd, .at = DATA};
+ for (uint64_t i = 0; ok && i < count; ++i) {
+ unsigned char *record = records + i * RECORD;
+ Entry document = {decode64(record + KEY), decode64(record + KEY + 8)};
+ unsigned document_height;
+ ok = root_height(&tree, document, &document_height) &&
+ write_subtree(&tree, writer, &document, document_height, true);
+ encode64(record + KEY, document.ref);
+ }
+ Entry fresh = {0};
+ ok = ok && rebuild(&tree, (Entry){0}, 0, 0, (PureSpan){records, (size_t)(count * RECORD)}, &fresh) &&
+ write_root(&tree, writer, &fresh, false) && flush(writer);
+ if (ok) {
+ Head compacted = {head->seq + 1, writer->at, fresh.ref, fresh.bytes, writer->at - DATA};
+ ok = install(directory, temporary, fd, &compacted);
+ fd = -1;
+ }
+ if (fd >= 0) close(fd);
+ if (*temporary) unlinkat(directory, temporary, 0);
+ free(records);
+ free(writer);
+ release_tree(&tree);
+}
+
+static bool lock_writer(int lock, bool wait) {
+ struct flock hold = {.l_type = F_WRLCK, .l_whence = SEEK_SET, .l_start = 0, .l_len = 0};
+ int rc;
+ do { rc = fcntl(lock, wait ? F_SETLKW : F_SETLK, &hold); } while (rc && wait && errno == EINTR);
+ return rc == 0;
+}
+
+PureResult pure_store_init(const char *path) {
+ if (!path || !*path) return PURE_STORE_ERROR;
+ bool created = mkdir(path, 0700) == 0;
+ if (!created && errno != EEXIST) return PURE_STORE_ERROR;
+ int directory = open(path, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
+ if (directory < 0) return PURE_STORE_ERROR; /* Never reinterpret an old DB file. */
+ int lock = openat(directory, ".writer.lock", O_RDWR | O_CREAT | O_NOFOLLOW | O_CLOEXEC, 0600);
+ struct stat info;
+ bool ok = lock >= 0 && fstat(lock, &info) == 0 && S_ISREG(info.st_mode) && lock_writer(lock, true);
+ if (ok) {
+ Head head;
+ int fd = openat(directory, tree_name, O_RDONLY | O_NOFOLLOW | O_CLOEXEC);
+ if (fd >= 0) {
+ ok = read_head(fd, &head); /* An unreadable file is never replaced. */
+ close(fd);
+ } else if (errno != ENOENT) ok = false;
+ else {
+ char temporary[96];
+ int created_fd = create_temporary(directory, temporary);
+ head = (Head){.seq = 1, .end = DATA};
+ ok = created_fd >= 0 && install(directory, temporary, created_fd, &head);
+ if (*temporary) unlinkat(directory, temporary, 0);
+ }
+ }
+ ok = ok && sync_fd(lock) && sync_fd(directory);
+ if (lock >= 0) close(lock);
+ close(directory);
+ if (ok && created) {
+ char *parent = strdup(path);
+ if (!parent) return PURE_STORE_ERROR;
+ size_t length = strlen(parent);
+ while (length > 1 && parent[length - 1] == '/') parent[--length] = 0;
+ char *slash = strrchr(parent, '/');
+ if (!slash) strcpy(parent, ".");
+ else if (slash == parent) parent[1] = 0;
+ else *slash = 0;
+ int fd = open(parent, O_RDONLY | O_DIRECTORY | O_CLOEXEC);
+ free(parent);
+ ok = fd >= 0 && sync_fd(fd);
+ if (fd >= 0) close(fd);
+ }
+ return ok ? PURE_OK : PURE_STORE_ERROR;
+}
+
+void pure_store_release(PureStoredView *view) {
+ if (view->mapped_size) munmap(view->mapping, view->mapped_size);
+ *view = (PureStoredView){0};
+}
+
+PureResult pure_store_select(const char *path, const char *key, size_t offset,
+ size_t length, bool whole, size_t minimum,
+ size_t available, PureStoredView *out) {
+ *out = (PureStoredView){0};
+ if (!valid_key(key)) return PURE_STORE_ERROR;
+ int directory = open(path, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
+ if (directory < 0) return PURE_STORE_ERROR;
+ int fd = openat(directory, tree_name, O_RDONLY | O_NOFOLLOW | O_CLOEXEC);
+ close(directory);
+ if (fd < 0) return PURE_STORE_ERROR;
+ Tree tree = {0};
+ Head head;
+ Entry document;
+ uint64_t index;
+ int64_t revision;
+ bool found;
+ unsigned height;
+ PureResult result = PURE_STORE_ERROR;
+ if (!read_head(fd, &head) || !map_tree(fd, &head, &tree) ||
+ !find(&tree, (Entry){head.catalog, head.catalog_bytes}, key, &index, &found,
+ &document, &revision)) goto finish;
+ if (!found) { result = PURE_NOT_FOUND; goto finish; }
+ size_t size = (size_t)document.bytes;
+ if (size < minimum || offset > size) { result = PURE_LIMIT; goto finish; }
+ if (whole) length = size - offset;
+ if (length > size - offset) { result = PURE_LIMIT; goto finish; }
+ if (length > available) { result = PURE_BUDGET; goto finish; }
+ if (length) {
+ if (!root_height(&tree, document, &height)) goto finish;
+ const unsigned char *direct = single_leaf(&tree, document, height, offset, length);
+ void *mapping;
+ size_t mapped_size, prefix = 0;
+ if (direct) {
+ /* One extent: map it in place, as the flat store does. */
+ long page = sysconf(_SC_PAGESIZE);
+ if (page <= 0) goto finish;
+ size_t start = (size_t)(direct - tree.map), aligned = start - start % (size_t)page;
+ prefix = start - aligned;
+ mapped_size = prefix + length;
+ mapping = mmap(NULL, mapped_size, PROT_READ, MAP_PRIVATE, fd, (off_t)aligned);
+ if (mapping == MAP_FAILED) goto finish;
+ } else {
+ /* A contiguous view of several extents is assembled for this API. */
+ mapped_size = length;
+ mapping = mmap(NULL, mapped_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+ if (mapping == MAP_FAILED) goto finish;
+ if (!copy_range(&tree, document, height, offset, length, mapping) ||
+ mprotect(mapping, mapped_size, PROT_READ)) {
+ munmap(mapping, mapped_size);
+ goto finish;
+ }
+ }
+ out->mapping = mapping;
+ out->mapped_size = mapped_size;
+ out->body = (PureSpan){(const unsigned char *)mapping + prefix, length};
+ }
+ out->revision = revision;
+ out->object_size = size;
+ result = PURE_OK;
+finish:
+ release_tree(&tree);
+ close(fd);
+ return result;
+}
+
+PureResult pure_store_get(const char *path, const char *key, size_t available, PureStoredView *out) {
+ return pure_store_select(path, key, 0, 0, true, 0, available, out);
+}
+
+PureResult pure_store_get_range(const char *path, const char *key, size_t offset,
+ size_t length, size_t available, PureStoredView *out) {
+ return pure_store_select(path, key, offset, length, false, 0, available, out);
+}
+
+/* A locked writer over the current published file. */
+typedef struct {
+ int directory, lock, fd;
+ Head head;
+ Tree tree;
+ struct stat file;
+} Session;
+
+static void close_session(Session *session) {
+ release_tree(&session->tree);
+ if (session->fd >= 0) close(session->fd);
+ if (session->lock >= 0) close(session->lock);
+ if (session->directory >= 0) close(session->directory);
+}
+
+static PureResult open_session(const char *path, Session *session) {
+ *session = (Session){.directory = -1, .lock = -1, .fd = -1};
+ session->directory = open(path, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
+ if (session->directory < 0) return PURE_STORE_ERROR;
+ session->lock = openat(session->directory, ".writer.lock", O_RDWR | O_NOFOLLOW | O_CLOEXEC);
+ struct stat info;
+ if (session->lock < 0 || fstat(session->lock, &info) || !S_ISREG(info.st_mode)) return PURE_STORE_ERROR;
+ if (!lock_writer(session->lock, false))
+ return errno == EACCES || errno == EAGAIN ? PURE_BUSY : PURE_STORE_ERROR;
+ session->fd = openat(session->directory, tree_name, O_RDWR | O_NOFOLLOW | O_CLOEXEC);
+ if (session->fd < 0 || !read_head(session->fd, &session->head) ||
+ !map_tree(session->fd, &session->head, &session->tree) || fstat(session->fd, &session->file))
+ return PURE_STORE_ERROR;
+ /* Bytes after the published head belong to a commit nobody acknowledged. */
+ if ((uint64_t)session->file.st_size > session->head.end &&
+ ftruncate(session->fd, (off_t)session->head.end)) return PURE_STORE_ERROR;
+ return PURE_OK;
+}
+
+/* The document's new nodes are already appended at writer. Record it in the
+ * catalog, then make the commit durable before its head becomes visible. */
+static PureResult publish(Session *session, Writer *writer, const char *key, uint64_t index,
+ bool found, Entry document, int64_t *revision) {
+ unsigned char record[RECORD] = {0};
+ Entry catalog = {session->head.catalog, session->head.catalog_bytes}, updated;
+ memcpy(record, key, strlen(key));
+ encode64(record + KEY, document.ref);
+ encode64(record + KEY + 8, document.bytes);
+ encode64(record + KEY + 16, (uint64_t)*revision + 1);
+ bool written = rebuild(&session->tree, catalog, index * RECORD, found ? RECORD : 0,
+ (PureSpan){record, RECORD}, &updated) &&
+ write_root(&session->tree, writer, &updated, false) && flush(writer);
+ if (!written || fault(PURE_AFTER_HEADER) || fault(PURE_BEFORE_FILE_SYNC) ||
+ !sync_fd(session->fd) || fault(PURE_BEFORE_RENAME)) {
+ if (ftruncate(session->fd, (off_t)session->head.end) == 0) sync_fd(session->fd);
+ return PURE_STORE_ERROR;
+ }
+ Head published = {session->head.seq + 1, writer->at, updated.ref, updated.bytes, session->head.live};
+ unsigned char slot[SLOT];
+ encode_head(slot, &published);
+ ++*revision; /* Past this point the head may already be visible. */
+ if (!write_at(session->fd, slot, SLOT, (published.seq % 2) * SLOT) ||
+ fault(PURE_BEFORE_DIR_SYNC) || !sync_fd(session->fd)) return PURE_UNCERTAIN;
+ if (published.end - DATA > 4 * published.live + COMPACT_SLACK)
+ compact(session->directory, session->fd, &published);
+ return PURE_OK;
+}
+
+static PureResult mutate(const char *path, const char *key, int64_t expected,
+ PureSpan body, bool splice, size_t offset, size_t remove,
+ size_t max_size, int64_t *revision) {
+ *revision = 0;
+ if (!valid_key(key) || (body.size && !body.data)) return PURE_STORE_ERROR;
+ if (body.size > PURE_OBJECT_LIMIT || max_size > PURE_OBJECT_LIMIT || expected < 0) return PURE_LIMIT;
+ Session session;
+ Writer *writer = NULL;
+ Entry document, next;
+ uint64_t index;
+ bool found;
+ PureResult result = open_session(path, &session);
+ if (result != PURE_OK) goto finish;
+ result = PURE_STORE_ERROR;
+ if (!find(&session.tree, (Entry){session.head.catalog, session.head.catalog_bytes}, key,
+ &index, &found, &document, revision)) goto finish;
+ if (*revision != expected) { result = PURE_CONFLICT; goto finish; }
+ if (*revision == INT64_MAX) { result = PURE_LIMIT; goto finish; }
+ uint64_t old_size = document.bytes;
+ if (splice && (offset > old_size || remove > old_size - offset ||
+ old_size - remove > max_size ||
+ body.size > max_size - (old_size - remove))) { result = PURE_LIMIT; goto finish; }
+ writer = malloc(sizeof(*writer));
+ if (!writer || !rebuild(&session.tree, splice ? document : (Entry){0}, splice ? offset : 0,
+ splice ? remove : 0, body, &next)) goto finish;
+ *writer = (Writer){.fd = session.fd, .at = session.head.end};
+ if (!write_root(&session.tree, writer, &next, false)) {
+ if (ftruncate(session.fd, (off_t)session.head.end) == 0) sync_fd(session.fd);
+ goto finish;
+ }
+ result = publish(&session, writer, key, index, found, next, revision);
+finish:
+ free(writer);
+ close_session(&session);
+ return result;
+}
+
+/* Keys come from the catalog in ascending byte order. */
+PureResult pure_store_keys(const char *path, bool (*visit)(void *, const char *key),
+ void *context) {
+ int directory = open(path, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
+ if (directory < 0) return PURE_STORE_ERROR;
+ int fd = openat(directory, tree_name, O_RDONLY | O_NOFOLLOW | O_CLOEXEC);
+ close(directory);
+ if (fd < 0) return PURE_STORE_ERROR;
+ Tree tree = {0};
+ Head head;
+ PureResult result = PURE_STORE_ERROR;
+ if (read_head(fd, &head) && map_tree(fd, &head, &tree)) {
+ Entry catalog = {head.catalog, head.catalog_bytes}, document;
+ unsigned char record[RECORD];
+ char key[KEY + 1] = {0};
+ int64_t revision;
+ result = PURE_OK;
+ for (uint64_t i = 0; i < catalog.bytes / RECORD; ++i) {
+ if (!read_record(&tree, catalog, i, record, &document, &revision)) { result = PURE_STORE_ERROR; break; }
+ memcpy(key, record, KEY);
+ if (!valid_key(key)) { result = PURE_STORE_ERROR; break; }
+ if (!visit(context, key)) break;
+ }
+ }
+ release_tree(&tree);
+ close(fd);
+ return result;
+}
+
+PureResult pure_store_put(const char *path, const char *key, int64_t expected,
+ PureSpan body, int64_t *revision) {
+ return mutate(path, key, expected, body, false, 0, 0, PURE_OBJECT_LIMIT, revision);
+}
+
+PureResult pure_store_splice(const char *path, const char *key, int64_t expected,
+ size_t offset, size_t remove, PureSpan insert,
+ size_t max_size, int64_t *revision) {
+ return mutate(path, key, expected, insert, true, offset, remove, max_size, revision);
+}
+
+#ifdef PURE_STORE_TEST
+static bool shape(const Tree *tree, Entry entry, unsigned height, bool root,
+ uint64_t *leaves, uint64_t *small) {
+ if (!height) {
+ ++*leaves;
+ if (!root && entry.bytes < LEAF_MIN) ++*small;
+ return leaf_bytes(tree, entry) != NULL;
+ }
+ Entry kids[FAN_MAX];
+ size_t count = children(tree, entry, height, kids);
+ if (!count || (root && count < 2)) return false;
+ if (!root && count < FAN_MIN) ++*small;
+ for (size_t i = 0; i < count; ++i)
+ if (!shape(tree, kids[i], height - 1, false, leaves, small)) return false;
+ return true;
+}
+
+/* Test-only: the stored height, leaf count and number of underfull nodes. */
+bool pure_tree_test_shape(const char *path, const char *key, unsigned *height,
+ uint64_t *leaves, uint64_t *small, uint64_t *file_end) {
+ int directory = open(path, O_RDONLY | O_DIRECTORY | O_CLOEXEC);
+ int fd = directory >= 0 ? openat(directory, tree_name, O_RDONLY | O_CLOEXEC) : -1;
+ if (directory >= 0) close(directory);
+ Tree tree = {0};
+ Head head;
+ Entry document;
+ uint64_t index;
+ int64_t revision;
+ bool found = false;
+ *leaves = *small = 0;
+ bool ok = fd >= 0 && read_head(fd, &head) && map_tree(fd, &head, &tree) &&
+ find(&tree, (Entry){head.catalog, head.catalog_bytes}, key, &index, &found,
+ &document, &revision) && found && root_height(&tree, document, height) &&
+ (!document.bytes || shape(&tree, document, *height, true, leaves, small));
+ if (ok) *file_end = head.end;
+ release_tree(&tree);
+ if (fd >= 0) close(fd);
+ return ok;
+}
+#endif
diff --git a/prototypes/connections/test-splice.c b/prototypes/connections/test-splice.c
index bf3c0a9..51970d3 100644
--- a/prototypes/connections/test-splice.c
+++ b/prototypes/connections/test-splice.c
@@ -126,12 +126,15 @@ static void storage(const char *path) {
expect(path, "work", revision, (PureSpan){oracle, length});
CHECK(pure_store_splice(path, "work", revision, 0, length, (PureSpan){0}, PURE_OBJECT_LIMIT, &revision) == PURE_OK);
expect(path, "work", revision, (PureSpan){0});
+#ifndef PURE_STORE_TREE
+ /* Per-object file formats; test-tree-store covers the same limit. */
char file[256];
snprintf(file, sizeof(file), "%s/obj-work", path);
int fd = open(file, O_WRONLY);
const unsigned char maximum[] = {255,255,255,255,255,255,255,127};
CHECK(fd >= 0 && pwrite(fd, maximum, 8, 8) == 8 && close(fd) == 0);
CHECK(pure_store_splice(path, "work", INT64_MAX, 0, 0, (PureSpan){0}, PURE_OBJECT_LIMIT, &revision) == PURE_LIMIT);
+#endif
}
static void runtime(const char *path) {
diff --git a/prototypes/connections/test-tree-store.c b/prototypes/connections/test-tree-store.c
new file mode 100644
index 0000000..3a1e8f0
--- /dev/null
+++ b/prototypes/connections/test-tree-store.c
@@ -0,0 +1,394 @@
+#define _POSIX_C_SOURCE 200809L
+#include "store.h"
+#include <assert.h>
+#include <dirent.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/wait.h>
+#include <unistd.h>
+
+#define CHECK(x) do { if (!(x)) { fprintf(stderr, "%s:%d: %s\n", __FILE__, __LINE__, #x); abort(); } } while (0)
+
+static int fail_stage = -1;
+static int crash;
+int pure_store_test_fault(PureStoreStage stage) {
+ if ((int)stage != fail_stage) return 0;
+ if (crash) _exit(77);
+ return 1;
+}
+bool pure_tree_test_shape(const char *path, const char *key, unsigned *height,
+ uint64_t *leaves, uint64_t *small, uint64_t *file_end);
+
+enum { KEYS = 4, CAPACITY = 1 << 20 };
+typedef struct { unsigned char *bytes; size_t size; int64_t revision; } Oracle;
+
+static uint32_t random_value(uint32_t *state) {
+ *state ^= *state << 13;
+ *state ^= *state >> 17;
+ *state ^= *state << 5;
+ return *state;
+}
+
+static void expect(const char *path, const char *key, const Oracle *o) {
+ PureStoredView view;
+ CHECK(pure_store_get(path, key, PURE_OBJECT_LIMIT, &view) == PURE_OK);
+ CHECK(view.revision == o->revision && view.body.size == o->size);
+ CHECK(!o->size || !memcmp(view.body.data, o->bytes, o->size));
+ pure_store_release(&view);
+}
+
+static uint64_t file_size(const char *path) {
+ char name[512];
+ snprintf(name, sizeof(name), "%s/pure.tree", path);
+ struct stat info;
+ CHECK(stat(name, &info) == 0);
+ return (uint64_t)info.st_size;
+}
+
+static void fill(unsigned char *bytes, size_t size, uint32_t *state) {
+ for (size_t i = 0; i < size; ++i) bytes[i] = (unsigned char)random_value(state);
+}
+
+/* Random edits against a flat oracle, including views held across later
+ * commits and compactions. Every stored node must stay balanced. */
+static void oracle(const char *path) {
+ static const char *keys[KEYS] = {"alpha", "b", "gamma-long_key-0123456789", "z"};
+ Oracle docs[KEYS] = {{0}};
+ unsigned char *insert = malloc(65536);
+ for (unsigned k = 0; k < KEYS; ++k) CHECK((docs[k].bytes = malloc(CAPACITY)));
+ CHECK(insert);
+ uint32_t state = 0x9e3779b9u;
+ PureStoredView held = {0};
+ unsigned char *held_copy = malloc(CAPACITY);
+ size_t held_size = 0;
+ unsigned max_height = 0;
+ size_t max_size = 0, splices = 0;
+ for (unsigned step = 0; step < 6000; ++step) {
+ unsigned k = random_value(&state) % KEYS;
+ Oracle *o = &docs[k];
+ uint32_t kind = random_value(&state) % 100;
+ size_t count = kind < 70 ? random_value(&state) % 24 :
+ kind < 95 ? random_value(&state) % 6000 : random_value(&state) % 65536;
+ int64_t revision;
+ fill(insert, count, &state);
+ if (kind >= 98 || !o->revision) {
+ CHECK(pure_store_put(path, keys[k], o->revision, (PureSpan){insert, count}, &revision) == PURE_OK);
+ memcpy(o->bytes, insert, count);
+ o->size = count;
+ } else {
+ size_t offset = o->size ? random_value(&state) % (o->size + 1) : 0;
+ size_t remove = o->size - offset ? random_value(&state) % (o->size - offset + 1) : 0;
+ if (random_value(&state) % 24) remove = remove % 64;
+ if (o->size - remove + count > CAPACITY) { count = 0; remove = o->size - offset; }
+ CHECK(pure_store_splice(path, keys[k], o->revision, offset, remove,
+ (PureSpan){insert, count}, PURE_OBJECT_LIMIT, &revision) == PURE_OK);
+ memmove(o->bytes + offset + count, o->bytes + offset + remove, o->size - offset - remove);
+ memcpy(o->bytes + offset, insert, count);
+ o->size = o->size - remove + count;
+ ++splices;
+ }
+ if (o->size > max_size) max_size = o->size;
+ CHECK(revision == o->revision + 1);
+ o->revision = revision;
+ expect(path, keys[k], o);
+ unsigned height;
+ uint64_t leaves, small, end;
+ CHECK(pure_tree_test_shape(path, keys[k], &height, &leaves, &small, &end));
+ CHECK(small == 0);
+ if (height > max_height) max_height = height;
+ if (step % 97 == 0) {
+ if (held.mapped_size) {
+ CHECK(held.body.size == held_size && !memcmp(held.body.data, held_copy, held_size));
+ pure_store_release(&held);
+ }
+ CHECK(pure_store_get(path, keys[k], PURE_OBJECT_LIMIT, &held) == PURE_OK);
+ held_size = held.body.size;
+ if (held_size) memcpy(held_copy, held.body.data, held_size);
+ }
+ }
+ if (held.mapped_size) {
+ CHECK(held.body.size == held_size && !memcmp(held.body.data, held_copy, held_size));
+ pure_store_release(&held);
+ }
+ for (unsigned k = 0; k < KEYS; ++k) { expect(path, keys[k], &docs[k]); free(docs[k].bytes); }
+ CHECK(max_height >= 2 && max_height <= 4 && max_size > 262144);
+ printf("{\"oracle_steps\":6000,\"splices\":%zu,\"max_document_bytes\":%zu,\"max_height\":%u}\n",
+ splices, max_size, max_height);
+ free(insert);
+ free(held_copy);
+}
+
+/* A one-byte insert, delete or replacement in a 1 MiB document appends only
+ * its changed leaf and path, not another copy of the document. */
+static void amplification(const char *path) {
+ size_t size = 1048576;
+ unsigned char *bytes = malloc(size + 1);
+ uint32_t state = 7;
+ CHECK(bytes);
+ fill(bytes, size, &state);
+ int64_t revision;
+ CHECK(pure_store_put(path, "large", 0, (PureSpan){bytes, size}, &revision) == PURE_OK);
+ Oracle o = {bytes, size, revision};
+ unsigned char byte = 'Q';
+ const size_t edits[][3] = {{size / 2, 0, 1}, {size / 3, 1, 0}, {size - 1, 1, 1}, {0, 0, 1}};
+ uint64_t worst = 0;
+ for (size_t i = 0; i < sizeof(edits) / sizeof(*edits); ++i) {
+ uint64_t before = file_size(path);
+ size_t offset = edits[i][0], remove = edits[i][1], count = edits[i][2];
+ CHECK(pure_store_splice(path, "large", o.revision, offset, remove, (PureSpan){&byte, count},
+ PURE_OBJECT_LIMIT, &revision) == PURE_OK);
+ memmove(bytes + offset + count, bytes + offset + remove, o.size - offset - remove);
+ memcpy(bytes + offset, &byte, count);
+ o.size = o.size - remove + count;
+ o.revision = revision;
+ uint64_t after = file_size(path);
+ if (after > before && after - before > worst) worst = after - before;
+ expect(path, "large", &o);
+ }
+ /* Compaction may shrink the file; growth per edit is the claim. */
+ CHECK(worst > 0 && worst < 16384);
+ unsigned height;
+ uint64_t leaves, small, end;
+ CHECK(pure_tree_test_shape(path, "large", &height, &leaves, &small, &end));
+ CHECK(small == 0 && height == 2 && leaves >= 256 && leaves <= 512);
+ printf("{\"one_byte_edit_max_appended_bytes\":%llu,\"height\":%u,\"leaves\":%llu}\n",
+ (unsigned long long)worst, height, (unsigned long long)leaves);
+ free(bytes);
+}
+
+static void faults(const char *path) {
+ unsigned char text[] = "fault-free";
+ int64_t revision;
+ CHECK(pure_store_put(path, "fault", 0, (PureSpan){text, 10}, &revision) == PURE_OK);
+ Oracle o = {text, 10, revision};
+ uint64_t size = file_size(path);
+ for (fail_stage = PURE_AFTER_HEADER; fail_stage <= PURE_BEFORE_RENAME; ++fail_stage) {
+ CHECK(pure_store_splice(path, "fault", o.revision, 1, 3, (PureSpan){text, 5},
+ PURE_OBJECT_LIMIT, &revision) == PURE_STORE_ERROR);
+ expect(path, "fault", &o);
+ CHECK(file_size(path) == size);
+ }
+ fail_stage = PURE_BEFORE_DIR_SYNC;
+ CHECK(pure_store_splice(path, "fault", o.revision, 10, 0, (PureSpan){text, 2},
+ PURE_OBJECT_LIMIT, &revision) == PURE_UNCERTAIN);
+ fail_stage = -1;
+ unsigned char next[] = "fault-freefa";
+ o = (Oracle){next, 12, o.revision + 1};
+ CHECK(revision == o.revision);
+ expect(path, "fault", &o);
+}
+
+/* A process that dies at each publication stage leaves either the previous
+ * or the next complete version, and its unpublished bytes are discarded. */
+static void crashes(const char *path) {
+ unsigned char text[] = "crash-base";
+ int64_t revision;
+ CHECK(pure_store_put(path, "crash", 0, (PureSpan){text, 10}, &revision) == PURE_OK);
+ Oracle o = {text, 10, revision};
+ for (int stage = PURE_AFTER_HEADER; stage <= PURE_BEFORE_DIR_SYNC; ++stage) {
+ unsigned height;
+ uint64_t leaves, small, before, end;
+ CHECK(pure_tree_test_shape(path, "crash", &height, &leaves, &small, &before));
+ pid_t child = fork();
+ CHECK(child >= 0);
+ if (!child) {
+ crash = 1;
+ fail_stage = stage;
+ pure_store_splice(path, "crash", o.revision, 0, 1, (PureSpan){(const unsigned char *)"C", 1},
+ PURE_OBJECT_LIMIT, &revision);
+ _exit(0);
+ }
+ int status;
+ CHECK(waitpid(child, &status, 0) == child && WIFEXITED(status) && WEXITSTATUS(status) == 77);
+ if (stage == PURE_BEFORE_DIR_SYNC) { text[0] = 'C'; ++o.revision; }
+ expect(path, "crash", &o);
+ uint64_t garbage = file_size(path);
+ CHECK(pure_store_splice(path, "crash", o.revision, 10, 0, (PureSpan){(const unsigned char *)"", 0},
+ PURE_OBJECT_LIMIT, &revision) == PURE_OK);
+ o.revision = revision;
+ CHECK(pure_tree_test_shape(path, "crash", &height, &leaves, &small, &end));
+ CHECK(end == file_size(path));
+ /* Same-sized commit: equal growth only if the dead writer's tail was cut. */
+ if (stage != PURE_BEFORE_DIR_SYNC) CHECK(garbage > before && end - before == garbage - before);
+ }
+}
+
+/* A torn newest head slot exposes the previous commit, and the next writer
+ * overwrites that slot and the unpublished tail. */
+static void torn_head(const char *path) {
+ unsigned char first[] = "first", second[] = "second";
+ int64_t revision;
+ CHECK(pure_store_put(path, "torn", 0, (PureSpan){first, 5}, &revision) == PURE_OK);
+ Oracle o = {first, 5, revision};
+ CHECK(pure_store_put(path, "torn", o.revision, (PureSpan){second, 6}, &revision) == PURE_OK);
+ char name[512];
+ snprintf(name, sizeof(name), "%s/pure.tree", path);
+ int fd = open(name, O_RDWR);
+ unsigned char slots[128];
+ CHECK(fd >= 0 && pread(fd, slots, 128, 0) == 128);
+ uint64_t a = 0, b = 0;
+ for (unsigned i = 0; i < 8; ++i) { a |= (uint64_t)slots[8 + i] << (8 * i); b |= (uint64_t)slots[72 + i] << (8 * i); }
+ off_t newest = a > b ? 0 : 64;
+ unsigned char torn = slots[newest + 20] ^ 0x5a;
+ CHECK(pwrite(fd, &torn, 1, newest + 20) == 1 && close(fd) == 0);
+ expect(path, "torn", &o);
+ CHECK(pure_store_put(path, "torn", o.revision + 1, (PureSpan){first, 1}, &revision) == PURE_CONFLICT);
+ CHECK(pure_store_put(path, "torn", o.revision, (PureSpan){second, 3}, &revision) == PURE_OK);
+ o = (Oracle){second, 3, revision};
+ expect(path, "torn", &o);
+}
+
+/* A reader's view stays byte-identical while another process commits and
+ * compacts; readers take no lock. */
+static void isolation(const char *path) {
+ size_t size = 300000;
+ unsigned char *bytes = malloc(size);
+ uint32_t state = 99;
+ CHECK(bytes);
+ fill(bytes, size, &state);
+ int64_t revision;
+ CHECK(pure_store_put(path, "shared", 0, (PureSpan){bytes, size}, &revision) == PURE_OK);
+ PureStoredView view;
+ CHECK(pure_store_get(path, "shared", size, &view) == PURE_OK);
+ uint64_t before = file_size(path);
+ pid_t child = fork();
+ CHECK(child >= 0);
+ if (!child) {
+ int64_t at = revision;
+ unsigned char byte = 'x';
+ for (unsigned i = 0; i < 400; ++i)
+ if (pure_store_splice(path, "shared", at, (i * 7919u) % size, 1, (PureSpan){&byte, 1},
+ PURE_OBJECT_LIMIT, &at) != PURE_OK) _exit(1);
+ _exit(0);
+ }
+ int status;
+ CHECK(waitpid(child, &status, 0) == child && WIFEXITED(status) && WEXITSTATUS(status) == 0);
+ CHECK(view.revision == revision && view.body.size == size && !memcmp(view.body.data, bytes, size));
+ pure_store_release(&view);
+ PureStoredView latest;
+ CHECK(pure_store_get(path, "shared", size, &latest) == PURE_OK && latest.revision == revision + 400);
+ pure_store_release(&latest);
+ /* About 2 MB of path copies without compaction would exceed this. */
+ CHECK(file_size(path) < before + 3 * 524288);
+ free(bytes);
+}
+
+/* Damaged branch bytes are reported as storage errors, never trusted. */
+static void corruption(const char *path) {
+ unsigned char *bytes = malloc(200000);
+ uint32_t state = 3;
+ CHECK(bytes);
+ fill(bytes, 200000, &state);
+ int64_t revision;
+ CHECK(pure_store_put(path, "damaged", 0, (PureSpan){bytes, 200000}, &revision) == PURE_OK);
+ char name[512];
+ snprintf(name, sizeof(name), "%s/pure.tree", path);
+ uint64_t size = file_size(path);
+ int fd = open(name, O_RDWR);
+ CHECK(fd >= 0);
+ unsigned errors = 0;
+ for (unsigned i = 0; i < 3000; ++i) {
+ off_t at = (off_t)(128 + random_value(&state) % (size - 128));
+ unsigned char original, damaged;
+ CHECK(pread(fd, &original, 1, at) == 1);
+ damaged = original ^ (unsigned char)(1u << (random_value(&state) % 8));
+ CHECK(pwrite(fd, &damaged, 1, at) == 1);
+ PureStoredView view;
+ PureResult result = pure_store_get(path, "damaged", PURE_OBJECT_LIMIT, &view);
+ CHECK(result == PURE_OK || result == PURE_STORE_ERROR || result == PURE_NOT_FOUND);
+ if (result == PURE_OK) pure_store_release(&view); else ++errors;
+ CHECK(pwrite(fd, &original, 1, at) == 1);
+ }
+ CHECK(close(fd) == 0 && errors > 0);
+ free(bytes);
+}
+
+static void many_keys(const char *path) {
+ int64_t revision;
+ char key[32];
+ for (unsigned i = 0; i < 1024; ++i) {
+ unsigned id = (i * 617u) % 1024u;
+ snprintf(key, sizeof(key), "doc-%04u", id);
+ CHECK(pure_store_put(path, key, 0, (PureSpan){(const unsigned char *)key, strlen(key)}, &revision) == PURE_OK);
+ }
+ for (unsigned id = 0; id < 1024; ++id) {
+ snprintf(key, sizeof(key), "doc-%04u", id);
+ Oracle o = {(unsigned char *)key, strlen(key), 1};
+ expect(path, key, &o);
+ }
+ PureStoredView view;
+ CHECK(pure_store_get(path, "doc-1024", 64, &view) == PURE_NOT_FOUND);
+ CHECK(pure_store_get(path, "doc-0001", 3, &view) == PURE_BUDGET);
+ CHECK(pure_store_splice(path, "doc-0002", 1, 9, 0, (PureSpan){0}, PURE_OBJECT_LIMIT, &revision) == PURE_LIMIT);
+ CHECK(pure_store_put(path, "doc-0003", 0, (PureSpan){0}, &revision) == PURE_CONFLICT && revision == 1);
+}
+
+/* The newest catalog of a one-key store is one leaf; forge its revision. */
+static void revision_limit(const char *path) {
+ int64_t revision;
+ CHECK(pure_store_put(path, "work", 0, (PureSpan){(const unsigned char *)"w", 1}, &revision) == PURE_OK);
+ char name[512];
+ snprintf(name, sizeof(name), "%s/pure.tree", path);
+ int fd = open(name, O_RDWR);
+ unsigned char slots[128];
+ CHECK(fd >= 0 && pread(fd, slots, 128, 0) == 128);
+ uint64_t seq[2] = {0, 0}, catalog = 0;
+ for (unsigned s = 0; s < 2; ++s)
+ for (unsigned i = 0; i < 8; ++i) seq[s] |= (uint64_t)slots[64 * s + 8 + i] << (8 * i);
+ unsigned newest = seq[0] > seq[1] ? 0 : 1;
+ for (unsigned i = 0; i < 8; ++i) catalog |= (uint64_t)slots[64 * newest + 24 + i] << (8 * i);
+ CHECK(catalog >> 63);
+ const unsigned char maximum[] = {255,255,255,255,255,255,255,127};
+ off_t record = (off_t)(catalog & ((UINT64_C(1) << 62) - 1));
+ CHECK(pwrite(fd, maximum, 8, record + 80) == 8 && close(fd) == 0);
+ PureStoredView view;
+ CHECK(pure_store_get(path, "work", 1, &view) == PURE_OK && view.revision == INT64_MAX);
+ pure_store_release(&view);
+ CHECK(pure_store_splice(path, "work", INT64_MAX, 0, 0, (PureSpan){0}, PURE_OBJECT_LIMIT, &revision) == PURE_LIMIT);
+}
+
+static bool count_key(void *context, const char *key) {
+ (void)key;
+ ++*(unsigned *)context;
+ return true;
+}
+
+static void keys(const char *path) {
+ unsigned count = 0;
+ CHECK(pure_store_keys(path, count_key, &count) == PURE_OK && count == 0);
+ int64_t revision;
+ const char *names[] = {"m", "a", "zz", "a-b", "A"};
+ for (unsigned i = 0; i < 5; ++i)
+ CHECK(pure_store_put(path, names[i], 0, (PureSpan){0}, &revision) == PURE_OK);
+ CHECK(pure_store_keys(path, count_key, &count) == PURE_OK && count == 5);
+}
+
+static void clean(const char *path) {
+ DIR *entries = opendir(path);
+ CHECK(entries);
+ struct dirent *entry;
+ while ((entry = readdir(entries))) {
+ if (!strcmp(entry->d_name, ".") || !strcmp(entry->d_name, "..")) continue;
+ CHECK(strncmp(entry->d_name, ".pending-", 9));
+ CHECK(unlinkat(dirfd(entries), entry->d_name, 0) == 0);
+ }
+ CHECK(closedir(entries) == 0 && rmdir(path) == 0);
+}
+
+int main(void) {
+ void (*checks[])(const char *) = {oracle, amplification, faults, crashes, torn_head,
+ isolation, corruption, many_keys, revision_limit, keys};
+ for (size_t i = 0; i < sizeof(checks) / sizeof(*checks); ++i) {
+ char path[] = "/tmp/pure-tree-check-XXXXXX";
+ CHECK(mkdtemp(path) && pure_store_init(path) == PURE_OK);
+ checks[i](path);
+ CHECK(pure_store_init(path) == PURE_OK);
+ clean(path);
+ }
+ puts("{\"tree_store\":\"random-oracle-balance-held-views-one-byte-amplification-faults-"
+ "process-crash-torn-head-lockless-isolation-compaction-corruption-1024-keys-revision-limit-key-listing\"}");
+ return 0;
+}
diff --git a/research/experiments/008-tree-store-edit-cost/2026-09-25-ext4-linux-aarch64.json b/research/experiments/008-tree-store-edit-cost/2026-09-25-ext4-linux-aarch64.json
new file mode 100644
index 0000000..708346f
--- /dev/null
+++ b/research/experiments/008-tree-store-edit-cost/2026-09-25-ext4-linux-aarch64.json
@@ -0,0 +1,346 @@
+{
+ "device": "/dev/mmcblk0p2",
+ "filesystem": "ext4",
+ "machine": "aarch64",
+ "kernel": "6.18.39+rpt-rpi-2712",
+ "compiler": "cc (Debian 14.2.0-19) 14.2.0",
+ "edits_per_scenario": 32,
+ "results": [
+ {
+ "store": "flat",
+ "op": "insert-middle",
+ "document_bytes": 256,
+ "edits": 32,
+ "edit_median_ms": 9.305,
+ "edit_p90_ms": 20.037,
+ "edit_max_ms": 154.578,
+ "write_call_bytes_per_edit": 296,
+ "device_write_bytes_per_edit": 16384,
+ "store_file_bytes_after": 312,
+ "whole_read_median_ms": 0.008,
+ "read_checksum": 7776
+ },
+ {
+ "store": "paged",
+ "op": "insert-middle",
+ "document_bytes": 256,
+ "edits": 32,
+ "edit_median_ms": 8.463,
+ "edit_p90_ms": 11.859,
+ "edit_max_ms": 64.024,
+ "write_call_bytes_per_edit": 296,
+ "device_write_bytes_per_edit": 16384,
+ "store_file_bytes_after": 312,
+ "whole_read_median_ms": 0.01,
+ "read_checksum": 7776
+ },
+ {
+ "store": "tree",
+ "op": "insert-middle",
+ "document_bytes": 256,
+ "edits": 32,
+ "edit_median_ms": 8.343,
+ "edit_p90_ms": 11.553,
+ "edit_max_ms": 65.974,
+ "write_call_bytes_per_edit": 432,
+ "device_write_bytes_per_edit": 32768,
+ "store_file_bytes_after": 12272,
+ "whole_read_median_ms": 0.011,
+ "read_checksum": 7776
+ },
+ {
+ "store": "flat",
+ "op": "delete-middle",
+ "document_bytes": 256,
+ "edits": 32,
+ "edit_median_ms": 8.506,
+ "edit_p90_ms": 10.021,
+ "edit_max_ms": 65.646,
+ "write_call_bytes_per_edit": 263,
+ "device_write_bytes_per_edit": 16384,
+ "store_file_bytes_after": 248,
+ "whole_read_median_ms": 0.008,
+ "read_checksum": 6720
+ },
+ {
+ "store": "paged",
+ "op": "delete-middle",
+ "document_bytes": 256,
+ "edits": 32,
+ "edit_median_ms": 8.344,
+ "edit_p90_ms": 9.873,
+ "edit_max_ms": 65.228,
+ "write_call_bytes_per_edit": 263,
+ "device_write_bytes_per_edit": 16384,
+ "store_file_bytes_after": 248,
+ "whole_read_median_ms": 0.01,
+ "read_checksum": 6720
+ },
+ {
+ "store": "tree",
+ "op": "delete-middle",
+ "document_bytes": 256,
+ "edits": 32,
+ "edit_median_ms": 6.933,
+ "edit_p90_ms": 10.713,
+ "edit_max_ms": 11.529,
+ "write_call_bytes_per_edit": 399,
+ "device_write_bytes_per_edit": 32768,
+ "store_file_bytes_after": 11216,
+ "whole_read_median_ms": 0.011,
+ "read_checksum": 6720
+ },
+ {
+ "store": "flat",
+ "op": "replace-end",
+ "document_bytes": 256,
+ "edits": 32,
+ "edit_median_ms": 8.321,
+ "edit_p90_ms": 9.878,
+ "edit_max_ms": 27.943,
+ "write_call_bytes_per_edit": 280,
+ "device_write_bytes_per_edit": 16384,
+ "store_file_bytes_after": 280,
+ "whole_read_median_ms": 0.008,
+ "read_checksum": 6944
+ },
+ {
+ "store": "paged",
+ "op": "replace-end",
+ "document_bytes": 256,
+ "edits": 32,
+ "edit_median_ms": 8.409,
+ "edit_p90_ms": 9.723,
+ "edit_max_ms": 62.134,
+ "write_call_bytes_per_edit": 280,
+ "device_write_bytes_per_edit": 16384,
+ "store_file_bytes_after": 280,
+ "whole_read_median_ms": 0.01,
+ "read_checksum": 6944
+ },
+ {
+ "store": "tree",
+ "op": "replace-end",
+ "document_bytes": 256,
+ "edits": 32,
+ "edit_median_ms": 9.141,
+ "edit_p90_ms": 16.762,
+ "edit_max_ms": 62.869,
+ "write_call_bytes_per_edit": 416,
+ "device_write_bytes_per_edit": 32768,
+ "store_file_bytes_after": 11744,
+ "whole_read_median_ms": 0.011,
+ "read_checksum": 6944
+ },
+ {
+ "store": "flat",
+ "op": "append-end",
+ "document_bytes": 256,
+ "edits": 32,
+ "edit_median_ms": 9.068,
+ "edit_p90_ms": 10.062,
+ "edit_max_ms": 62.784,
+ "write_call_bytes_per_edit": 296,
+ "device_write_bytes_per_edit": 16384,
+ "store_file_bytes_after": 312,
+ "whole_read_median_ms": 0.012,
+ "read_checksum": 7984
+ },
+ {
+ "store": "paged",
+ "op": "append-end",
+ "document_bytes": 256,
+ "edits": 32,
+ "edit_median_ms": 8.613,
+ "edit_p90_ms": 10.037,
+ "edit_max_ms": 59.767,
+ "write_call_bytes_per_edit": 296,
+ "device_write_bytes_per_edit": 16384,
+ "store_file_bytes_after": 312,
+ "whole_read_median_ms": 0.015,
+ "read_checksum": 7984
+ },
+ {
+ "store": "tree",
+ "op": "append-end",
+ "document_bytes": 256,
+ "edits": 32,
+ "edit_median_ms": 6.468,
+ "edit_p90_ms": 11.041,
+ "edit_max_ms": 61.441,
+ "write_call_bytes_per_edit": 432,
+ "device_write_bytes_per_edit": 32768,
+ "store_file_bytes_after": 12272,
+ "whole_read_median_ms": 0.017,
+ "read_checksum": 7984
+ },
+ {
+ "store": "flat",
+ "op": "insert-middle",
+ "document_bytes": 1048576,
+ "edits": 32,
+ "edit_median_ms": 78.508,
+ "edit_p90_ms": 82.768,
+ "edit_max_ms": 83.984,
+ "write_call_bytes_per_edit": 1048616,
+ "device_write_bytes_per_edit": 1064960,
+ "store_file_bytes_after": 1048632,
+ "whole_read_median_ms": 0.102,
+ "read_checksum": 28574496
+ },
+ {
+ "store": "paged",
+ "op": "insert-middle",
+ "document_bytes": 1048576,
+ "edits": 32,
+ "edit_median_ms": 86.573,
+ "edit_p90_ms": 103.801,
+ "edit_max_ms": 108.102,
+ "write_call_bytes_per_edit": 1065528,
+ "device_write_bytes_per_edit": 1081344,
+ "store_file_bytes_after": 1065528,
+ "whole_read_median_ms": 0.07,
+ "read_checksum": 28574496
+ },
+ {
+ "store": "tree",
+ "op": "insert-middle",
+ "document_bytes": 1048576,
+ "edits": 32,
+ "edit_median_ms": 8.58,
+ "edit_p90_ms": 16.685,
+ "edit_max_ms": 64.33,
+ "write_call_bytes_per_edit": 2732,
+ "device_write_bytes_per_edit": 35328,
+ "store_file_bytes_after": 1138480,
+ "whole_read_median_ms": 0.262,
+ "read_checksum": 28574496
+ },
+ {
+ "store": "flat",
+ "op": "delete-middle",
+ "document_bytes": 1048576,
+ "edits": 32,
+ "edit_median_ms": 78.755,
+ "edit_p90_ms": 81.697,
+ "edit_max_ms": 86.715,
+ "write_call_bytes_per_edit": 1048583,
+ "device_write_bytes_per_edit": 1060352,
+ "store_file_bytes_after": 1048568,
+ "whole_read_median_ms": 0.065,
+ "read_checksum": 28573440
+ },
+ {
+ "store": "paged",
+ "op": "delete-middle",
+ "document_bytes": 1048576,
+ "edits": 32,
+ "edit_median_ms": 85.293,
+ "edit_p90_ms": 93.573,
+ "edit_max_ms": 108.718,
+ "write_call_bytes_per_edit": 1049136,
+ "device_write_bytes_per_edit": 1064960,
+ "store_file_bytes_after": 1049136,
+ "whole_read_median_ms": 0.077,
+ "read_checksum": 28573440
+ },
+ {
+ "store": "tree",
+ "op": "delete-middle",
+ "document_bytes": 1048576,
+ "edits": 32,
+ "edit_median_ms": 8.512,
+ "edit_p90_ms": 11.869,
+ "edit_max_ms": 14.455,
+ "write_call_bytes_per_edit": 4907,
+ "device_write_bytes_per_edit": 37376,
+ "store_file_bytes_after": 1208072,
+ "whole_read_median_ms": 0.229,
+ "read_checksum": 28573440
+ },
+ {
+ "store": "flat",
+ "op": "replace-end",
+ "document_bytes": 1048576,
+ "edits": 32,
+ "edit_median_ms": 79.202,
+ "edit_p90_ms": 86.6,
+ "edit_max_ms": 90.952,
+ "write_call_bytes_per_edit": 1048600,
+ "device_write_bytes_per_edit": 1064960,
+ "store_file_bytes_after": 1048600,
+ "whole_read_median_ms": 0.063,
+ "read_checksum": 28573664
+ },
+ {
+ "store": "paged",
+ "op": "replace-end",
+ "document_bytes": 1048576,
+ "edits": 32,
+ "edit_median_ms": 13.959,
+ "edit_p90_ms": 23.871,
+ "edit_max_ms": 69.901,
+ "write_call_bytes_per_edit": 16944,
+ "device_write_bytes_per_edit": 32768,
+ "store_file_bytes_after": 1573424,
+ "whole_read_median_ms": 0.074,
+ "read_checksum": 28573664
+ },
+ {
+ "store": "tree",
+ "op": "replace-end",
+ "document_bytes": 1048576,
+ "edits": 32,
+ "edit_median_ms": 6.488,
+ "edit_p90_ms": 8.209,
+ "edit_max_ms": 60.301,
+ "write_call_bytes_per_edit": 4912,
+ "device_write_bytes_per_edit": 37376,
+ "store_file_bytes_after": 1208232,
+ "whole_read_median_ms": 0.37,
+ "read_checksum": 28573664
+ },
+ {
+ "store": "flat",
+ "op": "append-end",
+ "document_bytes": 1048576,
+ "edits": 32,
+ "edit_median_ms": 77.885,
+ "edit_p90_ms": 86.005,
+ "edit_max_ms": 89.824,
+ "write_call_bytes_per_edit": 1048616,
+ "device_write_bytes_per_edit": 1064960,
+ "store_file_bytes_after": 1048632,
+ "whole_read_median_ms": 0.062,
+ "read_checksum": 28574704
+ },
+ {
+ "store": "paged",
+ "op": "append-end",
+ "document_bytes": 1048576,
+ "edits": 32,
+ "edit_median_ms": 86.039,
+ "edit_p90_ms": 102.765,
+ "edit_max_ms": 121.611,
+ "write_call_bytes_per_edit": 1065528,
+ "device_write_bytes_per_edit": 1081344,
+ "store_file_bytes_after": 1065528,
+ "whole_read_median_ms": 0.07,
+ "read_checksum": 28574704
+ },
+ {
+ "store": "tree",
+ "op": "append-end",
+ "document_bytes": 1048576,
+ "edits": 32,
+ "edit_median_ms": 8.473,
+ "edit_p90_ms": 11.691,
+ "edit_max_ms": 60.672,
+ "write_call_bytes_per_edit": 2716,
+ "device_write_bytes_per_edit": 35328,
+ "store_file_bytes_after": 1137960,
+ "whole_read_median_ms": 0.225,
+ "read_checksum": 28574704
+ }
+ ]
+}
diff --git a/research/experiments/008-tree-store-edit-cost/2026-09-25-tui-ext4-linux-aarch64.json b/research/experiments/008-tree-store-edit-cost/2026-09-25-tui-ext4-linux-aarch64.json
new file mode 100644
index 0000000..6bea961
--- /dev/null
+++ b/research/experiments/008-tree-store-edit-cost/2026-09-25-tui-ext4-linux-aarch64.json
@@ -0,0 +1,285 @@
+{
+ "platform": "Linux-6.18.39+rpt-rpi-2712-aarch64-with-glibc2.41",
+ "filesystem": "/dev/mmcblk0p2 ext4",
+ "document_bytes": 1048576,
+ "rounds": 3,
+ "edits_per_session": 8,
+ "median_ms": {
+ "replace": {
+ "flat": 86.32,
+ "paged": 21.31,
+ "tree": 16.97
+ },
+ "append": {
+ "flat": 84.59,
+ "paged": 93.29,
+ "tree": 16.6
+ }
+ },
+ "binaries": {
+ "flat": {
+ "cli_sha256": "f69c7580104b873e0736d7503dcbbcbb6b180adf54920e6ba741d363262867b8",
+ "tui_sha256": "644980089e65e9db01ecdd21c4edf8a0ae7fe248114340b3b691e6c283e5e5b4"
+ },
+ "paged": {
+ "cli_sha256": "233311ee133bc1d5299963f4523131dac542d71013cb825f8a4eaa48de813ee8",
+ "tui_sha256": "77169c79a800881e901e5b1bba938543eed5f3f1f0914799e56c0635859af2d1"
+ },
+ "tree": {
+ "cli_sha256": "c0c783a1b30a1f8e88bfe9a09b6ae124f596bf6239b802e4aec5e56f374714b9",
+ "tui_sha256": "8030475c663b09791a5fa24bad2f8498f652d4361a6734a737b3c6a22e4659aa"
+ }
+ },
+ "runs": {
+ "replace": {
+ "flat": [
+ {
+ "samples_ns": [
+ 83883854,
+ 32709073,
+ 121001564,
+ 92289945,
+ 96340473,
+ 90943269,
+ 47708564,
+ 89206225
+ ],
+ "body_sha256": "1acbea9ad5e6321b7e4f8b3f85450d99cb2d82b655820160dcd18bcaae58e00f"
+ },
+ {
+ "samples_ns": [
+ 85403197,
+ 33045701,
+ 71709920,
+ 88249419,
+ 87284740,
+ 30588422,
+ 86124025,
+ 86515579
+ ],
+ "body_sha256": "1acbea9ad5e6321b7e4f8b3f85450d99cb2d82b655820160dcd18bcaae58e00f"
+ },
+ {
+ "samples_ns": [
+ 85907991,
+ 52699698,
+ 36407605,
+ 91580804,
+ 91115363,
+ 96859361,
+ 91057308,
+ 30358535
+ ],
+ "body_sha256": "1acbea9ad5e6321b7e4f8b3f85450d99cb2d82b655820160dcd18bcaae58e00f"
+ }
+ ],
+ "paged": [
+ {
+ "samples_ns": [
+ 21286170,
+ 21523669,
+ 20435102,
+ 21175041,
+ 24219650,
+ 19863957,
+ 21457280,
+ 20223807
+ ],
+ "body_sha256": "1acbea9ad5e6321b7e4f8b3f85450d99cb2d82b655820160dcd18bcaae58e00f"
+ },
+ {
+ "samples_ns": [
+ 20820044,
+ 18812854,
+ 22029072,
+ 19884550,
+ 21458169,
+ 21256596,
+ 25127069,
+ 73009317
+ ],
+ "body_sha256": "1acbea9ad5e6321b7e4f8b3f85450d99cb2d82b655820160dcd18bcaae58e00f"
+ },
+ {
+ "samples_ns": [
+ 21277781,
+ 23763283,
+ 18573541,
+ 21452318,
+ 21336634,
+ 75828726,
+ 18893984,
+ 21467613
+ ],
+ "body_sha256": "1acbea9ad5e6321b7e4f8b3f85450d99cb2d82b655820160dcd18bcaae58e00f"
+ }
+ ],
+ "tree": [
+ {
+ "samples_ns": [
+ 16691108,
+ 17111328,
+ 15930836,
+ 17254197,
+ 16401425,
+ 16596591,
+ 17847323,
+ 18489726
+ ],
+ "body_sha256": "1acbea9ad5e6321b7e4f8b3f85450d99cb2d82b655820160dcd18bcaae58e00f"
+ },
+ {
+ "samples_ns": [
+ 16340445,
+ 18211228,
+ 12615914,
+ 17297716,
+ 17568492,
+ 16601387,
+ 17259642,
+ 15886466
+ ],
+ "body_sha256": "1acbea9ad5e6321b7e4f8b3f85450d99cb2d82b655820160dcd18bcaae58e00f"
+ },
+ {
+ "samples_ns": [
+ 17321808,
+ 16796238,
+ 16831978,
+ 15132138,
+ 17392253,
+ 12584915,
+ 17342679,
+ 17423900
+ ],
+ "body_sha256": "1acbea9ad5e6321b7e4f8b3f85450d99cb2d82b655820160dcd18bcaae58e00f"
+ }
+ ]
+ },
+ "append": {
+ "flat": [
+ {
+ "samples_ns": [
+ 85268681,
+ 58706620,
+ 80969562,
+ 30847551,
+ 88231013,
+ 85485051,
+ 29563059,
+ 88786509
+ ],
+ "body_sha256": "bd42946a8dafa764907691fbebfd981ef4b012fbcbf5c25beab7d6b1c8d0cc30"
+ },
+ {
+ "samples_ns": [
+ 58646271,
+ 30913440,
+ 89689432,
+ 85278758,
+ 33291979,
+ 86740285,
+ 83913915,
+ 90844368
+ ],
+ "body_sha256": "bd42946a8dafa764907691fbebfd981ef4b012fbcbf5c25beab7d6b1c8d0cc30"
+ },
+ {
+ "samples_ns": [
+ 51920946,
+ 28831473,
+ 85487720,
+ 85906421,
+ 85835644,
+ 32848520,
+ 85315555,
+ 82713850
+ ],
+ "body_sha256": "bd42946a8dafa764907691fbebfd981ef4b012fbcbf5c25beab7d6b1c8d0cc30"
+ }
+ ],
+ "paged": [
+ {
+ "samples_ns": [
+ 91333586,
+ 37920058,
+ 92091377,
+ 63703624,
+ 95308984,
+ 40485355,
+ 99081866,
+ 93075370
+ ],
+ "body_sha256": "bd42946a8dafa764907691fbebfd981ef4b012fbcbf5c25beab7d6b1c8d0cc30"
+ },
+ {
+ "samples_ns": [
+ 96352460,
+ 93512017,
+ 65222596,
+ 38763552,
+ 98852258,
+ 131612999,
+ 138169789,
+ 131450371
+ ],
+ "body_sha256": "bd42946a8dafa764907691fbebfd981ef4b012fbcbf5c25beab7d6b1c8d0cc30"
+ },
+ {
+ "samples_ns": [
+ 40996002,
+ 93012356,
+ 63056206,
+ 93559778,
+ 37065306,
+ 98450597,
+ 112059339,
+ 111368899
+ ],
+ "body_sha256": "bd42946a8dafa764907691fbebfd981ef4b012fbcbf5c25beab7d6b1c8d0cc30"
+ }
+ ],
+ "tree": [
+ {
+ "samples_ns": [
+ 15267508,
+ 17113014,
+ 17454104,
+ 16570055,
+ 69532029,
+ 15171139,
+ 18091600,
+ 11101536
+ ],
+ "body_sha256": "bd42946a8dafa764907691fbebfd981ef4b012fbcbf5c25beab7d6b1c8d0cc30"
+ },
+ {
+ "samples_ns": [
+ 19964902,
+ 14639290,
+ 16407186,
+ 17425161,
+ 15285638,
+ 18012231,
+ 15061398,
+ 17329272
+ ],
+ "body_sha256": "bd42946a8dafa764907691fbebfd981ef4b012fbcbf5c25beab7d6b1c8d0cc30"
+ },
+ {
+ "samples_ns": [
+ 17466771,
+ 16352390,
+ 16629406,
+ 15132064,
+ 18861503,
+ 11662162,
+ 16306335,
+ 17256958
+ ],
+ "body_sha256": "bd42946a8dafa764907691fbebfd981ef4b012fbcbf5c25beab7d6b1c8d0cc30"
+ }
+ ]
+ }
+ }
+}
diff --git a/research/experiments/008-tree-store-edit-cost/bench.c b/research/experiments/008-tree-store-edit-cost/bench.c
new file mode 100644
index 0000000..f16f8db
--- /dev/null
+++ b/research/experiments/008-tree-store-edit-cost/bench.c
@@ -0,0 +1,104 @@
+/* Durable edit cost of one selected byte store, through its public API only.
+ * Built once per store source; the store directory must be on the device
+ * being measured (not tmpfs, where fsync is free). */
+#define _POSIX_C_SOURCE 200809L
+#include "store.h"
+#include <dirent.h>
+#include <inttypes.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <time.h>
+
+static double now_ms(void) {
+ struct timespec t;
+ clock_gettime(CLOCK_MONOTONIC, &t);
+ return (double)t.tv_sec * 1e3 + (double)t.tv_nsec / 1e6;
+}
+
+/* Linux accounting: wchar is bytes passed to write calls, write_bytes is
+ * bytes this process caused to be sent to the block layer. */
+static uint64_t io_field(const char *name) {
+ FILE *file = fopen("/proc/self/io", "r");
+ char line[128];
+ uint64_t value = 0;
+ size_t length = strlen(name);
+ while (file && fgets(line, sizeof(line), file))
+ if (!strncmp(line, name, length) && line[length] == ':') value = strtoull(line + length + 1, NULL, 10);
+ if (file) fclose(file);
+ return value;
+}
+
+static uint64_t store_bytes(const char *path) {
+ DIR *directory = opendir(path);
+ struct dirent *entry;
+ uint64_t total = 0;
+ char name[4096];
+ while (directory && (entry = readdir(directory))) {
+ struct stat info;
+ snprintf(name, sizeof(name), "%s/%s", path, entry->d_name);
+ if (!stat(name, &info) && S_ISREG(info.st_mode)) total += (uint64_t)info.st_size;
+ }
+ if (directory) closedir(directory);
+ return total;
+}
+
+static int compare(const void *a, const void *b) {
+ double x = *(const double *)a, y = *(const double *)b;
+ return (x > y) - (x < y);
+}
+
+int main(int argc, char **argv) {
+ if (argc != 6) {
+ fputs("bench STORE_NAME DIRECTORY SIZE insert-middle|delete-middle|replace-end|append-end COUNT\n", stderr);
+ return 2;
+ }
+ const char *name = argv[1], *path = argv[2], *op = argv[4];
+ size_t size = strtoull(argv[3], NULL, 10), count = strtoull(argv[5], NULL, 10);
+ unsigned char *body = malloc(size + 1);
+ double *samples = malloc(count * sizeof(*samples)), reads[16];
+ if (!body || !samples || pure_store_init(path) != PURE_OK) return 1;
+ for (size_t i = 0; i < size; ++i) body[i] = (unsigned char)('a' + i % 26);
+ int64_t revision;
+ if (pure_store_put(path, "doc", 0, (PureSpan){body, size}, &revision) != PURE_OK) return 1;
+ size_t length = size;
+ uint64_t wchar = io_field("wchar"), device = io_field("write_bytes");
+ for (size_t i = 0; i < count; ++i) {
+ size_t offset, remove, insert;
+ if (!strcmp(op, "insert-middle")) { offset = length / 2; remove = 0; insert = 1; }
+ else if (!strcmp(op, "delete-middle")) { offset = length / 2; remove = 1; insert = 0; }
+ else if (!strcmp(op, "replace-end")) { offset = length - 1; remove = 1; insert = 1; }
+ else if (!strcmp(op, "append-end")) { offset = length; remove = 0; insert = 1; }
+ else return 2;
+ unsigned char byte = (unsigned char)('A' + i % 26);
+ double start = now_ms();
+ PureResult result = pure_store_splice(path, "doc", revision, offset, remove,
+ (PureSpan){&byte, insert}, PURE_OBJECT_LIMIT, &revision);
+ samples[i] = now_ms() - start;
+ if (result != PURE_OK) { fprintf(stderr, "splice failed: %d\n", (int)result); return 1; }
+ length = length - remove + insert;
+ }
+ wchar = io_field("wchar") - wchar;
+ device = io_field("write_bytes") - device;
+ uint64_t checksum = 0;
+ for (size_t i = 0; i < 16; ++i) {
+ PureStoredView view;
+ double start = now_ms();
+ if (pure_store_get(path, "doc", PURE_OBJECT_LIMIT, &view) != PURE_OK || view.body.size != length) return 1;
+ for (size_t k = 0; k < view.body.size; k += 64) checksum += view.body.data[k];
+ pure_store_release(&view);
+ reads[i] = now_ms() - start;
+ }
+ qsort(samples, count, sizeof(*samples), compare);
+ qsort(reads, 16, sizeof(*reads), compare);
+ printf("{\"store\":\"%s\",\"op\":\"%s\",\"document_bytes\":%zu,\"edits\":%zu,"
+ "\"edit_median_ms\":%.3f,\"edit_p90_ms\":%.3f,\"edit_max_ms\":%.3f,"
+ "\"write_call_bytes_per_edit\":%" PRIu64 ",\"device_write_bytes_per_edit\":%" PRIu64 ","
+ "\"store_file_bytes_after\":%" PRIu64 ",\"whole_read_median_ms\":%.3f,\"read_checksum\":%" PRIu64 "}\n",
+ name, op, size, count, samples[count / 2], samples[count * 9 / 10], samples[count - 1],
+ wchar / count, device / count, store_bytes(path), reads[8], checksum);
+ free(samples);
+ free(body);
+ return 0;
+}
diff --git a/research/experiments/008-tree-store-edit-cost/run.py b/research/experiments/008-tree-store-edit-cost/run.py
new file mode 100755
index 0000000..1b9dfa8
--- /dev/null
+++ b/research/experiments/008-tree-store-edit-cost/run.py
@@ -0,0 +1,49 @@
+#!/usr/bin/env python3
+"""Build bench.c against each selected store and time durable edits on disk."""
+import argparse
+import json
+import platform
+import shutil
+import subprocess
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+CONNECTIONS = HERE.parents[2] / 'prototypes' / 'connections'
+STORES = {'flat': 'store.c', 'paged': 'store-paged.c', 'tree': 'store-tree.c'}
+OPS = ['insert-middle', 'delete-middle', 'replace-end', 'append-end']
+
+
+def main():
+ parser = argparse.ArgumentParser()
+ parser.add_argument('--work', type=Path, default=CONNECTIONS / 'build' / 'bench-tree-cost')
+ parser.add_argument('--count', type=int, default=32)
+ parser.add_argument('--sizes', default='256,1048576')
+ parser.add_argument('--output', type=Path)
+ args = parser.parse_args()
+ shutil.rmtree(args.work, ignore_errors=True)
+ args.work.mkdir(parents=True)
+ source = subprocess.check_output(['findmnt', '-n', '-T', str(args.work), '-o', 'SOURCE,FSTYPE'], text=True).split()
+ assert source[1] != 'tmpfs', 'fsync is free on tmpfs; choose a real device'
+ results = []
+ for name, store in STORES.items():
+ binary = args.work / f'bench-{name}'
+ subprocess.run(['cc', '-std=c17', '-O2', '-Wall', '-Wextra', '-Werror', f'-I{CONNECTIONS}',
+ str(HERE / 'bench.c'), str(CONNECTIONS / store), '-o', str(binary)], check=True)
+ for size in map(int, args.sizes.split(',')):
+ for op in OPS:
+ for name in STORES: # interleave stores within each scenario
+ directory = args.work / f'{name}-{op}-{size}'
+ line = subprocess.check_output([str(args.work / f'bench-{name}'), name, str(directory),
+ str(size), op, str(args.count)], text=True)
+ results.append(json.loads(line))
+ print(line.strip(), flush=True)
+ shutil.rmtree(directory)
+ report = {'device': source[0], 'filesystem': source[1], 'machine': platform.machine(),
+ 'kernel': platform.release(), 'compiler': subprocess.check_output(['cc', '--version'], text=True).splitlines()[0],
+ 'edits_per_scenario': args.count, 'results': results}
+ if args.output:
+ args.output.write_text(json.dumps(report, indent=2) + '\n')
+
+
+if __name__ == '__main__':
+ main()
diff --git a/research/experiments/008-tree-store-edit-cost/tui.py b/research/experiments/008-tree-store-edit-cost/tui.py
new file mode 100755
index 0000000..78c6ce0
--- /dev/null
+++ b/research/experiments/008-tree-store-edit-cost/tui.py
@@ -0,0 +1,128 @@
+#!/usr/bin/env python3
+"""Ctrl+S to completed frame in the real native terminal editor, per store.
+
+Same method as 006 (full draft comparison, runtime, durable store and redraw),
+with two edit shapes: replacing the final character (same size) and typing one
+new character at the end (the document grows, as ordinary typing does)."""
+import argparse
+import fcntl
+import hashlib
+import json
+import os
+from pathlib import Path
+import platform
+import pty
+import select
+import statistics
+import struct
+import subprocess
+import tempfile
+import termios
+import time
+
+
+def run(*args):
+ return subprocess.run([str(arg) for arg in args], check=True, capture_output=True, timeout=30)
+
+
+def frame(master, process):
+ output = bytearray()
+ deadline = time.monotonic() + 15
+ while time.monotonic() < deadline:
+ if select.select([master], [], [], 0.5)[0]:
+ output.extend(os.read(master, 65536))
+ if b'\x1b[?25h' in output:
+ return bytes(output)
+ if process.poll() is not None:
+ break
+ raise AssertionError((process.poll(), output[-1000:]))
+
+
+def one_session(cli, tui, root, size, edits, shape):
+ root.mkdir(parents=True)
+ store = root / 'store'
+ body = b'A' * size
+ initial = root / 'initial.bin'
+ initial.write_bytes(body)
+ assert json.loads(run(cli, '--store', store, 'edit', 'document', 0, 0, 0, initial).stdout)['revision'] == '1'
+ master, slave = pty.openpty()
+ fcntl.ioctl(slave, termios.TIOCSWINSZ, struct.pack('HHHH', 24, 80, 0, 0))
+ process = subprocess.Popen([str(tui), '--store', str(store), 'document'],
+ stdin=slave, stdout=slave, stderr=slave,
+ env={**os.environ, 'LANG': 'C.UTF-8', 'LC_ALL': 'C.UTF-8'})
+ samples, expected_body = [], bytearray(body)
+ try:
+ assert b'Pure | document' in frame(master, process)
+ for index in range(edits):
+ character = b'B' if index % 2 == 0 else b'C'
+ if shape == 'replace':
+ os.write(master, b'\x7f')
+ frame(master, process)
+ expected_body[-1:] = character
+ position = size - 1
+ else:
+ expected_body += character
+ position = size + index
+ os.write(master, character)
+ frame(master, process)
+ start = time.perf_counter_ns()
+ os.write(master, b'\x13')
+ shown = frame(master, process)
+ samples.append(time.perf_counter_ns() - start)
+ assert f'Saved 1-byte edit at byte {position}'.encode() in shown, shown[-1500:]
+ assert f'revision {index + 2}'.encode() in shown, shown[-1500:]
+ os.write(master, b'\x11')
+ process.wait(timeout=8)
+ assert process.returncode == 0
+ finally:
+ if process.poll() is None:
+ process.kill()
+ process.wait(timeout=5)
+ os.close(master)
+ os.close(slave)
+ final = run(cli, '--store', store, 'show', 'document')
+ assert final.stdout == bytes(expected_body)
+ assert json.loads(final.stderr)['revision'] == str(edits + 1)
+ return {'samples_ns': samples, 'body_sha256': hashlib.sha256(final.stdout).hexdigest()}
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument('--store', nargs=3, action='append', metavar=('NAME', 'CLI', 'TUI'), required=True)
+ parser.add_argument('--scratch-parent', type=Path, required=True)
+ parser.add_argument('--size', type=int, default=1_048_576)
+ parser.add_argument('--rounds', type=int, default=3)
+ parser.add_argument('--edits', type=int, default=8)
+ parser.add_argument('--output', type=Path)
+ args = parser.parse_args()
+ binaries = {name: (Path(cli).resolve(), Path(tui).resolve()) for name, cli, tui in args.store}
+ parent = args.scratch_parent.resolve()
+ parent.mkdir(parents=True, exist_ok=True)
+ filesystem = run('findmnt', '-T', parent, '-no', 'SOURCE,FSTYPE').stdout.decode().strip()
+ assert 'tmpfs' not in filesystem, 'fsync is free on tmpfs'
+ raw = {shape: {name: [] for name in binaries} for shape in ('replace', 'append')}
+ with tempfile.TemporaryDirectory(prefix='pure-tree-save-', dir=parent) as directory:
+ root = Path(directory)
+ for shape in raw:
+ for round_number in range(args.rounds):
+ names = list(binaries)
+ order = names[round_number % len(names):] + names[:round_number % len(names)]
+ for name in order:
+ cli, tui = binaries[name]
+ raw[shape][name].append(one_session(cli, tui, root / f'{shape}-{round_number}-{name}',
+ args.size, args.edits, shape))
+ medians = {shape: {name: round(statistics.median(s for session in sessions for s in session['samples_ns']) / 1e6, 2)
+ for name, sessions in stores.items()} for shape, stores in raw.items()}
+ result = {'platform': platform.platform(), 'filesystem': filesystem, 'document_bytes': args.size,
+ 'rounds': args.rounds, 'edits_per_session': args.edits, 'median_ms': medians,
+ 'binaries': {name: {'cli_sha256': hashlib.sha256(cli.read_bytes()).hexdigest(),
+ 'tui_sha256': hashlib.sha256(tui.read_bytes()).hexdigest()}
+ for name, (cli, tui) in binaries.items()},
+ 'runs': raw}
+ if args.output:
+ args.output.write_text(json.dumps(result, indent=2) + '\n')
+ print(json.dumps({'filesystem': filesystem, 'median_ms': medians}))
+
+
+if __name__ == '__main__':
+ main()