diff options
| author | eunhhu <[email protected]> | 2026-09-30 08:54:29 +0100 |
|---|---|---|
| committer | eunhhu <[email protected]> | 2026-09-30 08:54:29 +0100 |
| commit | 9840d27a9e437de32256024a2fd6a3219a239815 (patch) | |
| tree | eba57012bcdaac8e019d1421370a570a3e9f70d9 | |
| parent | 987d4d7aa08624e28d0ea18a1e66e359a0230c80 (diff) | |
| download | pure-9840d27a9e437de32256024a2fd6a3219a239815.tar.gz pure-9840d27a9e437de32256024a2fd6a3219a239815.zip | |
Say what snapshots cost the owner, and what reusing their nodes gave (107)
Most of it was the allocator: leaves past glibc's thread cache, freed by
whichever thread let the last snapshot go. jemalloc was 11-14% faster
at more memory; reusing the nodes in Pure gave a median +9.4% over six
pairs, 4-8% less CPU a transaction in all six, and 1.97x the state bytes
resident where it was 2.18x. TPC-C in RAM is now 4.0x a tuned SQLite at
W=1 and 3.5x at W=4; finding items is the owner's largest part now.
25 files changed, 400 insertions, 2 deletions
diff --git a/docs/research/107-snapshot-nodes-reused.md b/docs/research/107-snapshot-nodes-reused.md new file mode 100644 index 0000000..f9b385f --- /dev/null +++ b/docs/research/107-snapshot-nodes-reused.md @@ -0,0 +1,141 @@ +# 107 — 스냅숏이 놓은 노드를 목록이 다시 쓴다 + +2026-09-30. [106](106-tpcc-conditions-first.md)의 판정 2에서 이어진다. RAM에서 TPC-C는 소유자 스레드 하나가 병목이고, 표본으로 본 +소유자의 바쁜 시간 가운데 3분의 1이 스냅숏이 공유한 노드를 복사하고 놓는 데 갔다. 그 3분의 1이 무엇인지 나눠 보고 줄였다. + +기준(106의 표본에서 곧바로 이어진 일이라, 측정 전에 따로 커밋하지는 않았다): + +- **바꿀 결정.** 스냅숏에 딸린 비용을 코어 안에서 줄일지, 할당기를 바꿀지, 둘 다 하지 않을지. +- **넣는 조건.** 번갈아 잰 쌍에서 처리량과 거래당 CPU가 나아진다. 상태 바이트에 대한 상주 메모리가 늘지 않는다. 붙든 스냅숏이 + 읽는 값이 그대로다(테스트, ASan과 LSan). + +## 쉬운 말로 + +- **스냅숏 때문에 소유자가 쓰던 시간의 70%는 복사가 아니라 할당이었다.** 그 몫의 표본 20개 가운데 할당기가 14개, 복사가 4개였다. + - TPC-C의 잎(약 1.3 KB)은 glibc의 스레드 캐시(1,032 B까지)에 들지 않는다. 그래서 malloc과 free가 매번 아레나의 잠금을 잡는다. + - 스냅숏을 마지막으로 놓는 쪽이 연결 스레드면, 그 스레드가 소유자의 아레나에 free한다. 소유자의 malloc과 같은 잠금을 다툰다. +- **할당기를 jemalloc으로 바꾸면 11–14% 빨랐다.** 할당이 정말 비용이었다는 확인이다. 대신 상주가 상태 바이트의 2.17배에서 + 2.24배로 늘었다. +- **할당기를 바꾸지 않고, 스냅숏이 놓은 노드를 목록이 다시 쓰게 했다.** + - 목록에는 원래 제가 놓은 노드를 다음에 쓰는 자리(spare)가 있었다. 스냅숏이 놓은 노드는 거기 오지 않고 free됐다. 이제 온다. + - 번갈아 잰 여섯 쌍에서 처리량의 중앙값이 +9.4%다(다섯 쌍 +9.3–13.0%, 한 쌍 −2.5%). + - 거래당 CPU는 여섯 쌍 모두 4–8% 줄었다. + - 상주는 상태 바이트의 2.17–2.18배에서 1.97–1.98배로 줄었다. 빨라지면서 메모리도 덜 쓴다. +- **TPC-C의 RAM 비교가 바뀌었다.** W=1에서 초당 40,976건으로 조율한 SQLite(10,164건)의 4.0배다. W=4에서는 35,838건, 3.5배다. + 106에서는 36,567건과 32,765건이었다. +- 소유자 표본에서 스냅숏에 딸린 몫은 바쁜 시간의 33%에서 17%로 줄었다. 할당기 표본은 14개에서 7개가 됐다. + +## 무엇이 비쌌나 + +106의 소유자 표본([원시](../../research/experiments/022-tpcc-grouped/2026-09-30-owner-stacks-linux-aarch64.txt)) 가운데 +스냅숏에 딸린 20개를, 가장 안쪽에서 부른 할당기와 그 호출자로 나눴다. + +| 가장 안쪽 | 표본 | +|---|---:| +| `malloc` ← `new_node`(목록이 복사할 노드를 새로 얻는다) | 9 | +| `free` ← `pure_retired_free`(스냅숏이 놓인 뒤 그 노드를 돌려준다) | 4 | +| `realloc` ← `retire`(앱의 retired 벡터가 스냅숏마다 16칸부터 다시 자란다) | 1 | +| 복사 자체(`own` 안의 memcpy)와 그 밖 | 6 | + +같은 서버를 할당기만 바꿔 번갈아 돌렸다(RAM, W=1, 64터미널, 20초씩). + +| 할당기 | 초당 거래 | 거래당 CPU | 상주 / 상태 | +|---|---:|---:|---:| +| glibc | 37,280 | 48.2 µs | 2.17배 | +| jemalloc | 42,323 | 43.4 µs | 2.24배 | +| glibc, `tcache_max=4096` | 39,799 | 47.0 µs | 2.18배 | +| glibc | 40,515 | 46.8 µs | 2.18배 | +| jemalloc | 45,119 | 43.5 µs | 2.24배 | + +jemalloc은 glibc 바로 앞 실행보다 13.5%와 11.4% 빨랐다. glibc의 스레드 캐시를 키우는 설정은 아무것도 바꾸지 않았다(이 판의 +glibc는 1,032 B를 넘는 크기를 캐시하지 않는다). 같은 glibc 두 번이 8.7% 벌어진다. 이 기계에서 한 번의 실행은 ±5%쯤 흔들린다. + +## 한 것 + +[`core/module-list.c`](../../core/module-list.c), [`core/module.c`](../../core/module.c) (`6c87722`): + +1. **retired 항목이 제 노드를 설명한다.** 노드의 공유 세대(`made`), 크기, 잎인지를 항목에 적는다. 스냅숏을 놓을 때 넘기는 일과 + 되쓰는 일이 식은 노드를 읽지 않는다(전에는 넘길 때마다 노드의 `made`를 읽었다). +2. **놓인 것은 앱으로 돌아온다.** 스냅숏을 마지막으로 놓는 스레드는 아무 스냅숏도 읽지 않는 것을 free하지 않는다. 대신 앱의 + 레지스터(앱과 그 스냅숏들이 함께 쥔다)에 둔다. 레지스터의 스핀 잠금 아래이고, 전에도 넘기는 일은 이 잠금 아래였다. +3. **소유자가 나눠 준다.** 스냅숏을 만들 때마다 소유자는 돌아온 것을 크기가 맞는 목록의 spare로 나눠 준다. 목록은 최근에 만든 + 노드 수만큼 쥔다(돌아올 때마다 반으로 줄여 센다, 적어도 8개). 남는 것과 text는 free한다. +4. **spare는 포인터 배열이다.** 노드 안에 끼운 연결이면 나눠 줄 때마다 식은 노드에 써야 한다. +5. 앱의 retired 벡터는 칸을 두고 되쓴다. + +처음 판(커밋하지 않았다)은 spare를 노드 안의 연결로 두고, 돌아올 때마다 수요를 0으로 지웠다. 세 쌍에서 +7.4%에 그쳤다. 표본을 +보니 나눠 주는 고리가 식은 노드를 읽고 쓰느라 바빴고(표본 4개), 수요를 지우는 바람에 돌아온 것의 상당수를 free했다(표본 5개). +위의 1, 3, 4가 그것을 고친 것이다. + +공유되지 않은 노드를 놓을 때(`drop_node`)는 전처럼 8개까지만 spare로 쥔다. 스냅숏을 만들지 않는 앱에서는 수요가 반으로 줄 +기회가 없기 때문이다. + +## 안전 + +되쓰는 노드는 전에 free하던 바로 그것이다. 아무 스냅숏도 읽지 않는다는 판정(`pure_retired_pass`)은 바꾸지 않았다. + +- `snapshot_held`(이미 있던 테스트): 스냅숏 하나를 붙든 채 사건과 스냅숏을 4만 번 돌린 뒤, 붙든 스냅숏의 내용이 글자 하나 다르지 + 않다. +- `list_trees`: 무작위 변경 6,000번 사이에 복사본을 넷까지 붙들고, 가장 오래된 것을 놓으면 그 retired를 이제 목록이 되쓴다. + 붙든 복사본의 항목을 계속 확인한다. 아직 붙든 복사본의 retired를 되쓰게 바꾼 변이는 여기서 잡혔다. +- `retired_come_back`(새 테스트): 복사본이 놓은 잎으로 다음 복사를 만든다(주소가 같다). 붙든 복사본은 제 값을 읽는다. 되쓰지 + 않게 바꾼 변이는 여기서 잡혔다. +- 모듈 테스트는 ASan과 LSan으로도 돈다. Pure의 `make check`, Vega와 aup의 검사가 통과했다. + +## 결과 + +[원시 결과](../../research/experiments/023-node-reuse/)와 [`runs.sh`](../../research/experiments/023-node-reuse/runs.sh), +표는 `summary.py`가 만든다. RAM(tmpfs), W=1, 64터미널, 20초씩, 전(`c9fda91`)과 후(`6c87722`)를 번갈아. + +| 쌍 | 초당 거래(전 → 후) | 거래당 CPU(µs) | 상주 / 상태 | +|---|---:|---:|---:| +| 1 | 39,304 → 38,305 (−2.5%) | 47.4 → 45.6 | 2.18 → 1.97 | +| 2 | 38,863 → 43,811 (+12.7%) | 47.0 → 43.8 | 2.18 → 1.97 | +| 3 | 39,952 → 43,732 (+9.5%) | 47.0 → 43.9 | 2.18 → 1.97 | +| 4(후 먼저) | 37,931 → 41,485 (+9.4%) | 48.4 → 44.7 | 2.17 → 1.98 | +| 5(후 먼저) | 36,817 → 41,617 (+13.0%) | 48.2 → 45.0 | 2.17 → 1.97 | +| 6(후 먼저) | 38,679 → 42,292 (+9.3%) | 47.9 → 44.7 | 2.18 → 1.97 | + +첫 쌍의 "후" 실행에서는 소유자가 코어의 0.83만 썼다(다른 실행은 0.87–0.91). 그 실행에서도 거래당 CPU는 3.8% 줄었다. + +**106의 RAM 행렬을 다시.** SQLite는 그대로다. + +| | W=1 | W=4 | +|---|---:|---:| +| Pure, 16터미널 | 38,320 (106: 35,672) | 33,057 (32,765) | +| Pure, 64터미널 | 40,976 (36,567) | 35,838 (32,255) | +| 조율한 SQLite의 최고 | 10,164 | 10,168 | +| Pure의 상주 / 상태 | 2.01 (2.23) | 1.94 (2.10) | + +같은 거래 2만 건 뒤 세 쪽의 표가 모두 같았고, 모든 실행 뒤 일관성 조건이 성립했다. + +**소유자 표본**(RAM, W=1, 64터미널, 80개씩). + +| 소유자가 하던 일 | 106(전) | 지금 | +|---|---:|---:| +| 항목 찾기 | 24 | 20 | +| 요청 기다리기 | 19 | 17 | +| 스냅숏이 공유한 노드를 복사하기 | 15 | 10 | +| 스냅숏 게시와 놓기 | 5 | 1 | +| 해석기 | 5 | 10 | +| 항목 바꾸기 | 3 | 10 | +| 항목 읽기 | 3 | 0 | +| 답 넘기기 | 3 | 7 | +| 저장과 그 밖 | 3 | 5 | +| (할당기가 가장 안쪽인 표본) | (14) | (7) | + +스냅숏에 딸린 몫(복사와 게시)이 바쁜 시간의 33%에서 17%가 됐다. 남은 할당기 표본 7개 가운데 4개는 `new_node`의 malloc이다. +목록이 자라 새 잎이 필요하거나, 연결 스레드가 스냅숏을 오래 붙들어 돌아온 것이 늦을 때다. 칸마다의 차이는 ±3–4개가 흔들림이다. + +## 판정 + +- 넣었다. 처리량 중앙값 +9.4%, 거래당 CPU −4–8%, 상주/상태 −10%. jemalloc의 이득(11–14%)에 가깝고, 메모리는 jemalloc과 + 반대로 줄었다. Pure는 할당기를 바꾸지 않는다. +- 106의 기준으로 소유자는 여전히 병목이다. W=4, 64터미널에서 코어의 0.92를 쓰고, 처리량은 16터미널보다 8.4% 많다(기준 10%). + +## 다음 + +- 소유자의 바쁜 시간은 이제 찾기 32%, 바꾸기·복사·해석기 각 16%, 답 넘기기 11%, 저장 8%다. 가장 큰 몫은 B+트리를 내려가는 + 찾기다. 한 사건 안에서 같은 목록을 여러 번 찾는다(New-Order는 재고를 열 번). 찾은 자리를 기억하는 손가락은 읽기에만 있다. +- 남은 복사는 저장마다 스냅숏을 게시하는 한 든다. 줄이려면 잎을 통째로 복사하지 않는 방법이 필요하다. +- SD 카드에서는 소유자가 코어의 7–15%만 쓰므로(106) 이 변경이 처리량을 바꿀 까닭이 없다. 재지 않았다. diff --git a/research/experiments/022-tpcc-grouped/tables.py b/research/experiments/022-tpcc-grouped/tables.py index 2c018b0..fcfd468 100644 --- a/research/experiments/022-tpcc-grouped/tables.py +++ b/research/experiments/022-tpcc-grouped/tables.py @@ -69,5 +69,6 @@ def table(name, device): print() -for name in sys.argv[1:] or ['ram-w1', 'ram-w4', 'sd-w1-*', 'sd-w4-*']: - table(name, name.startswith('sd')) +if __name__ == '__main__': + for name in sys.argv[1:] or ['ram-w1', 'ram-w4', 'sd-w1-*', 'sd-w4-*']: + table(name, name.startswith('sd')) diff --git a/research/experiments/023-node-reuse/2026-09-30-after-1-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-after-1-linux-aarch64.jsonl new file mode 100644 index 0000000..bc8e270 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-after-1-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.73, "sqlite_load_s": 1.61, "sqlite_page_size": 4096, "pure_resident_mib": 38.2, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 38305, "new_orders_per_minute": 1024457, "undone_new_orders": 3268, "errors": 0, "new_order": {"done": 344754, "failed": 0, "p50_us": 1472, "p99_us": 5888}, "payment": {"done": 329403, "failed": 0, "p50_us": 1472, "p99_us": 5888}, "order_status": {"done": 30724, "failed": 0, "p50_us": 272, "p99_us": 2816}, "delivery": {"done": 30742, "failed": 0, "p50_us": 1472, "p99_us": 5888}, "stock_level": {"done": 30476, "failed": 0, "p50_us": 304, "p99_us": 3072}, "server_threads": "3", "cpu_us_per_transaction": 45.6, "core_shares": {"pure-owner": 0.83, "pure-checkpoint": 0.01}, "resident_mib": 407.4, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 407.4, "pure_state_mib": 206.7, "pure_store_mib": 214.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 24045, "burst_events": 701713, "checkpoints": 33}} diff --git a/research/experiments/023-node-reuse/2026-09-30-after-2-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-after-2-linux-aarch64.jsonl new file mode 100644 index 0000000..1a4e2fd --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-after-2-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.72, "sqlite_load_s": 1.6, "sqlite_page_size": 4096, "pure_resident_mib": 38.2, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 43811, "new_orders_per_minute": 1172150, "undone_new_orders": 3762, "errors": 0, "new_order": {"done": 394494, "failed": 0, "p50_us": 1472, "p99_us": 3200}, "payment": {"done": 376612, "failed": 0, "p50_us": 1472, "p99_us": 3200}, "order_status": {"done": 35142, "failed": 0, "p50_us": 256, "p99_us": 1152}, "delivery": {"done": 35080, "failed": 0, "p50_us": 1472, "p99_us": 3200}, "stock_level": {"done": 34926, "failed": 0, "p50_us": 288, "p99_us": 1280}, "server_threads": "3", "cpu_us_per_transaction": 43.8, "core_shares": {"pure-owner": 0.91, "pure-checkpoint": 0.01}, "resident_mib": 458.6, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 458.6, "pure_state_mib": 233.3, "pure_store_mib": 238.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 26432, "burst_events": 802509, "checkpoints": 39}} diff --git a/research/experiments/023-node-reuse/2026-09-30-after-3-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-after-3-linux-aarch64.jsonl new file mode 100644 index 0000000..5e21714 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-after-3-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.57, "pure_load_s": 0.71, "sqlite_load_s": 1.74, "sqlite_page_size": 4096, "pure_resident_mib": 38.2, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 43732, "new_orders_per_minute": 1170089, "undone_new_orders": 3746, "errors": 0, "new_order": {"done": 393776, "failed": 0, "p50_us": 1472, "p99_us": 3584}, "payment": {"done": 375969, "failed": 0, "p50_us": 1408, "p99_us": 3456}, "order_status": {"done": 35031, "failed": 0, "p50_us": 256, "p99_us": 1280}, "delivery": {"done": 35009, "failed": 0, "p50_us": 1472, "p99_us": 3328}, "stock_level": {"done": 34864, "failed": 0, "p50_us": 288, "p99_us": 1344}, "server_threads": "3", "cpu_us_per_transaction": 43.9, "core_shares": {"pure-owner": 0.9, "pure-checkpoint": 0.01}, "resident_mib": 458.0, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 458.0, "pure_state_mib": 232.9, "pure_store_mib": 238.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 26495, "burst_events": 801094, "checkpoints": 39}} diff --git a/research/experiments/023-node-reuse/2026-09-30-after-4-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-after-4-linux-aarch64.jsonl new file mode 100644 index 0000000..5753c70 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-after-4-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.58, "pure_load_s": 0.73, "sqlite_load_s": 1.72, "sqlite_page_size": 4096, "pure_resident_mib": 38.9, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 41485, "new_orders_per_minute": 1110139, "undone_new_orders": 3545, "errors": 0, "new_order": {"done": 373592, "failed": 0, "p50_us": 1472, "p99_us": 4608}, "payment": {"done": 356539, "failed": 0, "p50_us": 1472, "p99_us": 4352}, "order_status": {"done": 33350, "failed": 0, "p50_us": 272, "p99_us": 1920}, "delivery": {"done": 33193, "failed": 0, "p50_us": 1472, "p99_us": 4608}, "stock_level": {"done": 33021, "failed": 0, "p50_us": 304, "p99_us": 2048}, "server_threads": "3", "cpu_us_per_transaction": 44.7, "core_shares": {"pure-owner": 0.87, "pure-checkpoint": 0.01}, "resident_mib": 438.8, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 438.8, "pure_state_mib": 222.1, "pure_store_mib": 230.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 26186, "burst_events": 759848, "checkpoints": 37}} diff --git a/research/experiments/023-node-reuse/2026-09-30-after-5-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-after-5-linux-aarch64.jsonl new file mode 100644 index 0000000..f895f85 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-after-5-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.72, "sqlite_load_s": 1.57, "sqlite_page_size": 4096, "pure_resident_mib": 38.9, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 41617, "new_orders_per_minute": 1113588, "undone_new_orders": 3556, "errors": 0, "new_order": {"done": 374752, "failed": 0, "p50_us": 1472, "p99_us": 4096}, "payment": {"done": 357693, "failed": 0, "p50_us": 1472, "p99_us": 4096}, "order_status": {"done": 33407, "failed": 0, "p50_us": 256, "p99_us": 1600}, "delivery": {"done": 33347, "failed": 0, "p50_us": 1472, "p99_us": 4096}, "stock_level": {"done": 33138, "failed": 0, "p50_us": 304, "p99_us": 1856}, "server_threads": "3", "cpu_us_per_transaction": 45.0, "core_shares": {"pure-owner": 0.88, "pure-checkpoint": 0.01}, "resident_mib": 439.3, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 439.3, "pure_state_mib": 222.7, "pure_store_mib": 230.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 26099, "burst_events": 762316, "checkpoints": 37}} diff --git a/research/experiments/023-node-reuse/2026-09-30-after-6-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-after-6-linux-aarch64.jsonl new file mode 100644 index 0000000..6fe89a3 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-after-6-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.72, "sqlite_load_s": 1.58, "sqlite_page_size": 4096, "pure_resident_mib": 38.2, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42292, "new_orders_per_minute": 1131652, "undone_new_orders": 3611, "errors": 0, "new_order": {"done": 380829, "failed": 0, "p50_us": 1472, "p99_us": 3968}, "payment": {"done": 363536, "failed": 0, "p50_us": 1472, "p99_us": 3968}, "order_status": {"done": 33950, "failed": 0, "p50_us": 256, "p99_us": 1536}, "delivery": {"done": 33869, "failed": 0, "p50_us": 1472, "p99_us": 3968}, "stock_level": {"done": 33663, "failed": 0, "p50_us": 304, "p99_us": 1664}, "server_threads": "3", "cpu_us_per_transaction": 44.7, "core_shares": {"pure-owner": 0.89, "pure-checkpoint": 0.01}, "resident_mib": 445.2, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 445.2, "pure_state_mib": 226.0, "pure_store_mib": 230.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 26227, "burst_events": 774708, "checkpoints": 38}} diff --git a/research/experiments/023-node-reuse/2026-09-30-allocator-glibc-1-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-allocator-glibc-1-linux-aarch64.jsonl new file mode 100644 index 0000000..a156a94 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-allocator-glibc-1-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.71, "sqlite_load_s": 1.58, "sqlite_page_size": 4096, "pure_resident_mib": 38.1, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 37280, "new_orders_per_minute": 996985, "undone_new_orders": 3189, "errors": 0, "new_order": {"done": 335518, "failed": 0, "p50_us": 1664, "p99_us": 4352}, "payment": {"done": 320602, "failed": 0, "p50_us": 1664, "p99_us": 4352}, "order_status": {"done": 29884, "failed": 0, "p50_us": 272, "p99_us": 2176}, "delivery": {"done": 29906, "failed": 0, "p50_us": 1664, "p99_us": 4352}, "stock_level": {"done": 29689, "failed": 0, "p50_us": 320, "p99_us": 2560}, "server_threads": "3", "cpu_us_per_transaction": 48.2, "core_shares": {"pure-owner": 0.88, "pure-checkpoint": 0.01}, "resident_mib": 438.6, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 438.6, "pure_state_mib": 201.8, "pure_store_mib": 206.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 23125, "burst_events": 682923, "checkpoints": 32}} diff --git a/research/experiments/023-node-reuse/2026-09-30-allocator-glibc-2-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-allocator-glibc-2-linux-aarch64.jsonl new file mode 100644 index 0000000..a2785f8 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-allocator-glibc-2-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.72, "sqlite_load_s": 1.63, "sqlite_page_size": 4096, "pure_resident_mib": 38.1, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 40515, "new_orders_per_minute": 1083966, "undone_new_orders": 3476, "errors": 0, "new_order": {"done": 364798, "failed": 0, "p50_us": 1600, "p99_us": 3200}, "payment": {"done": 348245, "failed": 0, "p50_us": 1600, "p99_us": 3072}, "order_status": {"done": 32579, "failed": 0, "p50_us": 256, "p99_us": 1344}, "delivery": {"done": 32449, "failed": 0, "p50_us": 1600, "p99_us": 3200}, "stock_level": {"done": 32227, "failed": 0, "p50_us": 288, "p99_us": 1536}, "server_threads": "3", "cpu_us_per_transaction": 46.8, "core_shares": {"pure-owner": 0.92, "pure-checkpoint": 0.01}, "resident_mib": 474.3, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 474.3, "pure_state_mib": 217.4, "pure_store_mib": 222.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 24336, "burst_events": 742101, "checkpoints": 36}} diff --git a/research/experiments/023-node-reuse/2026-09-30-allocator-jemalloc-1-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-allocator-jemalloc-1-linux-aarch64.jsonl new file mode 100644 index 0000000..75074f5 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-allocator-jemalloc-1-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.71, "sqlite_load_s": 1.62, "sqlite_page_size": 4096, "pure_resident_mib": 51.4, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42323, "new_orders_per_minute": 1132316, "undone_new_orders": 3622, "errors": 0, "new_order": {"done": 381061, "failed": 0, "p50_us": 1408, "p99_us": 4608}, "payment": {"done": 363847, "failed": 0, "p50_us": 1408, "p99_us": 4608}, "order_status": {"done": 33946, "failed": 0, "p50_us": 272, "p99_us": 1984}, "delivery": {"done": 33913, "failed": 0, "p50_us": 1408, "p99_us": 4352}, "stock_level": {"done": 33695, "failed": 0, "p50_us": 304, "p99_us": 2176}, "server_threads": "3", "cpu_us_per_transaction": 43.4, "core_shares": {"pure-owner": 0.85, "pure-checkpoint": 0.01}, "resident_mib": 506.0, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 506.0, "pure_state_mib": 226.1, "pure_store_mib": 230.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 27007, "burst_events": 775283, "checkpoints": 38}} diff --git a/research/experiments/023-node-reuse/2026-09-30-allocator-jemalloc-2-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-allocator-jemalloc-2-linux-aarch64.jsonl new file mode 100644 index 0000000..a8e76a1 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-allocator-jemalloc-2-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.71, "sqlite_load_s": 1.61, "sqlite_page_size": 4096, "pure_resident_mib": 51.0, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 45119, "new_orders_per_minute": 1207006, "undone_new_orders": 3879, "errors": 0, "new_order": {"done": 406223, "failed": 0, "p50_us": 1408, "p99_us": 3456}, "payment": {"done": 387908, "failed": 0, "p50_us": 1408, "p99_us": 3456}, "order_status": {"done": 36167, "failed": 0, "p50_us": 256, "p99_us": 1344}, "delivery": {"done": 36140, "failed": 0, "p50_us": 1408, "p99_us": 3456}, "stock_level": {"done": 35967, "failed": 0, "p50_us": 288, "p99_us": 1408}, "server_threads": "3", "cpu_us_per_transaction": 43.5, "core_shares": {"pure-owner": 0.89, "pure-checkpoint": 0.01}, "resident_mib": 536.3, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 536.3, "pure_state_mib": 239.5, "pure_store_mib": 246.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 28362, "burst_events": 826477, "checkpoints": 40}} diff --git a/research/experiments/023-node-reuse/2026-09-30-allocator-tcache-1-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-allocator-tcache-1-linux-aarch64.jsonl new file mode 100644 index 0000000..4f93d87 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-allocator-tcache-1-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.72, "sqlite_load_s": 1.57, "sqlite_page_size": 4096, "pure_resident_mib": 38.0, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 39799, "new_orders_per_minute": 1064974, "undone_new_orders": 3407, "errors": 0, "new_order": {"done": 358401, "failed": 0, "p50_us": 1664, "p99_us": 3584}, "payment": {"done": 342088, "failed": 0, "p50_us": 1600, "p99_us": 3456}, "order_status": {"done": 31982, "failed": 0, "p50_us": 256, "p99_us": 1664}, "delivery": {"done": 31892, "failed": 0, "p50_us": 1600, "p99_us": 3456}, "stock_level": {"done": 31628, "failed": 0, "p50_us": 304, "p99_us": 1792}, "server_threads": "3", "cpu_us_per_transaction": 47.0, "core_shares": {"pure-owner": 0.91, "pure-checkpoint": 0.01}, "resident_mib": 465.9, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 465.9, "pure_state_mib": 214.0, "pure_store_mib": 222.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 24127, "burst_events": 729058, "checkpoints": 36}} diff --git a/research/experiments/023-node-reuse/2026-09-30-before-1-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-before-1-linux-aarch64.jsonl new file mode 100644 index 0000000..b50fef8 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-before-1-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.72, "sqlite_load_s": 1.56, "sqlite_page_size": 4096, "pure_resident_mib": 38.1, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 39304, "new_orders_per_minute": 1051376, "undone_new_orders": 3367, "errors": 0, "new_order": {"done": 353832, "failed": 0, "p50_us": 1664, "p99_us": 3584}, "payment": {"done": 337906, "failed": 0, "p50_us": 1600, "p99_us": 3584}, "order_status": {"done": 31594, "failed": 0, "p50_us": 256, "p99_us": 1536}, "delivery": {"done": 31495, "failed": 0, "p50_us": 1600, "p99_us": 3456}, "stock_level": {"done": 31267, "failed": 0, "p50_us": 304, "p99_us": 1792}, "server_threads": "3", "cpu_us_per_transaction": 47.4, "core_shares": {"pure-owner": 0.91, "pure-checkpoint": 0.01}, "resident_mib": 460.9, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 460.9, "pure_state_mib": 211.5, "pure_store_mib": 214.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 23739, "burst_events": 719932, "checkpoints": 35}} diff --git a/research/experiments/023-node-reuse/2026-09-30-before-2-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-before-2-linux-aarch64.jsonl new file mode 100644 index 0000000..5b6b754 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-before-2-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.72, "sqlite_load_s": 1.58, "sqlite_page_size": 4096, "pure_resident_mib": 38.1, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 38863, "new_orders_per_minute": 1039598, "undone_new_orders": 3326, "errors": 0, "new_order": {"done": 349872, "failed": 0, "p50_us": 1664, "p99_us": 3968}, "payment": {"done": 334120, "failed": 0, "p50_us": 1600, "p99_us": 3840}, "order_status": {"done": 31207, "failed": 0, "p50_us": 256, "p99_us": 1600}, "delivery": {"done": 31184, "failed": 0, "p50_us": 1664, "p99_us": 3840}, "stock_level": {"done": 30908, "failed": 0, "p50_us": 304, "p99_us": 1856}, "server_threads": "3", "cpu_us_per_transaction": 47.0, "core_shares": {"pure-owner": 0.9, "pure-checkpoint": 0.01}, "resident_mib": 456.2, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 456.2, "pure_state_mib": 209.4, "pure_store_mib": 214.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 23303, "burst_events": 711935, "checkpoints": 34}} diff --git a/research/experiments/023-node-reuse/2026-09-30-before-3-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-before-3-linux-aarch64.jsonl new file mode 100644 index 0000000..0cb9922 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-before-3-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.73, "sqlite_load_s": 1.59, "sqlite_page_size": 4096, "pure_resident_mib": 38.1, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 39952, "new_orders_per_minute": 1068823, "undone_new_orders": 3425, "errors": 0, "new_order": {"done": 359702, "failed": 0, "p50_us": 1600, "p99_us": 3584}, "payment": {"done": 343427, "failed": 0, "p50_us": 1600, "p99_us": 3456}, "order_status": {"done": 32145, "failed": 0, "p50_us": 256, "p99_us": 1600}, "delivery": {"done": 31999, "failed": 0, "p50_us": 1600, "p99_us": 3456}, "stock_level": {"done": 31779, "failed": 0, "p50_us": 288, "p99_us": 1728}, "server_threads": "3", "cpu_us_per_transaction": 47.0, "core_shares": {"pure-owner": 0.91, "pure-checkpoint": 0.01}, "resident_mib": 467.8, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 467.8, "pure_state_mib": 214.7, "pure_store_mib": 222.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 24088, "burst_events": 731789, "checkpoints": 36}} diff --git a/research/experiments/023-node-reuse/2026-09-30-before-4-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-before-4-linux-aarch64.jsonl new file mode 100644 index 0000000..74be71d --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-before-4-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.72, "sqlite_load_s": 1.58, "sqlite_page_size": 4096, "pure_resident_mib": 38.0, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 37931, "new_orders_per_minute": 1014602, "undone_new_orders": 3235, "errors": 0, "new_order": {"done": 341436, "failed": 0, "p50_us": 1664, "p99_us": 4096}, "payment": {"done": 326136, "failed": 0, "p50_us": 1664, "p99_us": 4096}, "order_status": {"done": 30443, "failed": 0, "p50_us": 272, "p99_us": 1984}, "delivery": {"done": 30413, "failed": 0, "p50_us": 1664, "p99_us": 4096}, "stock_level": {"done": 30195, "failed": 0, "p50_us": 320, "p99_us": 2176}, "server_threads": "3", "cpu_us_per_transaction": 48.4, "core_shares": {"pure-owner": 0.89, "pure-checkpoint": 0.01}, "resident_mib": 445.5, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 445.5, "pure_state_mib": 204.9, "pure_store_mib": 206.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 23368, "burst_events": 694830, "checkpoints": 33}} diff --git a/research/experiments/023-node-reuse/2026-09-30-before-5-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-before-5-linux-aarch64.jsonl new file mode 100644 index 0000000..8bba878 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-before-5-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.72, "sqlite_load_s": 1.69, "sqlite_page_size": 4096, "pure_resident_mib": 38.1, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 36817, "new_orders_per_minute": 984754, "undone_new_orders": 3146, "errors": 0, "new_order": {"done": 331418, "failed": 0, "p50_us": 1664, "p99_us": 4864}, "payment": {"done": 316613, "failed": 0, "p50_us": 1664, "p99_us": 4864}, "order_status": {"done": 29542, "failed": 0, "p50_us": 272, "p99_us": 2432}, "delivery": {"done": 29512, "failed": 0, "p50_us": 1664, "p99_us": 4608}, "stock_level": {"done": 29297, "failed": 0, "p50_us": 304, "p99_us": 2432}, "server_threads": "3", "cpu_us_per_transaction": 48.2, "core_shares": {"pure-owner": 0.88, "pure-checkpoint": 0.01}, "resident_mib": 433.5, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 433.5, "pure_state_mib": 199.6, "pure_store_mib": 206.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 22848, "burst_events": 674483, "checkpoints": 32}} diff --git a/research/experiments/023-node-reuse/2026-09-30-before-6-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-before-6-linux-aarch64.jsonl new file mode 100644 index 0000000..b455f82 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-before-6-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.57, "pure_load_s": 0.72, "sqlite_load_s": 1.63, "sqlite_page_size": 4096, "pure_resident_mib": 38.1, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 38679, "new_orders_per_minute": 1034696, "undone_new_orders": 3302, "errors": 0, "new_order": {"done": 348201, "failed": 0, "p50_us": 1664, "p99_us": 3968}, "payment": {"done": 332513, "failed": 0, "p50_us": 1600, "p99_us": 3968}, "order_status": {"done": 31063, "failed": 0, "p50_us": 256, "p99_us": 1920}, "delivery": {"done": 31057, "failed": 0, "p50_us": 1600, "p99_us": 3968}, "stock_level": {"done": 30750, "failed": 0, "p50_us": 304, "p99_us": 2048}, "server_threads": "3", "cpu_us_per_transaction": 47.9, "core_shares": {"pure-owner": 0.9, "pure-checkpoint": 0.01}, "resident_mib": 453.6, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 453.6, "pure_state_mib": 208.5, "pure_store_mib": 214.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 23746, "burst_events": 708553, "checkpoints": 34}} diff --git a/research/experiments/023-node-reuse/2026-09-30-owner-run-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-owner-run-linux-aarch64.jsonl new file mode 100644 index 0000000..97f3543 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-owner-run-linux-aarch64.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.73, "sqlite_load_s": 1.6, "sqlite_page_size": 4096, "pure_resident_mib": 38.2, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 39350, "new_orders_per_minute": 1051691, "undone_new_orders": 15845, "errors": 0, "new_order": {"done": 1593393, "failed": 0, "p50_us": 1536, "p99_us": 4096}, "payment": {"done": 1523624, "failed": 0, "p50_us": 1536, "p99_us": 4096}, "order_status": {"done": 141544, "failed": 0, "p50_us": 272, "p99_us": 1664}, "delivery": {"done": 141406, "failed": 0, "p50_us": 1536, "p99_us": 4096}, "stock_level": {"done": 141524, "failed": 0, "p50_us": 304, "p99_us": 1792}, "server_threads": "3", "cpu_us_per_transaction": 45.8, "core_shares": {"pure-owner": 0.86, "pure-checkpoint": 0.02}, "resident_mib": 1683.6, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 1739.4, "pure_state_mib": 874.3, "pure_store_mib": 932.5, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 108865, "burst_events": 3242662, "checkpoints": 166}} diff --git a/research/experiments/023-node-reuse/2026-09-30-owner-stacks-linux-aarch64.txt b/research/experiments/023-node-reuse/2026-09-30-owner-stacks-linux-aarch64.txt new file mode 100644 index 0000000..ed498a2 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-owner-stacks-linux-aarch64.txt @@ -0,0 +1,80 @@ +mprotect < ?? < ?? < malloc < list_changes < pure_save_commit < pure_app_save < save_burst < save_burst < execute < execute < run_owner < ?? < ?? +upper < pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < move_items < move_items < place < pure_list_insert < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +upper < pure_list_seek < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +upper < pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_timedwait < run_owner < ?? < ?? +?? < malloc < new_node < prepare < pure_list_insert < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_timedwait < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_wait < run_owner < ?? < ?? +item_field < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < hand_back < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_wait < run_owner < ?? < ?? +pure_log_append < pure_save_commit < pure_app_save < save_burst < save_burst < execute < execute < run_owner < ?? < ?? +upper < pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < malloc < new_node < prepare < pure_list_insert < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < own < descend < pure_list_set < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_wait < run_owner < ?? < ?? +?? < ?? < ?? < free < release_extras < place < pure_list_insert < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < hand_back < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < hand_back < execute < execute < run_owner < ?? < ?? +descend < prepare < pure_list_insert < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_timedwait < run_owner < ?? < ?? +child_at < leaf_of < pure_list_get < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +lower < pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_wait < run_owner < ?? < ?? +pure_list_count < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +apply_effect < execute < execute < run_owner < ?? < ?? +mark_event < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +lower < pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +own < descend < pure_list_set < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +mix < stripes < digest_add < entry_digest < pure_log_append < pure_save_commit < pure_app_save < save_burst < save_burst < execute < execute < run_owner < ?? < ?? +descend < prepare < pure_list_insert < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_wait < run_owner < ?? < ?? +child < leaf_of < pure_list_get < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +malloc@plt < new_node < own < descend < prepare < pure_list_insert < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < hand_back < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < hand_back < execute < execute < run_owner < ?? < ?? +pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_timedwait < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_timedwait < run_owner < ?? < ?? +sum < sum < sum_path < sum_path < place < pure_list_insert < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < hand_back < execute < execute < run_owner < ?? < ?? +run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_timedwait < run_owner < ?? < ?? +new_node < own < descend < pure_list_set < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +descend < prepare < pure_list_insert < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +descend < prepare < pure_list_insert < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_timedwait < run_owner < ?? < ?? +pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +journal_room < pure_list_set < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +upper < pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +malloc < new_node < prepare < pure_list_insert < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < malloc < new_node < own < descend < prepare < pure_list_insert < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_wait < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_timedwait < run_owner < ?? < ?? +?? < malloc < pure_list_share < pure_save_snapshot < pure_app_snapshot < publish < after_save < save_burst < save_burst < execute < execute < run_owner < ?? < ?? +lower < pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +lower < pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +own < descend < prepare < pure_list_insert < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +child < pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_timedwait < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_timedwait < run_owner < ?? < ?? +new_node < own < descend < pure_list_set < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +child < pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < hand_back < execute < execute < run_owner < ?? < ?? +descend < pure_list_set < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +?? < ?? < ?? < pthread_cond_timedwait < run_owner < ?? < ?? +put_compact < pure_list_insert < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +pure_save_commit < pure_app_save < save_burst < save_burst < execute < execute < run_owner < ?? < ?? +stored_list < pure_app_checkpoint < checkpoint_round < after_save < save_burst < save_burst < execute < execute < run_owner < ?? < ?? +upper < pure_list_seek < run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? +run < run < dispatch < message < pure_app_message_fields < apply_input < apply_effect < execute < execute < run_owner < ?? < ?? diff --git a/research/experiments/023-node-reuse/2026-09-30-ram-w1-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-ram-w1-linux-aarch64.jsonl new file mode 100644 index 0000000..c516273 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-ram-w1-linux-aarch64.jsonl @@ -0,0 +1,9 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.74, "sqlite_load_s": 1.59, "sqlite_page_size": 4096, "pure_resident_mib": 38.2, "sqlite_file_mib": 13.8} +{"step": "same transactions", "transactions": 20000, "pure": {"new_order": 9036, "payment": 8518, "order_status": 811, "delivery": 796, "stock_level": 839}, "sqlite": {"new_order": 9036, "payment": 8518, "order_status": 811, "delivery": 796, "stock_level": 839}, "undone_new_orders": [90, 90], "failed": [0, 0], "differing": {}, "grouped_sqlite_differing": {}, "consistency_failures": []} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 16, "transactions_per_second": 38320, "new_orders_per_minute": 1025360, "undone_new_orders": 3508, "errors": 0, "new_order": {"done": 345298, "failed": 0, "p50_us": 416, "p99_us": 1088}, "payment": {"done": 329116, "failed": 0, "p50_us": 400, "p99_us": 1024}, "order_status": {"done": 30751, "failed": 0, "p50_us": 100, "p99_us": 512}, "delivery": {"done": 30647, "failed": 0, "p50_us": 400, "p99_us": 1024}, "stock_level": {"done": 30604, "failed": 0, "p50_us": 144, "p99_us": 544}, "server_threads": "3", "cpu_us_per_transaction": 42.7, "core_shares": {"pure-owner": 0.81, "pure-checkpoint": 0.02}, "resident_mib": 425.4, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 40976, "new_orders_per_minute": 1096607, "undone_new_orders": 3513, "errors": 0, "new_order": {"done": 369053, "failed": 0, "p50_us": 1536, "p99_us": 3968}, "payment": {"done": 352111, "failed": 0, "p50_us": 1536, "p99_us": 3968}, "order_status": {"done": 32916, "failed": 0, "p50_us": 272, "p99_us": 1664}, "delivery": {"done": 32826, "failed": 0, "p50_us": 1536, "p99_us": 3968}, "stock_level": {"done": 32620, "failed": 0, "p50_us": 304, "p99_us": 1728}, "server_threads": "3", "cpu_us_per_transaction": 45.8, "core_shares": {"pure-owner": 0.89, "pure-checkpoint": 0.01}, "resident_mib": 821.5, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "run", "system": "sqlite 3.46.1", "synchronous": "full", "pragmas": "PRAGMA mmap_size=1073741824; PRAGMA cache_size=-65536; PRAGMA temp_store=MEMORY; PRAGMA wal_autocheckpoint=10000;", "warehouses": 1, "terminals": 1, "transactions_per_second": 10164, "batches": 0, "batched": 0, "new_orders_per_minute": 272459, "undone_new_orders": 964, "new_order": {"done": 91812, "failed": 0, "p50_us": 152, "p99_us": 256}, "payment": {"done": 87175, "failed": 0, "p50_us": 24, "p99_us": 44}, "order_status": {"done": 8073, "failed": 0, "p50_us": 16, "p99_us": 30}, "delivery": {"done": 8215, "failed": 0, "p50_us": 56, "p99_us": 352}, "stock_level": {"done": 8065, "failed": 0, "p50_us": 272, "p99_us": 432}, "cpu_seconds": 19.939, "peak_resident_kib": 99744, "written_bytes": 0, "cpu_us_per_transaction": 98.1, "peak_resident_mib": 97.4, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "run", "system": "sqlite 3.46.1", "synchronous": "full", "pragmas": "PRAGMA mmap_size=1073741824; PRAGMA cache_size=-65536; PRAGMA temp_store=MEMORY; PRAGMA wal_autocheckpoint=10000;", "warehouses": 1, "terminals": 4, "transactions_per_second": 9955, "batches": 0, "batched": 0, "new_orders_per_minute": 266365, "undone_new_orders": 948, "new_order": {"done": 90047, "failed": 0, "p50_us": 152, "p99_us": 304}, "payment": {"done": 85836, "failed": 0, "p50_us": 25, "p99_us": 52}, "order_status": {"done": 7969, "failed": 0, "p50_us": 20, "p99_us": 46}, "delivery": {"done": 8007, "failed": 0, "p50_us": 58, "p99_us": 136}, "stock_level": {"done": 7937, "failed": 0, "p50_us": 272, "p99_us": 608}, "cpu_seconds": 20.287, "peak_resident_kib": 42784, "written_bytes": 0, "cpu_us_per_transaction": 101.5, "peak_resident_mib": 41.8, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "run", "system": "sqlite 3.46.1, grouped", "synchronous": "full", "pragmas": "PRAGMA mmap_size=1073741824; PRAGMA cache_size=-65536; PRAGMA temp_store=MEMORY; PRAGMA wal_autocheckpoint=10000;", "warehouses": 1, "terminals": 16, "transactions_per_second": 9731, "batches": 15707, "batched": 179191, "new_orders_per_minute": 260697, "undone_new_orders": 882, "new_order": {"done": 87831, "failed": 0, "p50_us": 1600, "p99_us": 4352}, "payment": {"done": 83499, "failed": 0, "p50_us": 1472, "p99_us": 4096}, "order_status": {"done": 7845, "failed": 0, "p50_us": 136, "p99_us": 864}, "delivery": {"done": 7861, "failed": 0, "p50_us": 1536, "p99_us": 4096}, "stock_level": {"done": 7691, "failed": 0, "p50_us": 736, "p99_us": 1792}, "cpu_seconds": 26.473, "peak_resident_kib": 77728, "written_bytes": 0, "cpu_us_per_transaction": 135.9, "peak_resident_mib": 75.9, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "run", "system": "sqlite 3.46.1, grouped", "synchronous": "full", "pragmas": "PRAGMA mmap_size=1073741824; PRAGMA cache_size=-65536; PRAGMA temp_store=MEMORY; PRAGMA wal_autocheckpoint=10000;", "warehouses": 1, "terminals": 64, "transactions_per_second": 10073, "batches": 3703, "batched": 185882, "new_orders_per_minute": 268814, "undone_new_orders": 873, "new_order": {"done": 90762, "failed": 0, "p50_us": 5888, "p99_us": 12800}, "payment": {"done": 87012, "failed": 0, "p50_us": 5888, "p99_us": 12800}, "order_status": {"done": 8203, "failed": 0, "p50_us": 248, "p99_us": 1728}, "delivery": {"done": 8108, "failed": 0, "p50_us": 5888, "p99_us": 12800}, "stock_level": {"done": 8018, "failed": 0, "p50_us": 1152, "p99_us": 3840}, "cpu_seconds": 27.139, "peak_resident_kib": 129888, "written_bytes": 0, "cpu_us_per_transaction": 134.3, "peak_resident_mib": 126.8, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 821.5, "pure_state_mib": 408.9, "pure_store_mib": 447.2, "sqlite_files_mib": 82.6, "grouped_sqlite_files_mib": 80.9, "pure_counts": {"bursts": 133359, "burst_events": 1470390, "checkpoints": 77}} diff --git a/research/experiments/023-node-reuse/2026-09-30-ram-w4-linux-aarch64.jsonl b/research/experiments/023-node-reuse/2026-09-30-ram-w4-linux-aarch64.jsonl new file mode 100644 index 0000000..82d7c58 --- /dev/null +++ b/research/experiments/023-node-reuse/2026-09-30-ram-w4-linux-aarch64.jsonl @@ -0,0 +1,9 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 4, "rows": 1975536, "generate_s": 1.99, "pure_load_s": 2.64, "sqlite_load_s": 5.95, "sqlite_page_size": 4096, "pure_resident_mib": 104.1, "sqlite_file_mib": 52.8} +{"step": "same transactions", "transactions": 20000, "pure": {"new_order": 8875, "payment": 8786, "order_status": 777, "delivery": 777, "stock_level": 785}, "sqlite": {"new_order": 8875, "payment": 8786, "order_status": 777, "delivery": 777, "stock_level": 785}, "undone_new_orders": [93, 93], "failed": [0, 0], "differing": {}, "grouped_sqlite_differing": {}, "consistency_failures": []} +{"step": "run", "system": "pure", "warehouses": 4, "terminals": 16, "transactions_per_second": 33057, "new_orders_per_minute": 883329, "undone_new_orders": 2956, "errors": 0, "new_order": {"done": 297399, "failed": 0, "p50_us": 496, "p99_us": 1408}, "payment": {"done": 283706, "failed": 0, "p50_us": 448, "p99_us": 1344}, "order_status": {"done": 26855, "failed": 0, "p50_us": 84, "p99_us": 544}, "delivery": {"done": 26604, "failed": 0, "p50_us": 464, "p99_us": 1472}, "stock_level": {"done": 26583, "failed": 0, "p50_us": 144, "p99_us": 640}, "server_threads": "3", "cpu_us_per_transaction": 46.6, "core_shares": {"pure-owner": 0.82, "pure-checkpoint": 0.01}, "resident_mib": 459.0, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "run", "system": "pure", "warehouses": 4, "terminals": 64, "transactions_per_second": 35838, "new_orders_per_minute": 958218, "undone_new_orders": 3172, "errors": 0, "new_order": {"done": 322578, "failed": 0, "p50_us": 1856, "p99_us": 3584}, "payment": {"done": 308407, "failed": 0, "p50_us": 1792, "p99_us": 3584}, "order_status": {"done": 28642, "failed": 0, "p50_us": 272, "p99_us": 1216}, "delivery": {"done": 28624, "failed": 0, "p50_us": 1792, "p99_us": 3456}, "stock_level": {"done": 28509, "failed": 0, "p50_us": 304, "p99_us": 1344}, "server_threads": "3", "cpu_us_per_transaction": 50.2, "core_shares": {"pure-owner": 0.92, "pure-checkpoint": 0.01}, "resident_mib": 817.1, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "run", "system": "sqlite 3.46.1", "synchronous": "full", "pragmas": "PRAGMA mmap_size=1073741824; PRAGMA cache_size=-65536; PRAGMA temp_store=MEMORY; PRAGMA wal_autocheckpoint=10000;", "warehouses": 4, "terminals": 1, "transactions_per_second": 10168, "batches": 0, "batched": 0, "new_orders_per_minute": 272034, "undone_new_orders": 943, "new_order": {"done": 91652, "failed": 0, "p50_us": 152, "p99_us": 240}, "payment": {"done": 87378, "failed": 0, "p50_us": 25, "p99_us": 42}, "order_status": {"done": 8107, "failed": 0, "p50_us": 16, "p99_us": 28}, "delivery": {"done": 8122, "failed": 0, "p50_us": 56, "p99_us": 416}, "stock_level": {"done": 8167, "failed": 0, "p50_us": 272, "p99_us": 384}, "cpu_seconds": 19.95, "peak_resident_kib": 117472, "written_bytes": 0, "cpu_us_per_transaction": 98.1, "peak_resident_mib": 114.7, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "run", "system": "sqlite 3.46.1", "synchronous": "full", "pragmas": "PRAGMA mmap_size=1073741824; PRAGMA cache_size=-65536; PRAGMA temp_store=MEMORY; PRAGMA wal_autocheckpoint=10000;", "warehouses": 4, "terminals": 4, "transactions_per_second": 9732, "batches": 0, "batched": 0, "new_orders_per_minute": 259837, "undone_new_orders": 811, "new_order": {"done": 87684, "failed": 0, "p50_us": 160, "p99_us": 288}, "payment": {"done": 84068, "failed": 0, "p50_us": 25, "p99_us": 52}, "order_status": {"done": 7871, "failed": 0, "p50_us": 19, "p99_us": 38}, "delivery": {"done": 7773, "failed": 0, "p50_us": 60, "p99_us": 128}, "stock_level": {"done": 7830, "failed": 0, "p50_us": 272, "p99_us": 608}, "cpu_seconds": 20.416, "peak_resident_kib": 60048, "written_bytes": 0, "cpu_us_per_transaction": 104.6, "peak_resident_mib": 58.6, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "run", "system": "sqlite 3.46.1, grouped", "synchronous": "full", "pragmas": "PRAGMA mmap_size=1073741824; PRAGMA cache_size=-65536; PRAGMA temp_store=MEMORY; PRAGMA wal_autocheckpoint=10000;", "warehouses": 4, "terminals": 16, "transactions_per_second": 8750, "batches": 14079, "batched": 161044, "new_orders_per_minute": 234615, "undone_new_orders": 798, "new_order": {"done": 79047, "failed": 0, "p50_us": 1792, "p99_us": 8192}, "payment": {"done": 74842, "failed": 0, "p50_us": 1600, "p99_us": 8192}, "order_status": {"done": 7025, "failed": 0, "p50_us": 136, "p99_us": 928}, "delivery": {"done": 7155, "failed": 0, "p50_us": 1664, "p99_us": 8192}, "stock_level": {"done": 7024, "failed": 0, "p50_us": 736, "p99_us": 1920}, "cpu_seconds": 26.066, "peak_resident_kib": 116320, "written_bytes": 0, "cpu_us_per_transaction": 148.9, "peak_resident_mib": 113.6, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "run", "system": "sqlite 3.46.1, grouped", "synchronous": "full", "pragmas": "PRAGMA mmap_size=1073741824; PRAGMA cache_size=-65536; PRAGMA temp_store=MEMORY; PRAGMA wal_autocheckpoint=10000;", "warehouses": 4, "terminals": 64, "transactions_per_second": 8805, "batches": 3234, "batched": 162696, "new_orders_per_minute": 236411, "undone_new_orders": 770, "new_order": {"done": 79777, "failed": 0, "p50_us": 6656, "p99_us": 18432}, "payment": {"done": 75881, "failed": 0, "p50_us": 6400, "p99_us": 17408}, "order_status": {"done": 6928, "failed": 0, "p50_us": 232, "p99_us": 2048}, "delivery": {"done": 7038, "failed": 0, "p50_us": 6656, "p99_us": 17408}, "stock_level": {"done": 6924, "failed": 0, "p50_us": 1088, "p99_us": 3584}, "cpu_seconds": 26.154, "peak_resident_kib": 189424, "written_bytes": 0, "cpu_us_per_transaction": 148.1, "peak_resident_mib": 185.0, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 817.1, "pure_state_mib": 420.9, "pure_store_mib": 434.7, "sqlite_files_mib": 123.1, "grouped_sqlite_files_mib": 117.2, "pure_counts": {"bursts": 113964, "burst_events": 1279680, "checkpoints": 29}} diff --git a/research/experiments/023-node-reuse/runs.sh b/research/experiments/023-node-reuse/runs.sh new file mode 100644 index 0000000..d87302e --- /dev/null +++ b/research/experiments/023-node-reuse/runs.sh @@ -0,0 +1,54 @@ +#!/bin/sh +# The runs of docs/research/107, from the repository's root. BEFORE is +# pure-serve built at c9fda91 (nodes a snapshot let go were freed), AFTER at +# the commit that reuses them; copy each out of build/ first, since the +# names they run under do not matter (the owner is found by its thread). +set -e +out=research/experiments/023-node-reuse +before=${BEFORE:?pure-serve built before} +after=${AFTER:?pure-serve built after} +jemalloc=/usr/lib/aarch64-linux-gnu/libjemalloc.so.2 +run() { + name=$1 + shift + python3 benchmarks/tpcc/run.py "$@" --sqlite-threads '' --sqlite-group '' > $out/2026-09-30-$name-linux-aarch64.jsonl +} +alone() { + name=$1 + serve=$2 + run $name --serve $serve --warehouses 1 --check 0 --terminals 64 --seconds 20 +} + +# Is it the allocator? The same server under jemalloc, and with glibc's +# thread cache raised (it stays at 1,032 B: no change), against glibc. +printf '#!/bin/sh\nLD_PRELOAD=%s exec %s "$@"\n' $jemalloc $before > $out/serve-jemalloc.sh +printf '#!/bin/sh\nGLIBC_TUNABLES=glibc.malloc.tcache_max=4096 exec %s "$@"\n' $before > $out/serve-tcache.sh +chmod +x $out/serve-jemalloc.sh $out/serve-tcache.sh +alone allocator-glibc-1 $before +alone allocator-jemalloc-1 $out/serve-jemalloc.sh +alone allocator-tcache-1 $out/serve-tcache.sh +alone allocator-glibc-2 $before +alone allocator-jemalloc-2 $out/serve-jemalloc.sh +rm $out/serve-jemalloc.sh $out/serve-tcache.sh + +# Before and after, taking turns: three pairs before first, three after first. +for round in 1 2 3; do + alone before-$round $before + alone after-$round $after +done +for round in 4 5 6; do + alone after-$round $after + alone before-$round $before +done + +# Where the owner is now (as in 106): 80 samples while AFTER runs 90 seconds. +python3 research/experiments/022-tpcc-grouped/owner-samples.py 80 $out/2026-09-30-owner-stacks-linux-aarch64.txt & +run owner-run --serve $after --warehouses 1 --check 0 --terminals 64 --seconds 90 +wait + +# 106's RAM matrix again with AFTER (SQLite as it was). +both='--terminals 16,64 --sqlite-threads 1,4 --sqlite-group 16,64' +python3 benchmarks/tpcc/run.py --serve $after --warehouses 1 --check 20000 $both --seconds 20 \ + > $out/2026-09-30-ram-w1-linux-aarch64.jsonl +python3 benchmarks/tpcc/run.py --serve $after --warehouses 4 --check 20000 $both --seconds 20 \ + > $out/2026-09-30-ram-w4-linux-aarch64.jsonl diff --git a/research/experiments/023-node-reuse/summary.py b/research/experiments/023-node-reuse/summary.py new file mode 100644 index 0000000..e5fbbb0 --- /dev/null +++ b/research/experiments/023-node-reuse/summary.py @@ -0,0 +1,50 @@ +#!/usr/bin/env python3 +"""docs/research/107's numbers from the result files beside this one: the +allocators, before and after in turns (six pairs: runs.sh made the first +three, before first; three more followed, after first), and 106's RAM +matrix again (with 022-tpcc-grouped/tables.py).""" +import importlib.util +import json +from pathlib import Path + +HERE = Path(__file__).resolve().parent + + +def pure_run(name): + run = end = None + for line in (HERE / f'2026-09-30-{name}-linux-aarch64.jsonl').read_text().splitlines(): + r = json.loads(line) + run = r if r['step'] == 'run' else run + end = r if r['step'] == 'end' else end + return run, end + + +def row(label, name): + run, end = pure_run(name) + ratio = end['pure_resident_mib'] / end['pure_state_mib'] + print(f"| {label} | {run['transactions_per_second']:,} | {run['cpu_us_per_transaction']} | " + f"{run['core_shares']['pure-owner']:.2f} | {end['pure_resident_mib']:.0f} / {end['pure_state_mib']:.0f} MiB " + f"({ratio:.2f}) |") + return run['transactions_per_second'] + + +print('| | 초당 거래 | 거래당 CPU (µs) | 소유자의 코어 몫 | 상주 / 상태 (배수) |') +print('|---|---:|---:|---:|---:|') +for name, label in (('allocator-glibc-1', 'glibc'), ('allocator-jemalloc-1', 'jemalloc'), + ('allocator-tcache-1', 'glibc, tcache_max=4096'), ('allocator-glibc-2', 'glibc'), + ('allocator-jemalloc-2', 'jemalloc')): + row(label, name) +print() +print('| | 초당 거래 | 거래당 CPU (µs) | 소유자의 코어 몫 | 상주 / 상태 (배수) |') +print('|---|---:|---:|---:|---:|') +for round in range(1, 7): # 1-3 before first, 4-6 after first + before = row(f'전, {round}', f'before-{round}') + after = row(f'후, {round}', f'after-{round}') + print(f'| ({(after / before - 1) * 100:+.1f}%) | | | | |') +print() +spec = importlib.util.spec_from_file_location('tables', HERE.parent / '022-tpcc-grouped' / 'tables.py') +tables = importlib.util.module_from_spec(spec) +spec.loader.exec_module(tables) +tables.OUT = HERE +for name in ('ram-w1', 'ram-w4'): + tables.table(name, False) |
