diff options
| author | eunhhu <[email protected]> | 2026-10-04 16:10:43 +0100 |
|---|---|---|
| committer | eunhhu <[email protected]> | 2026-10-04 16:10:43 +0100 |
| commit | 50d8263492d69ca8521d3b1d51f60b551c76c443 (patch) | |
| tree | 722340f9d167ea985f9682442c9a923e6c3dbb28 | |
| parent | 90c8edece941f26cd280686c7063a7c9149a2212 (diff) | |
| download | pure-50d8263492d69ca8521d3b1d51f60b551c76c443.tar.gz pure-50d8263492d69ca8521d3b1d51f60b551c76c443.zip | |
Say what pushing views costs and how fresh it keeps them (019, research 120)
54 files changed, 458 insertions, 1 deletions
diff --git a/CHECKPOINT.md b/CHECKPOINT.md index 71d98ab..4b79218 100644 --- a/CHECKPOINT.md +++ b/CHECKPOINT.md @@ -14,6 +14,7 @@ |---|---|---| | 앱 표현 | 상태(수, `bool`, 정확한 64비트 정수 `i64`, text와 커서·마크, 레코드의 리스트와 키 있는 리스트, 인덱스), 입력(키, text, 필드 있는 메시지, 타이머), 원자적 이벤트 핸들러, 순수 보기(조건식, 표와 걸러진 표, count·sum·max·min·first), `fn`·`use` 라이브러리, text 비교, 시간(`now`, `every`, `floor`, `%`). 로드할 때 타입·스택·합류·커서와 인덱스의 소속을 모두 검증하고 이벤트의 실행량에 상한이 있다 | 008, [016](docs/design/016-bool-and-i64.md), aup `docs/design/002-dsl-direction.md`, aup `make check`(파이썬 참조 모델 대조, 진단 223건) | | 수명과 계산 | 값의 수명은 선언이 말한다: `state`(저장), `transient state`(앱이 도는 동안: 저장소에 쓰지 않고 다시 시작하면 초기값, SD 카드에서 쓰기 17배), `let`(이벤트 동안). 핸들러와 보기가 계산한다: 이벤트의 리스트(`let seen: list of Seen by node`), 세는 `for i in a .. b`, `break`, 8겹까지의 중첩, 그리고 `view 이름(인자) { … return … }`(스냅숏에서 돌아 소유자를 막지 않는다). 모든 반복은 시작할 때 횟수의 상한을 갖고, 모든 이벤트와 보기는 실행량 예산(정하지 않으면 2^28 명령)과 메모리 몫(2^28 바이트) 안에서 끝난다. 노드 10만·간선 100만의 BFS 한 이벤트 396 ms(CPython 441, C 9), TPC-C −1.7%. 같은 계산이 Pi·WSL2·M3 Mac에서 같다. 같은 탐색을 보기로 하면 쓰기는 CPU를 나누는 만큼만 줄고(이벤트로 하면 0.2%로 멈춘다), 보기를 계산하는 연결 스레드가 자기의 다른 연결을 기다리게 하는 꼬리가 남았다 | [018](docs/design/018-lifetimes-and-computation.md), [117](docs/research/117-transient-states.md), [118](docs/research/118-computing-in-one-event.md), [119](docs/research/119-views-that-compute.md) | +| 보기 밀기 | 보기의 `GET`에 `Accept: text/event-stream`이면 그 값을 민다(SSE, 브라우저의 `EventSource` 그대로). 같은 보기와 인자를 보는 구독자는 계산 한 번을 나누고, 값이 바뀔 때만, 많아야 10 ms마다 한 번 보낸다. 느린 구독자에게는 가장 새 값 하나만 기다리게 한다. 모듈이 바뀌어도 같은 이름의 보기로 이어 가고, 토큰을 빼면 끊는다. 구독자 1,000에 쓰기 초당 100번까지 신선도 p99 11 ms 이하, 초당 1번이면 100 ms 폴링의 약 10분의 1 CPU, 구독자 1만에 구독자마다 0.3–0.4 KiB, 1만에게 퍼지기까지 p99 45 ms | [019](docs/design/019-pushing-views.md), [120](docs/research/120-pushing-views.md) | | 영속 | 선언한 상태가 곧 영속 상태다. 목록은 이미지와 로그(저장마다 바뀐 기록 한 항목, 재사용하는 할당 구간, 앞 항목의 다이제스트로 이어짐)에, 나머지는 트리 저장소에(커밋당 한 연속 구간과 동기화 한 번). 이벤트 묶음마다 저장 한 번, 이미지는 소유자 밖에서 체크포인트. `kill -9`와 커밋 각 단계의 죽음 뒤 확인 응답한 이벤트가 모두 남는다. 레코드·키가 바뀐 모듈로 바꿔도 데이터가 이름으로 옮겨진다 | [076](docs/research/076-one-sync-commit.md), [081](docs/research/081-tail-records.md), [092](docs/research/092-tpcc.md), 008 상태 영속·교체 | | 인터페이스 | 한 명세가 TUI(셀 단위 갱신), CLI, 문장(NUI), 웹 GUI가 된다. 저장하면 실행 중인 앱이 상태를 이어 새 모듈로 바뀐다(`aup watch`, `vega … --watch`) | Vega `docs/design/001-first-component.md`, Vega `make check`(가상 터미널, 헤드리스 Chromium 포함) | | HTTP 서비스 | 보기가 GET, 입력이 POST다(JSON 본문과 그 안의 그룹 객체를 제자리에서 읽는다, 라우터 코드 없음). 앱의 `send`는 이벤트와 함께 저장되고, `--send 출력=URL`로 다른 앱이나 HTTP 끝점에 순서대로 묶어 전해지며, 받는 앱은 번호마다 한 번 적용한다(서버를 죽였다 살려도 정확히 한 번). `--seal 출력=공개키`면 받는 노드의 키로 봉인해(HPKE, 전달선마다 세션 하나, 보내는 노드의 키로 인증) 사이의 중계가 읽지도 바꾸지도 못하고, 받는 노드는 `--sender 이름=키`로 보낸 이를 가린다(봉인 전달 462k/s, 봉인 없이 468k/s). 답은 저장 뒤에 나간다. 상태를 가진 앱은 소유자 스레드 하나가 입력을 적용하고, 연결 스레드는 마지막 저장의 스냅숏으로 보기를 답한다(쓰기가 동기화하는 동안에도 읽기는 기다리지 않는다) | 008 HTTP 호스트, [077](docs/research/077-apps-as-http-services.md), [080](docs/research/080-snapshot-reads.md), [082](docs/research/082-json-in-place.md), [083](docs/research/083-delivery-over-http.md), [086](docs/research/086-sealed-sends.md), [088](docs/research/088-sealed-sessions.md) | diff --git a/docs/design/019-pushing-views.md b/docs/design/019-pushing-views.md index e94fd61..bf409ad 100644 --- a/docs/design/019-pushing-views.md +++ b/docs/design/019-pushing-views.md @@ -1,6 +1,6 @@ # 019 — 보기를 미는 길 (SSE) -작성일: 2026-10-04. 상태: 설계와 착수 기준. 1단계 구현, 검사로 고정했다. +작성일: 2026-10-04. 상태: 설계와 착수 기준. 1단계 구현, 검사로 고정하고 쟀다([120](../research/120-pushing-views.md)). [외부 리뷰 005](../outside-feedback/feedback-005.md)의 4장 (2) "실시간 클라이언트 푸시(SSE / WebSocket) 미지원"에서 시작한다. ## 문제 diff --git a/docs/research/120-pushing-views.md b/docs/research/120-pushing-views.md new file mode 100644 index 0000000..53339b8 --- /dev/null +++ b/docs/research/120-pushing-views.md @@ -0,0 +1,91 @@ +# 120 — 보기를 미는 길 + +2026-10-04. [019](../design/019-pushing-views.md)를 잰다. 기준은 구현하기 전에 적어 커밋했고(`f2675f0`), 구현하기 전에 한 번 +고쳐 커밋했다(`84483fe`: 신선도가 같은 둘을 견주게, 미는 간격을 두게, 모듈이 바뀌어도 스트림을 잇게). + +## 쉬운 말로 + +- `pure-serve`가 보기의 값을 민다. 브라우저의 `EventSource('/views/total')`가 그대로 붙는다(`Accept: text/event-stream`). 처음에 지금 + 값이 오고, 값이 바뀔 때마다 새 값이 온다. 같은 보기와 인자를 보는 구독자들은 계산 한 번을 나눈다. +- **신선도.** 구독자 1,000이 보는 카운터를 쓰는 쪽이 초당 1번, 10번, 100번 바꿀 때, 쓰기의 답을 받은 때부터 구독자가 그 값을 보기까지 + p99가 4–11 ms다(기준 20 ms). 절반 넘게는 쓰는 쪽이 답을 받기 전에 구독자가 먼저 본다. +- **값.** 대시보드처럼 초당 1번 바뀌는 값을 1,000이 볼 때, 미는 호스트의 CPU는 100 ms마다 폴링할 때의 약 10분의 1이다(두 묶음에서 + 0.093과 0.104, 여섯 라운드 모두 0.101). 기준(10분의 1 이하)의 경계에 있다. 같은 신선도라면(20 ms 폴링) 초당 10번 바뀔 때 13%, 초당 + 100번 바뀔 때 같은 CPU로 더 신선하다(p99 11 ms 대 20 ms). +- **크기.** 구독자 1만에서 호스트의 메모리는 구독자마다 0.29–0.42 KiB 늘었다(기준 1 KiB). 값 하나가 1만에게 다 닿기까지 p99는 + 37–45 ms다(기준 100 ms). +- **지금의 길.** TPC-C는 +0.3%, 은행 질의의 명령 수는 같다(±0.1%). 질의 하나(in_credit, 0.84 ms)의 시간이 처음 여섯 쌍에서 +4.2%, + 다음 여섯 쌍에서 +2.4%, 열두 쌍 모두 +3.0%였다. 명령 수가 같으니 일이 아니라 자리의 차이다. + +## 결과 + +원시 결과와 명령은 [035-push](../../research/experiments/035-push/)에 있다(`runs.sh`, `push.py`, `counter.aup`, 도구는 +`benchmarks/http/watch.c`). + +**방법.** `pure-serve`(연결 스레드 셋, 저장소는 RAM, 코어 0–2)에 카운터 앱(`counter.aup`). `watch`(코어 3)가 클라이언트 N을 붙이고 +(미는 쪽은 붙자마자 구독), 쓰는 연결 하나가 초당 RATE번 카운터를 올린다(10초). 쓰기 k의 답이 온 때부터, 각 클라이언트가 k 이상의 값을 +본 때까지가 그 쓰기의 신선도다(먼저 봤으면 0). 폴링은 클라이언트마다 MS마다 `GET`(keep-alive, 첫 요청은 첫 MS 안의 아무 때). 호스트의 +CPU는 `/proc/PID/stat`의 utime+stime, 메모리는 클라이언트를 붙이기 전과 후의 VmRSS 차이다. 여섯 가지를 번갈아 세 라운드(`fresh`). + +| 구독자 1,000, 중앙값 | 신선도 p99 | 최대 | 호스트 CPU(10초) | +|---|---:|---:|---:| +| 초당 1번, 밀기 | 4.4 ms | 6.4 ms | 0.13 s | +| 초당 1번, 100 ms 폴링 | 98 ms | 98 ms | 1.40 s | +| 초당 10번, 밀기 | 6.1 ms | 7.7 ms | 0.66 s | +| 초당 10번, 20 ms 폴링 | 19 ms | 29 ms | 5.17 s | +| 초당 100번, 밀기 | 10.8 ms | 19 ms | 5.38 s | +| 초당 100번, 20 ms 폴링 | 19 ms | 41 ms | 5.36 s | + +- 처음 세 라운드(`fresh-first-*`)는 구독자의 소켓을 쓸 수 있을 때마다 알리게 둔 채였다. 그러면 보낸 이벤트마다 상대의 확인이 스레드를 + 깨운다. 보낼 것을 쥔 동안만 알리게 고친 뒤(`watch_subscriber`) 다시 쟀다(위). 초당 1번 밀기의 CPU는 둘 다 0.13 s로 같았다. 비율이 + 0.104에서 0.093으로 바뀐 것은 폴링 쪽이 1.25 s에서 1.40 s로 흔들린 것이다. 그래서 이 기준은 경계에 있다고 적는다. +- 미는 값의 대부분은 커널이다: 초당 1번 바뀔 때 쓰기 하나에 약 13 ms, 구독자 하나에 약 13 µs. 루프백의 TCP 보내기 한 번(받는 쪽의 + 처리까지 보내는 스레드가 한다)의 값이다. +- 초당 100번이면 밀기는 초당 10만 번 보내고, 20 ms 폴링은 초당 5만 번 요청을 받아 답한다. CPU는 같고 밀기가 더 신선하다. 미는 + 간격(`--push-every-ms` 10 ms)이 쓰기의 빠르기와 상관없이 미는 일을 묶는다(초당 100번 쓰기에 미는 일 약 300번, 스레드 셋). + +**구독자 1만**(`many`, 세 라운드, 초당 1번). + +| 라운드 | 신선도 p50 | p99 | 최대 | 호스트 CPU(10초) | 구독자마다 메모리 | +|---|---:|---:|---:|---:|---:| +| 1 | 0 | 37 ms | 45 ms | 1.03 s | 0.42 KiB | +| 2 | 0 | 41 ms | 45 ms | 1.10 s | 0.33 KiB | +| 3 | 20 ms | 45 ms | 51 ms | 1.13 s | 0.29 KiB | + +구독자는 요청을 읽던 연결 기록(6,192 바이트)을 돌려주고 소켓과 구독만 남는다. 구독자 1,000의 런에서 구독자마다 1.5–4 KiB로 보인 것은 +붙는 동안 한꺼번에 생긴 연결 기록이 남은 것이다(돌려준 기록은 다음 연결이 다시 쓴다). 1만이면 그 몫이 묽어진다. + +**지금의 길**(전 `84483fe`, 후 `a7de450` 다음의 `watch_subscriber`를 넣은 빌드). TPC-C(RAM, W=1, 64터미널)와 은행 질의를 번갈아 +여섯 쌍, 은행 질의만 여섯 쌍 더. + +| | 전 | 후 | 바뀜 | +|---|---:|---:|---:| +| TPC-C 중앙값 | 42,542 | 42,679 | +0.3% | +| per_branch(12쌍) | | | +0.3% | +| in_credit(6쌍, 12쌍) | 0.835 ms, 0.84 ms | 0.87 ms, 0.865 ms | +4.2%, +3.0% | +| large(12쌍) | | | +1.3% | +| ranks(12쌍) | | | +1.2% | +| 은행 질의의 명령 수 | | | −0.09–+0.02% | + +in_credit의 후는 열두 쌍 가운데 세 번 1.01–1.14 ms였고 나머지는 전과 같은 0.82–0.87 ms다. + +**검사**(`tests/check.py`의 `server_push`). 구독하면 지금 값이 먼저 온다. 50 ms마다 쓰면 값이 하나도 빠짐없이 차례로 오고, 값을 바꾸지 +않는 쓰기에는 오지 않는다. 몰아 쓴 200번에는 이벤트 6개가 왔고 마지막 값이 왔으며 `id`는 늘기만 했다. 인자를 받는 보기, 표의 쪽(100행), +두 줄의 text(`data:` 두 줄), 계산하는 보기(018). 받기 버퍼를 작게 하고 읽지 않는 구독자에게 큰 표를 600번 바꿨을 때 호스트의 RSS는 +0 MiB 늘었고 기다리던 이벤트가 169–176번 바뀌었으며, 다시 읽자 마지막 값이 왔다. 여러 앱의 호스트에서: 토큰 없이 401, `read`로 +구독. 모듈을 바꾸면 같은 스트림에 새 모듈의 값(×10)이 왔다. 토큰 파일에서 그 토큰을 빼면 스트림이 끝났다. 보기가 없는 모듈로 바꾸면 +끝났다. 구독자가 모두 떠나면 호스트의 구독자 수가 0이다. + +## 판정 + +- **019는 들어간다.** 신선도(20 ms), 크기(1 KiB), 퍼뜨리기(100 ms), 지금의 길(TPC-C, 명령 수)은 기준 안이다. +- 대시보드의 값(폴링의 10분의 1 이하)은 경계에 있다(0.093–0.104). 미는 쪽의 값은 대부분 루프백의 TCP 보내기다. 그래도 폴링보다 + 열 배 가깝게 적은 CPU로 스무 배 신선하다. +- in_credit의 시간은 열두 쌍에서 기준의 경계(+3.0%)다. 명령 수가 같아 일의 차이는 아니다. + +## 다음 + +- 하트비트(프록시가 쉬는 스트림을 끊지 않게), 보기가 읽는 상태가 바뀌지 않은 스냅숏은 계산하지 않기, 긴 보기를 연결 스레드 밖에서 + 계산하기(119). 측정이 요구할 때. +- 구독자가 아주 많으면 보내기의 값(구독자마다 약 13 µs)이 남는다. 같은 이벤트를 여러 소켓에 한 번에 보내는 길은 리눅스에 없다 + (`io_uring`으로 묶는 것을 잴 수 있다). diff --git a/research/experiments/035-push/2026-10-04-counts.jsonl b/research/experiments/035-push/2026-10-04-counts.jsonl new file mode 100644 index 0000000..791304f --- /dev/null +++ b/research/experiments/035-push/2026-10-04-counts.jsonl @@ -0,0 +1,4 @@ +{"serve": "pure-serve", "per_branch": {"instructions": 27053801, "cycles": 11959324}, "in_credit": {"instructions": 3488159, "cycles": 1614350}, "large": {"instructions": 1458065, "cycles": 666917}, "ranks": {"instructions": 24211894, "cycles": 14848857}, "none": {"instructions": 606183, "cycles": 195200}, "field": {"instructions": 8927376, "cycles": 3255446}, "found": {"instructions": 15871922, "cycles": 12581393}} +{"serve": "pure-serve", "per_branch": {"instructions": 27084653, "cycles": 12093923}, "in_credit": {"instructions": 3495057, "cycles": 1682436}, "large": {"instructions": 1458197, "cycles": 663365}, "ranks": {"instructions": 24228166, "cycles": 15833361}, "none": {"instructions": 606307, "cycles": 193316}, "field": {"instructions": 8927500, "cycles": 3260106}, "found": {"instructions": 15872910, "cycles": 12607746}} +{"serve": "pure-serve", "per_branch": {"instructions": 27084257, "cycles": 12251672}, "in_credit": {"instructions": 3494865, "cycles": 1652308}, "large": {"instructions": 1458113, "cycles": 662572}, "ranks": {"instructions": 24227286, "cycles": 14894044}, "none": {"instructions": 606183, "cycles": 194389}, "field": {"instructions": 8927376, "cycles": 3261248}, "found": {"instructions": 15872386, "cycles": 12991628}} +{"serve": "pure-serve", "per_branch": {"instructions": 27023643, "cycles": 12061297}, "in_credit": {"instructions": 3481476, "cycles": 1717635}, "large": {"instructions": 1458205, "cycles": 668926}, "ranks": {"instructions": 24194608, "cycles": 15196260}, "none": {"instructions": 606307, "cycles": 194889}, "field": {"instructions": 8927602, "cycles": 3256729}, "found": {"instructions": 15871791, "cycles": 12573182}} diff --git a/research/experiments/035-push/2026-10-04-fresh-1.jsonl b/research/experiments/035-push/2026-10-04-fresh-1.jsonl new file mode 100644 index 0000000..a4b2437 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-fresh-1.jsonl @@ -0,0 +1,6 @@ +{"clients": 1000, "way": "push", "rate": 1, "writes": 11, "followed_to_end": 1000, "taken": 12000, "measured": 11000, "p50_us": 864, "p90_us": 3200, "p99_us": 4096, "max_us": 4608, "host_cpu_seconds": 0.13, "host_cpu_share": 0.015, "rss_kib_per_client": 2.88, "pushes": 54, "pushed": 12000, "replaced": 0, "requests": 1012} +{"clients": 1000, "way": "poll:100", "rate": 1, "writes": 11, "followed_to_end": 1000, "taken": 108400, "measured": 11000, "p50_us": 47104, "p90_us": 86016, "p99_us": 98304, "max_us": 98304, "host_cpu_seconds": 1.4, "host_cpu_share": 0.162, "rss_kib_per_client": 6.43, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 108412} +{"clients": 1000, "way": "push", "rate": 10, "writes": 101, "followed_to_end": 1000, "taken": 102000, "measured": 101000, "p50_us": 960, "p90_us": 3584, "p99_us": 4864, "max_us": 5632, "host_cpu_seconds": 0.62, "host_cpu_share": 0.072, "rss_kib_per_client": 1.47, "pushes": 327, "pushed": 102000, "replaced": 0, "requests": 1102} +{"clients": 1000, "way": "poll:20", "rate": 10, "writes": 101, "followed_to_end": 1000, "taken": 542739, "measured": 101000, "p50_us": 9728, "p90_us": 17408, "p99_us": 19456, "max_us": 20480, "host_cpu_seconds": 5.19, "host_cpu_share": 0.599, "rss_kib_per_client": 6.56, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 542843} +{"clients": 1000, "way": "push", "rate": 100, "writes": 1001, "followed_to_end": 1000, "taken": 995676, "measured": 1001000, "p50_us": 5120, "p90_us": 8704, "p99_us": 10752, "max_us": 13312, "host_cpu_seconds": 5.34, "host_cpu_share": 0.618, "rss_kib_per_client": 4.22, "pushes": 3002, "pushed": 995676, "replaced": 0, "requests": 2002} +{"clients": 1000, "way": "poll:20", "rate": 100, "writes": 1001, "followed_to_end": 1000, "taken": 541902, "measured": 1001000, "p50_us": 9728, "p90_us": 17408, "p99_us": 19456, "max_us": 25600, "host_cpu_seconds": 5.35, "host_cpu_share": 0.619, "rss_kib_per_client": 6.54, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 542906} diff --git a/research/experiments/035-push/2026-10-04-fresh-2.jsonl b/research/experiments/035-push/2026-10-04-fresh-2.jsonl new file mode 100644 index 0000000..8764e85 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-fresh-2.jsonl @@ -0,0 +1,6 @@ +{"clients": 1000, "way": "poll:20", "rate": 100, "writes": 1001, "followed_to_end": 1000, "taken": 542633, "measured": 1001000, "p50_us": 9728, "p90_us": 17408, "p99_us": 19456, "max_us": 22528, "host_cpu_seconds": 5.36, "host_cpu_share": 0.619, "rss_kib_per_client": 6.66, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 543635} +{"clients": 1000, "way": "push", "rate": 100, "writes": 1001, "followed_to_end": 1000, "taken": 987352, "measured": 1001000, "p50_us": 6656, "p90_us": 9728, "p99_us": 11264, "max_us": 19456, "host_cpu_seconds": 5.44, "host_cpu_share": 0.63, "rss_kib_per_client": 2.62, "pushes": 2978, "pushed": 987352, "replaced": 0, "requests": 2002} +{"clients": 1000, "way": "poll:20", "rate": 10, "writes": 101, "followed_to_end": 1000, "taken": 542060, "measured": 101000, "p50_us": 9728, "p90_us": 17408, "p99_us": 19456, "max_us": 28672, "host_cpu_seconds": 5.17, "host_cpu_share": 0.598, "rss_kib_per_client": 6.54, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 542164} +{"clients": 1000, "way": "push", "rate": 10, "writes": 101, "followed_to_end": 1000, "taken": 102000, "measured": 101000, "p50_us": 1216, "p90_us": 4864, "p99_us": 6656, "max_us": 7424, "host_cpu_seconds": 0.75, "host_cpu_share": 0.087, "rss_kib_per_client": 2.59, "pushes": 322, "pushed": 102000, "replaced": 0, "requests": 1102} +{"clients": 1000, "way": "poll:100", "rate": 1, "writes": 11, "followed_to_end": 1000, "taken": 108395, "measured": 11000, "p50_us": 47104, "p90_us": 86016, "p99_us": 98304, "max_us": 98304, "host_cpu_seconds": 1.31, "host_cpu_share": 0.151, "rss_kib_per_client": 6.42, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 108407} +{"clients": 1000, "way": "push", "rate": 1, "writes": 11, "followed_to_end": 1000, "taken": 12000, "measured": 11000, "p50_us": 0, "p90_us": 1984, "p99_us": 4352, "max_us": 4864, "host_cpu_seconds": 0.13, "host_cpu_share": 0.015, "rss_kib_per_client": 2.59, "pushes": 54, "pushed": 12000, "replaced": 0, "requests": 1012} diff --git a/research/experiments/035-push/2026-10-04-fresh-3.jsonl b/research/experiments/035-push/2026-10-04-fresh-3.jsonl new file mode 100644 index 0000000..6db2a8a --- /dev/null +++ b/research/experiments/035-push/2026-10-04-fresh-3.jsonl @@ -0,0 +1,6 @@ +{"clients": 1000, "way": "push", "rate": 1, "writes": 11, "followed_to_end": 1000, "taken": 12000, "measured": 11000, "p50_us": 1600, "p90_us": 3200, "p99_us": 5376, "max_us": 6400, "host_cpu_seconds": 0.12, "host_cpu_share": 0.014, "rss_kib_per_client": 2.59, "pushes": 51, "pushed": 12000, "replaced": 0, "requests": 1012} +{"clients": 1000, "way": "poll:100", "rate": 1, "writes": 11, "followed_to_end": 1000, "taken": 108508, "measured": 11000, "p50_us": 45056, "p90_us": 86016, "p99_us": 98304, "max_us": 98304, "host_cpu_seconds": 1.41, "host_cpu_share": 0.163, "rss_kib_per_client": 6.43, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 108520} +{"clients": 1000, "way": "push", "rate": 10, "writes": 101, "followed_to_end": 1000, "taken": 102000, "measured": 101000, "p50_us": 1408, "p90_us": 4352, "p99_us": 6144, "max_us": 7680, "host_cpu_seconds": 0.66, "host_cpu_share": 0.076, "rss_kib_per_client": 2.59, "pushes": 324, "pushed": 102000, "replaced": 0, "requests": 1102} +{"clients": 1000, "way": "poll:20", "rate": 10, "writes": 101, "followed_to_end": 1000, "taken": 542713, "measured": 101000, "p50_us": 9728, "p90_us": 17408, "p99_us": 20480, "max_us": 21504, "host_cpu_seconds": 5.15, "host_cpu_share": 0.594, "rss_kib_per_client": 6.43, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 542817} +{"clients": 1000, "way": "push", "rate": 100, "writes": 1001, "followed_to_end": 1000, "taken": 998340, "measured": 1001000, "p50_us": 6144, "p90_us": 8704, "p99_us": 10752, "max_us": 13312, "host_cpu_seconds": 5.38, "host_cpu_share": 0.623, "rss_kib_per_client": 3.82, "pushes": 3007, "pushed": 998340, "replaced": 0, "requests": 2002} +{"clients": 1000, "way": "poll:20", "rate": 100, "writes": 1001, "followed_to_end": 1000, "taken": 541936, "measured": 1001000, "p50_us": 9728, "p90_us": 17408, "p99_us": 19456, "max_us": 40960, "host_cpu_seconds": 5.43, "host_cpu_share": 0.628, "rss_kib_per_client": 6.48, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 542954} diff --git a/research/experiments/035-push/2026-10-04-fresh-first-1.jsonl b/research/experiments/035-push/2026-10-04-fresh-first-1.jsonl new file mode 100644 index 0000000..c7276a7 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-fresh-first-1.jsonl @@ -0,0 +1,6 @@ +{"clients": 1000, "way": "push", "rate": 1, "writes": 11, "followed_to_end": 1000, "taken": 12000, "measured": 11000, "p50_us": 960, "p90_us": 3712, "p99_us": 7424, "max_us": 8704, "host_cpu_seconds": 0.13, "host_cpu_share": 0.015, "rss_kib_per_client": 3.81, "pushes": 51, "pushed": 12000, "replaced": 0, "requests": 1012} +{"clients": 1000, "way": "poll:100", "rate": 1, "writes": 11, "followed_to_end": 1000, "taken": 108398, "measured": 11000, "p50_us": 47104, "p90_us": 86016, "p99_us": 98304, "max_us": 98304, "host_cpu_seconds": 1.23, "host_cpu_share": 0.142, "rss_kib_per_client": 6.42, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 108410} +{"clients": 1000, "way": "push", "rate": 10, "writes": 101, "followed_to_end": 1000, "taken": 102000, "measured": 101000, "p50_us": 2048, "p90_us": 5376, "p99_us": 7168, "max_us": 10752, "host_cpu_seconds": 0.68, "host_cpu_share": 0.079, "rss_kib_per_client": 2.59, "pushes": 327, "pushed": 102000, "replaced": 0, "requests": 1102} +{"clients": 1000, "way": "poll:20", "rate": 10, "writes": 101, "followed_to_end": 1000, "taken": 542653, "measured": 101000, "p50_us": 9728, "p90_us": 17408, "p99_us": 19456, "max_us": 20480, "host_cpu_seconds": 5.18, "host_cpu_share": 0.598, "rss_kib_per_client": 6.48, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 542757} +{"clients": 1000, "way": "push", "rate": 100, "writes": 1001, "followed_to_end": 1000, "taken": 988372, "measured": 1001000, "p50_us": 6656, "p90_us": 9728, "p99_us": 11264, "max_us": 14848, "host_cpu_seconds": 5.35, "host_cpu_share": 0.619, "rss_kib_per_client": 3.95, "pushes": 2978, "pushed": 988372, "replaced": 0, "requests": 2002} +{"clients": 1000, "way": "poll:20", "rate": 100, "writes": 1001, "followed_to_end": 1000, "taken": 541932, "measured": 1001000, "p50_us": 9728, "p90_us": 17408, "p99_us": 20480, "max_us": 22528, "host_cpu_seconds": 5.45, "host_cpu_share": 0.63, "rss_kib_per_client": 6.51, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 542935} diff --git a/research/experiments/035-push/2026-10-04-fresh-first-2.jsonl b/research/experiments/035-push/2026-10-04-fresh-first-2.jsonl new file mode 100644 index 0000000..ab5f187 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-fresh-first-2.jsonl @@ -0,0 +1,6 @@ +{"clients": 1000, "way": "poll:20", "rate": 100, "writes": 1001, "followed_to_end": 1000, "taken": 542345, "measured": 1001000, "p50_us": 9728, "p90_us": 17408, "p99_us": 19456, "max_us": 23552, "host_cpu_seconds": 5.36, "host_cpu_share": 0.619, "rss_kib_per_client": 6.59, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 543349} +{"clients": 1000, "way": "push", "rate": 100, "writes": 1001, "followed_to_end": 1000, "taken": 998653, "measured": 1001000, "p50_us": 5888, "p90_us": 8704, "p99_us": 10752, "max_us": 12800, "host_cpu_seconds": 5.36, "host_cpu_share": 0.62, "rss_kib_per_client": 5.06, "pushes": 3005, "pushed": 998653, "replaced": 0, "requests": 2002} +{"clients": 1000, "way": "poll:20", "rate": 10, "writes": 101, "followed_to_end": 1000, "taken": 541963, "measured": 101000, "p50_us": 9728, "p90_us": 17408, "p99_us": 19456, "max_us": 20480, "host_cpu_seconds": 5.26, "host_cpu_share": 0.608, "rss_kib_per_client": 6.46, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 542067} +{"clients": 1000, "way": "push", "rate": 10, "writes": 101, "followed_to_end": 1000, "taken": 102000, "measured": 101000, "p50_us": 2048, "p90_us": 5120, "p99_us": 6912, "max_us": 8192, "host_cpu_seconds": 0.67, "host_cpu_share": 0.078, "rss_kib_per_client": 1.55, "pushes": 330, "pushed": 102000, "replaced": 0, "requests": 1102} +{"clients": 1000, "way": "poll:100", "rate": 1, "writes": 11, "followed_to_end": 1000, "taken": 108395, "measured": 11000, "p50_us": 47104, "p90_us": 86016, "p99_us": 98304, "max_us": 98304, "host_cpu_seconds": 1.26, "host_cpu_share": 0.146, "rss_kib_per_client": 6.43, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 108407} +{"clients": 1000, "way": "push", "rate": 1, "writes": 11, "followed_to_end": 1000, "taken": 12000, "measured": 11000, "p50_us": 1344, "p90_us": 3840, "p99_us": 4608, "max_us": 5120, "host_cpu_seconds": 0.15, "host_cpu_share": 0.017, "rss_kib_per_client": 3.81, "pushes": 48, "pushed": 12000, "replaced": 0, "requests": 1012} diff --git a/research/experiments/035-push/2026-10-04-fresh-first-3.jsonl b/research/experiments/035-push/2026-10-04-fresh-first-3.jsonl new file mode 100644 index 0000000..9b8a1d9 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-fresh-first-3.jsonl @@ -0,0 +1,6 @@ +{"clients": 1000, "way": "push", "rate": 1, "writes": 11, "followed_to_end": 1000, "taken": 12000, "measured": 11000, "p50_us": 0, "p90_us": 272, "p99_us": 3968, "max_us": 4864, "host_cpu_seconds": 0.13, "host_cpu_share": 0.015, "rss_kib_per_client": 3.34, "pushes": 53, "pushed": 12000, "replaced": 0, "requests": 1012} +{"clients": 1000, "way": "poll:100", "rate": 1, "writes": 11, "followed_to_end": 1000, "taken": 108556, "measured": 11000, "p50_us": 45056, "p90_us": 86016, "p99_us": 98304, "max_us": 98304, "host_cpu_seconds": 1.25, "host_cpu_share": 0.144, "rss_kib_per_client": 6.43, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 108570} +{"clients": 1000, "way": "push", "rate": 10, "writes": 101, "followed_to_end": 1000, "taken": 102000, "measured": 101000, "p50_us": 0, "p90_us": 0, "p99_us": 5120, "max_us": 7680, "host_cpu_seconds": 0.71, "host_cpu_share": 0.082, "rss_kib_per_client": 5.01, "pushes": 315, "pushed": 102000, "replaced": 0, "requests": 1102} +{"clients": 1000, "way": "poll:20", "rate": 10, "writes": 101, "followed_to_end": 1000, "taken": 542711, "measured": 101000, "p50_us": 9728, "p90_us": 17408, "p99_us": 19456, "max_us": 20480, "host_cpu_seconds": 5.22, "host_cpu_share": 0.602, "rss_kib_per_client": 6.5, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 542813} +{"clients": 1000, "way": "push", "rate": 100, "writes": 1001, "followed_to_end": 1000, "taken": 997331, "measured": 1001000, "p50_us": 5376, "p90_us": 9216, "p99_us": 10752, "max_us": 15872, "host_cpu_seconds": 5.26, "host_cpu_share": 0.609, "rss_kib_per_client": 2.64, "pushes": 3007, "pushed": 997331, "replaced": 0, "requests": 2002} +{"clients": 1000, "way": "poll:20", "rate": 100, "writes": 1001, "followed_to_end": 1000, "taken": 541882, "measured": 1001000, "p50_us": 9728, "p90_us": 17408, "p99_us": 19456, "max_us": 25600, "host_cpu_seconds": 5.37, "host_cpu_share": 0.621, "rss_kib_per_client": 6.5, "pushes": 0, "pushed": 0, "replaced": 0, "requests": 542884} diff --git a/research/experiments/035-push/2026-10-04-load.txt b/research/experiments/035-push/2026-10-04-load.txt new file mode 100644 index 0000000..ef23532 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-load.txt @@ -0,0 +1,12 @@ +1 before 15:37:30 up 11:58, 1 user, load average: 0.63, 0.62, 0.72 +1 after 15:38:57 up 12:00, 1 user, load average: 1.42, 0.93, 0.83 +2 after 15:40:23 up 12:01, 1 user, load average: 1.45, 1.12, 0.91 +2 before 15:41:49 up 12:03, 1 user, load average: 1.24, 1.14, 0.94 +3 before 15:43:16 up 12:04, 1 user, load average: 1.40, 1.27, 1.01 +3 after 15:44:53 up 12:06, 1 user, load average: 1.33, 1.38, 1.08 +4 after 15:46:24 up 12:07, 1 user, load average: 1.43, 1.47, 1.15 +4 before 15:47:50 up 12:09, 1 user, load average: 1.33, 1.46, 1.17 +5 before 15:49:27 up 12:10, 1 user, load average: 1.45, 1.57, 1.25 +5 after 15:51:03 up 12:12, 1 user, load average: 1.43, 1.64, 1.31 +6 after 15:52:40 up 12:14, 1 user, load average: 1.33, 1.63, 1.35 +6 before 15:54:07 up 12:15, 1 user, load average: 1.45, 1.64, 1.38 diff --git a/research/experiments/035-push/2026-10-04-many-1.jsonl b/research/experiments/035-push/2026-10-04-many-1.jsonl new file mode 100644 index 0000000..0055029 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-many-1.jsonl @@ -0,0 +1 @@ +{"clients": 10000, "way": "push", "rate": 1, "writes": 11, "followed_to_end": 10000, "taken": 120000, "measured": 110000, "p50_us": 0, "p90_us": 16384, "p99_us": 36864, "max_us": 45056, "host_cpu_seconds": 1.03, "host_cpu_share": 0.127, "rss_kib_per_client": 0.42, "pushes": 317, "pushed": 120000, "replaced": 0, "requests": 10012} diff --git a/research/experiments/035-push/2026-10-04-many-2.jsonl b/research/experiments/035-push/2026-10-04-many-2.jsonl new file mode 100644 index 0000000..8292a8f --- /dev/null +++ b/research/experiments/035-push/2026-10-04-many-2.jsonl @@ -0,0 +1 @@ +{"clients": 10000, "way": "push", "rate": 1, "writes": 11, "followed_to_end": 10000, "taken": 120000, "measured": 110000, "p50_us": 0, "p90_us": 27648, "p99_us": 40960, "max_us": 45056, "host_cpu_seconds": 1.1, "host_cpu_share": 0.135, "rss_kib_per_client": 0.33, "pushes": 314, "pushed": 120000, "replaced": 0, "requests": 10012} diff --git a/research/experiments/035-push/2026-10-04-many-3.jsonl b/research/experiments/035-push/2026-10-04-many-3.jsonl new file mode 100644 index 0000000..1f376da --- /dev/null +++ b/research/experiments/035-push/2026-10-04-many-3.jsonl @@ -0,0 +1 @@ +{"clients": 10000, "way": "push", "rate": 1, "writes": 11, "followed_to_end": 10000, "taken": 120000, "measured": 110000, "p50_us": 20480, "p90_us": 38912, "p99_us": 45056, "max_us": 51200, "host_cpu_seconds": 1.13, "host_cpu_share": 0.138, "rss_kib_per_client": 0.29, "pushes": 318, "pushed": 120000, "replaced": 0, "requests": 10012} diff --git a/research/experiments/035-push/2026-10-04-queries-after-1.jsonl b/research/experiments/035-push/2026-10-04-queries-after-1.jsonl new file mode 100644 index 0000000..3b19ece --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-after-1.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.34, "pure_fastest_ms": 5.21, "sqlite_median_ms": 44.86, "sqlite_fastest_ms": 44.78, "sqlite_indexed_median_ms": 14.1, "sqlite_indexed_fastest_ms": 14.06} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.84, "pure_fastest_ms": 0.82, "sqlite_median_ms": 5.88, "sqlite_fastest_ms": 5.86, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.37, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.48, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.76, "pure_fastest_ms": 6.72, "sqlite_median_ms": 2827.0, "sqlite_fastest_ms": 2825.14, "sqlite_indexed_median_ms": 7.54, "sqlite_indexed_fastest_ms": 7.49} diff --git a/research/experiments/035-push/2026-10-04-queries-after-10.jsonl b/research/experiments/035-push/2026-10-04-queries-after-10.jsonl new file mode 100644 index 0000000..f79b20a --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-after-10.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.25, "pure_fastest_ms": 5.17, "sqlite_median_ms": 45.48, "sqlite_fastest_ms": 45.22, "sqlite_indexed_median_ms": 14.08, "sqlite_indexed_fastest_ms": 14.04} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.84, "pure_fastest_ms": 0.81, "sqlite_median_ms": 5.9, "sqlite_fastest_ms": 5.87, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.37, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.48, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.81, "pure_fastest_ms": 6.71, "sqlite_median_ms": 2835.56, "sqlite_fastest_ms": 2832.87, "sqlite_indexed_median_ms": 7.6, "sqlite_indexed_fastest_ms": 7.53} diff --git a/research/experiments/035-push/2026-10-04-queries-after-11.jsonl b/research/experiments/035-push/2026-10-04-queries-after-11.jsonl new file mode 100644 index 0000000..810a695 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-after-11.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.24, "pure_fastest_ms": 5.11, "sqlite_median_ms": 45.37, "sqlite_fastest_ms": 45.21, "sqlite_indexed_median_ms": 13.97, "sqlite_indexed_fastest_ms": 13.95} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.86, "pure_fastest_ms": 0.83, "sqlite_median_ms": 5.91, "sqlite_fastest_ms": 5.87, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.41, "pure_fastest_ms": 0.38, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.47, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.75, "pure_fastest_ms": 6.7, "sqlite_median_ms": 2840.04, "sqlite_fastest_ms": 2834.32, "sqlite_indexed_median_ms": 7.51, "sqlite_indexed_fastest_ms": 7.49} diff --git a/research/experiments/035-push/2026-10-04-queries-after-12.jsonl b/research/experiments/035-push/2026-10-04-queries-after-12.jsonl new file mode 100644 index 0000000..84f72cc --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-after-12.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.32, "pure_fastest_ms": 5.2, "sqlite_median_ms": 44.96, "sqlite_fastest_ms": 44.79, "sqlite_indexed_median_ms": 14.08, "sqlite_indexed_fastest_ms": 14.01} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.86, "pure_fastest_ms": 0.81, "sqlite_median_ms": 5.9, "sqlite_fastest_ms": 5.88, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.41, "pure_fastest_ms": 0.39, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.48, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.82, "pure_fastest_ms": 6.72, "sqlite_median_ms": 2831.82, "sqlite_fastest_ms": 2827.38, "sqlite_indexed_median_ms": 7.59, "sqlite_indexed_fastest_ms": 7.51} diff --git a/research/experiments/035-push/2026-10-04-queries-after-2.jsonl b/research/experiments/035-push/2026-10-04-queries-after-2.jsonl new file mode 100644 index 0000000..6f50ba0 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-after-2.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.27, "pure_fastest_ms": 5.18, "sqlite_median_ms": 44.92, "sqlite_fastest_ms": 44.77, "sqlite_indexed_median_ms": 14.09, "sqlite_indexed_fastest_ms": 14.03} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 1.03, "pure_fastest_ms": 0.93, "sqlite_median_ms": 5.87, "sqlite_fastest_ms": 5.86, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.42, "pure_fastest_ms": 0.38, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.47, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.77, "pure_fastest_ms": 6.72, "sqlite_median_ms": 2832.35, "sqlite_fastest_ms": 2830.51, "sqlite_indexed_median_ms": 7.55, "sqlite_indexed_fastest_ms": 7.49} diff --git a/research/experiments/035-push/2026-10-04-queries-after-3.jsonl b/research/experiments/035-push/2026-10-04-queries-after-3.jsonl new file mode 100644 index 0000000..763f261 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-after-3.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.27, "pure_fastest_ms": 5.13, "sqlite_median_ms": 44.89, "sqlite_fastest_ms": 44.73, "sqlite_indexed_median_ms": 14.08, "sqlite_indexed_fastest_ms": 14.05} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.87, "pure_fastest_ms": 0.82, "sqlite_median_ms": 5.89, "sqlite_fastest_ms": 5.86, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.4, "pure_fastest_ms": 0.38, "sqlite_median_ms": 1.49, "sqlite_fastest_ms": 1.48, "sqlite_indexed_median_ms": 1.49, "sqlite_indexed_fastest_ms": 1.49} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.72, "pure_fastest_ms": 6.65, "sqlite_median_ms": 2835.52, "sqlite_fastest_ms": 2831.03, "sqlite_indexed_median_ms": 7.55, "sqlite_indexed_fastest_ms": 7.51} diff --git a/research/experiments/035-push/2026-10-04-queries-after-4.jsonl b/research/experiments/035-push/2026-10-04-queries-after-4.jsonl new file mode 100644 index 0000000..a5c57b4 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-after-4.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.33, "pure_fastest_ms": 5.19, "sqlite_median_ms": 45.39, "sqlite_fastest_ms": 45.24, "sqlite_indexed_median_ms": 14.05, "sqlite_indexed_fastest_ms": 14.01} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.87, "pure_fastest_ms": 0.82, "sqlite_median_ms": 5.94, "sqlite_fastest_ms": 5.88, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.41, "pure_fastest_ms": 0.37, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.47, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.81, "pure_fastest_ms": 6.77, "sqlite_median_ms": 2837.06, "sqlite_fastest_ms": 2834.11, "sqlite_indexed_median_ms": 7.61, "sqlite_indexed_fastest_ms": 7.51} diff --git a/research/experiments/035-push/2026-10-04-queries-after-5.jsonl b/research/experiments/035-push/2026-10-04-queries-after-5.jsonl new file mode 100644 index 0000000..0a05a1d --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-after-5.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.26, "pure_fastest_ms": 5.15, "sqlite_median_ms": 45.13, "sqlite_fastest_ms": 44.82, "sqlite_indexed_median_ms": 14.02, "sqlite_indexed_fastest_ms": 13.98} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 1.01, "pure_fastest_ms": 0.96, "sqlite_median_ms": 6.03, "sqlite_fastest_ms": 5.88, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.38, "sqlite_median_ms": 1.47, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.48, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.83, "pure_fastest_ms": 6.64, "sqlite_median_ms": 2837.24, "sqlite_fastest_ms": 2833.39, "sqlite_indexed_median_ms": 7.59, "sqlite_indexed_fastest_ms": 7.52} diff --git a/research/experiments/035-push/2026-10-04-queries-after-6.jsonl b/research/experiments/035-push/2026-10-04-queries-after-6.jsonl new file mode 100644 index 0000000..e4d140f --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-after-6.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.28, "pure_fastest_ms": 5.16, "sqlite_median_ms": 45.27, "sqlite_fastest_ms": 45.15, "sqlite_indexed_median_ms": 14.03, "sqlite_indexed_fastest_ms": 14.0} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.82, "pure_fastest_ms": 0.8, "sqlite_median_ms": 5.88, "sqlite_fastest_ms": 5.86, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.37, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.48, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.76, "pure_fastest_ms": 6.67, "sqlite_median_ms": 2836.34, "sqlite_fastest_ms": 2832.81, "sqlite_indexed_median_ms": 7.55, "sqlite_indexed_fastest_ms": 7.5} diff --git a/research/experiments/035-push/2026-10-04-queries-after-7.jsonl b/research/experiments/035-push/2026-10-04-queries-after-7.jsonl new file mode 100644 index 0000000..bca43f8 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-after-7.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.23, "pure_fastest_ms": 5.17, "sqlite_median_ms": 45.28, "sqlite_fastest_ms": 45.1, "sqlite_indexed_median_ms": 14.21, "sqlite_indexed_fastest_ms": 14.04} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.87, "pure_fastest_ms": 0.81, "sqlite_median_ms": 5.89, "sqlite_fastest_ms": 5.87, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.4, "pure_fastest_ms": 0.38, "sqlite_median_ms": 1.49, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.57, "sqlite_indexed_fastest_ms": 1.5} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.81, "pure_fastest_ms": 6.75, "sqlite_median_ms": 2839.37, "sqlite_fastest_ms": 2835.54, "sqlite_indexed_median_ms": 7.65, "sqlite_indexed_fastest_ms": 7.52} diff --git a/research/experiments/035-push/2026-10-04-queries-after-8.jsonl b/research/experiments/035-push/2026-10-04-queries-after-8.jsonl new file mode 100644 index 0000000..27980e5 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-after-8.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 6.67, "pure_fastest_ms": 5.23, "sqlite_median_ms": 47.86, "sqlite_fastest_ms": 47.57, "sqlite_indexed_median_ms": 14.23, "sqlite_indexed_fastest_ms": 14.18} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.83, "pure_fastest_ms": 0.81, "sqlite_median_ms": 6.83, "sqlite_fastest_ms": 6.74, "sqlite_indexed_median_ms": 0.18, "sqlite_indexed_fastest_ms": 0.18} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.38, "sqlite_median_ms": 1.69, "sqlite_fastest_ms": 1.67, "sqlite_indexed_median_ms": 1.68, "sqlite_indexed_fastest_ms": 1.67} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.83, "pure_fastest_ms": 6.78, "sqlite_median_ms": 3331.22, "sqlite_fastest_ms": 3327.33, "sqlite_indexed_median_ms": 8.63, "sqlite_indexed_fastest_ms": 8.56} diff --git a/research/experiments/035-push/2026-10-04-queries-after-9.jsonl b/research/experiments/035-push/2026-10-04-queries-after-9.jsonl new file mode 100644 index 0000000..ae029a2 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-after-9.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.31, "pure_fastest_ms": 5.19, "sqlite_median_ms": 45.85, "sqlite_fastest_ms": 45.15, "sqlite_indexed_median_ms": 13.95, "sqlite_indexed_fastest_ms": 13.92} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 1.14, "pure_fastest_ms": 0.94, "sqlite_median_ms": 5.93, "sqlite_fastest_ms": 5.86, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.38, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.47, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.84, "pure_fastest_ms": 6.77, "sqlite_median_ms": 2833.07, "sqlite_fastest_ms": 2831.13, "sqlite_indexed_median_ms": 7.54, "sqlite_indexed_fastest_ms": 7.49} diff --git a/research/experiments/035-push/2026-10-04-queries-before-1.jsonl b/research/experiments/035-push/2026-10-04-queries-before-1.jsonl new file mode 100644 index 0000000..7aaf883 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-before-1.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.26, "pure_fastest_ms": 5.18, "sqlite_median_ms": 45.08, "sqlite_fastest_ms": 44.79, "sqlite_indexed_median_ms": 14.05, "sqlite_indexed_fastest_ms": 14.02} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.85, "pure_fastest_ms": 0.82, "sqlite_median_ms": 5.89, "sqlite_fastest_ms": 5.86, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.37, "sqlite_median_ms": 1.5, "sqlite_fastest_ms": 1.49, "sqlite_indexed_median_ms": 1.5, "sqlite_indexed_fastest_ms": 1.49} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.78, "pure_fastest_ms": 6.71, "sqlite_median_ms": 2834.27, "sqlite_fastest_ms": 2829.64, "sqlite_indexed_median_ms": 7.56, "sqlite_indexed_fastest_ms": 7.51} diff --git a/research/experiments/035-push/2026-10-04-queries-before-10.jsonl b/research/experiments/035-push/2026-10-04-queries-before-10.jsonl new file mode 100644 index 0000000..131f8df --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-before-10.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.28, "pure_fastest_ms": 5.18, "sqlite_median_ms": 45.29, "sqlite_fastest_ms": 45.14, "sqlite_indexed_median_ms": 14.08, "sqlite_indexed_fastest_ms": 14.04} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.87, "pure_fastest_ms": 0.81, "sqlite_median_ms": 5.88, "sqlite_fastest_ms": 5.87, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.4, "pure_fastest_ms": 0.38, "sqlite_median_ms": 1.47, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.47, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.74, "pure_fastest_ms": 6.67, "sqlite_median_ms": 2837.48, "sqlite_fastest_ms": 2827.98, "sqlite_indexed_median_ms": 7.57, "sqlite_indexed_fastest_ms": 7.5} diff --git a/research/experiments/035-push/2026-10-04-queries-before-11.jsonl b/research/experiments/035-push/2026-10-04-queries-before-11.jsonl new file mode 100644 index 0000000..4346850 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-before-11.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.27, "pure_fastest_ms": 5.2, "sqlite_median_ms": 45.42, "sqlite_fastest_ms": 45.27, "sqlite_indexed_median_ms": 14.04, "sqlite_indexed_fastest_ms": 14.03} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.84, "pure_fastest_ms": 0.81, "sqlite_median_ms": 5.95, "sqlite_fastest_ms": 5.88, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.38, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.47, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.73, "pure_fastest_ms": 6.65, "sqlite_median_ms": 2836.02, "sqlite_fastest_ms": 2834.36, "sqlite_indexed_median_ms": 7.58, "sqlite_indexed_fastest_ms": 7.53} diff --git a/research/experiments/035-push/2026-10-04-queries-before-12.jsonl b/research/experiments/035-push/2026-10-04-queries-before-12.jsonl new file mode 100644 index 0000000..95e5512 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-before-12.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.33, "pure_fastest_ms": 5.2, "sqlite_median_ms": 45.22, "sqlite_fastest_ms": 45.14, "sqlite_indexed_median_ms": 14.08, "sqlite_indexed_fastest_ms": 14.04} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.83, "pure_fastest_ms": 0.81, "sqlite_median_ms": 5.9, "sqlite_fastest_ms": 5.87, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.38, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.48, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.79, "pure_fastest_ms": 6.74, "sqlite_median_ms": 2834.98, "sqlite_fastest_ms": 2833.31, "sqlite_indexed_median_ms": 7.57, "sqlite_indexed_fastest_ms": 7.51} diff --git a/research/experiments/035-push/2026-10-04-queries-before-2.jsonl b/research/experiments/035-push/2026-10-04-queries-before-2.jsonl new file mode 100644 index 0000000..8594244 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-before-2.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.23, "pure_fastest_ms": 5.18, "sqlite_median_ms": 45.25, "sqlite_fastest_ms": 45.13, "sqlite_indexed_median_ms": 14.04, "sqlite_indexed_fastest_ms": 14.02} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.83, "pure_fastest_ms": 0.8, "sqlite_median_ms": 5.89, "sqlite_fastest_ms": 5.86, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.37, "sqlite_median_ms": 1.51, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.47, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 8.39, "pure_fastest_ms": 7.48, "sqlite_median_ms": 2831.03, "sqlite_fastest_ms": 2824.67, "sqlite_indexed_median_ms": 7.55, "sqlite_indexed_fastest_ms": 7.52} diff --git a/research/experiments/035-push/2026-10-04-queries-before-3.jsonl b/research/experiments/035-push/2026-10-04-queries-before-3.jsonl new file mode 100644 index 0000000..4af6cae --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-before-3.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.2, "pure_fastest_ms": 5.15, "sqlite_median_ms": 45.28, "sqlite_fastest_ms": 45.11, "sqlite_indexed_median_ms": 14.06, "sqlite_indexed_fastest_ms": 14.03} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.85, "pure_fastest_ms": 0.81, "sqlite_median_ms": 5.87, "sqlite_fastest_ms": 5.86, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.37, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.49, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.73, "pure_fastest_ms": 6.68, "sqlite_median_ms": 2837.24, "sqlite_fastest_ms": 2832.97, "sqlite_indexed_median_ms": 7.54, "sqlite_indexed_fastest_ms": 7.51} diff --git a/research/experiments/035-push/2026-10-04-queries-before-4.jsonl b/research/experiments/035-push/2026-10-04-queries-before-4.jsonl new file mode 100644 index 0000000..3aa9c7f --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-before-4.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.26, "pure_fastest_ms": 5.15, "sqlite_median_ms": 45.38, "sqlite_fastest_ms": 45.2, "sqlite_indexed_median_ms": 14.02, "sqlite_indexed_fastest_ms": 13.97} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.82, "pure_fastest_ms": 0.79, "sqlite_median_ms": 5.88, "sqlite_fastest_ms": 5.86, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.37, "pure_fastest_ms": 0.37, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.47, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.74, "pure_fastest_ms": 6.7, "sqlite_median_ms": 2838.94, "sqlite_fastest_ms": 2834.99, "sqlite_indexed_median_ms": 7.55, "sqlite_indexed_fastest_ms": 7.49} diff --git a/research/experiments/035-push/2026-10-04-queries-before-5.jsonl b/research/experiments/035-push/2026-10-04-queries-before-5.jsonl new file mode 100644 index 0000000..b6c6f55 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-before-5.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.27, "pure_fastest_ms": 5.17, "sqlite_median_ms": 44.88, "sqlite_fastest_ms": 44.76, "sqlite_indexed_median_ms": 14.09, "sqlite_indexed_fastest_ms": 14.03} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.8, "pure_fastest_ms": 0.79, "sqlite_median_ms": 5.91, "sqlite_fastest_ms": 5.86, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.38, "sqlite_median_ms": 1.47, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.47, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.73, "pure_fastest_ms": 6.7, "sqlite_median_ms": 2835.89, "sqlite_fastest_ms": 2832.44, "sqlite_indexed_median_ms": 7.56, "sqlite_indexed_fastest_ms": 7.49} diff --git a/research/experiments/035-push/2026-10-04-queries-before-6.jsonl b/research/experiments/035-push/2026-10-04-queries-before-6.jsonl new file mode 100644 index 0000000..84b7260 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-before-6.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.36, "pure_fastest_ms": 5.24, "sqlite_median_ms": 45.22, "sqlite_fastest_ms": 45.06, "sqlite_indexed_median_ms": 13.98, "sqlite_indexed_fastest_ms": 13.92} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.84, "pure_fastest_ms": 0.8, "sqlite_median_ms": 5.93, "sqlite_fastest_ms": 5.86, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.38, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.47, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.72, "pure_fastest_ms": 6.66, "sqlite_median_ms": 2835.45, "sqlite_fastest_ms": 2832.11, "sqlite_indexed_median_ms": 7.54, "sqlite_indexed_fastest_ms": 7.5} diff --git a/research/experiments/035-push/2026-10-04-queries-before-7.jsonl b/research/experiments/035-push/2026-10-04-queries-before-7.jsonl new file mode 100644 index 0000000..2a27a4e --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-before-7.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.24, "pure_fastest_ms": 5.15, "sqlite_median_ms": 44.99, "sqlite_fastest_ms": 44.85, "sqlite_indexed_median_ms": 14.02, "sqlite_indexed_fastest_ms": 14.0} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.87, "pure_fastest_ms": 0.82, "sqlite_median_ms": 5.93, "sqlite_fastest_ms": 5.87, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.38, "pure_fastest_ms": 0.37, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.48, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.67, "pure_fastest_ms": 6.56, "sqlite_median_ms": 2832.83, "sqlite_fastest_ms": 2825.13, "sqlite_indexed_median_ms": 7.61, "sqlite_indexed_fastest_ms": 7.52} diff --git a/research/experiments/035-push/2026-10-04-queries-before-8.jsonl b/research/experiments/035-push/2026-10-04-queries-before-8.jsonl new file mode 100644 index 0000000..15752fe --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-before-8.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.25, "pure_fastest_ms": 5.17, "sqlite_median_ms": 45.03, "sqlite_fastest_ms": 44.86, "sqlite_indexed_median_ms": 14.02, "sqlite_indexed_fastest_ms": 13.97} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.8, "pure_fastest_ms": 0.78, "sqlite_median_ms": 5.87, "sqlite_fastest_ms": 5.86, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.38, "pure_fastest_ms": 0.37, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.47, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.73, "pure_fastest_ms": 6.67, "sqlite_median_ms": 2835.51, "sqlite_fastest_ms": 2831.42, "sqlite_indexed_median_ms": 7.56, "sqlite_indexed_fastest_ms": 7.49} diff --git a/research/experiments/035-push/2026-10-04-queries-before-9.jsonl b/research/experiments/035-push/2026-10-04-queries-before-9.jsonl new file mode 100644 index 0000000..366c7a3 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-queries-before-9.jsonl @@ -0,0 +1,4 @@ +{"query": "per_branch", "rows": 10, "accounts": 100000, "pure_median_ms": 5.27, "pure_fastest_ms": 5.2, "sqlite_median_ms": 45.33, "sqlite_fastest_ms": 45.16, "sqlite_indexed_median_ms": 14.08, "sqlite_indexed_fastest_ms": 14.01} +{"query": "in_credit", "rows": 1, "accounts": 100000, "pure_median_ms": 0.84, "pure_fastest_ms": 0.81, "sqlite_median_ms": 5.9, "sqlite_fastest_ms": 5.87, "sqlite_indexed_median_ms": 0.16, "sqlite_indexed_fastest_ms": 0.16} +{"query": "large", "rows": 212, "accounts": 100000, "pure_median_ms": 0.39, "pure_fastest_ms": 0.37, "sqlite_median_ms": 1.48, "sqlite_fastest_ms": 1.47, "sqlite_indexed_median_ms": 1.48, "sqlite_indexed_fastest_ms": 1.47} +{"query": "ranks", "rows": 404, "accounts": 100000, "pure_median_ms": 6.71, "pure_fastest_ms": 6.65, "sqlite_median_ms": 2836.44, "sqlite_fastest_ms": 2834.17, "sqlite_indexed_median_ms": 7.56, "sqlite_indexed_fastest_ms": 7.51} diff --git a/research/experiments/035-push/2026-10-04-tpcc-after-1.jsonl b/research/experiments/035-push/2026-10-04-tpcc-after-1.jsonl new file mode 100644 index 0000000..5899ff0 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-tpcc-after-1.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.73, "sqlite_load_s": 1.56, "sqlite_page_size": 4096, "pure_resident_mib": 38.4, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 43328, "new_orders_per_minute": 1159284, "undone_new_orders": 3697, "errors": 0, "new_order": {"done": 390128, "failed": 0, "p50_us": 1472, "p99_us": 3456}, "payment": {"done": 372418, "failed": 0, "p50_us": 1472, "p99_us": 3328}, "order_status": {"done": 34753, "failed": 0, "p50_us": 256, "p99_us": 1088}, "delivery": {"done": 34741, "failed": 0, "p50_us": 1472, "p99_us": 3328}, "stock_level": {"done": 34517, "failed": 0, "p50_us": 288, "p99_us": 1152}, "server_threads": "3", "cpu_us_per_transaction": 43.7, "core_shares": {"pure-owner": 0.91, "pure-checkpoint": 0.01}, "resident_mib": 453.2, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 453.2, "pure_state_mib": 231.0, "pure_store_mib": 238.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 26156, "burst_events": 793675, "checkpoints": 38}} diff --git a/research/experiments/035-push/2026-10-04-tpcc-after-2.jsonl b/research/experiments/035-push/2026-10-04-tpcc-after-2.jsonl new file mode 100644 index 0000000..2ee652d --- /dev/null +++ b/research/experiments/035-push/2026-10-04-tpcc-after-2.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.74, "sqlite_load_s": 1.58, "sqlite_page_size": 4096, "pure_resident_mib": 38.3, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42684, "new_orders_per_minute": 1141854, "undone_new_orders": 3643, "errors": 0, "new_order": {"done": 384292, "failed": 0, "p50_us": 1536, "p99_us": 3584}, "payment": {"done": 367011, "failed": 0, "p50_us": 1472, "p99_us": 3584}, "order_status": {"done": 34267, "failed": 0, "p50_us": 256, "p99_us": 1088}, "delivery": {"done": 34181, "failed": 0, "p50_us": 1472, "p99_us": 3584}, "stock_level": {"done": 34005, "failed": 0, "p50_us": 288, "p99_us": 1152}, "server_threads": "3", "cpu_us_per_transaction": 44.5, "core_shares": {"pure-owner": 0.9, "pure-checkpoint": 0.01}, "resident_mib": 447.5, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 447.5, "pure_state_mib": 227.9, "pure_store_mib": 238.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 25541, "burst_events": 781926, "checkpoints": 38}} diff --git a/research/experiments/035-push/2026-10-04-tpcc-after-3.jsonl b/research/experiments/035-push/2026-10-04-tpcc-after-3.jsonl new file mode 100644 index 0000000..b194c7e --- /dev/null +++ b/research/experiments/035-push/2026-10-04-tpcc-after-3.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.73, "sqlite_load_s": 1.57, "sqlite_page_size": 4096, "pure_resident_mib": 38.4, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42674, "new_orders_per_minute": 1141634, "undone_new_orders": 3646, "errors": 0, "new_order": {"done": 384192, "failed": 0, "p50_us": 1536, "p99_us": 3584}, "payment": {"done": 366824, "failed": 0, "p50_us": 1472, "p99_us": 3584}, "order_status": {"done": 34237, "failed": 0, "p50_us": 256, "p99_us": 1216}, "delivery": {"done": 34210, "failed": 0, "p50_us": 1472, "p99_us": 3584}, "stock_level": {"done": 34017, "failed": 0, "p50_us": 288, "p99_us": 1344}, "server_threads": "3", "cpu_us_per_transaction": 44.8, "core_shares": {"pure-owner": 0.91, "pure-checkpoint": 0.01}, "resident_mib": 447.1, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 447.1, "pure_state_mib": 227.8, "pure_store_mib": 230.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 25781, "burst_events": 781665, "checkpoints": 38}} diff --git a/research/experiments/035-push/2026-10-04-tpcc-after-4.jsonl b/research/experiments/035-push/2026-10-04-tpcc-after-4.jsonl new file mode 100644 index 0000000..51b96d5 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-tpcc-after-4.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.73, "sqlite_load_s": 1.56, "sqlite_page_size": 4096, "pure_resident_mib": 38.3, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42146, "new_orders_per_minute": 1127708, "undone_new_orders": 3594, "errors": 0, "new_order": {"done": 379497, "failed": 0, "p50_us": 1536, "p99_us": 3840}, "payment": {"done": 362238, "failed": 0, "p50_us": 1472, "p99_us": 3840}, "order_status": {"done": 33860, "failed": 0, "p50_us": 256, "p99_us": 1216}, "delivery": {"done": 33756, "failed": 0, "p50_us": 1472, "p99_us": 3840}, "stock_level": {"done": 33565, "failed": 0, "p50_us": 288, "p99_us": 1280}, "server_threads": "3", "cpu_us_per_transaction": 44.5, "core_shares": {"pure-owner": 0.9, "pure-checkpoint": 0.01}, "resident_mib": 443.4, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 443.4, "pure_state_mib": 225.3, "pure_store_mib": 230.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 25353, "burst_events": 771983, "checkpoints": 38}} diff --git a/research/experiments/035-push/2026-10-04-tpcc-after-5.jsonl b/research/experiments/035-push/2026-10-04-tpcc-after-5.jsonl new file mode 100644 index 0000000..20de47f --- /dev/null +++ b/research/experiments/035-push/2026-10-04-tpcc-after-5.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.74, "sqlite_load_s": 1.59, "sqlite_page_size": 4096, "pure_resident_mib": 38.3, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42707, "new_orders_per_minute": 1142603, "undone_new_orders": 3645, "errors": 0, "new_order": {"done": 384513, "failed": 0, "p50_us": 1536, "p99_us": 3456}, "payment": {"done": 367089, "failed": 0, "p50_us": 1472, "p99_us": 3328}, "order_status": {"done": 34264, "failed": 0, "p50_us": 256, "p99_us": 1152}, "delivery": {"done": 34237, "failed": 0, "p50_us": 1472, "p99_us": 3328}, "stock_level": {"done": 34031, "failed": 0, "p50_us": 288, "p99_us": 1216}, "server_threads": "3", "cpu_us_per_transaction": 44.8, "core_shares": {"pure-owner": 0.9, "pure-checkpoint": 0.01}, "resident_mib": 447.9, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 447.9, "pure_state_mib": 228.0, "pure_store_mib": 238.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 25847, "burst_events": 782278, "checkpoints": 38}} diff --git a/research/experiments/035-push/2026-10-04-tpcc-after-6.jsonl b/research/experiments/035-push/2026-10-04-tpcc-after-6.jsonl new file mode 100644 index 0000000..682b35f --- /dev/null +++ b/research/experiments/035-push/2026-10-04-tpcc-after-6.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.73, "sqlite_load_s": 1.55, "sqlite_page_size": 4096, "pure_resident_mib": 39.1, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42303, "new_orders_per_minute": 1131722, "undone_new_orders": 3607, "errors": 0, "new_order": {"done": 380849, "failed": 0, "p50_us": 1536, "p99_us": 3712}, "payment": {"done": 363657, "failed": 0, "p50_us": 1472, "p99_us": 3712}, "order_status": {"done": 33979, "failed": 0, "p50_us": 256, "p99_us": 1216}, "delivery": {"done": 33883, "failed": 0, "p50_us": 1472, "p99_us": 3840}, "stock_level": {"done": 33692, "failed": 0, "p50_us": 288, "p99_us": 1344}, "server_threads": "3", "cpu_us_per_transaction": 44.5, "core_shares": {"pure-owner": 0.9, "pure-checkpoint": 0.01}, "resident_mib": 444.8, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 444.8, "pure_state_mib": 226.0, "pure_store_mib": 230.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 25762, "burst_events": 774860, "checkpoints": 38}} diff --git a/research/experiments/035-push/2026-10-04-tpcc-before-1.jsonl b/research/experiments/035-push/2026-10-04-tpcc-before-1.jsonl new file mode 100644 index 0000000..f7035b7 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-tpcc-before-1.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.73, "sqlite_load_s": 1.56, "sqlite_page_size": 4096, "pure_resident_mib": 38.3, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42852, "new_orders_per_minute": 1146481, "undone_new_orders": 3659, "errors": 0, "new_order": {"done": 385820, "failed": 0, "p50_us": 1536, "p99_us": 3456}, "payment": {"done": 368350, "failed": 0, "p50_us": 1472, "p99_us": 3456}, "order_status": {"done": 34393, "failed": 0, "p50_us": 256, "p99_us": 1088}, "delivery": {"done": 34322, "failed": 0, "p50_us": 1472, "p99_us": 3328}, "stock_level": {"done": 34150, "failed": 0, "p50_us": 288, "p99_us": 1216}, "server_threads": "3", "cpu_us_per_transaction": 43.8, "core_shares": {"pure-owner": 0.91, "pure-checkpoint": 0.01}, "resident_mib": 449.2, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 449.2, "pure_state_mib": 228.7, "pure_store_mib": 238.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 25811, "burst_events": 784920, "checkpoints": 38}} diff --git a/research/experiments/035-push/2026-10-04-tpcc-before-2.jsonl b/research/experiments/035-push/2026-10-04-tpcc-before-2.jsonl new file mode 100644 index 0000000..708403a --- /dev/null +++ b/research/experiments/035-push/2026-10-04-tpcc-before-2.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.73, "sqlite_load_s": 1.59, "sqlite_page_size": 4096, "pure_resident_mib": 38.3, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42502, "new_orders_per_minute": 1136994, "undone_new_orders": 3626, "errors": 0, "new_order": {"done": 382624, "failed": 0, "p50_us": 1536, "p99_us": 3456}, "payment": {"done": 365345, "failed": 0, "p50_us": 1472, "p99_us": 3456}, "order_status": {"done": 34136, "failed": 0, "p50_us": 256, "p99_us": 1216}, "delivery": {"done": 34060, "failed": 0, "p50_us": 1472, "p99_us": 3456}, "stock_level": {"done": 33868, "failed": 0, "p50_us": 288, "p99_us": 1216}, "server_threads": "3", "cpu_us_per_transaction": 44.9, "core_shares": {"pure-owner": 0.9, "pure-checkpoint": 0.01}, "resident_mib": 446.2, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 446.2, "pure_state_mib": 227.0, "pure_store_mib": 230.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 25478, "burst_events": 778489, "checkpoints": 38}} diff --git a/research/experiments/035-push/2026-10-04-tpcc-before-3.jsonl b/research/experiments/035-push/2026-10-04-tpcc-before-3.jsonl new file mode 100644 index 0000000..cb0a745 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-tpcc-before-3.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.73, "sqlite_load_s": 1.56, "sqlite_page_size": 4096, "pure_resident_mib": 38.3, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42898, "new_orders_per_minute": 1147897, "undone_new_orders": 3652, "errors": 0, "new_order": {"done": 386285, "failed": 0, "p50_us": 1536, "p99_us": 3456}, "payment": {"done": 368716, "failed": 0, "p50_us": 1472, "p99_us": 3456}, "order_status": {"done": 34422, "failed": 0, "p50_us": 256, "p99_us": 1088}, "delivery": {"done": 34365, "failed": 0, "p50_us": 1472, "p99_us": 3456}, "stock_level": {"done": 34182, "failed": 0, "p50_us": 288, "p99_us": 1216}, "server_threads": "3", "cpu_us_per_transaction": 44.8, "core_shares": {"pure-owner": 0.91, "pure-checkpoint": 0.01}, "resident_mib": 449.4, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 449.4, "pure_state_mib": 228.9, "pure_store_mib": 238.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 25989, "burst_events": 785795, "checkpoints": 38}} diff --git a/research/experiments/035-push/2026-10-04-tpcc-before-4.jsonl b/research/experiments/035-push/2026-10-04-tpcc-before-4.jsonl new file mode 100644 index 0000000..cab419c --- /dev/null +++ b/research/experiments/035-push/2026-10-04-tpcc-before-4.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.73, "sqlite_load_s": 1.58, "sqlite_page_size": 4096, "pure_resident_mib": 39.0, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42219, "new_orders_per_minute": 1129676, "undone_new_orders": 3602, "errors": 0, "new_order": {"done": 380164, "failed": 0, "p50_us": 1536, "p99_us": 3712}, "payment": {"done": 362870, "failed": 0, "p50_us": 1472, "p99_us": 3584}, "order_status": {"done": 33910, "failed": 0, "p50_us": 256, "p99_us": 1216}, "delivery": {"done": 33814, "failed": 0, "p50_us": 1472, "p99_us": 3584}, "stock_level": {"done": 33638, "failed": 0, "p50_us": 288, "p99_us": 1344}, "server_threads": "3", "cpu_us_per_transaction": 45.2, "core_shares": {"pure-owner": 0.9, "pure-checkpoint": 0.01}, "resident_mib": 444.7, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 444.7, "pure_state_mib": 225.6, "pure_store_mib": 230.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 25716, "burst_events": 773330, "checkpoints": 38}} diff --git a/research/experiments/035-push/2026-10-04-tpcc-before-5.jsonl b/research/experiments/035-push/2026-10-04-tpcc-before-5.jsonl new file mode 100644 index 0000000..8f7bb33 --- /dev/null +++ b/research/experiments/035-push/2026-10-04-tpcc-before-5.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.75, "sqlite_load_s": 1.59, "sqlite_page_size": 4096, "pure_resident_mib": 38.3, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42339, "new_orders_per_minute": 1132819, "undone_new_orders": 3605, "errors": 0, "new_order": {"done": 381212, "failed": 0, "p50_us": 1536, "p99_us": 3712}, "payment": {"done": 363969, "failed": 0, "p50_us": 1472, "p99_us": 3584}, "order_status": {"done": 33992, "failed": 0, "p50_us": 256, "p99_us": 1216}, "delivery": {"done": 33915, "failed": 0, "p50_us": 1472, "p99_us": 3712}, "stock_level": {"done": 33699, "failed": 0, "p50_us": 288, "p99_us": 1280}, "server_threads": "3", "cpu_us_per_transaction": 44.9, "core_shares": {"pure-owner": 0.9, "pure-checkpoint": 0.01}, "resident_mib": 444.8, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 444.8, "pure_state_mib": 226.2, "pure_store_mib": 230.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 25586, "burst_events": 775558, "checkpoints": 38}} diff --git a/research/experiments/035-push/2026-10-04-tpcc-before-6.jsonl b/research/experiments/035-push/2026-10-04-tpcc-before-6.jsonl new file mode 100644 index 0000000..ca2bd1a --- /dev/null +++ b/research/experiments/035-push/2026-10-04-tpcc-before-6.jsonl @@ -0,0 +1,3 @@ +{"step": "load", "directory": "/dev/shm", "warehouses": 1, "rows": 568758, "generate_s": 0.56, "pure_load_s": 0.73, "sqlite_load_s": 1.57, "sqlite_page_size": 4096, "pure_resident_mib": 38.3, "sqlite_file_mib": 13.8} +{"step": "run", "system": "pure", "warehouses": 1, "terminals": 64, "transactions_per_second": 42581, "new_orders_per_minute": 1139418, "undone_new_orders": 3638, "errors": 0, "new_order": {"done": 383447, "failed": 0, "p50_us": 1536, "p99_us": 3456}, "payment": {"done": 366000, "failed": 0, "p50_us": 1472, "p99_us": 3456}, "order_status": {"done": 34179, "failed": 0, "p50_us": 256, "p99_us": 1216}, "delivery": {"done": 34115, "failed": 0, "p50_us": 1472, "p99_us": 3456}, "stock_level": {"done": 33885, "failed": 0, "p50_us": 288, "p99_us": 1280}, "server_threads": "3", "cpu_us_per_transaction": 44.7, "core_shares": {"pure-owner": 0.91, "pure-checkpoint": 0.01}, "resident_mib": 446.8, "written_bytes_per_transaction": 0.0, "device_bytes_per_transaction": null, "consistency_failures": []} +{"step": "end", "pure_resident_mib": 446.8, "pure_state_mib": 227.4, "pure_store_mib": 230.7, "sqlite_files_mib": 13.8, "grouped_sqlite_files_mib": 0.0, "pure_counts": {"bursts": 25845, "burst_events": 780009, "checkpoints": 38}} diff --git a/research/experiments/035-push/counter.aup b/research/experiments/035-push/counter.aup new file mode 100644 index 0000000..8f70af9 --- /dev/null +++ b/research/experiments/035-push/counter.aup @@ -0,0 +1,8 @@ +# The counter of research 120 (docs/design/019): a view many clients follow, +# moved by one writer. +state count = 0 +input tap = text +on tap { + count = count + 1 +} +view total = count diff --git a/research/experiments/035-push/pairs.py b/research/experiments/035-push/pairs.py new file mode 100644 index 0000000..cc49772 --- /dev/null +++ b/research/experiments/035-push/pairs.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python3 +"""The pairs of runs.sh pairs as a table (research 120): each round's TPC-C +(transactions a second, CPU a transaction, the owner's core share) before +and after, the medians and the bank queries' median times. + +pairs.py DATE from DATE-tpcc-*.jsonl and DATE-queries-*.jsonl beside it +""" +import json +import statistics +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +day = sys.argv[1] + + +def tpcc(version, round_): + for line in (HERE / f'{day}-tpcc-{version}-{round_}.jsonl').read_text().splitlines(): + r = json.loads(line) + if r.get('step') == 'run' and r.get('system') == 'pure': + owner = max((share for name, share in r['core_shares'].items() if 'owner' in name), default=None) \ + if isinstance(r['core_shares'], dict) else None + return r['transactions_per_second'], r['cpu_us_per_transaction'], owner + + +def queries(version, round_): + return {r['query']: r['pure_median_ms'] for r in map(json.loads, (HERE / f'{day}-queries-{version}-{round_}.jsonl') + .read_text().splitlines()) if 'query' in r} + + +rounds = sorted({int(p.stem.rsplit('-', 1)[1]) for p in HERE.glob(f'{day}-tpcc-before-*.jsonl')}) +rows = {'before': [], 'after': []} +for round_ in rounds: + b, a = tpcc('before', round_), tpcc('after', round_) + rows['before'].append(b), rows['after'].append(a) + print(json.dumps({'round': round_, 'tps': [b[0], a[0]], 'change': round((a[0] / b[0] - 1) * 100, 1), + 'cpu_us': [b[1], a[1]], 'owner_share': [b[2], a[2]]})) +med = {v: statistics.median(t for t, _, _ in rows[v]) for v in rows} +cpu = {v: statistics.median(c for _, c, _ in rows[v]) for v in rows} +names = queries('before', rounds[0]).keys() +q = {v: {n: statistics.median(queries(v, r)[n] for r in rounds) for n in names} for v in rows} +print(json.dumps({'tps_median': [med['before'], med['after']], 'change': round((med['after'] / med['before'] - 1) * 100, 1), + 'cpu_us_median': [cpu['before'], cpu['after']], + 'queries_median_ms': {n: [q['before'][n], q['after'][n], + round((q['after'][n] / q['before'][n] - 1) * 100, 1)] for n in names}})) diff --git a/research/experiments/035-push/push.py b/research/experiments/035-push/push.py new file mode 100755 index 0000000..28334dd --- /dev/null +++ b/research/experiments/035-push/push.py @@ -0,0 +1,75 @@ +#!/usr/bin/env python3 +"""Pushing a view against polling it (docs/design/019, research 120). +pure-serve (connection threads THREADS, store in RAM, cores 0-2) holds +counter.aup; benchmarks/http/watch (core 3) has CLIENTS clients follow its +total view, by push (SSE) or by polling every MS, while one writer moves it +RATE times a second for SECONDS. Each run says the freshness (from the +writer's answer to a client seeing that value or a newer one), what the +host's process used of the CPU meanwhile (/proc/PID/stat), and its RSS +with the clients on less without them. + +push.py PURE_SERVE AUP THREADS SECONDS RUNS... RUNS: CLIENTS:RATE:push or CLIENTS:RATE:poll:MS, in turn ROUNDS times +""" +import json +import os +import shutil +import statistics +import subprocess +import sys +import tempfile +import time +from pathlib import Path + +HERE = Path(__file__).resolve().parent +WATCH = HERE.parents[2] / 'benchmarks' / 'http' / 'watch' +TICK = os.sysconf('SC_CLK_TCK') + + +def cpu(pid): + fields = Path(f'/proc/{pid}/stat').read_text().rsplit(')', 1)[1].split() + return (int(fields[11]) + int(fields[12])) / TICK # utime, stime + + +def rss(pid): + return next(int(line.split()[1]) for line in Path(f'/proc/{pid}/status').read_text().splitlines() + if line.startswith('VmRSS')) + + +def main(): + serve, aup, threads, seconds = Path(sys.argv[1]).resolve(), Path(sys.argv[2]).resolve(), sys.argv[3], sys.argv[4] + runs = sys.argv[5:] + work = Path(tempfile.mkdtemp(prefix='pure-035-', dir='/dev/shm')) + try: + subprocess.run([str(aup), 'build', str(HERE / 'counter.aup'), '-o', str(work / 'counter.pmod')], check=True) + for run in runs: + clients, rate, way = run.split(':', 2) + host = subprocess.Popen(['taskset', '-c', '0-2', str(serve), '--module', str(work / 'counter.pmod'), '--store', + str(work / f'store-{run.replace(":", "-")}'), '--port', '0', '--threads', threads, + '--connections', '32768', '--subscribers', '65536'], + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + port = json.loads(host.stdout.readline())['port'] + time.sleep(0.3) + idle = rss(host.pid) + watcher = subprocess.Popen(['taskset', '-c', '3', str(WATCH), str(port), clients, seconds, rate, way, + '/views/total', '/inputs/tap'], stdout=subprocess.PIPE, text=True) + time.sleep(2.0 + int(clients) / 5000) # every client on, its first value had + followed_rss, began = rss(host.pid), cpu(host.pid) + started = time.monotonic() + out, _ = watcher.communicate(timeout=600) + used = cpu(host.pid) - began + took = time.monotonic() - started + host.terminate() + _, err = host.communicate(timeout=60) + stats = json.loads(err.decode().strip().splitlines()[-1]) + result = json.loads(out) + result.update({'host_cpu_seconds': round(used, 2), 'host_cpu_share': round(used / took, 3), + 'rss_kib_per_client': round((followed_rss - idle) / int(clients), 2), + 'pushes': stats['pushes'], 'pushed': stats['pushed'], 'replaced': stats['push_replaced'], + 'requests': stats['requests']}) + print(json.dumps(result), flush=True) + finally: + shutil.rmtree(work) + + +if __name__ == '__main__': + main() diff --git a/research/experiments/035-push/runs.sh b/research/experiments/035-push/runs.sh new file mode 100755 index 0000000..11993a5 --- /dev/null +++ b/research/experiments/035-push/runs.sh @@ -0,0 +1,50 @@ +#!/bin/sh +# The runs of research 120 (docs/design/019), from the repository's root, +# in steps, each under research/guard.py: +# runs.sh fresh ROUND 1,000 clients follow the counter by push and by +# polling, at 1, 10 and 100 writes a second (the +# order turned in even rounds), 10 s each +# runs.sh many ROUND 10,000 subscribers, a write a second +# runs.sh build NAME COMMIT, pairs FIRST LAST, counts as 033's: the +# host before (84483fe) and after (a7de450) +set -e +out=research/experiments/035-push +builds=$HOME/.cache/pure-035 +day=$(date +%F) +case "$1" in +fresh) + runs="1000:1:push 1000:1:poll:100 1000:10:push 1000:10:poll:20 1000:100:push 1000:100:poll:20" + if [ $(($2 % 2)) = 0 ]; then runs=$(echo $runs | tr ' ' '\n' | tac | tr '\n' ' '); fi + until awk '{ exit !($1 < 1.5) }' /proc/loadavg; do sleep 10; done + python3 research/guard.py --most 2048 -- python3 $out/push.py build/pure-serve ../aup/build/aup 3 10 $runs \ + > "$out/$day-fresh-$2.jsonl" ;; +many) + until awk '{ exit !($1 < 1.5) }' /proc/loadavg; do sleep 10; done + python3 research/guard.py --most 3072 -- python3 $out/push.py build/pure-serve ../aup/build/aup 3 10 10000:1:push \ + > "$out/$day-many-$2.jsonl" ;; +build) + rm -rf "$builds/src-$2" + git worktree add -f -q --detach "$builds/src-$2" "$3" + make -s -C "$builds/src-$2" build/pure-serve build/pure-crypto.so + mkdir -p "$builds/$2" + cp "$builds/src-$2/build/pure-serve" "$builds/src-$2/build/pure-crypto.so" "$builds/$2/" + git worktree remove --force "$builds/src-$2" ;; +pairs) + for round in $(seq "$2" "$3"); do + first=before second=after + if [ $((round % 2)) = 0 ]; then first=after second=before; fi + for version in $first $second; do + until awk '{ exit !($1 < 1.5) }' /proc/loadavg; do sleep 10; done + echo "$round $version $(uptime)" >> "$out/$day-load.txt" + python3 research/guard.py -- python3 benchmarks/tpcc/run.py --serve "$builds/$version/pure-serve" --warehouses 1 \ + --check 0 --terminals 64 --sqlite-threads '' --sqlite-group '' --seconds 20 \ + > "$out/$day-tpcc-$version-$round.jsonl" + python3 research/guard.py -- python3 benchmarks/bank/queries.py --serve "$builds/$version/pure-serve" \ + > "$out/$day-queries-$version-$round.jsonl" + done + done ;; +counts) + python3 research/guard.py -- python3 research/experiments/025-i64-cost/count.py "$builds/before/pure-serve" \ + "$builds/after/pure-serve" "$builds/before/pure-serve" "$builds/after/pure-serve" > "$out/$day-counts.jsonl" ;; +*) echo "runs.sh fresh ROUND | many ROUND | build NAME COMMIT | pairs FIRST LAST | counts" >&2; exit 2 ;; +esac |
