pure/research/experiments/019-idle-links, branch masterVerified execution runtime unifying state, persistence, and replication.
http://git.yeowoo.org/pure/atom?h=master2026-10-01T07:52:43ZPush a turn's sends to a link at once, and round only the links that want one (017, 2b-1)2026-10-01T07:52:43Zeunhhunecpition205@gmail.com2026-10-01T07:52:43Zurn:sha1:94e7bb87bce6821e91734f1fcbb6cfd61a008049
The sending loop was woken by every record queued while it slept: the
owner queues one by one, the loop caught up and slept between them, and
every link took a round on every wake. Four links sent 64k packets of
12 records where threads sent 40k of 20, and ran at 257k sends/s
against 581k. The owner now queues a turn's sends and pushes each link
once; the loop rounds a link only when its socket has answers, it was
pushed, or its time came.
Let ngtcp2's handshake through a path that reorders2026-09-29T16:03:09Zeunhhunecpition205@gmail.com2026-09-29T16:03:09Zurn:sha1:d9e1fbddf9f92fc7850b0346f1f2511ddc0b6e54
OpenSSL 3.5's ClientHello offers ML-KEM with X25519 and takes two
Initials; when the path brought the second first, ngtcp2's server was
asked for a Retry and the harness gave up. The client now offers X25519
alone, as Pure's links agree, and the server lets go of a connection it
may not keep, so the client's next Initial begins another.
Put ngtcp2 beside quinn in M6, as it is in M52026-09-29T15:57:02Zeunhhunecpition205@gmail.com2026-09-29T15:57:02Zurn:sha1:1d6d2430960622589f6c11220e09ba4b6dba2989
drain.c takes a rate and seconds as quinn's client does (paced messages
stamped with the time they were written, and no more written past the
seconds), and its service writes quinn's line for each stream: the
messages read in each 100 ms and how long they took. 020's run.py runs
it as a third, on port 40005 behind the same bottleneck, and --who
picks whom to run.
Put ngtcp2 beside quinn in M5, as the design planned2026-09-29T15:13:48Zeunhhunecpition205@gmail.com2026-09-29T15:13:48Zurn:sha1:1e562fb73527adb46a6ce8f4f302e22460aedf95
ngtcp2 1.25.0 with OpenSSL 3.5's QUIC TLS (its ossl backend), fetched
and built into build/third by ngtcp2/build.sh; drain.c does in C what
the quinn harness does: one connection, one stream, "pure" and then the
messages each after its u32 length, counted by the service, packets in
GSO bursts. Its stream bytes are a function of their offset, so what
ngtcp2 keeps to send again points into a pattern that never changes.
run.py's drain_ngtcp2 measures both sides' CPU as drain_quic does.
Five rounds each, both sides' CPU a message: 61 B Pure 0.280, ngtcp2
0.270, quinn 0.310 us; 1 KiB 2.62, 3.54, 3.94; 16 KiB 40.2, 54.4, 60.2.
Say LOST when a run's epoch is not known, and let its sender begin again at once2026-09-29T14:59:44Zeunhhunecpition205@gmail.com2026-09-29T14:59:44Zurn:sha1:d0be140549b67708039f3c69d0b039440eb3a063
A receiver that started again without its memo could open none of its
senders' packets (their epochs were gone), said nothing, and each sender
waited 2 s and four probes, then offered with no records. Worse, those
packets failed to open, so each cost a token (256 a second, the flood
guard): ten thousand senders came back 256 a second. M2 without a memo
stopped 0.8 s and caught up in 7.7 s.
Now the receiver answers a packet of a run it does not know, in an epoch
past 0, with LOST: sealed in epoch 0 under the nodes' keys, naming the
packet. As QUIC's stateless reset, it is smaller than the packet and
comes out of a bucket of its own (4,096 a second). A sender that has that
packet out begins again as a run begins: epoch 0 with its records at
once, epoch 1 offered. Only the receiver can say LOST, and one of the past
names a packet no longer out; silence alone (an attacker's doing, maybe)
still gets offers only. Such a packet costs no key token: no keys were
made for it, and the secret made is remembered, once a sender.
M2 without a memo (ten thousand devices, twice): no 100 ms step stopped,
caught up in 0.4-0.5 s, p99 5.8-5.9 ms, the most 48-57 ms (was 4.1 and
7.8 s). test-link: a changed packet may get a LOST smaller than itself
and nothing else; half of a flood in epoch 1 got 267 LOSTs, 12 KB for its
6 MB; a record goes under the nodes' keys after an epoch began only after
the receiver forgot, until the next epoch.
Say that a receiver moved to another machine is followed, and how (100)2026-09-29T14:17:40Zeunhhunecpition205@gmail.com2026-09-29T14:17:40Zurn:sha1:7ad006f6f2d52c105ce6767d34fbd8a7abed4012
H3 of design 011 stands: with ten thousand devices sending 2,000 records
a second, the node's store, key and memo carried to another address,
nothing stopped for a 100 ms step, the moved node made no X25519, and
every record was applied once. Each device's first record after the move
waits one probe (30-45 ms here), as its first packet finds no way there.
The search found the listener answering from an address other than the
one a packet came to, and a link that looked for its receiver after its
probe went, losing every answer. H1-H4 are all measured now; 61 B's CPU
a delivery is at the edge of the criterion, to be looked at before step
5 (adoption and a specification).
Move M2's node to another machine under load (M3's harness)2026-09-29T14:17:40Zeunhhunecpition205@gmail.com2026-09-29T14:17:40Zurn:sha1:088f85ed72b61e879964a4cba6a0c0faa135facb
run.py --restart --move runs in namespaces of its own (unshare -rmn: a
network and an /etc/hosts): the devices send to receiver.test, which
names 127.0.0.2; at --kill-at the node is killed, its store, key and
memo copied elsewhere, 127.0.0.2 made unreachable, the name pointed at
127.0.0.3, and the copy started there. The load generator's devices
look for the node again as link/link.c does: after three probes in a
row unanswered, or a send the system said had no way there.
Say what epochs cost a link, and what the search for it found (099)2026-09-29T12:58:44Zeunhhunecpition205@gmail.com2026-09-29T12:58:30Zurn:sha1:0bbc12658c63f5f5fce2683798d4ac08b1e26e08
Forward secrecy, 098's first task left: keys that leak later open only
epoch 0. M2 after the epoch began and after a restart is as before it
(p50 0.4 ms, p99 0.8 ms, the same CPU, no X25519 on restart). It costs a
p50 of 0.7-1.0 ms more while ten thousand devices first meet and for
about 40 s of accepts after, and a receiver that lost its memo now
stalls 0.8 s and catches up in 7.7 s, not 0.2-0.7: it lost the epochs,
and each sender opens one again. The search also found a packet more a
device from each of M6's path probe and the security checks' challenge,
and a load generator whose own pass over its devices was the floor of
the latency it measured. 098's note that epochs narrow KCI to the first
epoch was wrong; corrected.
Move a link to a new epoch now and then, so keys that leak later open nothing past2026-09-29T12:58:06Zeunhhunecpition205@gmail.com2026-09-29T12:57:58Zurn:sha1:9791a1e70979ee4ba832851912f0de20301343a8
A sender offers the public half of a pair made for the next epoch
(OFFER) until the receiver accepts with a half of its own (ACCEPT); their
X25519 and the epoch's secret make the next one's, and the private
halves go. The header's low four bits say a packet's epoch; epoch 0 is
the nodes' static secret, as the keys were. The first epoch is offered in
the run's first packet (it begins with the first answer when the
receiver is not busy), then one an hour or GiB. One who later has both
nodes' keys opens only epoch 0, and no record went there after an epoch
began: test-link tries every packet sent under the static keys, with
epochs a second long, loss, reordering, restarts and replays.
The receiver writes the epoch it accepted in its memo (a place is 256
bytes now), so it goes on without an X25519 when it starts again; one
that forgot is not heard from, and its senders, after 2 s and four
probes unanswered, offer again in epoch 0 with no records until an epoch
begins. Keys of four epochs are tried (accepted, now, before, zero), as
two may share the four bits.
An accept costs an X25519 on each side, 220 µs on this Pi: one of a
sender with records costs a token (256 a second, 8 at once) and the
next packet offers again when there is none; one of a sender with
nothing else to send waits for none, one a second for it. The X25519 is
made after the turn's answers, never between a packet and its answer.
serve's report and M2's generator count keys, X25519s and epochs.
Give M2's devices a timer each, and say when the slow records were made2026-09-29T12:55:35Zeunhhunecpition205@gmail.com2026-09-29T12:55:35Zurn:sha1:e680e67dc99c33cfaa3152613bddf3742263e385
The load generator asked all ten thousand senders for their timers every
5 ms: 83 % of its CPU (47 ns a sender, mostly cache misses), and records
were timed from the start of that pass, so the pass (0.75-0.94 ms, longer
as the sender grew) was the floor of every latency it measured. A device
now keeps its sender's next timer, asked again only when the sender is
touched (the only time it changes), and the p50 of M2 falls from 0.8 to
0.4 ms for every version alike. It also says the p50 and p99 of the
records made in each second, and the keys and X25519s its senders made.