| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A receiver that took no records of a stream (a source the key may not
send as, no input of that name, an input of another shape) said nothing:
its sender heard them acknowledged, never stored, and sent them all
again every second, for ever, and its operator saw nothing. Now the
receiver answers REFUSED (0x0c: the names, and why), and the sender
sends no records for a second, then a packet of them to learn whether
they are taken now, the wait twice as long each time (64 s at most). A
refusal ends when that packet is acknowledged with no REFUSED beside, or
a record is stored. link.c says it once on stderr, as a wire says a
receiver does not take its sends; serve's input says which of the two
the app lacks.
test-link: each of the three is refused seven times in 90 s with nine
packets, and a stream refused until the app has its input goes on then,
all 200 records stored. check.py links a node to an input its receiver
lacks: said once, two packets in 2.5 s.
|
| |
|
|
|
|
|
|
|
|
|
| |
A receiver kept two runs a peer: the sender's, and the one before it
restarted. A node with three outputs to another sends in three runs with
one key, and they pushed one another out: each packet keyed its run
again, and an answer marked for one went to the run that took its place.
Now more places are made beside when the run seen longest ago is still
live (up to 64), and one is reused only once its run is quiet the idle
period. test-link's three outputs of one node key three runs, not seven,
and are all stored in 16 ms, not 1016.
|
| |
|
|
|
|
|
| |
A sender took sends up to 64 MiB, but a receiver holds 16 MiB of a
stream not yet stored, so a send between the two went again forever.
The most a send may be is now PURE_LINK_RECORD_MOST, what a receiver
holds; test-link sends one that large whole and is refused one byte more.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
A RECORDS frame of records in order, with nothing held and no record's
tail waiting, went through hold, on_line and line_append a record: the
same checks each time (heard, credit, the line's end, held, the memory
bound). Now they are made once for the frame and its records are copied
onto the line one after another. The receiver's CPU a 61-byte record in
M5 went from 0.15 to 0.133-0.143 us; both sides together 0.28 us, beside
ngtcp2's 0.27 and quinn's 0.31 in the same rounds (the M5 host here also
compares every record with what was sent). test-link's results are the
same to the byte.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A receiver that started again without its memo could open none of its
senders' packets (their epochs were gone), said nothing, and each sender
waited 2 s and four probes, then offered with no records. Worse, those
packets failed to open, so each cost a token (256 a second, the flood
guard): ten thousand senders came back 256 a second. M2 without a memo
stopped 0.8 s and caught up in 7.7 s.
Now the receiver answers a packet of a run it does not know, in an epoch
past 0, with LOST: sealed in epoch 0 under the nodes' keys, naming the
packet. As QUIC's stateless reset, it is smaller than the packet and
comes out of a bucket of its own (4,096 a second). A sender that has that
packet out begins again as a run begins: epoch 0 with its records at
once, epoch 1 offered. Only the receiver can say LOST, and one of the past
names a packet no longer out; silence alone (an attacker's doing, maybe)
still gets offers only. Such a packet costs no key token: no keys were
made for it, and the secret made is remembered, once a sender.
M2 without a memo (ten thousand devices, twice): no 100 ms step stopped,
caught up in 0.4-0.5 s, p99 5.8-5.9 ms, the most 48-57 ms (was 4.1 and
7.8 s). test-link: a changed packet may get a LOST smaller than itself
and nothing else; half of a flood in epoch 1 got 267 LOSTs, 12 KB for its
6 MB; a record goes under the nodes' keys after an epoch began only after
the receiver forgot, until the next epoch.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
A link that found the path carries 1,452 bytes kept sending them when a
route changed and larger packets were dropped with nothing said (a
black hole): it lost them one after another and stood still (test-link:
8 of 3,000 records in ten minutes). Now, as RFC 8899 §4.3 has it, it goes
back to 1,200 bytes when three larger packets in a row are lost while an
acknowledgment covers only smaller ones, or when two probes in a row go
unanswered, and tries the larger size again a second on (three tries,
then ten minutes). The case completes in 1.2 s of virtual time; the M6
simulator's results are the same to the byte in eight paths of loss,
reordering and a bottleneck.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Profiling both sides in one thread, without sockets (61-byte records):
the rings of records and flights divided for each place, the record a
number names was found three times over, a record in one part was
searched for its part, and the receiver walked the line record by record
to hand it over and again to let it go. The rings are powers of two long
and now masked; a record is found once; a one-part record's part is the
first; a line of a batch or less goes whole without a walk, and a
delivery applied whole lets its bytes go at once. The protocol alone:
0.171 to 0.162 us a record. M5 (the link and listener threads, sockets
too), five rounds against the commit before: 61 B 0.298 to 0.287 us, 1
KiB 2.63-2.90 to 2.48-2.79 us, 16 KiB the same or less; quinn measured
0.31, 3.85 and 60 us in the same rounds. test-link's results are the
same to the byte.
|
| |
|
|
|
|
|
|
|
|
| |
A send that fails with no route, nobody at the port, or the network
gone (ECONNREFUSED, EHOSTUNREACH, ENETUNREACH, ENETDOWN, EADDRNOTAVAIL)
now has the link look its receiver's name up again before its next
packet, not three unanswered probes later: a receiver moved away from an
address that is gone costs a sender one probe's wait, not three. Such a
failure no longer turns GSO off either (it said nothing of GSO). In the
check that moves a receiver, the sender sent 24 records again, not 243.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A link resolved its receiver's name once, at its start: a receiver whose
store moved to another machine (another address under the same name)
was never found, and a sender whose own network changed kept a socket
that no longer went anywhere. Now, when three probes in a row go
unanswered, and at each probe after (a second apart at most), the link
looks the name up again and sends from a new socket connected there.
It looks before the probe goes, so the probe's answer comes to the new
socket (looking after it lost every answer, one socket after another).
A sender at rest does not look: only probes unanswered count.
make check moves a receiver (H3 of docs/design/011-pure-transport.md):
in namespaces of its own, the sender links to pure://receiver.test at
127.0.0.2; the receiver is killed, its store, key and memo copied, the
old address made unreachable, the name pointed at 127.0.0.3 and the
copy started there. Every order is applied once, and the moved receiver
makes no X25519.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The listener's socket takes packets to any address of the node, and its
answers went out from whichever address the system chose for the way
back. A sender whose socket is connected to another of the node's
addresses (a node with many, or a name that points at a second one)
takes answers only from the address it sent to: it heard none, and its
records went on arriving unacknowledged. Now the listener asks for the
address each packet came to (IP_PKTINFO; IPv4 as ::ffff:a.b.c.d on a
socket of both) and answers from it. The path a receiver keeps for a run
is both addresses (48 bytes, not 28): a packet that comes to another of
the node's addresses moves it, and is checked, as one from elsewhere is.
Found by a receiver moved to 127.0.0.2 (the check that follows).
|
| |
|
|
|
|
|
|
| |
The window, threshold, the start of recovery and CUBIC's four values were
fields of their own, and an episode of congestion events copied each of
the seven aside and back. Now they are one Congestion: an episode keeps
a copy, and undoing it takes the copy back, the window grown since kept.
test-link's results and the M6 simulator's are the same to the byte.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A receiver's run kept seventeen fields of epochs: numbers, flags, secrets
and keys for now, the one before, the one accepted, and an offer waiting.
Now it keeps three epochs (now, before, next), each its number, secret
and the keys of the sender's packets in it, and every move is one of two:
made from a secret, or become the epoch now (the one it was goes before,
the answers are sealed in the new one). Using the one accepted, a chain
begun again from epoch 0, and a run begun from the memo are these moves.
The key of an offer waiting is the key offered: a new offer lets the one
accepted before go at once. A sender's moves (an epoch begun, one taken
for lost, the first) are one too: its packets sealed in epoch n.
test-link's results are the same to the byte.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A sender that took its epoch for lost offers epoch 1 again with a new
key. The receiver accepts it in the turn, after the answers; until then
its answers still carried the ACCEPT of the earlier offer, which named
the same epoch. The sender began its epoch from that one, the two sides
held different secrets, and 2 s later it took the epoch for lost again:
a turn of that each time the order fell so. Now an ACCEPT carries the
first four bytes of the key it answers, and a sender begins its epoch
only from one that answers its own.
test-link cuts the path for 3 s the moment the sender begins an epoch,
three times; a sender's counts say how many epochs it took for lost,
and it must be three. Before, it was four.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
nothing past
A sender offers the public half of a pair made for the next epoch
(OFFER) until the receiver accepts with a half of its own (ACCEPT); their
X25519 and the epoch's secret make the next one's, and the private
halves go. The header's low four bits say a packet's epoch; epoch 0 is
the nodes' static secret, as the keys were. The first epoch is offered in
the run's first packet (it begins with the first answer when the
receiver is not busy), then one an hour or GiB. One who later has both
nodes' keys opens only epoch 0, and no record went there after an epoch
began: test-link tries every packet sent under the static keys, with
epochs a second long, loss, reordering, restarts and replays.
The receiver writes the epoch it accepted in its memo (a place is 256
bytes now), so it goes on without an X25519 when it starts again; one
that forgot is not heard from, and its senders, after 2 s and four
probes unanswered, offer again in epoch 0 with no records until an epoch
begins. Keys of four epochs are tried (accepted, now, before, zero), as
two may share the four bits.
An accept costs an X25519 on each side, 220 µs on this Pi: one of a
sender with records costs a token (256 a second, 8 at once) and the
next packet offers again when there is none; one of a sender with
nothing else to send waits for none, one a second for it. The X25519 is
made after the turn's answers, never between a packet and its answer.
serve's report and M2's generator count keys, X25519s and epochs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
M2 (ten thousand devices, a record each every 5 s) sent 118k packets for
80k records. A link whose packets were never full still probed for a
1,452-byte path: nothing larger would have been carried. And a sender
said a receiver's challenge back in a packet of its own, one for each
device each time the receiver started. Now a link probes once a packet
of its was full, and a challenge rides in the next packet that goes
(records, or a probe). Until then the receiver's answers there are at
most three times what came, which the answers to one packet are not;
when they are held back, the sender's probe or record sent again says it
back. M2 sends 80,036 again (docs/research/099-link-epochs.md).
|
| |
|
|
|
|
|
|
| |
An epoch's keys (docs/design/011-pure-transport.md): a pair made for the
next epoch, held raw (a link may hold one for long and a node many, so
not in OpenSSL's 16 KiB secure heap), and the secret its private half
shares with the other side's public half, never all zeros. The crypto
table's version is 5.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
test-link now also plays an attacker: packets changed on the way,
packets of the past sent again from the sender's address and its own,
keys and sources not allowed, and a flood of made-up packets. It found:
- A nonce used twice: a receiver sealed every answer in its one run, and
a sender at rest coming back with the same run got a new slot whose
answers began at number 0 again under the same key. Each slot now
draws the run its answers are sealed in.
- A link stopped for good by one packet of the past: a stream that did
not know its last heard (after a restart or a rest) handed over from
its first record held, and a stale part of an old one never came
whole. It hands over from its first whole record.
- Answers taken away: any packet that opened, however old, moved where
answers went. Only the newest of its run moves it now; a new address
is sent a challenge the sender says back (CHALLENGE, RESPONSE), and
until then gets at most three times the bytes that came from it.
- An answer made by a packet that did not open: a number heard before
was acknowledged before the packet was opened. It is opened first.
- An old answer lowering the sender's credit, or its gap making it send
again: the sender now sends up to the most heard and credit it was
told, and takes a gap only from a word not older than it knows.
- Ten thousand made-up packets made ten thousand keys: keys made for a
run that did not open now cost a token, 256 a second.
Counts now say runs keyed and secrets made.
|
| |
|
|
|
|
|
| |
A sample of 0 (a path within one clock tick, or a test's) made the
pacing rate the window over 0, and its bytes infinity times 0: NaN,
after which the sender never sent again. CUBIC's time-based estimate
divided by it too. The RTT they take is at least a microsecond.
|
| |
|
|
|
| |
Linux's CUBIC and quinn's keep it so; RFC 9438's whole datagram once
past the window before the event is left out.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
acknowledgments
Packets go at the window an RTT, a quarter more (twice in slow start),
up to 2 ms of it at once (10 to 256 datagrams, RFC 9002 7.7); the sender
says when the next may go. The Reno-friendly estimate grows by the
acknowledgments (RFC 9438) or by time (RFC 8312), whichever is ahead, so
a link its app holds back to a few records a second does not fall
behind it.
In the M6 simulator (RTT 100 ms, 5% loss, 50 records a second) the
median time a record takes falls from 134 to 51-71 ms and the p99 from
about 900 to 290-350 ms; a backlog with 1% loss at RTT 20 ms goes at
8.3 Mbit/s. The transport's CPU a record is unchanged.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
acknowledged
A link starts at 1,200 bytes; once an RTT is known it sends a PING padded
to 1,452 (DPLPMTUD, RFC 8899), alone and with the don't-fragment bit,
and uses that size once it is acknowledged. A probe lost is no
congestion; it is tried again a second on, three times, then ten minutes
on. GSO is asked for each burst (a control message), not the socket, so
a packet sent alone is never cut. CUBIC counts in the datagram.
On loopback packets fall by 19-27%; the CPU a record falls 13% at 1 KiB
and 7% at 16 KiB; in the simulator of a 50 Mbit/s path with 1% loss the
goodput rises from 6.1 to 7.0 Mbit/s (5%: 2.3 to 3.0).
|
| |
|
|
|
|
|
|
|
| |
A record that does not fit what is left of a packet goes in part when at
least 64 bytes fit; its parts keep their cuts, so what a packet carried
is whole parts. The receiver keeps the spans it had of a record, and a
record coming in order from its start fills the line's tail with no
allocation. Packets fall by 8% for 1 KiB records and 10% for 16 KiB;
the CPU a record is the same at 61 B and 1 KiB and 5% less at 16 KiB.
|
| |
|
|
|
|
|
|
|
| |
020-lossy-paths/run.py runs each case in an unprivileged namespace: a
50 Mbit/s tbf bottleneck under netem's delay, jitter and loss, both
ways. A link may send without UDP GSO (pure_link_gso), as netem takes a
burst as one packet; quinn the same (QUIC_GSO=0). transport.c and the
quinn harness give the records handed over in each 100 ms, and at a
steady rate the time each took; TCP CUBIC flows run beside them.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A packet called lost stays known a while; acknowledged after all, it
came late: what it carried counts as received, the reordering awaited
grows (later packets acknowledged, and time past the RTT, as RACK), and
an episode of congestion events whose losses all came late is undone.
The window grows only when it was used (RFC 9002 7.8), and congestion
avoidance is CUBIC (RFC 9438), as quinn's.
In a path of 50 Mbit/s whose packets reorder by up to 5 ms, with no
loss, a link carried 1.5 Mbit/s at an RTT of 20 ms and 10 at 100 ms; now
43.6 and 40.1, as without reordering (research/experiments/020-lossy-paths,
virtual time). With 1% random loss: 4.9 to 6.1 at 20 ms, 1.0 to 1.3 at
100 ms. The transport's CPU a record is unchanged.
|
| |
|
|
|
|
|
|
|
|
| |
links.c was a second implementation of the packet format; each device is
now a PureLinkSender on a socket of its own, keyed with a secret made
outside (pure_link_sender_shared: a key kept elsewhere, or a test's
many). M1 (1k, 10k) and M2 (10k, kill -9) pass with it: no silent step,
caught up in 0.2 s, every order applied once; the slowest store after
the restart is now the sender's 1 s wait for a word of the save
(STORED_WAIT), where the old devices tried again after 200 ms.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
link/link-protocol.h is the protocol with no socket, thread or clock of
its own: packets and time in, packets out. link/link.h carries it over
UDP as before. A sender starts small (8 flights, 16 records, a 4 KiB
ring) so one thread can hold many. The transport's CPU a record is the
same within 2% (five rounds each way, 61 B to 16 KiB).
test-link runs both sides through a network of its own: loss, packets
held back and doubled, failed saves, an idle sweep, and both sides
starting again, every record applied once as sent.
|
| |
|
|
|
|
| |
A sender takes its messages' shape, and a listener asks its host which
input a stream goes to. The core gives the shape (pure_module_shape);
link/ now builds without the core's headers.
|
|
|
libpure-link.a holds them; pure-serve and the M5 harness link it instead
of naming its source files.
|
