Move a connection to its app's home thread at its first request (017)

With several connection threads, a host of apps had each app's
connections spread over all of them: every thread's round handed each
owner a share, and every turn handed answers back to every thread. The
apps took a third more CPU a request than as processes, with five times
the voluntary switches, and when two apps were busy each waited a
kernel tick at its 99th percentile (4.1 ms, against 1.5 ms as two
processes).

Each app now has a home thread (its place in --apps, modulo the
threads). A connection whose first request names an app at home
elsewhere moves there once, its head read and no ticket taken yet: the
home thread adopts its descriptor and the bytes read, and reads the
request again. Each app's requests then meet its owner from one
thread, as from a process of its own. Two busy apps: 99th percentile
1.4 ms, the same 145k writes a second.

server_apps restarts its apps on three connection threads, asks each
app from fresh connections, and checks that some moved and all were
answered right.

1 files changed, 13 insertions, 7 deletions

diff --git a/tests/check.py b/tests/check.py
index fdb4f56..b01b29b 100644
--- a/tests/check.py
+++ b/tests/check.py
@@ -3831,15 +3831,21 @@ twin   {work / 'apps.pmod'}   {work / 'store-a'}
         stats = host.stop()[-1]
     by_name = {app['name']: app for app in stats['apps']}
     assert by_name['busy']['busy'] == busy and by_name['a']['events'] > 0 and stats['workers'] == 2, stats['apps']
-    # Restarted: each app's states are back.
-    host = HttpHost([serve, '--apps', manifest])
+    # Restarted, on three connection threads: each app's states are back, and
+    # a connection whose first request is for an app at home on another
+    # thread moves there (and is answered there).
+    host = HttpHost([serve, '--apps', manifest, '--threads', 3])
     try:
-        assert get(host.port, '/a/views/total')[::2] == (200, b'3') and get(host.port, '/b/views/total')[::2] == (200, b'6')
-        assert get(host.port, '/a/views/count')[::2] == (200, b'10') and get(host.port, '/busy/views/total')[::2] == \
-            (200, str(ok).encode())
+        for _ in range(8):
+            assert get(host.port, '/a/views/total')[::2] == (200, b'3') and get(host.port, '/b/views/total')[::2] == (200, b'6')
+            assert get(host.port, '/a/views/count')[::2] == (200, b'10') and get(host.port, '/busy/views/total')[::2] == \
+                (200, str(ok).encode())
+            assert post(host.port, '/tight/inputs/add?step=1') == (200, 'none')
+        assert get(host.port, '/tight/views/total')[::2] == (200, b'8')
     finally:
-        host.stop()
-    return {'apps': len(running), 'stopped': 2, 'flooded': len(flood), 'busy': busy}
+        moved = host.stop()[-1]['moved_home']
+    assert moved > 0, moved
+    return {'apps': len(running), 'stopped': 2, 'flooded': len(flood), 'busy': busy, 'moved_home': moved}
 
 
 def main():