Wait on the monotonic clock everywhere a deadline is waited for

Review 004's timer fix (103) counted the deadlines on the monotonic
clock, but the owner still slept until a wall-clock time: with the
clock set back an hour a node's timers would stop for that hour. So
would a wire's pause between tries, and a lone commit after a group
would wait an hour for the others to join it. The test's LD_PRELOAD
moves only what the process reads, so there each wait ended at once
instead, and the timers went on ticking while the owner spun a core.

core/wait.h makes conditions that wait on the monotonic clock
(pthread_condattr_setclock; macOS keeps the wall clock) and deadlines
on it. The owner's waits, the wire's pauses and the group commit's
gathering use them.

clock-back.so can now move the clock forward too (CLOCK_SHIFT_S), and
check.py server_clocks moves it an hour each way under a node with a
timer and a wire to a port no one listens on, and under a document
server whose three threads commit together, then one alone. Before:
back, 159 % CPU and 50,886 tries in 1.2 s; forward, 1 tick, 1 try, and
a lone commit that never ended. After: 59 ticks, 0 % CPU, 4 tries and a
lone commit of 2.6 ms, both ways.

1 files changed, 44 insertions, 0 deletions

diff --git a/core/wait.h b/core/wait.h
new file mode 100644
index 0000000..241eaef
--- /dev/null
+++ b/core/wait.h
@@ -0,0 +1,44 @@
+#ifndef PURE_WAIT_H
+#define PURE_WAIT_H
+/* Timed waits that count on the monotonic clock. A deadline read from the
+ * wall clock moves when the clock is set (by a person, or NTP): set back an
+ * hour, a wait begun before lasts an hour more; set forward, it ends at once.
+ * A condition made by pure_cond_monotonic waits until a time pure_wait_until
+ * gives, and neither happens. (macOS has no clock for conditions: there both
+ * keep to the wall clock.) */
+#include <pthread.h>
+#include <time.h>
+
+#ifdef __APPLE__
+#define PURE_WAIT_CLOCK CLOCK_REALTIME
+#else
+#define PURE_WAIT_CLOCK CLOCK_MONOTONIC
+#endif
+
+static inline int pure_cond_monotonic(pthread_cond_t *cond) {
+#ifdef __APPLE__
+    return pthread_cond_init(cond, NULL);
+#else
+    pthread_condattr_t attr;
+    int result = pthread_condattr_init(&attr);
+    if (result) return result;
+    result = pthread_condattr_setclock(&attr, CLOCK_MONOTONIC);
+    if (!result) result = pthread_cond_init(cond, &attr);
+    pthread_condattr_destroy(&attr);
+    return result;
+#endif
+}
+
+/* The time ns from now, on the clock pure_cond_monotonic's conditions wait on. */
+static inline struct timespec pure_wait_until(long long ns) {
+    struct timespec t;
+    clock_gettime(PURE_WAIT_CLOCK, &t);
+    t.tv_sec += (time_t)(ns / 1000000000);
+    t.tv_nsec += (long)(ns % 1000000000);
+    if (t.tv_nsec >= 1000000000) {
+        ++t.tv_sec;
+        t.tv_nsec -= 1000000000;
+    }
+    return t;
+}
+#endif