PoC: detach execution context scheduler from running thread during blocking syscall

[ysbaddaden]

Some syscalls can block the current thread in certain circumstances, for example:

• open(2) when opening a fifo, pipe or character
  device until another end is connected (from another thread or process);
• getaddrinfo(3) until a DNS response (or error, or timeout) is received.

Move scheduler blocked on syscall

This proof of concept introduces a mechanism to declare the scheduler as "doing a syscall" which the monitor thread (SYSMON) can detect on its next iteration and will try to move the scheduler to another thread, so that only the fiber doing the syscall will be blocked, and the other fibers can be resumed.

Usually, the syscall should terminate before the monitor thread notices (for example opening a regular file), so the impact on performance is an atomic STORE + atomic CAS per syscalls. At worst, a thread will be blocked for 10ms (sysmon frequency). For example the updated opening fifo file spec takes ~11ms to complete.

It works for the MT execution contexts and the ST context. It doesn't invalidate the ST guarantee that fibers in the context will never run in parallel: the blocked fiber is blocked on a syscall and will be re-enqueued immediately after the syscall has completed; also the syscalls don't invoke callbacks that would execute crystal code, so AFAICT fibers still won't run in parallel (please correct me if I'm wrong).

Thread Pool

This PoC also introduces a pool of threads. It changes the behavior of threads: we don't start a thread to run a specific scheduler run loop, but each thread now has its own inner loop that basically switches to a scheduler loop then switches back to its inner loop to sleep.

The benefit of the global thread loop is that threads are kept around instead of being created and thrown away. If you regularly spawn an isolated fiber, it will likely keep reusing the same thread(s). Threads still eventually shutdown after some inactive time (configurable) except for the main thread (we need to keep the main fiber alive).

A potential evolution will park MT threads into the thread pool, instead of keeping them tied to the MT context, so they can be reused by any context that needs parallelism, or to boot a new isolated fiber or ST context.

Extracted to #15885.

NOTES

The isolated context expects to block, so the #syscall(&) method is a NOOP there.

There are probably other blocking syscalls that we might want to consider. For example reading from STDIN on Windows could be greatly simplified.

Another example is flock that is currently retried every 100ms when it doesn't block the current thread. We might want to be able to actively detach a scheduler when calling #syscall(&), so we could try once (non blocking) then on failure detach the scheduler and try again (blocking) without waiting for SYSMON to notice 🤔

The PR contains multiple commits that may be extracted into individual commit (at least the first one). Each commit is focused on one task and it might be easier to read each of them.

POTENTIAL ISSUE

I got one segfault in a gc call nested a libxml2 callback in one early run of the std specs (with -Dpreview_mt -Dexecution_context) but I couldn't reproduce it after fixing different issues in the PR.

Maybe it was a fluke (because of the bugs), or maybe it was just a regular MT issue with libxml2, or maybe sysmon moved the scheduler from the main thread to another thread then resumed a fiber doing something in libxml2, and the global thread local state couldn't be found?

This is the already known MT issue we have with libxml2. What's new is that the segfault might start happening in a ST environment 😢

Closes #15768.