Open special files in another thread #15768
Conversation
A bare thread doesn't have an execution context, yet may be capable to call Fiber#enqueue to enqueue a fiber back into its original context.
We can't reliably detect without significant overhead whether the file at *path* might block for a while before calling open; while O_NONBLOCK has no effect on regular disk files, special file types are a different story. Open with O_NONBLOCK will fail with ENXIO for O_WRONLY (no connected reader) but it will always succeed for O_RDONLY (regardless of a connected writer or not), then any attempt to read will return EOF, leaving no means to wait until a writer connects. We thus rely on the *blocking* arg: when false the file might be a special file type, so we check it; if it's a fifo (named pipe) or a character device, we open in another thread so we don't risk blocking the current thread (and thus other fibers) until a reader or writer is also connected. We need preview_mt to safely re-enqueue the current fiber from the thread.
| @@ -14,12 +14,37 @@ module Crystal::System::File | |||
| end | |||
| end | |||
|
|
|||
| {% if flag?(:preview_mt) && !flag?(:interpreted) %} | |||
| protected def self.async_open(filename, flags, permissions) | |||
There was a problem hiding this comment.
polish: "async" seems a big ambiguous in the context of the event loop. The method name could express better that this is about parallel open in a different thread. Maybe .open_threaded?
|
After talking with @straight-shoota this PR might be a bit rushed/shallow. We should take execution contexts into account.
That would allow a more general and useful solution. For example we could wrap |
We can't detect without significant overhead whether the file at path might block for a while before calling open.
While
O_NONBLOCKhas no effect on regular disk files, special file types are a different story. Open withO_NONBLOCKwill fail withENXIOforO_WRONLY(no connected reader) but it will always succeed forO_RDONLY(regardless of a connected writer or not), then any attempt to read will return EOF, leaving no means to wait until a writer connects.I thus rely on the blocking arg: when
nilthe file might be a special file type, so I check it's type, whenfalseI consider it's a special file. If the file is a fifo (named pipe) or a character device, I open it in another thread to not risk blocking the current thread (and thus other fibers) until a reader or writer is also connected.Opening regular files is unaffected: we still open directly, while opening a special file will now avoid blocking the current thread.
NOTE: we need the
preview_mtflag to safely re-enqueue the current fiber from the bare thread.Depends on #15754 and #15767.