[3.8] bpo-40826: Fix GIL usage in PyOS_Readline() (GH-20613)

[vstinner]

• bpo-40826: Fix GIL usage in PyOS_Readline() (bpo-40826: Fix GIL usage in PyOS_Readline() #20579)

Fix GIL usage in PyOS_Readline(): lock the GIL to set an exception.

Pass tstate to my_fgets() and _PyOS_WindowsConsoleReadline(). Cleanup
these functions.

(cherry picked from commit c353764)

• bpo-40826: Add _PyOS_InterruptOccurred(tstate) function (bpo-40826: Add _PyOS_InterruptOccurred(tstate) function #20599)

my_fgets() now calls _PyOS_InterruptOccurred(tstate) to check for
pending signals, rather calling PyOS_InterruptOccurred().

my_fgets() is called with the GIL released, whereas
PyOS_InterruptOccurred() must be called with the GIL held.

test_repl: use text=True and avoid SuppressCrashReport in
test_multiline_string_parsing().

Fix my_fgets() on Windows: fgets(fp) does crash if fileno(fp) is closed.

(cherry picked from commit fa7ab6a)

https://bugs.python.org/issue40826

[vstinner]

I validated manually on Linux and Windows that this change fix https://bugs.python.org/issue40826 using the interactive REPL: type import os; os.close(0), Python exits without crashing.

I made the bug more likely with this local patch:

diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
index 119fc355ff..8e4998d459 100644
--- a/Modules/signalmodule.c
+++ b/Modules/signalmodule.c
@@ -1737,11 +1737,12 @@ PyOS_FiniInterrupts(void)
 int
 _PyOS_InterruptOccurred(PyThreadState *tstate)
 {
+    _PyRuntimeState *runtime = &_PyRuntime;
+    if (!is_main_interp(runtime, tstate->interp)) {
+        return 0;
+    }
+
     if (_Py_atomic_load_relaxed(&Handlers[SIGINT].tripped)) {
-        _PyRuntimeState *runtime = &_PyRuntime;
-        if (!is_main_interp(runtime, tstate->interp)) {
-            return 0;
-        }
         _Py_atomic_store_relaxed(&Handlers[SIGINT].tripped, 0);
         return 1;
     }

I also reproduced https://bugs.python.org/issue40826 crash using this patch (on top of this change):

diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
index 119fc355ff..b1d35d8c71 100644
--- a/Modules/signalmodule.c
+++ b/Modules/signalmodule.c
@@ -1737,11 +1737,12 @@ PyOS_FiniInterrupts(void)
 int
 _PyOS_InterruptOccurred(PyThreadState *tstate)
 {
+    _PyRuntimeState *runtime = &_PyRuntime;
+    if (!is_main(runtime)) {
+        return 0;
+    }
+
     if (_Py_atomic_load_relaxed(&Handlers[SIGINT].tripped)) {
-        _PyRuntimeState *runtime = &_PyRuntime;
-        if (!is_main_interp(runtime, tstate->interp)) {
-            return 0;
-        }
         _Py_atomic_store_relaxed(&Handlers[SIGINT].tripped, 0);
         return 1;
     }

With these tests, I confirm that this change fix https://bugs.python.org/issue40826

[vstinner]

The PR has a NEWS entry, but the bot checking for NEWS looks sick. So I just added "skip news" label :-p