bpo-42340: Document issues around KeyboardInterrupt (GH-23255)

Update documentation to note that in some circumstances,
KeyboardInterrupt may cause code to enter an inconsistent state. Also
document sample workaround to avoid KeyboardInterrupt, if needed.

Author: benfogle

Doc/library/exceptions.rst

@@ -254,6 +254,15 @@ The following exceptions are the exceptions that are usually raised.
    accidentally caught by code that catches :exc:`Exception` and thus prevent
    the interpreter from exiting.
 
+   .. note::
+
+      Catching a :exc:`KeyboardInterrupt` requires special consideration.
+      Because it can be raised at unpredictable points, it may, in some
+      circumstances, leave the running program in an inconsistent state. It is
+      generally best to allow :exc:`KeyboardInterrupt` to end the program as
+      quickly as possible or avoid raising it entirely. (See
+      :ref:`handlers-and-exceptions`.)
+
 
 .. exception:: MemoryError
 

Doc/library/signal.rst

@@ -46,6 +46,9 @@ This has consequences:
   arbitrary amount of time, regardless of any signals received.  The Python
   signal handlers will be called when the calculation finishes.
 
+* If the handler raises an exception, it will be raised "out of thin air" in
+  the main thread. See the :ref:`note below <handlers-and-exceptions>` for a
+  discussion.
 
 .. _signals-and-threads:
 
@@ -712,3 +715,70 @@ Do not set :const:`SIGPIPE`'s disposition to :const:`SIG_DFL`
 in order to avoid :exc:`BrokenPipeError`.  Doing that would cause
 your program to exit unexpectedly also whenever any socket connection
 is interrupted while your program is still writing to it.
+
+.. _handlers-and-exceptions:
+
+Note on Signal Handlers and Exceptions
+--------------------------------------
+
+If a signal handler raises an exception, the exception will be propagated to
+the main thread and may be raised after any :term:`bytecode` instruction. Most
+notably, a :exc:`KeyboardInterrupt` may appear at any point during execution.
+Most Python code, including the standard library, cannot be made robust against
+this, and so a :exc:`KeyboardInterrupt` (or any other exception resulting from
+a signal handler) may on rare occasions put the program in an unexpected state.
+
+To illustrate this issue, consider the following code::
+
+    class SpamContext:
+        def __init__(self):
+            self.lock = threading.Lock()
+
+        def __enter__(self):
+            # If KeyboardInterrupt occurs here, everything is fine
+            self.lock.acquire()
+            # If KeyboardInterrupt occcurs here, __exit__ will not be called
+            ...
+            # KeyboardInterrupt could occur just before the function returns
+
+        def __exit__(self, exc_type, exc_val, exc_tb):
+            ...
+            self.lock.release()
+
+For many programs, especially those that merely want to exit on
+:exc:`KeyboardInterrupt`, this is not a problem, but applications that are
+complex or require high reliability should avoid raising exceptions from signal
+handlers. They should also avoid catching :exc:`KeyboardInterrupt` as a means
+of gracefully shutting down.  Instead, they should install their own
+:const:`SIGINT` handler. Below is an example of an HTTP server that avoids
+:exc:`KeyboardInterrupt`::
+
+    import signal
+    import socket
+    from selectors import DefaultSelector, EVENT_READ
+    from http.server import HTTPServer, SimpleHTTPRequestHandler
+
+    interrupt_read, interrupt_write = socket.socketpair()
+
+    def handler(signum, frame):
+        print('Signal handler called with signal', signum)
+        interrupt_write.send(b'\0')
+    signal.signal(signal.SIGINT, handler)
+
+    def serve_forever(httpd):
+        sel = DefaultSelector()
+        sel.register(interrupt_read, EVENT_READ)
+        sel.register(httpd, EVENT_READ)
+
+        while True:
+            for key, _ in sel.select():
+                if key.fileobj == interrupt_read:
+                    interrupt_read.recv(1)
+                    return
+                if key.fileobj == httpd:
+                    httpd.handle_request()
+
+    print("Serving on port 8000")
+    httpd = HTTPServer(('', 8000), SimpleHTTPRequestHandler)
+    serve_forever(httpd)
+    print("Shutdown...")

Misc/NEWS.d/next/Documentation/2020-11-12-21-26-31.bpo-42340.apumUL.rst

@@ -0,0 +1,3 @@
+Document that in some circumstances :exc:`KeyboardInterrupt` may cause the
+code to enter an inconsistent state. Provided a sample workaround to avoid
+it if needed.