Merge branch 'main' into setobject-note

Author: iritkatriel

Doc/c-api/init.rst

@@ -513,7 +513,7 @@ Process-wide parameters
    program name is ``'/usr/local/bin/python'``, the prefix is ``'/usr/local'``. The
    returned string points into static storage; the caller should not modify its
    value.  This corresponds to the :makevar:`prefix` variable in the top-level
-   :file:`Makefile` and the ``--prefix`` argument to the :program:`configure`
+   :file:`Makefile` and the :option:`--prefix` argument to the :program:`configure`
    script at build time.  The value is available to Python code as ``sys.prefix``.
    It is only useful on Unix.  See also the next function.
 

Doc/c-api/intro.rst

@@ -78,19 +78,19 @@ used by extension writers. Structure member names do not have a reserved prefix.
 
 The header files are typically installed with Python.  On Unix, these  are
 located in the directories :file:`{prefix}/include/pythonversion/` and
-:file:`{exec_prefix}/include/pythonversion/`, where :envvar:`prefix` and
-:envvar:`exec_prefix` are defined by the corresponding parameters to Python's
+:file:`{exec_prefix}/include/pythonversion/`, where :option:`prefix <--prefix>` and
+:option:`exec_prefix <--exec-prefix>` are defined by the corresponding parameters to Python's
 :program:`configure` script and *version* is
 ``'%d.%d' % sys.version_info[:2]``.  On Windows, the headers are installed
-in :file:`{prefix}/include`, where :envvar:`prefix` is the installation
+in :file:`{prefix}/include`, where ``prefix`` is the installation
 directory specified to the installer.
 
 To include the headers, place both directories (if different) on your compiler's
 search path for includes.  Do *not* place the parent directories on the search
 path and then use ``#include <pythonX.Y/Python.h>``; this will break on
 multi-platform builds since the platform independent headers under
-:envvar:`prefix` include the platform specific headers from
-:envvar:`exec_prefix`.
+:option:`prefix <--prefix>` include the platform specific headers from
+:option:`exec_prefix <--exec-prefix>`.
 
 C++ users should note that although the API is defined entirely using C, the
 header files properly declare the entry points to be ``extern "C"``. As a result,

Doc/library/sys.rst

@@ -1323,7 +1323,7 @@ always available.
 
    A string giving the site-specific directory prefix where the platform
    independent Python files are installed; on Unix, the default is
-   ``'/usr/local'``.  This can be set at build time with the ``--prefix``
+   :file:`/usr/local`. This can be set at build time with the :option:`--prefix`
    argument to the :program:`configure` script.  See
    :ref:`installation_paths` for derived paths.
 

Doc/library/venv.rst

@@ -61,7 +61,7 @@ running from a virtual environment.
 A virtual environment may be "activated" using a script in its binary directory
 (``bin`` on POSIX; ``Scripts`` on Windows).
 This will prepend that directory to your :envvar:`!PATH`, so that running
-:program:`!python` will invoke the environment's Python interpreter
+:program:`python` will invoke the environment's Python interpreter
 and you can run installed scripts without having to use their full path.
 The invocation of the activation script is platform-specific
 (:samp:`{<venv>}` must be replaced by the path to the directory
@@ -84,7 +84,7 @@ containing the virtual environment):
 +-------------+------------+--------------------------------------------------+
 
 .. versionadded:: 3.4
-   :program:`!fish` and :program:`!csh` activation scripts.
+   :program:`fish` and :program:`csh` activation scripts.
 
 .. versionadded:: 3.8
    PowerShell activation scripts installed under POSIX for PowerShell Core

Doc/tutorial/modules.rst

@@ -438,7 +438,7 @@ When importing the package, Python searches through the directories on
 
 The :file:`__init__.py` files are required to make Python treat directories
 containing the file as packages.  This prevents directories with a common name,
-such as ``string``, unintentionally hiding valid modules that occur later
+such as ``string``, from unintentionally hiding valid modules that occur later
 on the module search path. In the simplest case, :file:`__init__.py` can just be
 an empty file, but it can also execute initialization code for the package or
 set the ``__all__`` variable, described later.

Doc/using/configure.rst

@@ -216,6 +216,22 @@ WebAssembly Options
 Install Options
 ---------------
 
+.. cmdoption:: --prefix=PREFIX
+
+   Install architecture-independent files in PREFIX. On Unix, it
+   defaults to :file:`/usr/local`.
+
+   This value can be retrived at runtime using :data:`sys.prefix`.
+
+   As an example, one can use ``--prefix="$HOME/.local/"`` to install
+   a Python in its home directory.
+
+.. cmdoption:: --exec-prefix=EPREFIX
+
+   Install architecture-dependent files in EPREFIX, defaults to :option:`--prefix`.
+
+   This value can be retrived at runtime using :data:`sys.exec_prefix`.
+
 .. cmdoption:: --disable-test-modules
 
    Don't build nor install test modules, like the :mod:`test` package or the

Doc/using/unix.rst

@@ -93,7 +93,7 @@ Python-related paths and files
 ==============================
 
 These are subject to difference depending on local installation conventions;
-:envvar:`prefix` (``${prefix}``) and :envvar:`exec_prefix` (``${exec_prefix}``)
+:option:`prefix <--prefix>` and :option:`exec_prefix <--exec-prefix>`
 are installation-dependent and should be interpreted as for GNU software; they
 may be the same.
 

Doc/whatsnew/3.12.rst

@@ -982,6 +982,11 @@ Porting to Python 3.12
   effects, these side effects are no longer duplicated.
   (Contributed by Victor Stinner in :gh:`98724`.)
 
+* The interpreter's error indicator is now always normalized. This means
+  that :c:func:`PyErr_SetObject`, :c:func:`PyErr_SetString` and the other
+  functions that set the error indicator now normalize the exception
+  before storing it. (Contributed by Mark Shannon in :gh:`101578`.)
+
 Deprecated
 ----------
 

Lib/asyncio/taskgroups.py

@@ -10,7 +10,21 @@
 
 
 class TaskGroup:
+    """Asynchronous context manager for managing groups of tasks.
 
+    Example use:
+
+        async with asyncio.TaskGroup() as group:
+            task1 = group.create_task(some_coroutine(...))
+            task2 = group.create_task(other_coroutine(...))
+        print("Both tasks have completed now.")
+
+    All tasks are awaited when the context manager exits.
+
+    Any exceptions other than `asyncio.CancelledError` raised within
+    a task will cancel all remaining tasks and wait for them to exit.
+    The exceptions are then combined and raised as an `ExceptionGroup`.
+    """
     def __init__(self):
         self._entered = False
         self._exiting = False
@@ -135,6 +149,10 @@ async def __aexit__(self, et, exc, tb):
                 self._errors = None
 
     def create_task(self, coro, *, name=None, context=None):
+        """Create a new task in this group and return it.
+
+        Similar to `asyncio.create_task`.
+        """
         if not self._entered:
             raise RuntimeError(f"TaskGroup {self!r} has not been entered")
         if self._exiting and not self._tasks:

Lib/test/test_calendar.py

@@ -568,11 +568,15 @@ def test_locale_calendar_formatweekday(self):
         try:
             # formatweekday uses different day names based on the available width.
             cal = calendar.LocaleTextCalendar(locale='en_US')
+            # For really short widths, the abbreviated name is truncated.
+            self.assertEqual(cal.formatweekday(0, 1), "M")
+            self.assertEqual(cal.formatweekday(0, 2), "Mo")
             # For short widths, a centered, abbreviated name is used.
+            self.assertEqual(cal.formatweekday(0, 3), "Mon")
             self.assertEqual(cal.formatweekday(0, 5), " Mon ")
-            # For really short widths, even the abbreviated name is truncated.
-            self.assertEqual(cal.formatweekday(0, 2), "Mo")
+            self.assertEqual(cal.formatweekday(0, 8), "  Mon   ")
             # For long widths, the full day name is used.
+            self.assertEqual(cal.formatweekday(0, 9), "  Monday ")
             self.assertEqual(cal.formatweekday(0, 10), "  Monday  ")
         except locale.Error:
             raise unittest.SkipTest('cannot set the en_US locale')

Lib/test/test_enum.py

@@ -4579,15 +4579,14 @@ class Double(Enum):
             TWO = 2
         self.assertEqual(Double.__doc__, None)
 
-
-    def test_doc_1(self):
+    def test_doc_3(self):
         class Triple(Enum):
             ONE = 1
             TWO = 2
             THREE = 3
         self.assertEqual(Triple.__doc__, None)
 
-    def test_doc_1(self):
+    def test_doc_4(self):
         class Quadruple(Enum):
             ONE = 1
             TWO = 2

Lib/test/test_os.py

@@ -2683,12 +2683,17 @@ def test_listvolumes(self):
 
     def test_listmounts(self):
         for volume in os.listvolumes():
-            mounts = os.listmounts(volume)
-            self.assertIsInstance(mounts, list)
-            self.assertSetEqual(
-                set(mounts),
-                self.known_mounts & set(mounts),
-            )
+            try:
+                mounts = os.listmounts(volume)
+            except OSError as ex:
+                if support.verbose:
+                    print("Skipping", volume, "because of", ex)
+            else:
+                self.assertIsInstance(mounts, list)
+                self.assertSetEqual(
+                    set(mounts),
+                    self.known_mounts & set(mounts),
+                )
 
 
 @unittest.skipUnless(hasattr(os, 'readlink'), 'needs os.readlink()')

Lib/test/test_typing.py

@@ -3809,6 +3809,51 @@ class Y(C[int]):
         self.assertEqual(Y.__qualname__,
                          'GenericTests.test_repr_2.<locals>.Y')
 
+    def test_repr_3(self):
+        T = TypeVar('T')
+        T1 = TypeVar('T1')
+        P = ParamSpec('P')
+        P2 = ParamSpec('P2')
+        Ts = TypeVarTuple('Ts')
+
+        class MyCallable(Generic[P, T]):
+            pass
+
+        class DoubleSpec(Generic[P, P2, T]):
+            pass
+
+        class TsP(Generic[*Ts, P]):
+            pass
+
+        object_to_expected_repr = {
+            MyCallable[P, T]:                         "MyCallable[~P, ~T]",
+            MyCallable[Concatenate[T1, P], T]:        "MyCallable[typing.Concatenate[~T1, ~P], ~T]",
+            MyCallable[[], bool]:                     "MyCallable[[], bool]",
+            MyCallable[[int], bool]:                  "MyCallable[[int], bool]",
+            MyCallable[[int, str], bool]:             "MyCallable[[int, str], bool]",
+            MyCallable[[int, list[int]], bool]:       "MyCallable[[int, list[int]], bool]",
+            MyCallable[Concatenate[*Ts, P], T]:       "MyCallable[typing.Concatenate[*Ts, ~P], ~T]",
+
+            DoubleSpec[P2, P, T]:                     "DoubleSpec[~P2, ~P, ~T]",
+            DoubleSpec[[int], [str], bool]:           "DoubleSpec[[int], [str], bool]",
+            DoubleSpec[[int, int], [str, str], bool]: "DoubleSpec[[int, int], [str, str], bool]",
+
+            TsP[*Ts, P]:                              "TsP[*Ts, ~P]",
+            TsP[int, str, list[int], []]:             "TsP[int, str, list[int], []]",
+            TsP[int, [str, list[int]]]:               "TsP[int, [str, list[int]]]",
+
+            # These lines are just too long to fit:
+            MyCallable[Concatenate[*Ts, P], int][int, str, [bool, float]]:
+                                                      "MyCallable[[int, str, bool, float], int]",
+        }
+
+        for obj, expected_repr in object_to_expected_repr.items():
+            with self.subTest(obj=obj, expected_repr=expected_repr):
+                self.assertRegex(
+                    repr(obj),
+                    fr"^{re.escape(MyCallable.__module__)}.*\.{re.escape(expected_repr)}$",
+                )
+
     def test_eq_1(self):
         self.assertEqual(Generic, Generic)
         self.assertEqual(Generic[T], Generic[T])

Lib/typing.py

@@ -230,16 +230,17 @@ def _type_repr(obj):
     typically enough to uniquely identify a type.  For everything
     else, we fall back on repr(obj).
     """
-    if isinstance(obj, types.GenericAlias):
-        return repr(obj)
     if isinstance(obj, type):
         if obj.__module__ == 'builtins':
             return obj.__qualname__
         return f'{obj.__module__}.{obj.__qualname__}'
     if obj is ...:
-        return('...')
+        return '...'
     if isinstance(obj, types.FunctionType):
         return obj.__name__
+    if isinstance(obj, tuple):
+        # Special case for `repr` of types with `ParamSpec`:
+        return '[' + ', '.join(_type_repr(t) for t in obj) + ']'
     return repr(obj)
 
 

Makefile.pre.in

@@ -1485,6 +1485,7 @@ regen-cases:
 	PYTHONPATH=$(srcdir)/Tools/cases_generator \
 	$(PYTHON_FOR_REGEN) \
 	    $(srcdir)/Tools/cases_generator/generate_cases.py \
+		--emit-line-directives \
 		-o $(srcdir)/Python/generated_cases.c.h.new \
 		-m $(srcdir)/Python/opcode_metadata.h.new \
 		$(srcdir)/Python/bytecodes.c

Misc/NEWS.d/3.12.0a2.rst

@@ -959,7 +959,7 @@ Fix ``make regen-test-levenshtein`` for out-of-tree builds.
 
 Don't use vendored ``libmpdec`` headers if :option:`--with-system-libmpdec`
 is passed to :program:`configure`. Don't use vendored ``libexpat`` headers
-if :option:`--with-system-expat` is passed to :program:`!configure`.
+if :option:`--with-system-expat` is passed to :program:`configure`.
 
 ..
 

Misc/NEWS.d/next/Core and Builtins/2023-03-02-13-49-21.gh-issue-102281.QCuu2N.rst

@@ -0,0 +1 @@
+Fix potential nullptr dereference and use of uninitialized memory in fileutils. Patch by Max Bachmann.

Misc/NEWS.d/next/Library/2023-03-13-12-05-55.gh-issue-102615.NcA_ZL.rst

@@ -0,0 +1,3 @@
+Typing: Improve the ``repr`` of generic aliases for classes generic over a
+:class:`~typing.ParamSpec`. (Use square brackets to represent a parameter
+list.)

Misc/NEWS.d/next/Tests/2023-01-27-18-10-40.gh-issue-101377.IJGpqh.rst

@@ -0,0 +1 @@
+Improved test_locale_calendar_formatweekday of calendar.

Modules/getpath.c

@@ -446,7 +446,10 @@ getpath_realpath(PyObject *Py_UNUSED(self) , PyObject *args)
             if (s) {
                 *s = L'\0';
             }
-            path2 = _Py_normpath(_Py_join_relfile(path, resolved), -1);
+            path2 = _Py_join_relfile(path, resolved);
+            if (path2) {
+                path2 = _Py_normpath(path2, -1);
+            }
             PyMem_RawFree((void *)path);
             path = path2;
         }

Python/fileutils.c

@@ -2233,7 +2233,10 @@ _Py_join_relfile(const wchar_t *dirname, const wchar_t *relfile)
     }
     assert(wcslen(dirname) < MAXPATHLEN);
     assert(wcslen(relfile) < MAXPATHLEN - wcslen(dirname));
-    join_relfile(filename, bufsize, dirname, relfile);
+    if (join_relfile(filename, bufsize, dirname, relfile) < 0) {
+        PyMem_RawFree(filename);
+        return NULL;
+    }
     return filename;
 }
 
@@ -2271,6 +2274,7 @@ _Py_find_basename(const wchar_t *filename)
 wchar_t *
 _Py_normpath(wchar_t *path, Py_ssize_t size)
 {
+    assert(path != NULL);
     if (!path[0] || size == 0) {
         return path;
     }