Merge remote-tracking branch 'origin/master' into bpo_39337_new_2

* origin/master: (147 commits)
  Fix the attribute names in the docstring of GenericAlias (pythonGH-22594)
  bpo-39337: Add a test case for normalizing of codec names (pythonGH-19069)
  bpo-41557: Update Windows installer to use SQLite 3.33.0 (pythonGH-21960)
  bpo-41976: Fix the fallback to gcc of ctypes.util.find_library when using gcc>9 (pythonGH-22598)
  bpo-41306: Allow scale value to not be rounded (pythonGH-21715)
  bpo-41970: Avoid test failure in test_lib2to3 if the module is already imported (pythonGH-22595)
  bpo-41376: Fix the documentation of `site.getusersitepackages()` (pythonGH-21602)
  Revert "bpo-26680: Incorporate is_integer in all built-in and standard library numeric types (pythonGH-6121)" (pythonGH-22584)
  bpo-41923: PEP 613: Add TypeAlias to typing module (python#22532)
  Fix comment about PyObject_IsTrue. (pythonGH-22343)
  bpo-38605: Make 'from __future__ import annotations' the default (pythonGH-20434)
  bpo-41905: Add abc.update_abstractmethods() (pythonGH-22485)
  bpo-41944: No longer call eval() on content received via HTTP in the UnicodeNames tests (pythonGH-22575)
  bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (pythonGH-22566)
  Post 3.10.0a1
  Python 3.10.0a1
  bpo-41584: clarify when the reflected method of a binary arithemtic operator is called (python#22505)
  bpo-41939: Fix test_site.test_license_exists_at_url() (python#22559)
  bpo-41774: Tweak new programming FAQ entry (pythonGH-22562)
  bpo-41936. Remove macros Py_ALLOW_RECURSION/Py_END_ALLOW_RECURSION (pythonGH-22552)
  ...

Author: shihai1991

.travis.yml

@@ -51,10 +51,7 @@ matrix:
       env: TESTING=docs
       before_script:
         - cd Doc
-        # Sphinx is pinned so that new versions that introduce new warnings won't suddenly cause build failures.
-        # (Updating the version is fine as long as no warnings are raised by doing so.)
-        # The theme used by the docs is stored separately, so we need to install that as well.
-        - python -m pip install sphinx==2.2.0 blurb python-docs-theme
+        - make venv PYTHON=python
       script:
         - make check suspicious html SPHINXOPTS="-q -W -j4"
     - name: "Documentation tests"

Doc/Makefile

@@ -143,7 +143,7 @@ clean:
 venv:
 	$(PYTHON) -m venv $(VENVDIR)
 	$(VENVDIR)/bin/python3 -m pip install -U pip setuptools
-	$(VENVDIR)/bin/python3 -m pip install -U Sphinx==3.2.1 blurb python-docs-theme
+	$(VENVDIR)/bin/python3 -m pip install -r requirements.txt
 	@echo "The venv has been created in the $(VENVDIR) directory"
 
 dist:

Doc/c-api/codec.rst

@@ -10,6 +10,14 @@ Codec registry and support functions
    As side effect, this tries to load the :mod:`encodings` package, if not yet
    done, to make sure that it is always first in the list of search functions.
 
+.. c:function:: int PyCodec_Unregister(PyObject *search_function)
+
+   Unregister a codec search function and clear the registry's cache.
+   If the search function is not registered, do nothing.
+   Return 0 on success. Raise an exception and return -1 on error.
+
+   .. versionadded:: 3.10
+
 .. c:function:: int PyCodec_KnownEncoding(const char *encoding)
 
    Return ``1`` or ``0`` depending on whether there is a registered codec for

Doc/c-api/datetime.rst

@@ -185,6 +185,11 @@ must not be ``NULL``, and the type is not checked:
 
    Return the microsecond, as an int from 0 through 999999.
 
+.. c:function:: PyObject* PyDateTime_DATE_GET_TZINFO(PyDateTime_DateTime *o)
+
+   Return the tzinfo (which may be ``None``).
+
+   .. versionadded:: 3.10
 
 Macros to extract fields from time objects.  The argument must be an instance of
 :c:data:`PyDateTime_Time`, including subclasses. The argument must not be ``NULL``,
@@ -209,6 +214,12 @@ and the type is not checked:
 
    Return the microsecond, as an int from 0 through 999999.
 
+.. c:function:: PyObject* PyDateTime_TIME_GET_TZINFO(PyDateTime_Time *o)
+
+   Return the tzinfo (which may be ``None``).
+
+   .. versionadded:: 3.10
+
 
 Macros to extract fields from time delta objects.  The argument must be an
 instance of :c:data:`PyDateTime_Delta`, including subclasses. The argument must

Doc/c-api/dict.rst

@@ -81,14 +81,16 @@ Dictionary Objects
 .. c:function:: int PyDict_DelItem(PyObject *p, PyObject *key)
 
    Remove the entry in dictionary *p* with key *key*. *key* must be hashable;
-   if it isn't, :exc:`TypeError` is raised.  Return ``0`` on success or ``-1``
-   on failure.
+   if it isn't, :exc:`TypeError` is raised.
+   If *key* is not in the dictionary, :exc:`KeyError` is raised.
+   Return ``0`` on success or ``-1`` on failure.
 
 
 .. c:function:: int PyDict_DelItemString(PyObject *p, const char *key)
 
-   Remove the entry in dictionary *p* which has a key specified by the string
-   *key*.  Return ``0`` on success or ``-1`` on failure.
+   Remove the entry in dictionary *p* which has a key specified by the string *key*.
+   If *key* is not in the dictionary, :exc:`KeyError` is raised.
+   Return ``0`` on success or ``-1`` on failure.
 
 
 .. c:function:: PyObject* PyDict_GetItem(PyObject *p, PyObject *key)

Doc/c-api/gen.rst

@@ -15,6 +15,11 @@ than explicitly calling :c:func:`PyGen_New` or :c:func:`PyGen_NewWithQualName`.
    The C structure used for generator objects.
 
 
+.. c:type:: PySendResult
+
+   The enum value used to represent different results of :c:func:`PyGen_Send`.
+
+
 .. c:var:: PyTypeObject PyGen_Type
 
    The type object corresponding to generator objects.
@@ -42,3 +47,13 @@ than explicitly calling :c:func:`PyGen_New` or :c:func:`PyGen_NewWithQualName`.
    with ``__name__`` and ``__qualname__`` set to *name* and *qualname*.
    A reference to *frame* is stolen by this function.  The *frame* argument
    must not be ``NULL``.
+
+.. c:function:: PySendResult PyGen_Send(PyGenObject *gen, PyObject *arg, PyObject **presult)
+
+   Sends the *arg* value into the generator *gen*. Coroutine objects
+   are also allowed to be as the *gen* argument but they need to be
+   explicitly casted to PyGenObject*. Returns:
+
+   - ``PYGEN_RETURN`` if generator returns. Return value is returned via *presult*.
+   - ``PYGEN_NEXT`` if generator yields. Yielded value is returned via *presult*.
+   - ``PYGEN_ERROR`` if generator has raised and exception. *presult* is set to ``NULL``.

Doc/data/refcounts.dat

@@ -959,6 +959,11 @@ PyGen_NewWithQualName:PyFrameObject*:frame:0:
 PyGen_NewWithQualName:PyObject*:name:0:
 PyGen_NewWithQualName:PyObject*:qualname:0:
 
+PyGen_Send:int:::
+PyGen_Send:PyGenObject*:gen:0:
+PyGen_Send:PyObject*:arg:0:
+PyGen_Send:PyObject**:presult:+1:
+
 PyCoro_CheckExact:int:::
 PyCoro_CheckExact:PyObject*:ob:0:
 

Doc/faq/programming.rst

@@ -1164,6 +1164,21 @@ This converts the list into a set, thereby removing duplicates, and then back
 into a list.
 
 
+How do you remove multiple items from a list
+--------------------------------------------
+
+As with removing duplicates, explicitly iterating in reverse with a
+delete condition is one possibility.  However, it is easier and faster
+to use slice replacement with an implicit or explicit forward iteration.
+Here are three variations.::
+
+   mylist[:] = filter(keep_function, mylist)
+   mylist[:] = (x for x in mylist if keep_condition)
+   mylist[:] = [x for x in mylist if keep_condition]
+
+The list comprehension may be fastest.
+
+
 How do you make an array in Python?
 -----------------------------------
 
@@ -1176,7 +1191,7 @@ difference is that a Python list can contain objects of many different types.
 
 The ``array`` module also provides methods for creating arrays of fixed types
 with compact representations, but they are slower to index than lists.  Also
-note that the Numeric extensions and others define array-like structures with
+note that NumPy and other third party packages define array-like structures with
 various characteristics as well.
 
 To get Lisp-style linked lists, you can emulate cons cells using tuples::
@@ -1504,20 +1519,19 @@ Most :meth:`__setattr__` implementations must modify ``self.__dict__`` to store
 local state for self without causing an infinite recursion.
 
 
-How do I call a method defined in a base class from a derived class that overrides it?
---------------------------------------------------------------------------------------
+How do I call a method defined in a base class from a derived class that extends it?
+------------------------------------------------------------------------------------
 
 Use the built-in :func:`super` function::
 
    class Derived(Base):
        def meth(self):
-           super(Derived, self).meth()
+           super().meth()  # calls Base.meth
 
-For version prior to 3.0, you may be using classic classes: For a class
-definition such as ``class Derived(Base): ...`` you can call method ``meth()``
-defined in ``Base`` (or one of ``Base``'s base classes) as ``Base.meth(self,
-arguments...)``.  Here, ``Base.meth`` is an unbound method, so you need to
-provide the ``self`` argument.
+In the example, :func:`super` will automatically determine the instance from
+which it was called (the ``self`` value), look up the :term:`method resolution
+order` (MRO) with ``type(self).__mro__``, and return the next in line after
+``Derived`` in the MRO: ``Base``.
 
 
 How can I organize my code to make it easier to change the base class?

Doc/glossary.rst

@@ -1084,19 +1084,15 @@ Glossary
       Type aliases are useful for simplifying :term:`type hints <type hint>`.
       For example::
 
-         from typing import List, Tuple
-
          def remove_gray_shades(
-                 colors: List[Tuple[int, int, int]]) -> List[Tuple[int, int, int]]:
+                 colors: list[tuple[int, int, int]]) -> list[tuple[int, int, int]]:
              pass
 
       could be made more readable like this::
 
-         from typing import List, Tuple
-
-         Color = Tuple[int, int, int]
+         Color = tuple[int, int, int]
 
-         def remove_gray_shades(colors: List[Color]) -> List[Color]:
+         def remove_gray_shades(colors: list[Color]) -> list[Color]:
              pass
 
       See :mod:`typing` and :pep:`484`, which describe this functionality.

Doc/library/abc.rst

@@ -174,10 +174,11 @@ The :mod:`abc` module also provides the following decorator:
    to declare abstract methods for properties and descriptors.
 
    Dynamically adding abstract methods to a class, or attempting to modify the
-   abstraction status of a method or class once it is created, are not
-   supported.  The :func:`abstractmethod` only affects subclasses derived using
-   regular inheritance; "virtual subclasses" registered with the ABC's
-   :meth:`register` method are not affected.
+   abstraction status of a method or class once it is created, are only
+   supported using the :func:`update_abstractmethods` function.  The
+   :func:`abstractmethod` only affects subclasses derived using regular
+   inheritance; "virtual subclasses" registered with the ABC's :meth:`register`
+   method are not affected.
 
    When :func:`abstractmethod` is applied in combination with other method
    descriptors, it should be applied as the innermost decorator, as shown in
@@ -235,7 +236,6 @@ The :mod:`abc` module also provides the following decorator:
       super-call in a framework that uses cooperative
       multiple-inheritance.
 
-
 The :mod:`abc` module also supports the following legacy decorators:
 
 .. decorator:: abstractclassmethod
@@ -335,6 +335,22 @@ The :mod:`abc` module also provides the following functions:
 
    .. versionadded:: 3.4
 
+.. function:: update_abstractmethods(cls)
+   A function to recalculate an abstract class's abstraction status. This
+   function should be called if a class's abstract methods have been
+   implemented or changed after it was created. Usually, this function should
+   be called from within a class decorator.
+
+   Returns *cls*, to allow usage as a class decorator.
+
+   If *cls* is not an instance of ABCMeta, does nothing.
+
+   .. note::
+
+      This function assumes that *cls*'s superclasses are already updated.
+      It does not update any subclasses.
+
+   .. versionadded:: 3.10
 
 .. rubric:: Footnotes
 

Doc/library/array.rst

@@ -257,7 +257,6 @@ Examples::
       Packing and unpacking of External Data Representation (XDR) data as used in some
       remote procedure call systems.
 
-   `The Numerical Python Documentation <https://docs.scipy.org/doc/>`_
-      The Numeric Python extension (NumPy) defines another array type; see
-      http://www.numpy.org/ for further information about Numerical Python.
+   `NumPy <https://numpy.org/>`_
+      The NumPy package defines another array type.
 

Doc/library/ast.rst

@@ -1586,6 +1586,9 @@ and classes for traversing abstract syntax trees:
    .. versionchanged:: 3.9
       Now supports creating empty sets with ``'set()'``.
 
+   .. versionchanged:: 3.10
+      For string inputs, leading spaces and tabs are now stripped.
+
 
 .. function:: get_docstring(node, clean=True)
 
@@ -1820,4 +1823,4 @@ to stdout.  Otherwise, the content is read from stdin.
     `Parso <https://parso.readthedocs.io>`_ is a Python parser that supports
     error recovery and round-trip parsing for different Python versions (in
     multiple Python versions). Parso is also able to list multiple syntax errors
-    in your python file.
+    in your python file.

Doc/library/codecs.rst

@@ -163,11 +163,14 @@ function:
    :class:`CodecInfo` object. In case a search function cannot find
    a given encoding, it should return ``None``.
 
-   .. note::
 
-      Search function registration is not currently reversible,
-      which may cause problems in some cases, such as unit testing or
-      module reloading.
+.. function:: unregister(search_function)
+
+   Unregister a codec search function and clear the registry's cache.
+   If the search function is not registered, do nothing.
+
+   .. versionadded:: 3.10
+
 
 While the builtin :func:`open` and the associated :mod:`io` module are the
 recommended approach for working with encoded text files, this module

Doc/library/configparser.rst

@@ -135,6 +135,30 @@ involves the ``DEFAULT`` section which provides default values for all other
 sections [1]_.  Note also that keys in sections are
 case-insensitive and stored in lowercase [1]_.
 
+It is possible to read several configurations into a single
+:class:`ConfigParser`, where the most recently added configuration has the
+highest priority. Any conflicting keys are taken from the more recent
+configuration while the previously existing keys are retained.
+
+.. doctest::
+
+   >>> another_config = configparser.ConfigParser()
+   >>> another_config.read('example.ini')
+   ['example.ini']
+   >>> another_config['topsecret.server.com']['Port']
+   '50022'
+   >>> another_config.read_string("[topsecret.server.com]\nPort=48484")
+   >>> another_config['topsecret.server.com']['Port']
+   '48484'
+   >>> another_config.read_dict({"topsecret.server.com": {"Port": 21212}})
+   >>> another_config['topsecret.server.com']['Port']
+   '21212'
+   >>> another_config['topsecret.server.com']['ForwardX11']
+   'no'
+
+This behaviour is equivalent to a :meth:`ConfigParser.read` call with several
+files passed to the *filenames* parameter.
+
 
 Supported Datatypes
 -------------------

Doc/library/constants.rst

@@ -19,19 +19,21 @@ A small number of constants live in the built-in namespace.  They are:
 
 .. data:: None
 
-   The sole value of the type ``NoneType``.  ``None`` is frequently used to
-   represent the absence of a value, as when default arguments are not passed to a
-   function. Assignments to ``None`` are illegal and raise a :exc:`SyntaxError`.
+   An object frequently used to represent the absence of a value, as when
+   default arguments are not passed to a function. Assignments to ``None``
+   are illegal and raise a :exc:`SyntaxError`.
+   ``None`` is the sole instance of the :data:`NoneType` type.
 
 
 .. data:: NotImplemented
 
-   Special value which should be returned by the binary special methods
+   A special value which should be returned by the binary special methods
    (e.g. :meth:`__eq__`, :meth:`__lt__`, :meth:`__add__`, :meth:`__rsub__`,
    etc.) to indicate that the operation is not implemented with respect to
    the other type; may be returned by the in-place binary special methods
    (e.g. :meth:`__imul__`, :meth:`__iand__`, etc.) for the same purpose.
    It should not be evaluated in a boolean context.
+   ``NotImplemented`` is the sole instance of the :data:`types.NotImplementedType` type.
 
    .. note::
 
@@ -59,8 +61,9 @@ A small number of constants live in the built-in namespace.  They are:
 .. index:: single: ...; ellipsis literal
 .. data:: Ellipsis
 
-   The same as the ellipsis literal "``...``".  Special value used mostly in conjunction
+   The same as the ellipsis literal "``...``". Special value used mostly in conjunction
    with extended slicing syntax for user-defined container data types.
+   ``Ellipsis`` is the sole instance of the :data:`types.EllipsisType` type.
 
 
 .. data:: __debug__

Doc/library/dataclasses.rst

@@ -188,7 +188,7 @@ Module-level decorators, classes, and functions
 
      @dataclass
      class C:
-         mylist: List[int] = field(default_factory=list)
+         mylist: list[int] = field(default_factory=list)
 
      c = C()
      c.mylist += [1, 2, 3]
@@ -301,7 +301,7 @@ Module-level decorators, classes, and functions
 
      @dataclass
      class C:
-          mylist: List[Point]
+          mylist: list[Point]
 
      p = Point(10, 20)
      assert asdict(p) == {'x': 10, 'y': 20}

Doc/library/difflib.rst

@@ -18,8 +18,8 @@
 --------------
 
 This module provides classes and functions for comparing sequences. It
-can be used for example, for comparing files, and can produce difference
-information in various formats, including HTML and context and unified
+can be used for example, for comparing files, and can produce information
+about file differences in various formats, including HTML and context and unified
 diffs. For comparing directories and files, see also, the :mod:`filecmp` module.