Merge branch 'master' into disallow-yield-in-comprehencions

Author: serhiy-storchaka

Doc/c-api/init.rst

@@ -58,8 +58,9 @@ The following functions can be safely called before Python is initialized:
 
    The following functions **should not be called** before
    :c:func:`Py_Initialize`: :c:func:`Py_EncodeLocale`, :c:func:`Py_GetPath`,
-   :c:func:`Py_GetPrefix`, :c:func:`Py_GetExecPrefix` and
-   :c:func:`Py_GetProgramFullPath` and :c:func:`Py_GetPythonHome`.
+   :c:func:`Py_GetPrefix`, :c:func:`Py_GetExecPrefix`,
+   :c:func:`Py_GetProgramFullPath`, :c:func:`Py_GetPythonHome` and
+   :c:func:`PyEval_InitThreads`.
 
 
 .. _global-conf-vars:

Doc/c-api/memory.rst

@@ -100,9 +100,10 @@ The following function sets are wrappers to the system allocator. These
 functions are thread-safe, the :term:`GIL <global interpreter lock>` does not
 need to be held.
 
-The default raw memory block allocator uses the following functions:
-:c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` and :c:func:`free`; call
-``malloc(1)`` (or ``calloc(1, 1)``) when requesting zero bytes.
+The :ref:`default raw memory allocator <default-memory-allocators>` uses
+the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc`
+and :c:func:`free`; call ``malloc(1)`` (or ``calloc(1, 1)``) when requesting
+zero bytes.
 
 .. versionadded:: 3.4
 
@@ -165,7 +166,8 @@ The following function sets, modeled after the ANSI C standard, but specifying
 behavior when requesting zero bytes, are available for allocating and releasing
 memory from the Python heap.
 
-By default, these functions use :ref:`pymalloc memory allocator <pymalloc>`.
+The :ref:`default memory allocator <default-memory-allocators>` uses the
+:ref:`pymalloc memory allocator <pymalloc>`.
 
 .. warning::
 
@@ -270,7 +272,8 @@ The following function sets, modeled after the ANSI C standard, but specifying
 behavior when requesting zero bytes, are available for allocating and releasing
 memory from the Python heap.
 
-By default, these functions use :ref:`pymalloc memory allocator <pymalloc>`.
+The :ref:`default object allocator <default-memory-allocators>` uses the
+:ref:`pymalloc memory allocator <pymalloc>`.
 
 .. warning::
 
@@ -326,6 +329,31 @@ By default, these functions use :ref:`pymalloc memory allocator <pymalloc>`.
    If *p* is *NULL*, no operation is performed.
 
 
+.. _default-memory-allocators:
+
+Default Memory Allocators
+=========================
+
+Default memory allocators:
+
+===============================  ====================  ==================  =====================  ====================
+Configuration                    Name                  PyMem_RawMalloc     PyMem_Malloc           PyObject_Malloc
+===============================  ====================  ==================  =====================  ====================
+Release build                    ``"pymalloc"``        ``malloc``          ``pymalloc``           ``pymalloc``
+Debug build                      ``"pymalloc_debug"``  ``malloc`` + debug  ``pymalloc`` + debug   ``pymalloc`` + debug
+Release build, without pymalloc  ``"malloc"``          ``malloc``          ``malloc``             ``malloc``
+Release build, without pymalloc  ``"malloc_debug"``    ``malloc`` + debug  ``malloc`` + debug     ``malloc`` + debug
+===============================  ====================  ==================  =====================  ====================
+
+Legend:
+
+* Name: value for :envvar:`PYTHONMALLOC` environment variable
+* ``malloc``: system allocators from the standard C library, C functions:
+  :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` and :c:func:`free`
+* ``pymalloc``: :ref:`pymalloc memory allocator <pymalloc>`
+* "+ debug": with debug hooks installed by :c:func:`PyMem_SetupDebugHooks`
+
+
 Customize Memory Allocators
 ===========================
 
@@ -431,7 +459,8 @@ Customize Memory Allocators
    displayed if :mod:`tracemalloc` is tracing Python memory allocations and the
    memory block was traced.
 
-   These hooks are installed by default if Python is compiled in debug
+   These hooks are :ref:`installed by default <default-memory-allocators>` if
+   Python is compiled in debug
    mode. The :envvar:`PYTHONMALLOC` environment variable can be used to install
    debug hooks on a Python compiled in release mode.
 
@@ -453,9 +482,9 @@ to 512 bytes) with a short lifetime. It uses memory mappings called "arenas"
 with a fixed size of 256 KiB. It falls back to :c:func:`PyMem_RawMalloc` and
 :c:func:`PyMem_RawRealloc` for allocations larger than 512 bytes.
 
-*pymalloc* is the default allocator of the :c:data:`PYMEM_DOMAIN_MEM` (ex:
-:c:func:`PyMem_Malloc`) and :c:data:`PYMEM_DOMAIN_OBJ` (ex:
-:c:func:`PyObject_Malloc`) domains.
+*pymalloc* is the :ref:`default allocator <default-memory-allocators>` of the
+:c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) and
+:c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) domains.
 
 The arena allocator uses the following functions:
 

Doc/includes/shoddy.c

@@ -17,7 +17,7 @@ Shoddy_increment(Shoddy *self, PyObject *unused)
 static PyMethodDef Shoddy_methods[] = {
     {"increment", (PyCFunction)Shoddy_increment, METH_NOARGS,
      PyDoc_STR("increment state counter")},
-    {NULL,	NULL},
+    {NULL},
 };
 
 static int

Doc/library/2to3.rst

@@ -351,7 +351,7 @@ and off individually.  They are described here in more detail.
    ==================================  =============================================
    From                                To
    ==================================  =============================================
-   ``operator.isCallable(obj)``        ``hasattr(obj, '__call__')``
+   ``operator.isCallable(obj)``        ``callable(obj)``
    ``operator.sequenceIncludes(obj)``  ``operator.contains(obj)``
    ``operator.isSequenceType(obj)``    ``isinstance(obj, collections.abc.Sequence)``
    ``operator.isMappingType(obj)``     ``isinstance(obj, collections.abc.Mapping)``

Doc/library/asyncio-eventloop.rst

@@ -967,10 +967,7 @@ Wait until a file descriptor received some data using the
 :meth:`AbstractEventLoop.add_reader` method and then close the event loop::
 
     import asyncio
-    try:
-        from socket import socketpair
-    except ImportError:
-        from asyncio.windows_utils import socketpair
+    from socket import socketpair
 
     # Create a pair of connected file descriptors
     rsock, wsock = socketpair()

Doc/library/asyncio-protocol.rst

@@ -690,10 +690,7 @@ Wait until a socket receives data using the
 the event loop ::
 
     import asyncio
-    try:
-        from socket import socketpair
-    except ImportError:
-        from asyncio.windows_utils import socketpair
+    from socket import socketpair
 
     # Create a pair of connected sockets
     rsock, wsock = socketpair()

Doc/library/asyncio-stream.rst

@@ -426,10 +426,7 @@ Coroutine waiting until a socket receives data using the
 :func:`open_connection` function::
 
     import asyncio
-    try:
-        from socket import socketpair
-    except ImportError:
-        from asyncio.windows_utils import socketpair
+    from socket import socketpair
 
     @asyncio.coroutine
     def wait_for_data(loop):

Doc/library/development.rst

@@ -24,3 +24,6 @@ The list of modules described in this chapter is:
    unittest.mock-examples.rst
    2to3.rst
    test.rst
+
+See also the Python development mode: the :option:`-X` ``dev`` option and
+:envvar:`PYTHONDEVMODE` environment variable.

Doc/library/multiprocessing.rst

@@ -1837,8 +1837,8 @@ Running the following commands creates a server for a single shared queue which
 remote clients can access::
 
    >>> from multiprocessing.managers import BaseManager
-   >>> import queue
-   >>> queue = queue.Queue()
+   >>> from queue import Queue
+   >>> queue = Queue()
    >>> class QueueManager(BaseManager): pass
    >>> QueueManager.register('get_queue', callable=lambda:queue)
    >>> m = QueueManager(address=('', 50000), authkey=b'abracadabra')

Doc/library/re.rst

@@ -689,11 +689,11 @@ form.
    splits occur, and the remainder of the string is returned as the final element
    of the list. ::
 
-      >>> re.split('\W+', 'Words, words, words.')
+      >>> re.split(r'\W+', 'Words, words, words.')
       ['Words', 'words', 'words', '']
-      >>> re.split('(\W+)', 'Words, words, words.')
+      >>> re.split(r'(\W+)', 'Words, words, words.')
       ['Words', ', ', 'words', ', ', 'words', '.', '']
-      >>> re.split('\W+', 'Words, words, words.', 1)
+      >>> re.split(r'\W+', 'Words, words, words.', 1)
       ['Words', 'words, words.']
       >>> re.split('[a-f]+', '0a3B9', flags=re.IGNORECASE)
       ['0', '3', '9']
@@ -702,7 +702,7 @@ form.
    the string, the result will start with an empty string.  The same holds for
    the end of the string::
 
-      >>> re.split('(\W+)', '...words, words...')
+      >>> re.split(r'(\W+)', '...words, words...')
       ['', '...', 'words', ', ', 'words', '...', '']
 
    That way, separator components are always found at the same relative

Doc/library/ssl.rst

@@ -429,6 +429,10 @@ Certificate handling
       Matching of IP addresses, when present in the subjectAltName field
       of the certificate, is now supported.
 
+   .. versionchanged:: 3.7
+      Allow wildcard when it is the leftmost and the only character
+      in that segment.
+
 .. function:: cert_time_to_seconds(cert_time)
 
    Return the time in seconds since the Epoch, given the ``cert_time``

Doc/library/sys.rst

@@ -334,6 +334,7 @@ always available.
    :const:`bytes_warning`        :option:`-b`
    :const:`quiet`                :option:`-q`
    :const:`hash_randomization`   :option:`-R`
+   :const:`dev_mode`             :option:`-X` ``dev``
    ============================= =============================
 
    .. versionchanged:: 3.2
@@ -345,6 +346,9 @@ always available.
    .. versionchanged:: 3.3
       Removed obsolete ``division_warning`` attribute.
 
+   .. versionchanged:: 3.7
+      Added ``dev_mode`` attribute for the new :option:`-X` ``dev`` flag.
+
 
 .. data:: float_info
 

Doc/library/tracemalloc.rst

@@ -650,8 +650,8 @@ Traceback
 
 .. class:: Traceback
 
-   Sequence of :class:`Frame` instances sorted from the most recent frame to
-   the oldest frame.
+   Sequence of :class:`Frame` instances sorted from the oldest frame to the
+   most recent frame.
 
    A traceback contains at least ``1`` frame. If the ``tracemalloc`` module
    failed to get a frame, the filename ``"<unknown>"`` at line number ``0`` is
@@ -663,11 +663,17 @@ Traceback
    The :attr:`Trace.traceback` attribute is an instance of :class:`Traceback`
    instance.
 
-   .. method:: format(limit=None)
+   .. versionchanged:: 3.7
+      Frames are now sorted from the oldest to the most recent, instead of most recent to oldest.
 
-      Format the traceback as a list of lines with newlines.  Use the
-      :mod:`linecache` module to retrieve lines from the source code.  If
-      *limit* is set, only format the *limit* most recent frames.
+   .. method:: format(limit=None, most_recent_first=False)
+
+      Format the traceback as a list of lines with newlines. Use the
+      :mod:`linecache` module to retrieve lines from the source code.
+      If *limit* is set, format the *limit* most recent frames if *limit*
+      is positive. Otherwise, format the ``abs(limit)`` oldest frames.
+      If *most_recent_first* is ``True``, the order of the formatted frames
+      is reversed, returning the most recent frame first instead of last.
 
       Similar to the :func:`traceback.format_tb` function, except that
       :meth:`.format` does not include newlines.

Doc/library/typing.rst

@@ -146,6 +146,8 @@ See :pep:`484` for more details.
    ``Derived`` is expected. This is useful when you want to prevent logic
    errors with minimal runtime cost.
 
+.. versionadded:: 3.5.2
+
 Callable
 --------
 
@@ -494,6 +496,8 @@ The module defines the following classes, functions and decorators:
    ``Type[Any]`` is equivalent to ``Type`` which in turn is equivalent
    to ``type``, which is the root of Python's metaclass hierarchy.
 
+   .. versionadded:: 3.5.2
+
 .. class:: Iterable(Generic[T_co])
 
     A generic version of :class:`collections.abc.Iterable`.
@@ -674,6 +678,8 @@ The module defines the following classes, functions and decorators:
 
    A generic version of :class:`collections.defaultdict`.
 
+   .. versionadded:: 3.5.2
+
 .. class:: Counter(collections.Counter, Dict[T, int])
 
    A generic version of :class:`collections.Counter`.
@@ -762,6 +768,8 @@ The module defines the following classes, functions and decorators:
        def add_unicode_checkmark(text: Text) -> Text:
            return text + u' \u2713'
 
+   .. versionadded:: 3.5.2
+
 .. class:: io
 
    Wrapper namespace for I/O stream types.
@@ -847,6 +855,8 @@ The module defines the following classes, functions and decorators:
       UserId = NewType('UserId', int)
       first_user = UserId(1)
 
+   .. versionadded:: 3.5.2
+
 .. function:: cast(typ, val)
 
    Cast a value to a type.
@@ -1054,3 +1064,5 @@ The module defines the following classes, functions and decorators:
    "forward reference", to hide the ``expensive_mod`` reference from the
    interpreter runtime.  Type annotations for local variables are not
    evaluated, so the second annotation does not need to be enclosed in quotes.
+
+   .. versionadded:: 3.5.2

Doc/library/uuid.rst

@@ -156,10 +156,18 @@ The :mod:`uuid` module defines the following functions:
 
    Get the hardware address as a 48-bit positive integer.  The first time this
    runs, it may launch a separate program, which could be quite slow.  If all
-   attempts to obtain the hardware address fail, we choose a random 48-bit number
-   with its eighth bit set to 1 as recommended in RFC 4122.  "Hardware address"
-   means the MAC address of a network interface, and on a machine with multiple
-   network interfaces the MAC address of any one of them may be returned.
+   attempts to obtain the hardware address fail, we choose a random 48-bit
+   number with the multicast bit (least significant bit of the first octet)
+   set to 1 as recommended in RFC 4122.  "Hardware address" means the MAC
+   address of a network interface.  On a machine with multiple network
+   interfaces, universally administered MAC addresses (i.e. where the second
+   least significant bit of the first octet is *unset*) will be preferred over
+   locally administered MAC addresses, but with no other ordering guarantees.
+
+   .. versionchanged:: 3.7
+      Universally administered MAC addresses are preferred over locally
+      administered MAC addresses, since the former are guaranteed to be
+      globally unique, while the latter are not.
 
 .. index:: single: getnode