Make --namespace-packages the default (#9636)

Author: hauntsaninja

docs/source/command_line.rst

@@ -129,30 +129,12 @@ Import discovery
 The following flags customize how exactly mypy discovers and follows
 imports.
 
-.. option:: --namespace-packages
-
-    This flag enables import discovery to use namespace packages (see
-    :pep:`420`).  In particular, this allows discovery of imported
-    packages that don't have an ``__init__.py`` (or ``__init__.pyi``)
-    file.
-
-    Namespace packages are found (using the PEP 420 rules, which
-    prefers "classic" packages over namespace packages) along the
-    module search path -- this is primarily set from the source files
-    passed on the command line, the ``MYPYPATH`` environment variable,
-    and the :confval:`mypy_path` config option.
-
-    This flag affects how mypy finds modules and packages explicitly passed on
-    the command line. It also affects how mypy determines fully qualified module
-    names for files passed on the command line. See :ref:`Mapping file paths to
-    modules <mapping-paths-to-modules>` for details.
-
 .. option:: --explicit-package-bases
 
     This flag tells mypy that top-level packages will be based in either the
     current directory, or a member of the ``MYPYPATH`` environment variable or
     :confval:`mypy_path` config option. This option is only useful in
-    conjunction with :option:`--namespace-packages`. See :ref:`Mapping file
+    in the absence of `__init__.py`. See :ref:`Mapping file
     paths to modules <mapping-paths-to-modules>` for details.
 
 .. option:: --ignore-missing-imports
@@ -236,6 +218,18 @@ imports.
     setting the :option:`--fast-module-lookup` option.
 
 
+.. option:: --no-namespace-packages
+
+    This flag disables import discovery of namespace packages (see :pep:`420`).
+    In particular, this prevents discovery of packages that don't have an
+    ``__init__.py`` (or ``__init__.pyi``) file.
+
+    This flag affects how mypy finds modules and packages explicitly passed on
+    the command line. It also affects how mypy determines fully qualified module
+    names for files passed on the command line. See :ref:`Mapping file paths to
+    modules <mapping-paths-to-modules>` for details.
+
+
 .. _platform-configuration:
 
 Platform configuration

docs/source/config_file.rst

@@ -254,10 +254,11 @@ section of the command line docs.
 .. confval:: namespace_packages
 
     :type: boolean
-    :default: False
+    :default: True
 
     Enables :pep:`420` style namespace packages.  See the
-    corresponding flag :option:`--namespace-packages <mypy --namespace-packages>` for more information.
+    corresponding flag :option:`--no-namespace-packages <mypy --no-namespace-packages>`
+    for more information.
 
     This option may only be set in the global section (``[mypy]``).
 
@@ -269,7 +270,7 @@ section of the command line docs.
     This flag tells mypy that top-level packages will be based in either the
     current directory, or a member of the ``MYPYPATH`` environment variable or
     :confval:`mypy_path` config option. This option is only useful in
-    conjunction with :confval:`namespace_packages`. See :ref:`Mapping file
+    the absence of `__init__.py`. See :ref:`Mapping file
     paths to modules <mapping-paths-to-modules>` for details.
 
     This option may only be set in the global section (``[mypy]``).

docs/source/running_mypy.rst

@@ -26,10 +26,6 @@ Specifying code to be checked
 
 Mypy lets you specify what files it should type check in several different ways.
 
-Note that if you use namespace packages (in particular, packages without
-``__init__.py``), you'll need to specify :option:`--namespace-packages <mypy
---namespace-packages>`.
-
 1.  First, you can pass in paths to Python files and directories you
     want to type check. For example::
 
@@ -322,11 +318,6 @@ this error, try:
     you must run ``mypy ~/foo-project/src`` (or set the ``MYPYPATH`` to
     ``~/foo-project/src``.
 
-4.  If you are using namespace packages -- packages which do not contain
-    ``__init__.py`` files within each subfolder -- using the
-    :option:`--namespace-packages <mypy --namespace-packages>` command
-    line flag.
-
 In some rare cases, you may get the "Cannot find implementation or library
 stub for module" error even when the module is installed in your system.
 This can happen when the module is both missing type hints and is installed
@@ -423,10 +414,10 @@ to modules to type check.
   added to mypy's module search paths.
 
 How mypy determines fully qualified module names depends on if the options
-:option:`--namespace-packages <mypy --namespace-packages>` and
+:option:`--no-namespace-packages <mypy --no-namespace-packages>` and
 :option:`--explicit-package-bases <mypy --explicit-package-bases>` are set.
 
-1. If :option:`--namespace-packages <mypy --namespace-packages>` is off,
+1. If :option:`--no-namespace-packages <mypy --no-namespace-packages>` is set,
    mypy will rely solely upon the presence of ``__init__.py[i]`` files to
    determine the fully qualified module name. That is, mypy will crawl up the
    directory tree for as long as it continues to find ``__init__.py`` (or
@@ -436,12 +427,13 @@ How mypy determines fully qualified module names depends on if the options
    would require ``pkg/__init__.py`` and ``pkg/subpkg/__init__.py`` to exist in
    order correctly associate ``mod.py`` with ``pkg.subpkg.mod``
 
-2. If :option:`--namespace-packages <mypy --namespace-packages>` is on, but
-   :option:`--explicit-package-bases <mypy --explicit-package-bases>` is off,
-   mypy will allow for the possibility that directories without
-   ``__init__.py[i]`` are packages. Specifically, mypy will look at all parent
-   directories of the file and use the location of the highest
-   ``__init__.py[i]`` in the directory tree to determine the top-level package.
+2. The default case. If :option:`--namespace-packages <mypy
+   --no-namespace-packages>` is on, but :option:`--explicit-package-bases <mypy
+   --explicit-package-bases>` is off, mypy will allow for the possibility that
+   directories without ``__init__.py[i]`` are packages. Specifically, mypy will
+   look at all parent directories of the file and use the location of the
+   highest ``__init__.py[i]`` in the directory tree to determine the top-level
+   package.
 
    For example, say your directory tree consists solely of ``pkg/__init__.py``
    and ``pkg/a/b/c/d/mod.py``. When determining ``mod.py``'s fully qualified

mypy/build.py

@@ -1401,8 +1401,8 @@ def validate_meta(
         st = manager.get_stat(path)
     except OSError:
         return None
-    if not (stat.S_ISREG(st.st_mode) or stat.S_ISDIR(st.st_mode)):
-        manager.log(f"Metadata abandoned for {id}: file {path} does not exist")
+    if not stat.S_ISDIR(st.st_mode) and not stat.S_ISREG(st.st_mode):
+        manager.log(f"Metadata abandoned for {id}: file or directory {path} does not exist")
         return None
 
     manager.add_stats(validate_stat_time=time.time() - t0)

mypy/main.py

@@ -544,8 +544,9 @@ def add_invertible_flag(
         title="Import discovery", description="Configure how imports are discovered and followed."
     )
     add_invertible_flag(
-        "--namespace-packages",
-        default=False,
+        "--no-namespace-packages",
+        dest="namespace_packages",
+        default=True,
         help="Support namespace packages (PEP 420, __init__.py-less)",
         group=imports_group,
     )

mypy/options.py

@@ -95,7 +95,7 @@ def __init__(self) -> None:
         # This allows definitions of packages without __init__.py and allows packages to span
         # multiple directories. This flag affects both import discovery and the association of
         # input files/modules/packages to the relevant file and fully qualified module name.
-        self.namespace_packages = False
+        self.namespace_packages = True
         # Use current directory and MYPYPATH to determine fully qualified module names of files
         # passed by automatically considering their subdirectories as packages. This is only
         # relevant if namespace packages are enabled, since otherwise examining __init__.py's is

test-data/unit/check-modules.test

@@ -2665,12 +2665,13 @@ from foo.bar import x
 x = 0
 
 [case testClassicNotPackage]
+# flags: --no-namespace-packages
 from foo.bar import x
 [file foo/bar.py]
 x = 0
 [out]
-main:1: error: Cannot find implementation or library stub for module named "foo.bar"
-main:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
+main:2: error: Cannot find implementation or library stub for module named "foo.bar"
+main:2: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
 
 [case testNamespacePackage]
 # flags: --namespace-packages

test-data/unit/fine-grained-modules.test

@@ -845,7 +845,7 @@ main:2: error: Argument 1 to "f" has incompatible type "int"; expected "str"
 ==
 main:1: error: Cannot find implementation or library stub for module named "p.a"
 main:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
-main:1: error: Cannot find implementation or library stub for module named "p"
+main:2: error: "object" has no attribute "a"
 
 [case testDeletePackage2]
 import p

test-data/unit/semanal-errors.test

@@ -316,15 +316,16 @@ x = y
 tmp/k.py:2: error: Name "y" is not defined
 
 [case testPackageWithoutInitFile]
+# flags: --no-namespace-packages
 import typing
 import m.n
 m.n.x
 [file m/n.py]
 x = 1
 [out]
-main:2: error: Cannot find implementation or library stub for module named "m.n"
-main:2: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
-main:2: error: Cannot find implementation or library stub for module named "m"
+main:3: error: Cannot find implementation or library stub for module named "m.n"
+main:3: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
+main:3: error: Cannot find implementation or library stub for module named "m"
 
 [case testBreakOutsideLoop]
 break