Merge branch 'master' into literal_hash1

Author: elazarg

.travis.yml

@@ -15,6 +15,7 @@ python:
 
 install:
   - pip install -r test-requirements.txt
+  - python2 -m pip install --user typing
   - python setup.py install
 
 script:

CONTRIBUTING.md

@@ -30,7 +30,7 @@ place to start a discussion of any of the above or most any other
 topic concerning the project.
 
 For less formal discussion we have a chat room on
-[gitter.im](https://gitter.im/python/mypy).  Some Mypy core developers
+[gitter.im](https://gitter.im/python/typing).  Some Mypy core developers
 are almost always present; feel free to find us there and we're happy
 to chat.  Substantive technical discussion will be directed to the
 issue tracker.

LICENSE

@@ -27,8 +27,7 @@ DEALINGS IN THE SOFTWARE.
 = = = = =
 
 Portions of mypy are licensed under different licenses.  The files
-under stdlib-samples and lib-typing are licensed under the PSF 2
-License, reproduced below.
+under stdlib-samples are licensed under the PSF 2 License, reproduced below.
 
 = = = = =
 

README.md

@@ -2,14 +2,14 @@ Mypy: Optional Static Typing for Python
 =======================================
 
 [![Build Status](https://travis-ci.org/python/mypy.svg)](https://travis-ci.org/python/mypy)
-[![Chat at https://gitter.im/python/mypy](https://badges.gitter.im/python/mypy.svg)](https://gitter.im/python/mypy?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
+[![Chat at https://gitter.im/python/typing](https://badges.gitter.im/python/typing.svg)](https://gitter.im/python/typing?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 
 
 Got a question? Join us on Gitter!
 ----------------------------------
 
 We don't have a mailing list; but we are always happy to answer
-questions on [gitter chat](https://gitter.im/python/mypy).  If you are
+questions on [gitter chat](https://gitter.im/python/typing).  If you are
 sure you've found a bug please search our issue trackers for a
 duplicate before filing a new issue:
 
@@ -97,6 +97,23 @@ programs, even if they have type errors:
 [statically typed parts]: http://mypy.readthedocs.io/en/latest/basics.html#function-signatures
 
 
+IDE & Linter Integrations
+-------------------------
+
+Mypy can be integrated into popular IDEs:
+
+* Vim: [vim-mypy](https://github.com/Integralist/vim-mypy)
+* Emacs: using [Flycheck](https://github.com/flycheck/) and [Flycheck-mypy](https://github.com/lbolla/emacs-flycheck-mypy/issues)
+* Sublime Text: [SublimeLinter-contrib-mypy]
+* Atom: [linter-mypy](https://atom.io/packages/linter-mypy)
+* PyCharm: PyCharm integrates [its own implementation of PEP 484](https://www.jetbrains.com/help/pycharm/2017.1/type-hinting-in-pycharm.html).
+
+Mypy can also be integrated into [Flake8] using [flake8-mypy].
+
+[Flake8]: http://flake8.pycqa.org/
+[flake8-mypy]: https://github.com/ambv/flake8-mypy
+
+
 Web site and documentation
 --------------------------
 
@@ -157,6 +174,15 @@ If you want to contribute, first clone the mypy git repository:
 
     $ git clone --recurse-submodules https://github.com/python/mypy.git
 
+If you've already cloned the repo without `--recurse-submodules`,
+you need to pull in the typeshed repo as follows:
+
+    $ git submodule init
+    $ git submodule update
+
+Either way you should now have a subdirectory `typeshed` containing a
+clone of the typeshed repo (`https://github.com/python/typeshed`).
+
 From the mypy directory, use pip to install mypy:
 
     $ cd mypy
@@ -170,10 +196,6 @@ the above as root. For example, in Ubuntu:
 Now you can use the `mypy` program just as above.  In case of trouble
 see "Troubleshooting" above.
 
-The mypy wiki contains some useful information for contributors:
-
-  https://github.com/python/mypy/wiki/Developer-Guides
-
 
 Working with the git version of mypy
 ------------------------------------

ROADMAP.md

@@ -0,0 +1,96 @@
+# Mypy Roadmap
+
+The goal of the roadmap is to document areas the mypy core team is
+planning to work on in the future or is currently working on. PRs
+targeting these areas are very welcome, but please check first with a
+core team member that nobody else is working on the same thing.
+
+**Note:** This doesn’t include everything that the core team will work
+on, and everything is subject to change. Near-term plans are likely
+more accurate.
+
+## April-June 2017
+
+- Add more comprehensive testing for `--incremental` and `--quick`
+  modes to improve reliability. At least write more unit tests with
+  focus on areas that have previously had bugs.
+  ([issue](https://github.com/python/mypy/issues/3455))
+
+- Speed up `--quick` mode to better support million+ line codebases
+  through some of these:
+
+  - Make it possible to use remote caching for incremental cache
+    files. This would speed up a cold run with no local cache data.
+    We need to update incremental cache to use hashes to determine
+    whether files have changes to allow
+    [sharing cache data](https://github.com/python/mypy/issues/3403).
+
+  - See if we can speed up deserialization of incremental cache
+    files. Initial experiments aren’t very promising though so there
+    might not be any easy wins left.
+    ([issue](https://github.com/python/mypy/issues/3456))
+
+- Improve support for complex signatures such as `open(fn, 'rb')` and
+  specific complex decorators such as `contextlib.contextmanager`
+  through type checker plugins/hooks.
+  ([issue](https://github.com/python/mypy/issues/1240))
+
+- Document basic properties of all type operations used within mypy,
+  including compatibility, proper subtyping, joins and meets.
+  ([issue](https://github.com/python/mypy/issues/3454))
+
+- Make TypedDict an officially supported mypy feature. This makes it
+  possible to give precise types for dictionaries that represent JSON
+  objects, such as `{"path": "/dir/fnam.ext", "size": 1234}`.
+  ([issue](https://github.com/python/mypy/issues/3453))
+
+- Make error messages more useful and informative.
+  ([issue](https://github.com/python/mypy/labels/topic-usability))
+
+- Resolve [#2008](https://github.com/python/mypy/issues/2008) (we are
+  converging on approach 4).
+
+## July-December 2017
+
+- Invest some effort into systematically filling in missing
+  annotations and stubs in typeshed, with focus on features heavily
+  used at Dropbox. Better support for ORMs will be a separate
+  project.
+
+- Improve opt-in warnings about `Any` types to make it easier to keep
+  code free from unwanted `Any` types. For example, warn about using
+  `list` (instead of `List[x]`) and calling `open` if we can’t infer a
+  precise return type, or using types imported from ignored modules
+  (they are implicitly `Any`).
+
+- Add support for protocols and structural subtyping (PEP 544).
+
+- Switch completely to pytest and remove the custom testing framework.
+  ([issue](https://github.com/python/mypy/issues/1673))
+
+- Make it possible to run mypy as a daemon to avoid reprocessing the
+  entire program on each run. This will improve performance
+  significantly. Even when using the incremental mode, processing a
+  large number of files is not cheap.
+
+- Refactor and simplify specific tricky parts of mypy internals, such
+  as the [conditional type binder](https://github.com/python/mypy/issues/3457),
+  [symbol tables](https://github.com/python/mypy/issues/3458) or
+  the various [semantic analysis passes](https://github.com/python/mypy/issues/3459).
+
+- Implement a general type system plugin architecture. It should be
+  able to support some typical ORM features at least, such as
+  metaclasses that add methods with automatically inferred signatures
+  and complex descriptors such as those used by Django models.
+  ([issue](https://github.com/python/mypy/issues/1240))
+
+- Add support for statically typed
+  [protobufs](https://developers.google.com/protocol-buffers/).
+
+- Provide much faster, reliable interactive feedback through
+  fine-grained incremental type checking, built on top the daemon
+  mode.
+
+- Start work on editor plugins and support for selected IDE features.
+
+- Turn on `--strict-optional` by default.

appveyor.yml

@@ -1,15 +1,10 @@
 environment:
   matrix:
 
-    - PYTHON: "C:\\Python36"
-      PYTHON_VERSION: "3.6.x"
-      PYTHON_ARCH: "32"
-
     - PYTHON: "C:\\Python36-x64"
       PYTHON_VERSION: "3.6.x"
       PYTHON_ARCH: "64"
 
-
 install:
     - "git config core.symlinks true"
     - "git reset --hard"

conftest.py

@@ -1,3 +1,13 @@
+import os.path
+
+import pytest
+
 pytest_plugins = [
     'mypy.test.data',
 ]
+
+
+def pytest_configure(config):
+    mypy_source_root = os.path.dirname(os.path.abspath(__file__))
+    if os.getcwd() != mypy_source_root:
+        os.chdir(mypy_source_root)

docs/source/cheat_sheet.rst

@@ -149,6 +149,18 @@ When you're puzzled or when things are complicated
    reveal_type(c)  # -> error: Revealed type is 'builtins.list[builtins.str]'
    print(c)  # -> [4] the object is not cast
 
+   # if you want dynamic attributes on your class, have it override __setattr__ or __getattr__
+   # in a stub or in your source code.
+   # __setattr__ allows for dynamic assignment to names
+   # __getattr__ allows for dynamic access to names
+   class A:
+       # this will allow assignment to any A.x, if x is the same type as `value`
+       def __setattr__(self, name, value):
+           # type: (str, int) -> None
+           ...
+   a.foo = 42  # works
+   a.bar = 'Ex-parrot'  # fails type checking
+
    # TODO: explain "Need type annotation for variable" when
    # initializing with None or an empty container
 
@@ -220,15 +232,17 @@ Other stuff
 
    # Use AnyStr for functions that should accept any kind of string
    # without allowing different kinds of strings to mix.
-   def concat(a: AnyStr, b: AnyStr) -> AnyStr:
+   def concat(a, b):
+       # type: (AnyStr, AnyStr) -> AnyStr
        return a + b
    concat(u"foo", u"bar")  # type: unicode
    concat(b"foo", b"bar")  # type: bytes
 
    # Use IO[] for functions that should accept or return any
    # object that comes from an open() call. The IO[] does not
    # distinguish between reading, writing or other modes.
-   def get_sys_IO(mode='w') -> IO[str]:
+   def get_sys_IO(mode='w'):
+       # type: (str) -> IO[str]
        if mode == 'w':
            return sys.stdout
        elif mode == 'r':

docs/source/cheat_sheet_py3.rst

@@ -142,6 +142,19 @@ When you're puzzled or when things are complicated
    reveal_type(c)  # -> error: Revealed type is 'builtins.list[builtins.str]'
    print(c)  # -> [4] the object is not cast
 
+   # if you want dynamic attributes on your class, have it override __setattr__ or __getattr__
+   # in a stub or in your source code.
+   # __setattr__ allows for dynamic assignment to names
+   # __getattr__ allows for dynamic access to names
+   class A:
+       # this will allow assignment to any A.x, if x is the same type as `value`
+       def __setattr__(self, name: str, value: int) -> None: ...
+       # this will allow access to any A.x, if x is compatible with the return type
+       def __getattr__(self, name: str) -> int: ...
+   a.foo = 42  # works
+   a.bar = 'Ex-parrot'  # fails type checking
+
+
    # TODO: explain "Need type annotation for variable" when
    # initializing with None or an empty container
 

docs/source/command_line.rst

@@ -274,6 +274,36 @@ Here are some more useful flags:
   re-check your code without ``--strict-optional`` to ensure new type errors
   are not introduced.
 
+.. _disallow-any:
+
+- ``--disallow-any`` disallows various types of ``Any`` in a module.
+  The option takes a comma-separated list of the following values:
+  ``unimported``, ``unannotated``, ``expr``, ``decorated``, ``explicit``.
+
+  ``unimported`` disallows usage of types that come from unfollowed imports
+  (such types become aliases for ``Any``). Unfollowed imports occur either
+  when the imported module does not exist or when ``--follow-imports=skip``
+  is set.
+
+  ``unannotated`` disallows function definitions that are not fully
+  typed (i.e. that are missing an explicit type annotation for any
+  of the parameters or the return type). ``unannotated`` option is
+  interchangeable with ``--disallow-untyped-defs``.
+
+  ``expr`` disallows all expressions in the module that have type ``Any``.
+  If an expression of type ``Any`` appears anywhere in the module
+  mypy will output an error unless the expression is immediately
+  used as an argument to ``cast`` or assigned to a variable with an
+  explicit type annotation. In addition, declaring a variable of type ``Any``
+  or casting to type ``Any`` is not allowed. Note that calling functions
+  that take parameters of type ``Any`` is still allowed.
+
+  ``decorated`` disallows functions that have ``Any`` in their signature
+  after decorator transformation.
+
+  ``explicit`` disallows explicit ``Any`` in type positions such as type
+  annotations and generic type parameters.
+
 - ``--disallow-untyped-defs`` reports an error whenever it encounters
   a function definition without type annotations.
 
@@ -391,6 +421,13 @@ Here are some more useful flags:
 - ``--strict`` mode enables all optional error checking flags.  You can see the
   list of flags enabled by strict mode in the full ``mypy -h`` output.
 
+- ``--shadow-file SOURCE_FILE SHADOW_FILE`` makes mypy typecheck SHADOW_FILE in
+  place of SOURCE_FILE.  Primarily intended for tooling.  Allows tooling to
+  make transformations to a file before type checking without having to change
+  the file in-place.  (For example, tooling could use this to display the type
+  of an expression by wrapping it with a call to reveal_type in the shadow
+  file and then parsing the output.)
+
 For the remaining flags you can read the full ``mypy -h`` output.
 
 .. note::

docs/source/common_issues.rst

@@ -180,7 +180,7 @@ not support ``sort()``) as a list and sort it in-place:
         # Type of x is List[int] here.
         x.sort()  # Okay!
 
-.. _invariance-vs-covariance:
+.. _variance:
 
 Invariance vs covariance
 ------------------------

docs/source/config_file.rst

@@ -147,6 +147,12 @@ overridden by the pattern sections matching the module name.
 - ``almost_silent`` (Boolean, deprecated) equivalent to
   ``follow_imports=skip``.
 
+- ``disallow_any`` (Comma-separated list, default empty) is an option to
+  disallow various types of ``Any`` in a module. The flag takes a
+  comma-separated list of the following arguments: ``unimported``,
+  ``unannotated``, ``expr``, ``decorated``, ``explicit``. For explanations
+  see the discussion for the :ref:`--disallow-any <disallow-any>` option.
+
 - ``disallow_untyped_calls`` (Boolean, default False) disallows
   calling functions without type annotations from functions with type
   annotations.
@@ -178,6 +184,8 @@ overridden by the pattern sections matching the module name.
 - ``strict_boolean`` (Boolean, default False) makes using non-boolean
   expressions in conditions an error.
 
+- ``no_implicit_optional`` (Boolean, default false) changes the treatment of
+  arguments with a default value of None by not implicitly making their type Optional
 
 Example
 *******