Merge stuff from may release back into master (#12233)

* Revert vscode-extension-telemetry changes for the release (#11602) (#11656)

* Revert "Fix slashes in telemetry unit tests (#11572)"

This reverts commit 7431c9c.

* Revert "Use vscode-extension-telemetry for our exceptions & error telemetry (#11524)"

This reverts commit d5065e6.

* Remove from changelog

* Port storage fix to release branch (#11673)

* Fix storage not being used (#11649)

* Fix storage not being used

* Add disposable to storage so it won't write after shutdown

* Fix dirty title

* Hack to get tests to pass

* Another way to get run all to not interfere

* Update changelog

* Port scrolling fix to release (#11688)

* Fix scrolling (#11681)

* Fix scrolling

* Review feedback - fix scrolling on expand/collapse

* Update changelog

* Update package.json

Co-authored-by: Jim Griesmer <[email protected]>

* Cherry-pick pipenv changes and pythonpath prompt changes to release (#11700)

* Show the prompt again if user clicks on more info (#11664)

* Show the prompt again if user clicks on more info

* Review feedback

* Use Learn more as text for the link.

* Leave pipenv in a corner until the user decides to select an interpreter (#11654)

* add onSuggestion option
* Swap onActivation with onSuggestion
* Update unit tests
* Remove registration of IPipenvService
* Move didTriggerInterpreterSuggestions logic inside pipenv locator
* Fix existing unit tests
* Add new unit tests
* Replace typemoq any param with object
* Shorten the tests
* Fix warning
* Duplicate teardown

Co-authored-by: Kim-Adeline Miguel <[email protected]>

* Update extension version (#11730)

* Update extension version

* Update date in changelog.

* Update change log with additional notes. (#11764)

* Cherry picks and version updates for bug fix release (#11878)

* Do not execute shebang as an interpreter until user has clicked on the codelens enclosing the shebang (#11816)

* Do not execute shebang as an interpreter until user has clicked on the codelens enclosing the shebang

* Rename

* Oops

* Update src/test/providers/shebangCodeLenseProvider.unit.test.ts

Co-authored-by: Karthik Nadig <[email protected]>

Co-authored-by: Karthik Nadig <[email protected]>

* Update version and change log for bugfix release

Co-authored-by: Kartik Raj <[email protected]>

* Cherry-pick ExP platform work from master (#12160)

* User cannot belong to all experiments in an experiment group (#11945)

* 10790 prep - Create an experiments/ folder  (#11980)

* experiments.ts -> experiments/manager.ts
* eexperimentGroups -> experiments/experimentGroups
* test/experiments.u.t -> test/experiments/manager.u.t
* experimentGroups -> groups
* Whoops committed one file too many

* Add support for VS Code's experiment service (#11979)

* Add npm package
* News entry
* experiments.ts -> experiments/manager.ts
* Wrong issue number
* eexperimentGroups -> experiments/experimentGroups
* Move experiments.unit.tests.ts -> experiments/
* Commit new files
* Merge gone sideways
* Add types
* Add opt in/out handling
* Activation (service registry)
* Don't pin tas-client to one version
* Unit tests + fixes
* Lol forgot to remove a comment + add headers
* Forgot 'use strict' in service.ts
* Use IApplicationEnvironment instead of IExtensions
* Apply suggestions from code review
Co-authored-by: Don Jayamanne <[email protected]>
* Remove unnecessary formatted props
* n e v e r m i n d
* Aight fixed it
* flight -> experiment
* Check stub calls instead of ctor impl
* removed getExperimentService stub check
* Set shared properties for all telemetry events
* Add test for shared properties

Co-authored-by: Don Jayamanne <[email protected]>

* Fix merge issues

* Fix index unit tests

* Revert "Fix index unit tests"

This reverts commit 2fb61fc.

* Make MPLS and vscode-extension-telemetry work together 🤝 (#11823)

* Revert "Revert vscode-extension-telemetry changes for the release (#11602)"
This reverts commit 71d1793.
* Remove entry from changelog + add new news entry
* Update LS code to use periods instead of slashes
* Revert "Update LS code to use periods instead of slashes"
This reverts commit 1356651.
* Replace slashes before sending telemetry instead
* Too fast too furious

* Fix more merge issues

Co-authored-by: Don Jayamanne <[email protected]>
Co-authored-by: Kim-Adeline Miguel <[email protected]>

* Update version and changelog for point release (#12171)

* Ensure extension features are started when in Deprecate PythonPath experiment and opening a file without any folder opened (#12182)

* Ensure extension features are started when in Deprecate PythonPath experiment and opening a file without any folder opened.

* Added comments

* Update change log (#12190)

* Cherry pick fix for hasInterpreters and update change log (#12198)

* Double-check for interpreters when running diagnostics (#12158)

* Get interpreters if hasInterpreters returns false
* Undo ignoreErrors()
* Add unit tests
* Fixed tests
* Newline

* Fix merge issues.

* Update change log

Co-authored-by: Kim-Adeline Miguel <[email protected]>

* Update version for point release (#12259)

Co-authored-by: Kim-Adeline Miguel <[email protected]>
Co-authored-by: Rich Chiodo <[email protected]>
Co-authored-by: Jim Griesmer <[email protected]>
Co-authored-by: Kartik Raj <[email protected]>
Co-authored-by: Don Jayamanne <[email protected]>

Author: 6 people

CHANGELOG.md

@@ -6,8 +6,6 @@
 
 1. Removed `python.jediEnabled` setting in favor of `python.languageServer`. Instead of `"python.jediEnabled": true` please use `"python.languageServer": "Jedi"`.
    ([#7010](https://github.com/Microsoft/vscode-python/issues/7010))
-1. Integrate VS Code experiment framework in the extension.
-   ([#10790](https://github.com/Microsoft/vscode-python/issues/10790))
 1. Added a start page for the extension. It opens to new users or when there is a new release. It can be disabled with the setting 'Python: Show Start Page'.
    ([#11057](https://github.com/Microsoft/vscode-python/issues/11057))
 1. Preliminary support using other languages for the kernel.
@@ -61,7 +59,7 @@
    ([#11751](https://github.com/Microsoft/vscode-python/issues/11751))
 1. When switching to an invalid kernel (one that is registered but cannot start) in raw mode respect the launch timeout that is passed in.
    ([#11752](https://github.com/Microsoft/vscode-python/issues/11752))
-1. Make ```python.dataScience.textOutputLimit``` apply on subsequent rerun. We were letting the 'outputPrepend' metadata persist from run to run.
+1. Make `python.dataScience.textOutputLimit` apply on subsequent rerun. We were letting the 'outputPrepend' metadata persist from run to run.
    (thanks [Barry Nolte](https://github.com/BarryNolte))
    ([#11777](https://github.com/Microsoft/vscode-python/issues/11777))
 1. Use `${command:python.interpreterPath}` to get selected interpreter path in `launch.json` and `tasks.json`.
@@ -74,12 +72,8 @@
    ([#11800](https://github.com/Microsoft/vscode-python/issues/11800))
 1. Make sure to use webView.cspSource for all csp sources.
    ([#11855](https://github.com/Microsoft/vscode-python/issues/11855))
-1. Double-check for interpreters when running diagnostics before displaying the "Python is not installed" message.
-   ([#11870](https://github.com/Microsoft/vscode-python/issues/11870))
 1. Use command line arguments to launch our raw kernels as opposed to a connection file. The connection file seems to be causing issues in particular on windows CI machines with permissions.
    ([#11883](https://github.com/Microsoft/vscode-python/issues/11883))
-1. Ensure user cannot belong to all experiments in an experiment group.
-   ([#11943](https://github.com/Microsoft/vscode-python/issues/11943))
 1. Improve our status reporting when launching and connecting to a raw kernel.
    ([#11951](https://github.com/Microsoft/vscode-python/issues/11951))
 1. Prewarm raw kernels based on raw kernel support and don't prewarm if jupyter autostart is disabled.
@@ -102,8 +96,6 @@
    ([#11058](https://github.com/Microsoft/vscode-python/issues/11058))
 1. Reenable CDN unit tests.
    ([#11442](https://github.com/Microsoft/vscode-python/issues/11442))
-1. Update telemetry on errors and exceptions to use [vscode-extension-telemetry](https://www.npmjs.com/package/vscode-extension-telemetry).
-   ([#11597](https://github.com/Microsoft/vscode-python/issues/11597))
 1. Run by line for notebook cells minimal implementation.
    ([#11607](https://github.com/Microsoft/vscode-python/issues/11607))
 1. Get shape and count when showing debugger variables.
@@ -184,6 +176,136 @@ And finally thanks to the [Python](https://www.python.org/) development team and
 community for creating a fantastic programming language and community to be a
 part of!
 
+## 2020.5.3 (10 June 2020)
+
+1. Update `debugpy` to use `1.0.0b11` or greater.
+
+### Thanks
+
+Thanks to the following projects which we fully rely on to provide some of
+our features:
+
+-   [debugpy](https://pypi.org/project/debugpy/)
+-   [isort](https://pypi.org/project/isort/)
+-   [jedi](https://pypi.org/project/jedi/)
+    and [parso](https://pypi.org/project/parso/)
+-   [Microsoft Python Language Server](https://github.com/microsoft/python-language-server)
+-   [ptvsd](https://pypi.org/project/ptvsd/)
+-   [exuberant ctags](http://ctags.sourceforge.net/) (user-installed)
+-   [rope](https://pypi.org/project/rope/) (user-installed)
+
+Also thanks to the various projects we provide integrations with which help
+make this extension useful:
+
+-   Debugging support:
+    [Django](https://pypi.org/project/Django/),
+    [Flask](https://pypi.org/project/Flask/),
+    [gevent](https://pypi.org/project/gevent/),
+    [Jinja](https://pypi.org/project/Jinja/),
+    [Pyramid](https://pypi.org/project/pyramid/),
+    [PySpark](https://pypi.org/project/pyspark/),
+    [Scrapy](https://pypi.org/project/Scrapy/),
+    [Watson](https://pypi.org/project/Watson/)
+-   Formatting:
+    [autopep8](https://pypi.org/project/autopep8/),
+    [black](https://pypi.org/project/black/),
+    [yapf](https://pypi.org/project/yapf/)
+-   Interpreter support:
+    [conda](https://conda.io/),
+    [direnv](https://direnv.net/),
+    [pipenv](https://pypi.org/project/pipenv/),
+    [pyenv](https://github.com/pyenv/pyenv),
+    [venv](https://docs.python.org/3/library/venv.html#module-venv),
+    [virtualenv](https://pypi.org/project/virtualenv/)
+-   Linting:
+    [bandit](https://pypi.org/project/bandit/),
+    [flake8](https://pypi.org/project/flake8/),
+    [mypy](https://pypi.org/project/mypy/),
+    [prospector](https://pypi.org/project/prospector/),
+    [pylint](https://pypi.org/project/pylint/),
+    [pydocstyle](https://pypi.org/project/pydocstyle/),
+    [pylama](https://pypi.org/project/pylama/)
+-   Testing:
+    [nose](https://pypi.org/project/nose/),
+    [pytest](https://pypi.org/project/pytest/),
+    [unittest](https://docs.python.org/3/library/unittest.html#module-unittest)
+
+And finally thanks to the [Python](https://www.python.org/) development team and
+community for creating a fantastic programming language and community to be a
+part of!
+
+## 2020.5.2 (8 June 2020)
+
+### Fixes
+
+1. Double-check for interpreters when running diagnostics before displaying the "Python is not installed" message.
+   ([#11870](https://github.com/Microsoft/vscode-python/issues/11870))
+1. Ensure user cannot belong to all experiments in an experiment group.
+   ([#11943](https://github.com/Microsoft/vscode-python/issues/11943))
+1. Ensure extension features are started when in `Deprecate PythonPath` experiment and opening a file without any folder opened.
+   ([#12177](https://github.com/Microsoft/vscode-python/issues/12177))
+
+### Code Health
+
+1. Integrate VS Code experiment framework in the extension.
+   ([#10790](https://github.com/Microsoft/vscode-python/issues/10790))
+1. Update telemetry on errors and exceptions to use [vscode-extension-telemetry](https://www.npmjs.com/package/vscode-extension-telemetry).
+   ([#11597](https://github.com/Microsoft/vscode-python/issues/11597))
+
+### Thanks
+
+Thanks to the following projects which we fully rely on to provide some of
+our features:
+
+-   [debugpy](https://pypi.org/project/debugpy/)
+-   [isort](https://pypi.org/project/isort/)
+-   [jedi](https://pypi.org/project/jedi/)
+    and [parso](https://pypi.org/project/parso/)
+-   [Microsoft Python Language Server](https://github.com/microsoft/python-language-server)
+-   [ptvsd](https://pypi.org/project/ptvsd/)
+-   [exuberant ctags](http://ctags.sourceforge.net/) (user-installed)
+-   [rope](https://pypi.org/project/rope/) (user-installed)
+
+Also thanks to the various projects we provide integrations with which help
+make this extension useful:
+
+-   Debugging support:
+    [Django](https://pypi.org/project/Django/),
+    [Flask](https://pypi.org/project/Flask/),
+    [gevent](https://pypi.org/project/gevent/),
+    [Jinja](https://pypi.org/project/Jinja/),
+    [Pyramid](https://pypi.org/project/pyramid/),
+    [PySpark](https://pypi.org/project/pyspark/),
+    [Scrapy](https://pypi.org/project/Scrapy/),
+    [Watson](https://pypi.org/project/Watson/)
+-   Formatting:
+    [autopep8](https://pypi.org/project/autopep8/),
+    [black](https://pypi.org/project/black/),
+    [yapf](https://pypi.org/project/yapf/)
+-   Interpreter support:
+    [conda](https://conda.io/),
+    [direnv](https://direnv.net/),
+    [pipenv](https://pypi.org/project/pipenv/),
+    [pyenv](https://github.com/pyenv/pyenv),
+    [venv](https://docs.python.org/3/library/venv.html#module-venv),
+    [virtualenv](https://pypi.org/project/virtualenv/)
+-   Linting:
+    [bandit](https://pypi.org/project/bandit/),
+    [flake8](https://pypi.org/project/flake8/),
+    [mypy](https://pypi.org/project/mypy/),
+    [prospector](https://pypi.org/project/prospector/),
+    [pylint](https://pypi.org/project/pylint/),
+    [pydocstyle](https://pypi.org/project/pydocstyle/),
+    [pylama](https://pypi.org/project/pylama/)
+-   Testing:
+    [nose](https://pypi.org/project/nose/),
+    [pytest](https://pypi.org/project/pytest/),
+    [unittest](https://docs.python.org/3/library/unittest.html#module-unittest)
+
+And finally thanks to the [Python](https://www.python.org/) development team and
+community for creating a fantastic programming language and community to be a
+part of!
+
 ## 2020.5.1 (19 May 2020)
 
 ### Fixes

src/client/common/interpreterPathService.ts

@@ -9,6 +9,7 @@ import { ConfigurationChangeEvent, ConfigurationTarget, Event, EventEmitter, Uri
 import { IWorkspaceService } from './application/types';
 import { PythonSettings } from './configSettings';
 import { isTestExecution } from './constants';
+import { traceError } from './logger';
 import { FileSystemPaths } from './platform/fs-paths';
 import {
     IDisposable,
@@ -105,7 +106,8 @@ export class InterpreterPathService implements IInterpreterPathService {
             return;
         }
         if (!resource) {
-            throw new Error('Cannot update workspace settings as no workspace is opened');
+            traceError('Cannot update workspace settings as no workspace is opened');
+            return;
         }
         const settingKey = this.getSettingKey(resource, configTarget);
         const persistentSetting = this.persistentStateFactory.createGlobalPersistentState<string | undefined>(
@@ -149,7 +151,11 @@ export class InterpreterPathService implements IInterpreterPathService {
 
     public async _copyWorkspaceFolderValueToNewStorage(resource: Resource, value: string | undefined): Promise<void> {
         // Copy workspace folder setting into the new storage if it hasn't been copied already
-        const workspaceFolderKey = this.workspaceService.getWorkspaceFolderIdentifier(resource);
+        const workspaceFolderKey = this.workspaceService.getWorkspaceFolderIdentifier(resource, '');
+        if (workspaceFolderKey === '') {
+            // No workspace folder is opened, simply return.
+            return;
+        }
         const flaggedWorkspaceFolderKeysStorage = this.persistentStateFactory.createGlobalPersistentState<string[]>(
             workspaceFolderKeysForWhichTheCopyIsDone_Key,
             []

src/client/telemetry/index.ts

@@ -282,8 +282,8 @@ function serializeStackTrace(ex: Error): string {
             trace += '\n\tat <anonymous>';
         }
     }
-    // Ensure we always use `/` as path seperators.
-    // This way stack traces (with relative paths) comming from different OS will always look the same.
+    // Ensure we always use `/` as path separators.
+    // This way stack traces (with relative paths) coming from different OS will always look the same.
     return trace.trim().replace(/\\/g, '/');
 }
 

src/test/common/interpreterPathService.unit.test.ts

@@ -92,7 +92,7 @@ suite('Interpreter Path Service', async () => {
     test('If the one-off transfer to new storage has not happened yet for the workspace folder, do it and record the transfer', async () => {
         const update = sinon.stub(InterpreterPathService.prototype, 'update');
         const persistentState = TypeMoq.Mock.ofType<IPersistentState<string[]>>();
-        workspaceService.setup((w) => w.getWorkspaceFolderIdentifier(resource)).returns(() => resource.fsPath);
+        workspaceService.setup((w) => w.getWorkspaceFolderIdentifier(resource, '')).returns(() => resource.fsPath);
         persistentStateFactory
             .setup((p) => p.createGlobalPersistentState<string[]>(workspaceFolderKeysForWhichTheCopyIsDone_Key, []))
             .returns(() => persistentState.object);
@@ -112,7 +112,7 @@ suite('Interpreter Path Service', async () => {
     test('If the one-off transfer to new storage has already happened for the workspace folder, do not update and simply return', async () => {
         const update = sinon.stub(InterpreterPathService.prototype, 'update');
         const persistentState = TypeMoq.Mock.ofType<IPersistentState<string[]>>();
-        workspaceService.setup((w) => w.getWorkspaceFolderIdentifier(resource)).returns(() => resource.fsPath);
+        workspaceService.setup((w) => w.getWorkspaceFolderIdentifier(resource, '')).returns(() => resource.fsPath);
         persistentStateFactory
             .setup((p) => p.createGlobalPersistentState<string[]>(workspaceFolderKeysForWhichTheCopyIsDone_Key, []))
             .returns(() => persistentState.object);
@@ -126,6 +126,23 @@ suite('Interpreter Path Service', async () => {
         persistentState.verifyAll();
     });
 
+    test('If no folder is opened, do not do the one-off transfer to new storage for the workspace folder', async () => {
+        const update = sinon.stub(InterpreterPathService.prototype, 'update');
+        const persistentState = TypeMoq.Mock.ofType<IPersistentState<string[]>>();
+        workspaceService.setup((w) => w.getWorkspaceFolderIdentifier(resource, '')).returns(() => '');
+        persistentStateFactory
+            .setup((p) => p.createGlobalPersistentState<string[]>(workspaceFolderKeysForWhichTheCopyIsDone_Key, []))
+            .returns(() => persistentState.object);
+        persistentState.setup((p) => p.value).returns(() => ['...storedWorkspaceKeys']);
+        persistentState.setup((p) => p.updateValue(TypeMoq.It.isAny())).verifiable(TypeMoq.Times.never());
+
+        interpreterPathService = new InterpreterPathService(persistentStateFactory.object, workspaceService.object, []);
+        await interpreterPathService._copyWorkspaceFolderValueToNewStorage(resource, 'workspaceFolderPythonPath');
+
+        assert(update.notCalled);
+        persistentState.verifyAll();
+    });
+
     test('If the one-off transfer to new storage has not happened yet for the workspace, do it and record the transfer', async () => {
         const workspaceFileUri = Uri.parse('path/to/workspaceFile');
         const expectedWorkspaceKey = fs.normCase(workspaceFileUri.fsPath);
@@ -395,7 +412,7 @@ suite('Interpreter Path Service', async () => {
         _didChangeInterpreterEmitter.verifyAll();
     });
 
-    test('Updating workspace settings throws error if no workspace is opened', async () => {
+    test('Updating workspace settings simply returns if no workspace is opened', async () => {
         const expectedSettingKey = `WORKSPACE_FOLDER_INTERPRETER_PATH_${resource.fsPath}`;
         const persistentState = TypeMoq.Mock.ofType<IPersistentState<string | undefined>>();
         workspaceService.setup((w) => w.workspaceFolders).returns(() => undefined);
@@ -408,18 +425,13 @@ suite('Interpreter Path Service', async () => {
             .returns(() => Promise.resolve())
             .verifiable(TypeMoq.Times.never());
 
-        const promise = interpreterPathService.update(
-            resourceOutsideOfWorkspace,
-            ConfigurationTarget.Workspace,
-            interpreterPath
-        );
-        await expect(promise).to.eventually.be.rejectedWith(Error);
+        await interpreterPathService.update(resourceOutsideOfWorkspace, ConfigurationTarget.Workspace, interpreterPath);
 
         persistentState.verifyAll();
         persistentStateFactory.verifyAll();
     });
 
-    test('Updating workspace folder settings throws error if no workspace is opened', async () => {
+    test('Updating workspace folder settings simply returns if no workspace is opened', async () => {
         const expectedSettingKey = `WORKSPACE_FOLDER_INTERPRETER_PATH_${resource.fsPath}`;
         const persistentState = TypeMoq.Mock.ofType<IPersistentState<string | undefined>>();
         workspaceService.setup((w) => w.workspaceFolders).returns(() => undefined);
@@ -432,12 +444,7 @@ suite('Interpreter Path Service', async () => {
             .returns(() => Promise.resolve())
             .verifiable(TypeMoq.Times.never());
 
-        const promise = interpreterPathService.update(
-            resourceOutsideOfWorkspace,
-            ConfigurationTarget.Workspace,
-            interpreterPath
-        );
-        await expect(promise).to.eventually.be.rejectedWith(Error);
+        await interpreterPathService.update(resourceOutsideOfWorkspace, ConfigurationTarget.Workspace, interpreterPath);
 
         persistentState.verifyAll();
         persistentStateFactory.verifyAll();