NOT TO BE MERGED Ds custom editor squashed (#1543)

[DonJayamanne]

• Setup usage of the new API

• Implement base custom editor support (Implement base custom editor support #9812)

• Setup usage of the new API

• Partial ideas

• Idea for splitting

• Idea in place

• Get all of the tests to build

• Enable proposed api

• Still not working but setting more options.

• Get web views to load

• Fix save and save as to use VS code commands

• Fix unit tests

• Fix a number of functional tests

• REmove autoSave tests as not needed anymore

• Add news entry

• Review comments

• Code review comments

• Update PR validation list

• Fix nyc compiler problems.

• Try fixing the toggle markdown test

• Ids of new cells must be provided to actions (Ids of new cells must be provided to actions #9833)

• ids of new cells must be provided to actions

• Comments

• Oops

• Use new filesystem in DataScience functional tests (Use new filesystem in DataScience functional tests  #9839)

• Fix merge problem

• Run tests against insiders (Run tests against insiders #9872)

• Run tests against insiders

• Disable noisy logging

• Restore changes

• Ensure we have a base type for all action payloads (Ensure we have a base type for all action payloads #9840)

For #9340

Ensures all actions have a base type, to pass information such whether the action was invoked to sync state for multiple editors (pointing to same file) in same session or across sessions.
This PR only adds the ability to pass such new information to all actions.

• Remove root level flags used to track cell selection and focus (Remove root level flags used to track cell selection and focus #9882)

For #9340
To be merged after #9840

Basically I got rid of the root level state info that keeps track of the cell that's selected and focused.
This required us to keep the two in sync along with the state at the cell level.
When syncing information between multiple editors this got messy. removing this removes the issues where things could go out of sync, i.e. two ways to storing the same thing. Now we get selected information from the cells directly - single source of truth.

• Synchronize contents in multiple editors for same nb file (Synchronize contents in multiple editors for same nb file #9886)

For #9340
Note

This PR adds the infrastructure necessary to sync edits & the like across editors for the same file
Code has been commented out to ensure some remaining bits work properly
Waiting on some of the new changes from @rchiodo from his undo PR.
Remaining changes

Ensuring we do not end up in a crazy loop that will cause one editor to propagate changes to the other, then that one back to this & so on
Sync cell execution (currently creation of new cells isn't sync'ed across)
Find a bullet proof solution for syncing focused states (this messes up everything - we don't need this in syncing editors, but if required for live share, then we need to ensure focus states work correctly - depending on whether liveshare even needs such a feature?)

• Address code review comments

• Fix test

• Fix linter

• Remove unused property

• More changes

• Run tests against insiders (Run tests against insiders #9919)

• We have code in extension that displays a dialog (preventing extension from loading) in insiders version of VSC.

• Run tests against insiders

• Restore env variable

• Implement undo support for the custom editor (Implement undo support for the custom editor #9946)

• Partially working undo/redo

• Concept created using update message/command

• Put the storage object in the global map

• Look for more data during undo/redo

• New idea with reverse edit and disable undo/redo in monaco

• Potentially working with tracking edits

• Fix completions for interactive

• Fix unit tests

• First working functional test of undo

• More testing of undo

• More tests

• Close suggestions on undo

• Fix other functional tests

• Fix undo misses

• Fix focus changes on undo

• Fixup after merge

• Add news entry

• Put back launch.json

• Get rid of asyncness of sendCellsToWebView

• Add a comment about why we remove \r

• Prevent swapCells from doing anything if same cell

• Fix changed event for clear

• Actually remove all state from editor class. Try to fix undo on save

• Fix func test in DS failing due to FS issues (Fix func test in DS failing due to FS issues #9995)

• Fix dirty state not matching save button state (Fix dirty state not matching save button state #9993)

• Fix dirty state not matching save button state

• Fix modify to update UI too
  Make a 'save' point for the undo count. Looks like this is what the editor is doing.

• Minor changes to dispatchers (action creators will dispatch multiple actions) (Minor changes to dispatchers (action creators will dispatch multiple actions) #9920)

• Fixes

• Remove cyclic messages

• Misc

• Address code review comments

• Oops

• More oops

• More fixes

• Address code review comments

• Address code review comments

• Fix tests

• Retry flaky test 3 times.

• Fixes

• Fix problem with undoing entire stack (Fix problem with undoing entire stack #10018)

• Refactor how messages are posted to reducers and to the extension (Refactor how messages are posted to reducers and to the extension #10019)

Rather than having a developer call create and then arg.queue, this replaces that with one function.
Hides the intricacies of how something is posted or queued.
Also allows us to add additional logic to control what messages can be posted (for synchronization) instead of developer having to do this in every reducer.

Simple harmless refactor

• Complete refactoring of creation and queueing of actions (Complete refactoring of creation and queueing of actions #10036)

• Some how I had not included some of the other remaining arg.queueAction into the original
  PR Refactor how messages are posted to reducers and to the extension #10019

• This PR merely brings them from (I had these in the same branch, but failed to include them).

• Same as Refactor how messages are posted to reducers and to the extension #10019, just a refactor to use a common function to create an action and dispatch it in one step. & removed some of the old syncing stuff.

• Prettier

• Get everything building again

• Fix source maps and undoing the first cell

• Fix order of cells inserted (Fix order of cells inserted #10229)

Cells weren't inserted into the model in the right order.

• Insert cell using a
• Insert cell using b
• Save NB and reopen
• All the new cells are at the very bottom

Added https://github.com/microsoft/vscode-python/issues/10230 and moved into backlog and assigned to ds_editor epic. I think in light of this issue it must be done. Else it'll be bad for users to report this.

• Fixes to ds custom editor (Fixes to ds custom editor #10257)

• Fix more ds functional tests (Fix more ds functional tests #10260)

• Eureka

• misc

• Merge master into ds custom (Merge master into ds custom #10261)

• fixing a typo in CONTRIBUTING.md (fixing a typo in CONTRIBUTING.md #10044)

• fix for button backgrounds (Fix Native Editor button backgrounds #10234)

• add kernel connection sys info string (add kernel connection sys info string #10236)

• Fix to return env variables of interpreter that is not current interpreter (Fix to return env variables of interpreter that is not current interpreter #10251)

For #10250

• Bug fix

• Support jupyter output in the remote scenario too (Support jupyter output in the remote scenario too #10241)

• Add some output for remote situations

• Add news entry

• Skip flaky FS test (Skip flaky FS test #10244)

Merely disables test for #10240

• Switch experiments settings scope to machine (Switch experiments settings scope to machine #10237)

• Switch experiments settings scope to machine

• Add news item

• Show quickfixes for launch.json (Show quickfixes for launch.json #10245)

• Added implementation

• Added tests

• Moved into providers

• Add code actions vscode mock and convert .test.ts into unit.test.ts

• Rename folders

• Dispose registered services

• News entry

• Added unit tests

• Rename core service

• Reduce the number of build agents used by CI. (Reduce the number of build agents used by CI. #10221)

• Drop extra jobs from the PR-validation pipeline.

This reduces the number of build agents we are using most frequently,
without sacrificing much coverage. Note that this relies on 2 things:

• sufficient unit test coverage
• manual "full CI" runs when there may be OS-specific concerns

With this change:

• all tests get run on 3.x on linux (including smoke)
• for 2.7 only the unit and functional tests are run (and only on linux)
• on Windows and OSX, only functional and "single workspace" tests are run

The total number of test jobs (and hence agents) drops from 13 to 8.

• Drop extra jobs from the PR-merge pipeline.

This change relies on the nightly CI run covering the full matrix.

The total number of test jobs (and hence agents) drops from 39 to 24.

• Limit # of parallel jobs in each job matrix in the PR-merge and nightly pipelines.

This reduces the max number of agents used by the PR-merge pipeline to 8
(from 24). For the nightly pipeline it goes down from 84 to 12. Both
will take longer to complete, but that shouldn't be a major problem.

• Add a "manual build" CI pipeline for faster turnaround.

• Do not test internal tools on lower than Python 3.7.

• Drop the temporary workaround jobs.

• (again) Do not test internal tools on lower than Python 3.7.

• Improve the perf of functional tests running with real jupyter (Improve the perf of functional tests running with real jupyter #10242)

• Improve the perf of subsequent tests by caching interpreters

• Add back the nightly flake

• Remove coverage

• Add news entry

• Use a static map to allow promise to be cleared on new interpreters (as it was before)

Co-authored-by: Panpan Lin [email protected]
Co-authored-by: Ian Huff [email protected]
Co-authored-by: Rich Chiodo [email protected]
Co-authored-by: Karthik Nadig [email protected]
Co-authored-by: Kartik Raj [email protected]
Co-authored-by: Eric Snow [email protected]

• Fix startup and shutdown tests (Fix DS startup and shutdown functional tests #10284)

• Merge master again into ds/custom_editor (Merge master again into ds/custom_editor #10295)

• fixing a typo in CONTRIBUTING.md (fixing a typo in CONTRIBUTING.md #10044)

• fix for button backgrounds (Fix Native Editor button backgrounds #10234)

• add kernel connection sys info string (add kernel connection sys info string #10236)

• Fix to return env variables of interpreter that is not current interpreter (Fix to return env variables of interpreter that is not current interpreter #10251)

For #10250

• Bug fix

• Support jupyter output in the remote scenario too (Support jupyter output in the remote scenario too #10241)

• Add some output for remote situations

• Add news entry

• Skip flaky FS test (Skip flaky FS test #10244)

Merely disables test for #10240

• Switch experiments settings scope to machine (Switch experiments settings scope to machine #10237)

• Switch experiments settings scope to machine

• Add news item

• Show quickfixes for launch.json (Show quickfixes for launch.json #10245)

• Added implementation

• Added tests

• Moved into providers

• Add code actions vscode mock and convert .test.ts into unit.test.ts

• Rename folders

• Dispose registered services

• News entry

• Added unit tests

• Rename core service

• Reduce the number of build agents used by CI. (Reduce the number of build agents used by CI. #10221)

• Drop extra jobs from the PR-validation pipeline.

This reduces the number of build agents we are using most frequently,
without sacrificing much coverage. Note that this relies on 2 things:

• sufficient unit test coverage
• manual "full CI" runs when there may be OS-specific concerns

With this change:

• all tests get run on 3.x on linux (including smoke)
• for 2.7 only the unit and functional tests are run (and only on linux)
• on Windows and OSX, only functional and "single workspace" tests are run

The total number of test jobs (and hence agents) drops from 13 to 8.

• Drop extra jobs from the PR-merge pipeline.

This change relies on the nightly CI run covering the full matrix.

The total number of test jobs (and hence agents) drops from 39 to 24.

• Limit # of parallel jobs in each job matrix in the PR-merge and nightly pipelines.

This reduces the max number of agents used by the PR-merge pipeline to 8
(from 24). For the nightly pipeline it goes down from 84 to 12. Both
will take longer to complete, but that shouldn't be a major problem.

• Add a "manual build" CI pipeline for faster turnaround.

• Do not test internal tools on lower than Python 3.7.

• Drop the temporary workaround jobs.

• (again) Do not test internal tools on lower than Python 3.7.

• Improve the perf of functional tests running with real jupyter (Improve the perf of functional tests running with real jupyter #10242)

• Improve the perf of subsequent tests by caching interpreters

• Add back the nightly flake

• Remove coverage

• Add news entry

• Use a static map to allow promise to be cleared on new interpreters (as it was before)

• Merge release branch into master (Merge release branch into master #10258)

• Davidkutu/ds interactive window focus fix (Davidkutu/ds interactive window focus fix #9895)

• Fixed the focus on the interactive window
  when pressing ctrl +2

• the new prop 'focusPending' needs to change in
  order to give focus

• added functional test

• In test: force focus to buttons to take it away from

• finished the functional test

• updated changelog

• deleted bug#.md

• Fix variables on mac and linux (Fix variables on mac and linux #9897) (Fix variables on mac and linux  #9903)

• Fix regexes to work on other platforms

• Fix data frame viewer

• Fix linter problems

Co-authored-by: Don Jayamanne [email protected]

Co-authored-by: Don Jayamanne [email protected]

• Fix focusing of cells when navigating cells using up/down arrow (Fix focusing of cells when navigating cells using up/down arrow #9885) (Fix focusing of cells when navigating cells using up/down arrow (#9885) #9908)

For #9884

• Disable webHostNotebook experiment and fix name of exp (Disable webHostNotebook experiment and fix name of exp #9894) (Disable webHostNotebook experiment and fix name of exp (#9894) #9907)

Disable an experiment
Fix name of an existing experiment

• Davidkutu/ds scrapbook render fix (Davidkutu/ds scrapbook render fix #9912)

• check if we support mimetypes before
  loading html to avoid crashing

• Silently skip rendering of the scapbook mime types

• added news file

• updated changelog and deleted news file

• Disable the terminal activation experiment (Disable the terminal activation experiment #9937)

Disable until next release of VSC (for hidden terminals).
Disabled because a terminal gets displayed to the user and then hidden (doesn't happen in insiders, we will need to wait for their new API to hide terms)

• Notebook should use kernel from metadata (Notebook should use kernel from metadata #9936)

For #9927

• Notebook should use kernel from metadata

• Fix formatting

• Port server cache fix to release (Port server cache fix to release #9932)

• Port variable restart fix (Port variable restart fix #9938)

• Fix variable explorer when restarting a kernel (Fix variable explorer when restarting a kernel #9931)

• Make sure to clear variable list on reset

• Add news entry

• Update changelog

• Update changelog

• Do not override kernels info in other notebooks (Do not override kernels info in other notebooks #9958)

For #9951, #9950, #9949

• Do not override kernels info in other notebooks

• Fixes

• Disables the use of a terminal to activate an environment (Disables the use of a terminal to activate an environment #9967)

• Disables the use of a terminal to activate an environment

• Fix tests

• Fix linter

• Port server name fix to release (Port server name fix to release #9970)

• When starting Jupyter activate conda env (When starting Jupyter activate conda env #9972)

• If interpreter is provided use that to check if is conda environment

• Fix tests
  For #9973
  Currently conda isn't getting activated when auto starting Jupyter server.

• Porting K8 fix to release (Porting K8 fix to release #9987)

• Update version and changelog date for release (Update version and changelog date for release #10022)

• Ignore FS errors from stat() in some cases. (Ignore FS errors from stat() in some cases. #10032)

(for #9901)

The earlier FS code ignored errors in a couple of situations. Recent changes inadvertently stopped ignoring them. This change restores the earlier behavior (and adds logging of the errors).

This is a backport of #10021.

• Update chang log (Update change log for release #10041)

• Install jupyter instead of installing kernel spec (Install jupyter instead of installing kernel spec #10080) (Install jupyter instead of installing kernel spec (#10080) #10083)

• Install jupyter instead of installing kernel spec
  For #10071

• Fix for zh-tw loading issue. (Fix for zh-tw loading issue. #10085) (cherrypick into release: Fix for zh-tw loading issue. (#10085) #10089)

• Update version and changelog for point release. (Update version and changelog for point release. #10091)

• Update version and changelog for point release.

• Add note to change log.

• Port fixes for second point release. (Port fixes for second point release. #10146)

• Better messaging on notebook fail (Better messaging on notebook fail #10056)

• Install jupyter instead of installing kernel spec (Install jupyter instead of installing kernel spec #10080)

• Install jupyter instead of installing kernel spec
  For #10071

• Eliminate variable value when computing data frame info (Eliminate variable value when computing data frame info #10081)

• Fix ndarray types to be viewable again (Fix ndarray types to be viewable again #10093)

• Eliminate variable value when computing data frame info

• Fix ndarrays to work again
  Add test to make sure we don't regress this again

• Rchiodo/kernel telemetry (Rchiodo/kernel telemetry #10115)

• Add duration to select local/remote kernel

• Add notebook language telemetry

• Add news entries

• #9883 telemetry

• News entry

• Another spot for kernel spec failure

• Add telemetry on product install

• Fix install telemetry

• Undo launch.json change

• Handle other cases

• Better way to handle case

• Wrong event for jupyter install

• Fix unit tests

• Clear variables when restarting regardless if visible or not (Clear variables when restarting regardless if visible or not #10117)

• Use different method for checking if kernelspec is available (Use different method for checking if kernelspec is available #10114)

• Use different method for checking if kernelspec is available

• Fix unit tests

• More logging for kernelspec problems (More logging for kernelspec problems #10132)

• More logging for kernelspec problems

• Actually capture the exception on the new code

• Not actually using output if first exception still there.

• Actually only return output on one of the expected calls.

• Fix nightly flake

• Check our saved jupyter interpreters before allowing any of them to be used as active interpreters (Check our saved jupyter interpreters before allowing any of them to be used as active interpreters #10113)

• Update changelog and package.json

• Missing part of changelog

• Fix tests and linter problems

Co-authored-by: Ian Huff [email protected]
Co-authored-by: Don Jayamanne [email protected]

• Cherry pick more fixes for feb point2 (Cherry pick more fixes for feb point2 #10183)

• Ensure DS auto start code is non-blocking (Ensure DS auto start code is non-blocking #10171)

• Perf fixes

• Add news entry

• Include interperter name in message (Include interpreter name in message #10174)

For #10071
Fix to ensure interpreter is included in message, here's the actual message seen by the user.

Error: Jupyter cannot be started. Error attempting to locate jupyter: 'Kernelspec' module not installed in the selected interpreter ({0}).

Note the {0}

• Ignore display name when searching interpreters (Ignore display name when searching kernels matching an interpreter #10175)

Partial fix for #10173
Comparing against path of interpreter alone is sufficient.

Figured there's no harm in trying to minimise occurrences of dup kernels (basically just compare against python executable path, as thats sufficient and always accurate)

Display name:

Can change from version to version of Python extension (i.e. it shouldn't have been used as a unique identifier)
Display name can change after extension loads and more information about interpreter is available.

• Track cold/warm times to execute notebook cells (Track cold/warm times to execute notebook cells #10180)

For #10176

• fix SVG mimetype name (fix SVG mimetype name #10181) (Port to Release: fix SVG mimetype name (#10181) #10185)

• Update change log

• Some more ds telemetry (Some more ds telemetry #10218)

• Add telemetry to capture perceived time taken to start jupyter and run cell (Add telemetry to capture perceived time taken to start jupyter and run cell #10217)

• Add telemetry

• Only for nb
  For #10098
  For #10212

• Fix change log

• Fix linter

• Update date in change log

• Point release with fix for Issue 10250 (Point release with fix for Issue 10250 #10254)

• Fix to return env variables of interpreter that is not current interpreter (Fix to return env variables of interpreter that is not current interpreter #10251)
  For #10250

• Bug fix

• Update change log

• Fix date

• Oops

• Fix linter

• Clean up

• More clean up

• Some more text fixes

Co-authored-by: David Kutugata [email protected]
Co-authored-by: Rich Chiodo [email protected]
Co-authored-by: Don Jayamanne [email protected]
Co-authored-by: Ian Huff [email protected]
Co-authored-by: Eric Snow [email protected]

Co-authored-by: Panpan Lin [email protected]
Co-authored-by: Ian Huff [email protected]
Co-authored-by: Rich Chiodo [email protected]
Co-authored-by: Karthik Nadig [email protected]
Co-authored-by: Kartik Raj [email protected]
Co-authored-by: Eric Snow [email protected]
Co-authored-by: David Kutugata [email protected]

• Restore old webview provider

• Ooops

• Misc

• Fix typos and linter issues

• Save and save as

• Fix test

• More fixes

• Add undo/redo

Co-authored-by: Rich Chiodo [email protected]
Co-authored-by: Panpan Lin [email protected]
Co-authored-by: Ian Huff [email protected]
Co-authored-by: Karthik Nadig [email protected]
Co-authored-by: Kartik Raj [email protected]
Co-authored-by: Eric Snow [email protected]
Co-authored-by: David Kutugata [email protected]

For #

• Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR).
• Title summarizes what is changing.
• Has a news entry file (remember to thank yourself!).
• Appropriate comments and documentation strings in the code.
• Has sufficient logging.
• Has telemetry for enhancements.
• Unit tests & system/integration tests are added/updated.
• Test plan is updated as appropriate.
• package-lock.json has been regenerated by running npm install (if dependencies have changed).
• The wiki is updated with any design decisions/details.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
3 Code Smells

No Coverage information
1.4% Duplication