Skip to content

Detect tfevent files in workspace and prompt to launch TensorBoard #14752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 41 commits into from
Nov 25, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
e5b0540
Prototype
joyceerhl Nov 16, 2020
741c1f3
Fixes
joyceerhl Nov 17, 2020
de479fd
Detect tfevent files and prompt to launch tensorboard
joyceerhl Nov 17, 2020
6d67e96
Format and cleanup
joyceerhl Nov 20, 2020
e8532a2
Prepopulate and validate logDir
joyceerhl Nov 20, 2020
98fcc16
Merge branch 'launch-tensorboard' into tfeventfiles
joyceerhl Nov 20, 2020
11d8bc7
Persist prompt state at workspace level
joyceerhl Nov 20, 2020
c93f0dd
Linter
joyceerhl Nov 20, 2020
47d9980
Add localization keys
joyceerhl Nov 20, 2020
94ef018
Merge branch 'launch-tensorboard' into tfeventfiles
joyceerhl Nov 20, 2020
92e2150
Localization keys
joyceerhl Nov 20, 2020
aaf29fe
Reformat with black
joyceerhl Nov 20, 2020
522b398
Merge branch 'launch-tensorboard' into tfeventfiles
joyceerhl Nov 20, 2020
ba2d841
Weird, typescript didn't complain about this?
joyceerhl Nov 20, 2020
ab468aa
Merge branch 'launch-tensorboard' into tfeventfiles
joyceerhl Nov 20, 2020
2134836
Add disposables to registry and catch errors
joyceerhl Nov 23, 2020
5481d5f
Put command behind feature flag
joyceerhl Nov 23, 2020
ab3c322
Merge branch 'launch-tensorboard' into tfeventfiles
joyceerhl Nov 23, 2020
158d034
Add feature flag
joyceerhl Nov 23, 2020
275be2e
* Add cancellable progress indicator when starting session
joyceerhl Nov 23, 2020
98166bd
Merge branch 'launch-tensorboard' into tfeventfiles
joyceerhl Nov 23, 2020
3eca8ba
Redundant code
joyceerhl Nov 23, 2020
19a31ed
Merge branch 'main' of https://github.com/microsoft/vscode-python int…
joyceerhl Nov 23, 2020
5cc0a14
Merge branch 'launch-tensorboard' into tfeventfiles
joyceerhl Nov 23, 2020
cf0757d
Whoops
joyceerhl Nov 23, 2020
94afab0
Merge branch 'launch-tensorboard' into tfeventfiles
joyceerhl Nov 23, 2020
f0328e3
More linter problems
joyceerhl Nov 23, 2020
bcccc19
Merge branch 'launch-tensorboard' into tfeventfiles
joyceerhl Nov 23, 2020
53a8437
Merge branch 'main' of https://github.com/microsoft/vscode-python int…
joyceerhl Nov 23, 2020
48d3648
Add news and check if in exp in provider
joyceerhl Nov 23, 2020
0746bcf
Merge branch 'launch-tensorboard' into tfeventfiles
joyceerhl Nov 23, 2020
6e82355
Don't create filewatchers if not in experiment
joyceerhl Nov 23, 2020
3c2516c
Add news entry
joyceerhl Nov 23, 2020
89c8294
Remove unnecessary async/await
joyceerhl Nov 23, 2020
037d572
Merge branch 'launch-tensorboard' into tfeventfiles
joyceerhl Nov 23, 2020
7c88cec
Search current dir and one level down only
joyceerhl Nov 23, 2020
217be56
Fixes
joyceerhl Nov 24, 2020
4176b98
Oops
joyceerhl Nov 24, 2020
fa7eb6a
Pass search.exclude
joyceerhl Nov 24, 2020
bd472fc
Merge branch 'launch-tensorboard' into tfeventfiles
joyceerhl Nov 24, 2020
9de9e01
Merge branch 'main' of https://github.com/microsoft/vscode-python int…
joyceerhl Nov 25, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions news/1 Enhancements/14807.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Detect tfevent files in workspace and prompt to launch native TensorBoard session.
4 changes: 2 additions & 2 deletions package.nls.json
Original file line number Diff line number Diff line change
Expand Up @@ -222,9 +222,9 @@
"StartPage.badWebPanelFormatString": "<html><body><h1>{0} is not a valid file name</h1></body></html>",
"Jupyter.extensionRequired": "The Jupyter extension is required to perform that task. Click Yes to open the Jupyter extension installation page.",
"TensorBoard.logDirectoryPrompt" : "Please select a log directory to start TensorBoard with.",
"TensorBoard.installPrompt" : "The package TensorBoard is required in order to launch a TensorBoard session. Would you like to install it?",
"TensorBoard.failedToStartSessionError" : "We failed to start a TensorBoard session due to the following error: {0}",
"TensorBoard.progressMessage" : "Starting TensorBoard session...",
"TensorBoard.failedToStartSessionError" : "We failed to start a TensorBoard session due to the following error: {0}",
"TensorBoard.nativeTensorBoardPrompt" : "VS Code now has native TensorBoard support. Would you like to launch TensorBoard?",
"TensorBoard.usingCurrentWorkspaceFolder": "We are using the current workspace folder as the log directory for your TensorBoard session.",
"TensorBoard.selectAFolder": "Select a folder"
}
3 changes: 2 additions & 1 deletion src/client/common/application/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -795,7 +795,8 @@ export interface IWorkspaceService {
* will be matched against the file paths of resulting matches relative to their workspace. Use a [relative pattern](#RelativePattern)
* to restrict the search results to a [workspace folder](#WorkspaceFolder).
* @param exclude A [glob pattern](#GlobPattern) that defines files and folders to exclude. The glob pattern
* will be matched against the file paths of resulting matches relative to their workspace.
* will be matched against the file paths of resulting matches relative to their workspace. If `undefined` is passed,
* the glob patterns excluded in the `search.exclude` setting will be applied.
* @param maxResults An upper-bound for the result.
* @param token A token that can be used to signal cancellation to the underlying search engine.
* @return A thenable that resolves to an array of resource identifiers. Will return no results if no
Expand Down
13 changes: 10 additions & 3 deletions src/client/common/application/workspace.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,13 +52,13 @@ export class WorkspaceService implements IWorkspaceService {
}
public createFileSystemWatcher(
globPattern: GlobPattern,
_ignoreCreateEvents?: boolean,
ignoreCreateEvents?: boolean,
ignoreChangeEvents?: boolean,
ignoreDeleteEvents?: boolean
): FileSystemWatcher {
return workspace.createFileSystemWatcher(
globPattern,
ignoreChangeEvents,
ignoreCreateEvents,
ignoreChangeEvents,
ignoreDeleteEvents
);
Expand All @@ -69,7 +69,8 @@ export class WorkspaceService implements IWorkspaceService {
maxResults?: number,
token?: CancellationToken
): Thenable<Uri[]> {
return workspace.findFiles(include, exclude, maxResults, token);
const excludePattern = exclude === undefined ? this.searchExcludes : exclude;
return workspace.findFiles(include, excludePattern, maxResults, token);
}
public getWorkspaceFolderIdentifier(resource: Resource, defaultValue: string = ''): string {
const workspaceFolder = resource ? workspace.getWorkspaceFolder(resource) : undefined;
Expand All @@ -79,4 +80,10 @@ export class WorkspaceService implements IWorkspaceService {
)
: defaultValue;
}

private get searchExcludes() {
const searchExcludes = this.getConfiguration('search.exclude');
const enabledSearchExcludes = Object.keys(searchExcludes).filter((key) => searchExcludes.get(key) === true);
return `{${enabledSearchExcludes.join(',')}}`;
}
}
8 changes: 4 additions & 4 deletions src/client/common/utils/localize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -139,14 +139,14 @@ export namespace TensorBoard {
'Please select a log directory to start TensorBoard with.'
);
export const progressMessage = localize('TensorBoard.progressMessage', 'Starting TensorBoard session...');
export const installTensorBoardPrompt = localize(
'TensorBoard.installPrompt',
'The package TensorBoard is required in order to launch a TensorBoard session. Would you like to install it?'
);
export const failedToStartSessionError = localize(
'TensorBoard.failedToStartSessionError',
'We failed to start a TensorBoard session due to the following error: {0}'
);
export const nativeTensorBoardPrompt = localize(
'TensorBoard.nativeTensorBoardPrompt',
'VS Code now has native TensorBoard support. Would you like to launch TensorBoard?'
);
export const usingCurrentWorkspaceFolder = localize(
'TensorBoard.usingCurrentWorkspaceFolder',
'We are using the current workspace folder as the log directory for your TensorBoard session.'
Expand Down
7 changes: 7 additions & 0 deletions src/client/tensorBoard/serviceRegistry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,18 @@

import { IExtensionSingleActivationService } from '../activation/types';
import { IServiceManager } from '../ioc/types';
import { TensorBoardFileWatcher } from './tensorBoardFileWatcher';
import { TensorBoardPrompt } from './tensorBoardPrompt';
import { TensorBoardSessionProvider } from './tensorBoardSessionProvider';

export function registerTypes(serviceManager: IServiceManager) {
serviceManager.addSingleton<IExtensionSingleActivationService>(
IExtensionSingleActivationService,
TensorBoardSessionProvider
);
serviceManager.addSingleton<IExtensionSingleActivationService>(
IExtensionSingleActivationService,
TensorBoardFileWatcher
);
serviceManager.addSingleton<TensorBoardPrompt>(TensorBoardPrompt, TensorBoardPrompt);
}
98 changes: 98 additions & 0 deletions src/client/tensorBoard/tensorBoardFileWatcher.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

import { inject, injectable } from 'inversify';
import { FileSystemWatcher, RelativePattern, WorkspaceFolder, WorkspaceFoldersChangeEvent } from 'vscode';
import { IExtensionSingleActivationService } from '../activation/types';
import { IWorkspaceService } from '../common/application/types';
import { NativeTensorBoard } from '../common/experiments/groups';
import { traceError } from '../common/logger';
import { IDisposableRegistry, IExperimentService } from '../common/types';
import { TensorBoardPrompt } from './tensorBoardPrompt';

@injectable()
export class TensorBoardFileWatcher implements IExtensionSingleActivationService {
private fileSystemWatchers = new Map<WorkspaceFolder, FileSystemWatcher[]>();
private globPattern1 = '*tfevents*';
private globPattern2 = '*/*tfevents*';

constructor(
@inject(IWorkspaceService) private workspaceService: IWorkspaceService,
@inject(TensorBoardPrompt) private tensorBoardPrompt: TensorBoardPrompt,
@inject(IDisposableRegistry) private readonly disposables: IDisposableRegistry,
@inject(IExperimentService) private experimentService: IExperimentService
) {}

public async activate() {
if (!(await this.experimentService.inExperiment(NativeTensorBoard.experiment))) {
return;
}

const folders = this.workspaceService.workspaceFolders;
if (!folders) {
return;
}

// Look for pre-existing tfevent files, as the file watchers will only pick up files
// created or changed after they have been registered and hooked up. Just one will do.
await this.promptIfWorkspaceHasPreexistingFiles();

// If the user creates or changes tfevent files, listen for those too
for (const folder of folders) {
this.createFileSystemWatcher(folder);
}

// If workspace folders change, ensure we update our FileSystemWatchers
this.disposables.push(
this.workspaceService.onDidChangeWorkspaceFolders((e) => this.updateFileSystemWatchers(e))
);
}

private async promptIfWorkspaceHasPreexistingFiles() {
try {
for (const pattern of [this.globPattern1, this.globPattern2]) {
const matches = await this.workspaceService.findFiles(pattern, undefined, 1);
if (matches.length > 0) {
await this.tensorBoardPrompt.showNativeTensorBoardPrompt();
return;
}
}
} catch (e) {
traceError(
`Failed to prompt to launch TensorBoard session based on preexisting tfevent files in workspace: ${e}`
);
}
}

private async updateFileSystemWatchers(event: WorkspaceFoldersChangeEvent) {
for (const added of event.added) {
this.createFileSystemWatcher(added);
}
for (const removed of event.removed) {
const fileSystemWatchers = this.fileSystemWatchers.get(removed);
if (fileSystemWatchers) {
fileSystemWatchers.forEach((fileWatcher) => fileWatcher.dispose());
this.fileSystemWatchers.delete(removed);
}
}
}

private createFileSystemWatcher(folder: WorkspaceFolder) {
const fileWatchers = [];
for (const pattern of [this.globPattern1, this.globPattern2]) {
const relativePattern = new RelativePattern(folder, pattern);
const fileSystemWatcher = this.workspaceService.createFileSystemWatcher(relativePattern);

// When a file is created or changed that matches `this.globPattern`, try to show our prompt
this.disposables.push(
fileSystemWatcher.onDidCreate((_uri) => this.tensorBoardPrompt.showNativeTensorBoardPrompt())
);
this.disposables.push(
fileSystemWatcher.onDidChange((_uri) => this.tensorBoardPrompt.showNativeTensorBoardPrompt())
);
this.disposables.push(fileSystemWatcher);
fileWatchers.push(fileSystemWatcher);
}
this.fileSystemWatchers.set(folder, fileWatchers);
}
}
65 changes: 65 additions & 0 deletions src/client/tensorBoard/tensorBoardPrompt.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

import { inject, injectable } from 'inversify';
import { IApplicationShell, ICommandManager } from '../common/application/types';
import { Commands } from '../common/constants';
import { IPersistentState, IPersistentStateFactory } from '../common/types';
import { Common, TensorBoard } from '../common/utils/localize';

enum TensorBoardPromptStateKeys {
ShowNativeTensorBoardPrompt = 'showNativeTensorBoardPrompt'
}

@injectable()
export class TensorBoardPrompt {
private state: IPersistentState<boolean>;
private enabled: Promise<boolean> | undefined;
private waitingForUserSelection: boolean = false;

constructor(
@inject(IApplicationShell) private applicationShell: IApplicationShell,
@inject(ICommandManager) private commandManager: ICommandManager,
@inject(IPersistentStateFactory) private persistentStateFactory: IPersistentStateFactory
) {
this.state = this.persistentStateFactory.createWorkspacePersistentState<boolean>(
TensorBoardPromptStateKeys.ShowNativeTensorBoardPrompt,
true
);
this.enabled = this.isPromptEnabled();
}

public async showNativeTensorBoardPrompt() {
if ((await this.enabled) && !this.waitingForUserSelection) {
const yes = Common.bannerLabelYes();
const no = Common.bannerLabelNo();
const doNotAskAgain = Common.doNotShowAgain();
const options = [yes, no, doNotAskAgain];
this.waitingForUserSelection = true;
const selection = await this.applicationShell.showInformationMessage(
TensorBoard.nativeTensorBoardPrompt(),
...options
);
this.waitingForUserSelection = false;
switch (selection) {
case yes:
await this.commandManager.executeCommand(Commands.LaunchTensorBoard);
await this.disablePrompt();
break;
case doNotAskAgain:
await this.disablePrompt();
break;
default:
break;
}
}
}

private async isPromptEnabled() {
return this.state.value;
}

private async disablePrompt() {
await this.state.updateValue(false);
}
}