Skip to content

Instead of sending a message to the index log when an indexing task finishes, stream results as they come in #1382

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 4, 2024

Conversation

ahoppen
Copy link
Member

@ahoppen ahoppen commented May 31, 2024

This also means that you can use the index log to view which tasks are currently being executed.

Since we only have a single log stream we can write to, I decided to prefix every line in the index log with two colored emojis that an easy visual association of every log line to the task that generated them.

This is how an index log now looks like in VS Code

[Info  - 9:49:31 AM] ⬜️🟥 [1/1] Compiling plugin Format Source Code
[Info  - 9:49:31 AM] ⬜️🟥 Finished in 0.51599975 seconds
[Info  - 9:49:31 AM] 🟥🟥 Preparing GenerateManual for dummy
/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swift build --package-path /Users/alex/src/sourcekit-lsp --scratch-path /Users/alex/src/sourcekit-lsp/.index-build --disable-index-store --target GenerateManual
[Info  - 9:49:32 AM] 🟧🟩 Indexing /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-format/Plugins/FormatPlugin/plugin.swift
/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swiftc -swift-version 5 -I /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/ManifestAPI -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk -package-description-version 5.6.0 /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-format/Plugins/FormatPlugin/plugin.swift -index-file -index-file-path /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-format/Plugins/FormatPlugin/plugin.swift -disable-batch-mode -index-unit-output-path /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-format/Plugins/FormatPlugin/plugin.swift.o
[Info  - 9:49:32 AM] 🟧🟩 /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-format/Plugins/FormatPlugin/plugin.swift:1:8: error: no such module 'PackagePlugin'
[Info  - 9:49:32 AM] 🟧🟩  1 | import PackagePlugin
[Info  - 9:49:32 AM] 🟧🟩    |        `- error: no such module 'PackagePlugin'
[Info  - 9:49:32 AM] 🟧🟩  2 | import Foundation
[Info  - 9:49:32 AM] 🟧🟩  3 | 
[Info  - 9:49:32 AM] 🟧🟩 Finished in 0.205038459 seconds
[Info  - 9:49:32 AM] 🟥🟥 [1/1] Compiling plugin GenerateManual
[Info  - 9:49:32 AM] 🟥🟥 Finished in 0.522686625 seconds
[Info  - 9:49:32 AM] 🟦🟨 Preparing _CryptoExtras for dummy
/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swift build --package-path /Users/alex/src/sourcekit-lsp --scratch-path /Users/alex/src/sourcekit-lsp/.index-build --disable-index-store --target _CryptoExtras
[Info  - 9:49:32 AM] 🟩⬜️ Indexing /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPlugin.swift
/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swiftc -swift-version 5 -I /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/ManifestAPI -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk -package-description-version 5.6.0 /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPlugin.swift /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPluginError.swift /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/PackagePlugin+Helpers.swift -index-file -index-file-path /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPlugin.swift -disable-batch-mode -index-unit-output-path /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPlugin.swift.o
[Info  - 9:49:32 AM] 🟧⬛️ Indexing /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPluginError.swift
/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swiftc -swift-version 5 -I /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/ManifestAPI -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk -package-description-version 5.6.0 /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPlugin.swift /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPluginError.swift /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/PackagePlugin+Helpers.swift -index-file -index-file-path /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPluginError.swift -disable-batch-mode -index-unit-output-path /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPluginError.swift.o
[Info  - 9:49:32 AM] 🟩🟪 Indexing /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/PackagePlugin+Helpers.swift
/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swiftc -swift-version 5 -I /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/ManifestAPI -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk -package-description-version 5.6.0 /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPlugin.swift /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPluginError.swift /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/PackagePlugin+Helpers.swift -index-file -index-file-path /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/PackagePlugin+Helpers.swift -disable-batch-mode -index-unit-output-path /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/PackagePlugin+Helpers.swift.o
[Info  - 9:49:32 AM] 🟩🟪 /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPlugin.swift:12:8: error: no such module 'PackagePlugin'
[Info  - 9:49:32 AM] 🟩🟪 10 | //===----------------------------------------------------------------------===//
[Info  - 9:49:32 AM] 🟩🟪 11 | 
[Info  - 9:49:32 AM] 🟩🟪 12 | import PackagePlugin
[Info  - 9:49:32 AM] 🟩🟪    |        `- error: no such module 'PackagePlugin'
[Info  - 9:49:32 AM] 🟩🟪 13 | 
[Info  - 9:49:32 AM] 🟩🟪 14 | @main
[Info  - 9:49:32 AM] 🟩⬜️ /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPlugin.swift:12:8: error: no such module 'PackagePlugin'
[Info  - 9:49:32 AM] 🟩⬜️ 10 | //===----------------------------------------------------------------------===//
[Info  - 9:49:32 AM] 🟩⬜️ 11 | 
[Info  - 9:49:32 AM] 🟩⬜️ 12 | import PackagePlugin
[Info  - 9:49:32 AM] 🟩⬜️    |        `- error: no such module 'PackagePlugin'
[Info  - 9:49:32 AM] 🟩⬜️ 13 | 
[Info  - 9:49:32 AM] 🟩⬜️ 14 | @main
[Info  - 9:49:32 AM] 🟧⬛️ /Users/alex/src/sourcekit-lsp/.index-build/checkouts/swift-argument-parser/Plugins/GenerateManual/GenerateManualPlugin.swift:12:8: error: no such module 'PackagePlugin'
[Info  - 9:49:32 AM] 🟧⬛️ 10 | //===----------------------------------------------------------------------===//
[Info  - 9:49:32 AM] 🟧⬛️ 11 | 
[Info  - 9:49:32 AM] 🟧⬛️ 12 | import PackagePlugin
[Info  - 9:49:32 AM] 🟧⬛️    |        `- error: no such module 'PackagePlugin'
[Info  - 9:49:32 AM] 🟧⬛️ 13 | 
[Info  - 9:49:32 AM] 🟧⬛️ 14 | @main
[Info  - 9:49:32 AM] 🟩⬜️ Finished in 0.159845792 seconds
[Info  - 9:49:32 AM] 🟧⬛️ Finished in 0.159851625 seconds
[Info  - 9:49:32 AM] 🟩🟪 Finished in 0.159853 seconds

@ahoppen ahoppen requested review from bnbarham and hamishknight May 31, 2024 16:53
@ahoppen ahoppen requested a review from benlangmuir as a code owner May 31, 2024 16:53
@ahoppen
Copy link
Member Author

ahoppen commented May 31, 2024

@swift-ci Please test

@ahoppen ahoppen force-pushed the stream-index-log branch from 63079dd to 8480b4e Compare May 31, 2024 22:57
@ahoppen
Copy link
Member Author

ahoppen commented May 31, 2024

@swift-ci Please test

@ahoppen
Copy link
Member Author

ahoppen commented May 31, 2024

@swift-ci Please test Windows

ahoppen added a commit to ahoppen/sourcekit-lsp that referenced this pull request Jun 2, 2024
The `showActivePreparationTasksInProgress` didn’t turn out to be terribly useful and with the streaming index log (swiftlang#1382), we should have the functionality to view which index tasks are currently running. So, we should remove the feature.

rdar://129109830
ahoppen added a commit to ahoppen/sourcekit-lsp that referenced this pull request Jun 3, 2024
The `showActivePreparationTasksInProgress` didn’t turn out to be terribly useful and with the streaming index log (swiftlang#1382), we should have the functionality to view which index tasks are currently running. So, we should remove the feature.

rdar://129109830
Comment on lines 950 to 951
logMessageToIndexLog: { taskID, message in
self.logMessageToIndexLog(taskID: taskID, message: message)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No weak self?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, should have made this comment sooner, but should we consider using a delegate or something to cut down on the amount of closures we have to pass around?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m not a fan of delegates for exactly the weak self reason. If this becomes a delegate then it’s the responsibility of Workspace to decide whether self (or the delegate) should be captured strongly or weakly. But I think that decision should be made on every call-side.


private func handleDataFromPipeImpl(_ newData: Data) {
self.buffer += newData
while let newlineIndex = self.buffer.firstIndex(of: UInt8(ascii: "\n")) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scanning the whole buffer every time doesn't seem ideal, but I guess lines aren't that long in practice?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the expectation is that lines aren’t long and I can’t think of common scenarios where you get less than a full line from a pipe.

@@ -931,8 +947,8 @@ extension SourceKitLSPServer {
options: options,
indexOptions: self.options.indexOptions,
indexTaskScheduler: indexTaskScheduler,
indexProcessDidProduceResult: { [weak self] in
self?.indexTaskDidProduceResult($0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is indexTaskDidProduceResult unused now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, yes. Thanks for catching! Could also remove IndexProcessResult now.

@ahoppen ahoppen force-pushed the stream-index-log branch 3 times, most recently from 5af1e0c to 90f1f2f Compare June 3, 2024 15:55
@ahoppen
Copy link
Member Author

ahoppen commented Jun 3, 2024

@swift-ci Please test

@ahoppen
Copy link
Member Author

ahoppen commented Jun 3, 2024

@swift-ci Please test Windows

@ahoppen ahoppen enabled auto-merge June 3, 2024 17:34
…inishes, stream results as they come in

This also means that you can use the index log to view which tasks are currently being executed.

Since we only have a single log stream we can write to, I decided to prefix every line in the index log with two colored emojis that an easy visual association of every log line to the task that generated them.
@ahoppen ahoppen force-pushed the stream-index-log branch from 90f1f2f to 09ad77b Compare June 3, 2024 20:22
@ahoppen
Copy link
Member Author

ahoppen commented Jun 3, 2024

@swift-ci Please test

@ahoppen
Copy link
Member Author

ahoppen commented Jun 3, 2024

@swift-ci Please test Windows

@ahoppen
Copy link
Member Author

ahoppen commented Jun 4, 2024

@swift-ci Please test macOS

@ahoppen ahoppen merged commit fff9eb5 into swiftlang:main Jun 4, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants