Use native VSCode coverage APIs, cover swift-testing tests #807
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
plemarquand
force-pushed
the
swift-testing-coverage-3
branch
from
b171b9b to
8af5e10
Compare
plemarquand
requested review from
adam-fowler,
daveyc123,
matthewbastien and
award999
plemarquand
force-pushed
the
swift-testing-coverage-3
branch
5 times, most recently
from
aa0fe65 to
6c0aa09
Compare
plemarquand
force-pushed
the
swift-testing-coverage-3
branch
from
48d2dca to
12613c9
Compare
Move to the new VSCode coverage APIs that were introduced in VSCode version 1.88. This allows us to remove the coverage renderer and coverage report files. This functionality is now built in to the test explorer and VSCode's document rendering. This patch also adds support for producing test coverage from swift-testing tests. XCTests and swift-testing tests are each run in their respective binaries and then the coverage data is merged into a profdata file before being exported to LCOV format. Finally we transform the LCOV JSON into the coverage structures for VSCode and associate this data with the test run.
Refactor common debug config creation
Instead of a build followed by a test run with --skip-build, use a SwiftExecution task to do swift test with the appropriate arguments. This shows the build progress in the status bar.
plemarquand
force-pushed
the
swift-testing-coverage-3
branch
from
12613c9 to
4c44a38
Compare
adam-fowler
reviewed
May 28, 2024
There was a problem hiding this comment.
I'm not sure how ready this is for review, but I can get the coverage view to appear now which looks nice, I don't get inline rendering of coverage in the text editor though. Outside of coverage, parallel testing crashes VSCode which is quite impressive and debugging tests doesn't work anymore.
I think you are going to need to reimplement the two phase build and test instead of relying on swift test to do everything (outside of coverage testing), because you cannot use swift test when debugging tests.
adam-fowler
reviewed
May 29, 2024
plemarquand
force-pushed
the
swift-testing-coverage-3
branch
from
bf0e5d8 to
1c3a35d
Compare
plemarquand
force-pushed
the
swift-testing-coverage-3
branch
2 times, most recently
from
686c1d6 to
40a6ff2
Compare
Use a combination of the xunit xml output and the test output to capture test pass/fail state and issues.
plemarquand
force-pushed
the
swift-testing-coverage-3
branch
from
40a6ff2 to
b78d7a2
Compare
plemarquand
force-pushed
the
swift-testing-coverage-3
branch
from
2aff937 to
1a0a3a3
Compare
|
@adam-fowler thanks for taking such a thorough look. I think everything should be addressed. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Move to the new VSCode coverage APIs that were introduced in VSCode version 1.88. This allows us to remove the coverage renderer and coverage report files. This functionality is now built in to the test explorer and VSCode's document rendering.
This patch also adds support for producing test coverage from swift-testing tests. XCTests and swift-testing tests are each run in their respective binaries and then the coverage data is merged into a profdata file before being exported to LCOV format.
Finally we transform the LCOV JSON into the coverage structures for VSCode and associate this data with the test run.