Skip to content

Discover test content stored in the test content section of loaded images. #893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 7, 2025

Conversation

grynspan
Copy link
Contributor

@grynspan grynspan commented Jan 6, 2025

This PR implements discovery, but not emission, of test content that has been added to a loaded image's test content metadata section at compile time.

Loading this data from a dedicated section has several benefits over our current model, which involves walking Swift's type metadata table looking for types that conform to a protocol:

  1. We don't need to define that protocol as public API in Swift Testing,
  2. We don't need to emit type metadata (much larger than what we really need) for every test function,
  3. We don't need to duplicate a large chunk of the Swift ABI sources in order to walk the type metadata table correctly, and
  4. Almost all the new code is written in Swift, whereas the code it is intended to replace could not be fully represented in Swift and needed to be written in C++.

The change also opens up the possibility of supporting generic types in the future because we can emit metadata without needing to emit a nested type (which is not always valid in a generic context.) That's a "future direction" and not covered by this PR specifically.

I've defined a layout for entries in the new swift5_tests section that should be flexible enough for us in the short-to-medium term and which lets us define additional arbitrary test content record types. The layout of this section is covered in depth in the new TestContent.md article.

This PR does not include the code necessary to emit test content records into images at compile time. That part of the change is covered by #880 and requires a new language feature to control which section data is emitted to. An experimental version of that language feature is currently available under the "SymbolLinkageMarkers" label.

Because there is no test content in the test content section yet, this PR does not remove the "legacy" codepaths that discover tests in the type metadata section.

Note

This change is experimental.

See Also

#880 #735 swiftlang/swift#76698
swiftlang/swift#78411

Checklist:

  • Code and documentation should follow the style of the Style Guide.
  • If public symbols are renamed or modified, DocC references should be updated.

…ages.

This PR implements discovery, _but not emission_, of test content that has been
added to a loaded image's test content metadata section at compile time.

Loading this data from a dedicated section has several benefits over our current
model, which involves walking Swift's type metadata table looking for types that
conform to a protocol:

1. We don't need to define that protocol as public API in Swift Testing,
1. We don't need to emit type metadata (much larger than what we really need)
   for every test function,
1. We don't need to duplicate a large chunk of the Swift ABI sources in order to
   walk the type metadata table correctly, and
1. Almost all the new code is written in Swift, whereas the code it is intended
   to replace could not be fully represented in Swift and needed to be written
   in C++.

The change also opens up the possibility of supporting generic types in the
future because we can emit metadata without needing to emit a nested type (which
is not always valid in a generic context.) That's a "future direction" and not
covered by this PR specifically.

I've defined a layout for entries in the new `swift5_tests` section that should
be flexible enough for us in the short-to-medium term and which lets us define
additional arbitrary test content record types. The layout of this section is
covered in depth in the new [TestContent.md](Documentation/ABI/TestContent.md)
article.

This PR does **not** include the code necessary to _emit_ test content records
into images at compile time. That part of the change is covered by #880 and
requires a new language feature to control which section data is emitted to. An
experimental version of that language feature is currently available under the
`"SymbolLinkageMarkers"` label.

Because there is no test content in the test content section yet, this PR does
not remove the "legacy" codepaths that discover tests in the type metadata
section.

> [!NOTE]
> This change is experimental.

 ## See Also

#880
#735
swiftlang/swift#76698
swiftlang/swift#78411
@grynspan grynspan added enhancement New feature or request tools integration 🛠️ Integration of swift-testing into tools/IDEs windows 🪟 Windows support linux 🐧 Linux support (all distros) darwin 🍎 macOS, iOS, watchOS, tvOS, and visionOS support performance 🏎️ Performance issues embedded-swift 📟 Embedded Swift issues wasi/wasm 🧭 WebAssembly support android 🤖 Android support freebsd 😈 FreeBSD support openbsd 🐡 OpenBSD support labels Jan 6, 2025
@grynspan grynspan added this to the Swift 6.x milestone Jan 6, 2025
@grynspan grynspan self-assigned this Jan 6, 2025
@grynspan
Copy link
Contributor Author

grynspan commented Jan 6, 2025

@swift-ci test

@grynspan
Copy link
Contributor Author

grynspan commented Jan 7, 2025

@swift-ci test

@grynspan grynspan merged commit c8998c7 into main Jan 7, 2025
3 checks passed
@grynspan grynspan deleted the jgrynspan/test-section-discovery branch January 7, 2025 16:45
@grynspan grynspan added the less-c++ 🐀 Work to reduce the size of our C++ codebase and/or dependencies label Jan 19, 2025
@grynspan grynspan added the discovery 🔎 test content discovery label Mar 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
android 🤖 Android support darwin 🍎 macOS, iOS, watchOS, tvOS, and visionOS support discovery 🔎 test content discovery embedded-swift 📟 Embedded Swift issues enhancement New feature or request freebsd 😈 FreeBSD support less-c++ 🐀 Work to reduce the size of our C++ codebase and/or dependencies linux 🐧 Linux support (all distros) openbsd 🐡 OpenBSD support performance 🏎️ Performance issues tools integration 🛠️ Integration of swift-testing into tools/IDEs wasi/wasm 🧭 WebAssembly support windows 🪟 Windows support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants