Runtime: Implement wrapper function to call metadata access functions #30024

Azoy · 2020-02-23T21:33:09Z

See here: https://forums.swift.org/t/bitcast-pointer-to-swift-function/32324

This is a runtime function that takes an arbitrary metadata access function along with the request and generics and calls said access function. This is useful for those creating their own Swift metadata libraries to be able to call these access functions.

Right now this is using the Swift cc so that I could play around with something in an interpreter test, but it might make more sense to give this the C cc.

cc: @jckarter @rjmccall

jckarter · 2020-02-24T20:21:44Z

Seems like a reasonable helper to me.

I wonder if the dispatch code could be factored in such a way that the basic logic could be phrased in plain C and exposed into the standard library via SwiftShims. It would be nice if it could be inlined into Swift code. (It probably doesn't make that big a difference though.)

jckarter · 2020-02-24T20:21:53Z

@swift-ci Please smoke test

Azoy · 2020-02-25T00:12:47Z

I originally had this in in SwiftShims, but it couldn't find swift/ABI/Metadata.h. This could be moved into there if we reimplemented things like MetadataResponse and MetadataAccessFunction in there as well.

include/swift/Runtime/Metadata.h

Azoy · 2020-04-06T21:10:37Z

I've gone ahead and moved this to SwiftShims! Let me know if there's anything else that needs changing.

jckarter

Looks good, thanks!

jckarter · 2020-04-06T22:13:01Z

@swift-ci Please test

CodaFi · 2020-04-15T02:28:44Z

⛵

shahmishal · 2020-04-15T08:15:49Z

We are seeing failure on Swift in the OS bot - https://ci.swift.org/job/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/2127/

Note this bot is using dylib from the OS not just built.

20:26:03 FAIL: Swift(macosx-x86_64) :: Interpreter/metadata_access.swift (11046 of 13138)
20:26:03 ******************** TEST 'Swift(macosx-x86_64) :: Interpreter/metadata_access.swift' FAILED ********************
20:26:03 Script:
20:26:03 --
20:26:03 : 'RUN: at line 1';   rm -rf "/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/test-macosx-x86_64/Interpreter/Output/metadata_access.swift.tmp" && mkdir -p "/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/test-macosx-x86_64/Interpreter/Output/metadata_access.swift.tmp" && xcrun --toolchain default --sdk '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk' /Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/bin/swiftc -target x86_64-apple-macosx10.9  -module-cache-path '/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/swift-test-results/x86_64-apple-macosx10.9/clang-module-cache'  -F '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks' -toolchain-stdlib-rpath -Xlinker -rpath -Xlinker '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks'  -swift-version 4  -Xfrontend -ignore-module-source-info  -F '/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/lib' -Xlinker -rpath -Xlinker '/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/lib' -module-cache-path '/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/swift-test-results/x86_64-apple-macosx10.9/clang-module-cache' /Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/swift/test/Interpreter/metadata_access.swift -o /Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/test-macosx-x86_64/Interpreter/Output/metadata_access.swift.tmp/a.out -module-name main  && codesign -f -s - /Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/test-macosx-x86_64/Interpreter/Output/metadata_access.swift.tmp/a.out && /usr/bin/env DYLD_LIBRARY_PATH='/usr/lib/swift:/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/lib/swift/macosx' LD_LIBRARY_PATH='/usr/lib/swift:/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/lib/swift/macosx:' SIMCTL_CHILD_DYLD_LIBRARY_PATH='/usr/lib/swift:/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/lib/swift/macosx'  /Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/test-macosx-x86_64/Interpreter/Output/metadata_access.swift.tmp/a.out | '/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python' '/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/swift/utils/PathSanitizingFileCheck' --sanitize BUILD_DIR='/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64' --sanitize SOURCE_DIR='/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/swift' --use-filecheck '/Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/llvm-macosx-x86_64/bin/FileCheck'  /Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/swift/test/Interpreter/metadata_access.swift
20:26:03 --
20:26:03 Exit Code: 2
20:26:03 
20:26:03 Command Output (stderr):
20:26:03 --
20:26:03 dyld: lazy symbol binding failed: Symbol not found: __swift_metadataAccessorCall
20:26:03   Referenced from: /Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/test-macosx-x86_64/Interpreter/Output/metadata_access.swift.tmp/a.out
20:26:03   Expected in: /usr/lib/swift/libswiftCore.dylib
20:26:03 
20:26:03 dyld: Symbol not found: __swift_metadataAccessorCall
20:26:03   Referenced from: /Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/swift-macosx-x86_64/test-macosx-x86_64/Interpreter/Output/metadata_access.swift.tmp/a.out
20:26:03   Expected in: /usr/lib/swift/libswiftCore.dylib
20:26:03 
20:26:03 FileCheck error: '-' is empty.
20:26:03 FileCheck command line:  /Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/buildbot-custom/llvm-macosx-x86_64/bin/FileCheck /Users/buildnode/jenkins/workspace/oss-swift_tools-RA_stdlib-RA-test-simulator-swift-in-the-os/swift/test/Interpreter/metadata_access.swift
20:26:03 
20:26:03 --
20:26:03 
20:26:03 ********************

Azoy · 2020-04-15T13:36:23Z

@shahmishal is this just a matter of forcing the test to use the just built libraries? How would one achieve that?

jckarter · 2020-04-15T22:07:27Z

@Azoy You will want to update your test so that it only runs on a newer Swift runtime that has your added metadata call. For unreleased features, you can do this by requiring version 9999, by putting a guard #available(macOS 9999, iOS 9999, tvOS 9999, watchOS 9999, *) else { exit(0) } before your test, and decorating the shim header declaration with an API_AVAILABLE(macos(9999), ios(9999), watchos(9999), tvos(9999)) attribute.

This was introduced by swiftlang#30024. rdar://61814566

rjmccall · 2020-04-24T19:24:01Z

I don't think we should've accepted this; it's a pretty limited API that's never going to handle arbitrary generics well. What exactly are you trying to do at a high level?

I have some ideas about supporting higher-kinded types as a first-class language feature that would allow you to abstractly pass around uninstantiated generics. That would naturally result in code-generation like what you've got here, while also working correctly with constrained generics.

Alternatively, if you're trying to do something more inherently reflective, I think we should think much more carefully about how we want to satisfy that. This API does not seem like a good stepping-stone for that; it's actually extremely unsafe.

…-accessor-from-swift!" This reverts commit 5fd6e98, reversing changes made to 3aee49d. Revert "XFAIL test/Interpreter/metadata_access.swift on arm64e" This reverts commit 8fe216b. Revert "XFAIl test on os stdlib bots" This reverts commit aea5fa4.

[5.3] Revert "Merge pull request #30024 from Azoy/calling-metadata-accessor-from-swift"

Revert "Merge pull request #30024 from Azoy/calling-metadata-accessor…-from-swift"

jckarter reviewed Feb 25, 2020

View reviewed changes

include/swift/Runtime/Metadata.h Outdated Show resolved Hide resolved

Azoy added 2 commits April 4, 2020 20:45

Runtime: Implement wrapper function for metadata access functions

e82a0b9

Move metadataAccessorCall to SwiftShims

f35efee

Azoy force-pushed the calling-metadata-accessor-from-swift! branch from 3b55c45 to f35efee Compare April 6, 2020 21:09

Azoy requested a review from jckarter April 6, 2020 21:10

jckarter approved these changes Apr 6, 2020

View reviewed changes

CodaFi merged commit 5fd6e98 into swiftlang:master Apr 15, 2020

aschwaighofer added a commit to aschwaighofer/swift that referenced this pull request Apr 16, 2020

XFAIl test on os stdlib bots

aea5fa4

This was introduced by swiftlang#30024. rdar://61814566

aschwaighofer mentioned this pull request Apr 16, 2020

XFAIL test on os stdlib bots #31072

Merged

Azoy deleted the calling-metadata-accessor-from-swift! branch April 17, 2020 00:52

aschwaighofer mentioned this pull request Apr 24, 2020

Revert "Merge pull request #30024 from Azoy/calling-metadata-accessor…-from-swift" #31280

Merged

aschwaighofer added a commit that referenced this pull request Apr 25, 2020

Merge pull request #31282 from aschwaighofer/revert_pr30024_5.3

08e5969

[5.3] Revert "Merge pull request #30024 from Azoy/calling-metadata-accessor-from-swift"

aschwaighofer added a commit that referenced this pull request Apr 25, 2020

Merge pull request #31280 from aschwaighofer/revert_pr30024

6244a81

Revert "Merge pull request #30024 from Azoy/calling-metadata-accessor…-from-swift"

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Runtime: Implement wrapper function to call metadata access functions #30024

Runtime: Implement wrapper function to call metadata access functions #30024

Uh oh!

Azoy commented Feb 23, 2020

Uh oh!

jckarter commented Feb 24, 2020

Uh oh!

jckarter commented Feb 24, 2020

Uh oh!

Azoy commented Feb 25, 2020

Uh oh!

Uh oh!

Azoy commented Apr 6, 2020

Uh oh!

jckarter left a comment

Uh oh!

jckarter commented Apr 6, 2020

Uh oh!

CodaFi commented Apr 15, 2020

Uh oh!

shahmishal commented Apr 15, 2020

Uh oh!

Azoy commented Apr 15, 2020

Uh oh!

jckarter commented Apr 15, 2020

Uh oh!

rjmccall commented Apr 24, 2020

Uh oh!

Uh oh!

Runtime: Implement wrapper function to call metadata access functions #30024

Runtime: Implement wrapper function to call metadata access functions #30024

Uh oh!

Conversation

Azoy commented Feb 23, 2020

Uh oh!

jckarter commented Feb 24, 2020

Uh oh!

jckarter commented Feb 24, 2020

Uh oh!

Azoy commented Feb 25, 2020

Uh oh!

Uh oh!

Azoy commented Apr 6, 2020

Uh oh!

jckarter left a comment

Choose a reason for hiding this comment

Uh oh!

jckarter commented Apr 6, 2020

Uh oh!

CodaFi commented Apr 15, 2020

Uh oh!

shahmishal commented Apr 15, 2020

Uh oh!

Azoy commented Apr 15, 2020

Uh oh!

jckarter commented Apr 15, 2020

Uh oh!

rjmccall commented Apr 24, 2020

Uh oh!

Uh oh!