Skip to content

[AutoDiff] fix _Differentiation rpath #31183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 21, 2020

Conversation

marcrasi
Copy link

I'm not very confident that this is the right thing to do. @compnerd could you take a look and let me know what you think?

The problem that this is fixing:

# In a SwiftPM package with a single main file:
$ cat Sources/testpackage/main.swift
import _Differentiation
print(gradient(at: Float(6), in: { $0 * $0 }))
$ swift run
dyld: Library not loaded: /usr/lib/swift/libswift_Differentiation.dylib
  Referenced from: /Users/marcrasi/testpackage/.build/x86_64-apple-macosx/debug/testpackage
  Reason: image not found
Abort trap: 6

Some commands that I ran for debugging:

$ otool -L /Users/marcrasi/testpackage/.build/x86_64-apple-macosx/debug/testpackage
/Users/marcrasi/testpackage/.build/x86_64-apple-macosx/debug/testpackage:
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1675.129.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
	/usr/lib/swift/libswift_Differentiation.dylib (compatibility version 1.0.0, current version 0.0.0)
	@rpath/libswiftCore.dylib (compatibility version 1.0.0, current version 0.0.0)
	@rpath/libswiftDarwin.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
	@rpath/libswiftSwiftOnoneSupport.dylib (compatibility version 1.0.0, current version 0.0.0)

$ otool -l /Users/marcrasi/testpackage/.build/x86_64-apple-macosx/debug/testpackage | grep RPATH -A2
          cmd LC_RPATH
      cmdsize 32
         path /usr/lib/swift (offset 12)
--
          cmd LC_RPATH
      cmdsize 32
         path @loader_path (offset 12)
--
          cmd LC_RPATH
      cmdsize 192
         path /Users/marcrasi/swift-base/build/Ninja-ReleaseAssert/toolchain-macosx-x86_64/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx (offset 12)

$ otool -l /Users/marcrasi/swift-base/build/Ninja-ReleaseAssert/toolchain-macosx-x86_64/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/libswift_Differentiation.dylib | grep ID_DYLIB -A2
          cmd LC_ID_DYLIB
      cmdsize 72
         name /usr/lib/swift/libswift_Differentiation.dylib (offset 24)

This PR fixes the problem by changing the "install name" of libswift_Differentiation.dylib to @rpath/libswift_Differentiation.dylib which makes it get found in the toolchain. Here are the same commands after rebuilding the toolchain with this PR:

$ swift run
12.0

$ otool -L /Users/marcrasi/testpackage/.build/x86_64-apple-macosx/debug/testpackage
/Users/marcrasi/testpackage/.build/x86_64-apple-macosx/debug/testpackage:
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1675.129.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
	@rpath/libswiftCore.dylib (compatibility version 1.0.0, current version 0.0.0)
	@rpath/libswiftDarwin.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
	@rpath/libswiftSwiftOnoneSupport.dylib (compatibility version 1.0.0, current version 0.0.0)
	@rpath/libswift_Differentiation.dylib (compatibility version 1.0.0, current version 0.0.0)

$ otool -l /Users/marcrasi/swift-base/build/Ninja-ReleaseAssert/toolchain-macosx-x86_64/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/libswift_Differentiation.dylib | grep ID_DYLIB -A2
          cmd LC_ID_DYLIB
      cmdsize 64
         name @rpath/libswift_Differentiation.dylib (offset 24)

@marcrasi
Copy link
Author

@swift-ci please test

@marcrasi marcrasi merged commit 4f3eab3 into swiftlang:master Apr 21, 2020
@marcrasi marcrasi deleted the differentiation-rpath branch April 21, 2020 22:30
edymtt added a commit to edymtt/swift that referenced this pull request Oct 7, 2020
This way we match the way we build other Swift libraries, to allow the
dylib to have an absolute install name and at the same time be found
when used as part of a local toolchain (e.g. swiftlang#31183)
@edymtt
Copy link
Contributor

edymtt commented Oct 7, 2020

Apologies for chiming in late -- it turns out we need a different change (#34216), since we need the install name in the dylib to be absolute (while allowing this scenario to work)

@lorentey
Copy link
Member

lorentey commented Oct 7, 2020

I think /usr/lib/swift/libFoo.dylib is the right install_name for these libraries -- after all, this is where they are expected to install when they get released, and it matches the configuration of other libraries. (Ignoring the pre-5.0 compatibility overrides in magic-symbols-for-install-name, which don't apply here.)

To test locally built libraries, our test configuration uses DYLD_LIBRARY_PATH to override the install_name. Would setting that be an acceptable workflow?

edymtt added a commit that referenced this pull request Oct 12, 2020
This effectively reverts #31183 -- we need to match the install name convention of the other stdlib libraries.

From the review feedback:

> The right way to load the stdlib & runtime libraries from a custom toolchain is to set `DYLD_LIBRARY_PATH` when executing the generated binary. This is how we run tests against the just-built libraries and this is how downloadable toolchain snapshots are currently configured in Xcode -- see #33178
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants