[dsymutil] Filter our swiftinterface files from the toolchain. #71205
+79
−26
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@llvm/pr-subscribers-debuginfo Author: Adrian Prantl (adrian-prantl) ChangesDsymutil already avoids copying textual Swift interface files from the SDK, since any consumer would have to have a matching SDK installed anyway. It should also do the same thing with interfaces found in the toolchain itself, which includes the compiler built-in libraries such as Swift (=the standard library), and _Concurrency. rdar://117881604 Full diff: https://github.com/llvm/llvm-project/pull/71205.diff 4 Files Affected:
diff --git a/llvm/lib/DWARFLinker/DWARFLinker.cpp b/llvm/lib/DWARFLinker/DWARFLinker.cpp
index 80a4e2adefa6cb6..24fe082bbd357e7 100644
--- a/llvm/lib/DWARFLinker/DWARFLinker.cpp
+++ b/llvm/lib/DWARFLinker/DWARFLinker.cpp
@@ -177,6 +177,21 @@ static void resolveRelativeObjectPath(SmallVectorImpl<char> &Buf, DWARFDie CU) {
sys::path::append(Buf, dwarf::toString(CU.find(dwarf::DW_AT_comp_dir), ""));
}
+/// Make a best effort to guess the
+/// Xcode.app/Contents/Developer/Toolchains/ path from an SDK path.
+static SmallString<128> guessToolChainBaseDir(StringRef SysRoot) {
+ SmallString<128> Result;
+ // Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
+ StringRef Base = SysRoot;
+ Base = sys::path::parent_path(Base);
+ if (sys::path::filename(Base) != "SDKs")
+ return Result;
+ Base = sys::path::parent_path(Base);
+ Result = Base;
+ sys::path::append(Result, "Toolchains");
+ return Result;
+}
+
/// Collect references to parseable Swift interfaces in imported
/// DW_TAG_module blocks.
static void analyzeImportedModule(
@@ -198,6 +213,11 @@ static void analyzeImportedModule(
SysRoot = CU.getSysRoot();
if (!SysRoot.empty() && Path.startswith(SysRoot))
return;
+ // Don't track interfaces that are part of the toolchain.
+ // For example: Swift, _Concurrency, ...
+ SmallString<128> ToolChain = guessToolChainBaseDir(SysRoot);
+ if (!ToolChain.empty() && Path.startswith(ToolChain))
+ return;
std::optional<const char *> Name =
dwarf::toString(DIE.find(dwarf::DW_AT_name));
if (!Name)
diff --git a/llvm/lib/DWARFLinkerParallel/DWARFLinkerCompileUnit.cpp b/llvm/lib/DWARFLinkerParallel/DWARFLinkerCompileUnit.cpp
index 96017bac31342aa..5a45163b9f82292 100644
--- a/llvm/lib/DWARFLinkerParallel/DWARFLinkerCompileUnit.cpp
+++ b/llvm/lib/DWARFLinkerParallel/DWARFLinkerCompileUnit.cpp
@@ -167,6 +167,21 @@ void CompileUnit::cleanupDataAfterClonning() {
getOrigUnit().clear();
}
+/// Make a best effort to guess the
+/// Xcode.app/Contents/Developer/Toolchains/ path from an SDK path.
+static SmallString<128> guessToolChainBaseDir(StringRef SysRoot) {
+ SmallString<128> Result;
+ // Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
+ StringRef Base = SysRoot;
+ Base = sys::path::parent_path(Base);
+ if (sys::path::filename(Base) != "SDKs")
+ return Result;
+ Result = Base;
+ Base = sys::path::parent_path(Base);
+ sys::path::append(Result, "Toolchains");
+ return Result;
+}
+
/// Collect references to parseable Swift interfaces in imported
/// DW_TAG_module blocks.
void CompileUnit::analyzeImportedModule(const DWARFDebugInfoEntry *DieEntry) {
@@ -187,6 +202,11 @@ void CompileUnit::analyzeImportedModule(const DWARFDebugInfoEntry *DieEntry) {
SysRoot = getSysRoot();
if (!SysRoot.empty() && Path.startswith(SysRoot))
return;
+ // Don't track interfaces that are part of the toolchain.
+ // For example: Swift, _Concurrency, ...
+ SmallString<128> ToolChain = guessToolChainBaseDir(SysRoot);
+ if (!ToolChain.empty() && Path.startswith(ToolChain))
+ return;
if (std::optional<DWARFFormValue> Val = find(DieEntry, dwarf::DW_AT_name)) {
Expected<const char *> Name = Val->getAsCString();
if (!Name) {
diff --git a/llvm/test/tools/dsymutil/Inputs/swift-interface.s b/llvm/test/tools/dsymutil/Inputs/swift-interface.s
index a69f008a6cf3e9d..099f8396d40a7d5 100644
--- a/llvm/test/tools/dsymutil/Inputs/swift-interface.s
+++ b/llvm/test/tools/dsymutil/Inputs/swift-interface.s
@@ -15,17 +15,19 @@
##; !llvm.dbg.cu = !{!0}
##; !swift.module.flags = !{!14}
##; !llvm.module.flags = !{!20, !21, !24}
-##;
-##; !0 = distinct !DICompileUnit(language: DW_LANG_Swift, file: !1, isOptimized: false, runtimeVersion: 5, emissionKind: FullDebug, enums: !2, imports: !3, sysroot: "/SDK")
+##;
+##; !0 = distinct !DICompileUnit(language: DW_LANG_Swift, file: !1, isOptimized: false, runtimeVersion: 5, emissionKind: FullDebug, enums: !2, imports: !3, sysroot: "/Xcode.app/Contents/Developer/SDKs/MacOSX.sdk")
##; !1 = !DIFile(filename: "ParseableInterfaceImports.swift", directory: "/")
##; !2 = !{}
-##; !3 = !{!4, !6, !8}
+##; !3 = !{!4, !6, !8, !10}
##; !4 = !DIImportedEntity(tag: DW_TAG_imported_module, scope: !1, entity: !5, file: !1)
##; !5 = !DIModule(scope: null, name: "Foo", includePath: "/Foo/x86_64.swiftinterface")
##; !6 = !DIImportedEntity(tag: DW_TAG_imported_module, scope: !1, entity: !7, file: !1)
-##; !7 = !DIModule(scope: null, name: "Swift", includePath: "/SDK/Swift.swiftmodule/x86_64.swiftinterface")
-##; !8 = !DIImportedEntity(tag: DW_TAG_imported_module, scope: !1, entity: !7, file: !1)
-##; !9 = !DIModule(scope: null, name: "Foundation", includePath: "/SDK/Foundation.swiftmodule")
+##; !7 = !DIModule(scope: null, name: "Swift", includePath: "/Xcode.app/Contents/Developer/SDKs/MacOSX.sdk/Swift.swiftmodule/x86_64.swiftinterface")
+##; !8 = !DIImportedEntity(tag: DW_TAG_imported_module, scope: !1, entity: !9, file: !1)
+##; !9 = !DIModule(scope: null, name: "Foundation", includePath: "/Xcode.app/Contents/Developer/SDKs/MacOSX.sdk/Foundation.swiftmodule")
+##; !10 = !DIImportedEntity(tag: DW_TAG_imported_module, scope: !1, entity: !11, file: !1)
+##; !11 = !DIModule(scope: null, name: "_Concurrency", includePath: "/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/_Concurrency.swiftmodule/x86_64-apple-macos.swiftinterface")
##; !14 = !{!"standard-library", i1 false}
##; !20 = !{i32 2, !"Dwarf Version", i32 4}
##; !21 = !{i32 2, !"Debug Info Version", i32 3}
@@ -36,6 +38,7 @@
##; !35 = !DILocation(line: 0, scope: !36)
##; !36 = !DILexicalBlockFile(scope: !29, file: !37, discriminator: 0)
##; !37 = !DIFile(filename: "<compiler-generated>", directory: "")
+
.section __TEXT,__text,regular,pure_instructions
.macosx_version_min 10, 9
.globl _main ## -- Begin function main
@@ -145,21 +148,21 @@ Ldebug_info_start0:
.set Lset1, Lsection_abbrev-Lsection_abbrev ## Offset Into Abbrev. Section
.long Lset1
.byte 8 ## Address Size (in bytes)
- .byte 1 ## Abbrev [1] 0xb:0x60 DW_TAG_compile_unit
+ .byte 1 ## Abbrev [1] 0xb:0x77 DW_TAG_compile_unit
.long 0 ## DW_AT_producer
.short 30 ## DW_AT_language
.long 1 ## DW_AT_name
.long 33 ## DW_AT_LLVM_sysroot
.set Lset2, Lline_table_start0-Lsection_line ## DW_AT_stmt_list
.long Lset2
- .long 38 ## DW_AT_comp_dir
+ .long 79 ## DW_AT_comp_dir
.byte 5 ## DW_AT_APPLE_major_runtime_vers
.quad Lfunc_begin0 ## DW_AT_low_pc
.set Lset3, Lfunc_end0-Lfunc_begin0 ## DW_AT_high_pc
.long Lset3
.byte 2 ## Abbrev [2] 0x2f:0x23 DW_TAG_module
- .long 40 ## DW_AT_name
- .long 44 ## DW_AT_LLVM_include_path
+ .long 81 ## DW_AT_name
+ .long 85 ## DW_AT_LLVM_include_path
.byte 3 ## Abbrev [3] 0x38:0x19 DW_TAG_subprogram
.quad Lfunc_begin0 ## DW_AT_low_pc
.set Lset4, Lfunc_end0-Lfunc_begin0 ## DW_AT_high_pc
@@ -167,8 +170,8 @@ Ldebug_info_start0:
## DW_AT_APPLE_omit_frame_ptr
.byte 1 ## DW_AT_frame_base
.byte 87
- .long 122 ## DW_AT_linkage_name
- .long 122 ## DW_AT_name
+ .long 112 ## DW_AT_linkage_name
+ .long 112 ## DW_AT_name
.byte 1 ## DW_AT_decl_file
.byte 1 ## DW_AT_decl_line
## DW_AT_external
@@ -176,25 +179,37 @@ Ldebug_info_start0:
.byte 4 ## Abbrev [4] 0x52:0x5 DW_TAG_imported_module
.long 47 ## DW_AT_import
.byte 5 ## Abbrev [5] 0x57:0x9 DW_TAG_module
- .long 71 ## DW_AT_name
- .long 77 ## DW_AT_LLVM_include_path
+ .long 117 ## DW_AT_name
+ .long 123 ## DW_AT_LLVM_include_path
.byte 4 ## Abbrev [4] 0x60:0x5 DW_TAG_imported_module
.long 87 ## DW_AT_import
- .byte 4 ## Abbrev [4] 0x65:0x5 DW_TAG_imported_module
- .long 87 ## DW_AT_import
+ .byte 5 ## Abbrev [5] 0x65:0x9 DW_TAG_module
+ .long 209 ## DW_AT_name
+ .long 220 ## DW_AT_LLVM_include_path
+ .byte 4 ## Abbrev [4] 0x6e:0x5 DW_TAG_imported_module
+ .long 101 ## DW_AT_import
+ .byte 5 ## Abbrev [5] 0x73:0x9 DW_TAG_module
+ .long 289 ## DW_AT_name
+ .long 302 ## DW_AT_LLVM_include_path
+ .byte 4 ## Abbrev [4] 0x7c:0x5 DW_TAG_imported_module
+ .long 115 ## DW_AT_import
.byte 0 ## End Of Children Mark
Ldebug_info_end0:
.section __DWARF,__debug_str,regular,debug
Linfo_string:
.byte 0 ## string offset=0
.asciz "ParseableInterfaceImports.swift" ## string offset=1
- .asciz "/SDK" ## string offset=33
- .asciz "/" ## string offset=38
- .asciz "Foo" ## string offset=40
- .asciz "/Foo/x86_64.swiftinterface" ## string offset=44
- .asciz "Swift" ## string offset=71
- .asciz "/SDK/Swift.swiftmodule/x86_64.swiftinterface" ## string offset=77
- .asciz "main" ## string offset=122
+ .asciz "/Xcode.app/Contents/Developer/SDKs/MacOSX.sdk" ## string offset=33
+ .asciz "/" ## string offset=79
+ .asciz "Foo" ## string offset=81
+ .asciz "/Foo/x86_64.swiftinterface" ## string offset=85
+ .asciz "main" ## string offset=112
+ .asciz "Swift" ## string offset=117
+ .asciz "/Xcode.app/Contents/Developer/SDKs/MacOSX.sdk/Swift.swiftmodule/x86_64.swiftinterface" ## string offset=123
+ .asciz "Foundation" ## string offset=209
+ .asciz "/Xcode.app/Contents/Developer/SDKs/MacOSX.sdk/Foundation.swiftmodule" ## string offset=220
+ .asciz "_Concurrency" ## string offset=289
+ .asciz "/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/_Concurrency.swiftmodule/x86_64-apple-macos.swiftinterface" ## string offset=302
.section __DWARF,__apple_names,regular,debug
Lnames_begin:
.long 1212240712 ## Header Magic
@@ -212,7 +227,7 @@ Lnames_begin:
.set Lset5, LNames0-Lnames_begin ## Offset in Bucket 0
.long Lset5
LNames0:
- .long 122 ## main
+ .long 112 ## main
.long 1 ## Num DIEs
.long 56
.long 0
diff --git a/llvm/test/tools/dsymutil/X86/swift-interface.test b/llvm/test/tools/dsymutil/X86/swift-interface.test
index 6aa6e8c3174f9ef..23b27bd3e12d02e 100644
--- a/llvm/test/tools/dsymutil/X86/swift-interface.test
+++ b/llvm/test/tools/dsymutil/X86/swift-interface.test
@@ -12,9 +12,9 @@
# RUN: cat %t.dir/swift-interface.dSYM/Contents/Resources/Swift/x86_64/Foo.swiftinterface \
# RUN: | FileCheck %s --check-prefix=INTERFACE
-# WARNINGS-NOT: cannot copy parseable Swift interface {{.*}}{{Swift|Foundation}}
+# WARNINGS-NOT: cannot copy parseable Swift interface {{.*}}{{Swift|Foundation|_Concurrency}}
# WARNINGS: cannot copy parseable Swift interface {{.*}}Foo
-# WARNINGS-NOT: cannot copy parseable Swift interface {{.*}}{{Swift|Foundation}}
+# WARNINGS-NOT: cannot copy parseable Swift interface {{.*}}{{Swift|Foundation|_Concurrency}}
# INTERFACE: module Foo
---
|
JDevlieghere
approved these changes
Nov 3, 2023
Comment on lines
170
to
182
| /// Make a best effort to guess the | ||
| /// Xcode.app/Contents/Developer/Toolchains/ path from an SDK path. | ||
| static SmallString<128> guessToolChainBaseDir(StringRef SysRoot) { | ||
| SmallString<128> Result; | ||
| // Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk | ||
| StringRef Base = SysRoot; | ||
| Base = sys::path::parent_path(Base); | ||
| if (sys::path::filename(Base) != "SDKs") | ||
| return Result; | ||
| Result = Base; | ||
| Base = sys::path::parent_path(Base); | ||
| sys::path::append(Result, "Toolchains"); | ||
| return Result; | ||
| } |
There was a problem hiding this comment.
I really wish we could avoid duplicating this. Out of scope for this patch, but I think the parallel dwarf linker should live under DWARFLinker so we can facilitate sharing files. (CC @avl-llvm)
There was a problem hiding this comment.
Agreed, we need a way to share things. I would suggest a bit another solution. Let`s have DWARFLinkerBase library. Which would be linked to DWARFLinker and DWARFLinkerParallel. That library will contain all shared code. It also potentially could be used by other part trying to link DWARF. f.e. BOLT does similar things.
What do you think on creating such a common library?
There was a problem hiding this comment.
Sounds good, but I still think we should move all that in a DWARFLinker directory (i.e. on the filesystem) to avoid the proliferation of DWARFLinker* dirs. They can still be different libraries. The Transforms directory for example is good prior art for that.
There was a problem hiding this comment.
f.e.
DWARFLinker/
DWARFLinker/Apple
DWARFLinker/Parallel
DWARFLinker would contain code which goes to DWARFLinkerBase library.
DWARFLinker/Apple would contain code for apple DWARFLinker and will use DWARFLinkerBase library.
DWARFLinker/Parallel would contain code for llvm DWARFLinker and will use DWARFLinkerBase library.
?
augusto2112
reviewed
Nov 3, 2023
| @@ -167,6 +167,21 @@ void CompileUnit::cleanupDataAfterClonning() { | |||
| getOrigUnit().clear(); | |||
| } | |||
|
|
|||
| /// Make a best effort to guess the | |||
| /// Xcode.app/Contents/Developer/Toolchains/ path from an SDK path. | |||
| static SmallString<128> guessToolChainBaseDir(StringRef SysRoot) { | |||
There was a problem hiding this comment.
Is there unify this function since it exists both in DWARFLinker.cpp and here?
There was a problem hiding this comment.
Yes, see the discussion above. That's out-of-scope for this patch though.
adrian-prantl
force-pushed
the
117881604
branch
from
076dabf to
69efb61
Compare
Dsymutil already avoids copying textual Swift interface files from the SDK, since any consumer would have to have a matching SDK installed anyway. It should also do the same thing with interfaces found in the toolchain itself, which includes the compiler built-in libraries such as Swift (=the standard library), and _Concurrency. rdar://117881604
adrian-prantl
force-pushed
the
117881604
branch
from
69efb61 to
fd10a5f
Compare
adrian-prantl
added a commit
to adrian-prantl/llvm-project
that referenced
this pull request
Nov 3, 2023
…71205) Dsymutil already avoids copying textual Swift interface files from the SDK, since any consumer would have to have a matching SDK installed anyway. It should also do the same thing with interfaces found in the toolchain itself, which includes the compiler built-in libraries such as Swift (=the standard library), and _Concurrency. rdar://117881604 (cherry picked from commit 71a1367) Conflicts: llvm/lib/DWARFLinkerParallel/DWARFLinkerCompileUnit.cpp
|
Hi @adrian-prantl, the test you added https://lab.llvm.org/buildbot/#/builders/117/builds/16428 |
|
I made an attempt to fix this here: commit 2d460f2 (HEAD -> main) |
That looks like it did the trick (https://lab.llvm.org/buildbot/#/builders/216/builds/29804), thanks! |
adrian-prantl
added a commit
to swiftlang/llvm-project
that referenced
this pull request
Nov 14, 2023
[dsymutil] Filter our swiftinterface files from the toolchain. (llvm#71205)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Dsymutil already avoids copying textual Swift interface files from the SDK, since any consumer would have to have a matching SDK installed anyway. It should also do the same thing with interfaces found in the toolchain itself, which includes the compiler built-in libraries such as Swift (=the standard library), and _Concurrency.
rdar://117881604