[Tools] Create swift-inspect, a debugging tool for dumping Swift runtime data from another process #31468

mikeash · 2020-05-01T18:51:15Z

Create swift-inspect. This is a tool which can inspect a Swift process and dump information about the Swift runtime in that process.

It currently supports inspecting two kinds of information:

It can dump the protocol conformance cache.
It can dump all metadata allocations, and print the name and size of the metadata allocated by the generic metadata cache.

It's meant to grow more functionality over time.

The design of the tool places all of the runtime-related smarts in Remote Mirror. swiftdt is then a small that connects the Remote Mirror functionality with remote process inspection. The remote process inspection parts are necessarily OS-specific. Currently they're only implemented for Apple OSes. Everything is cleanly separated out so that someone who knows the appropriate APIs for another OS should be able to add support for that OS without much difficulty.

Update: after much soul searching, we have renamed this from swiftdt to swift-inspect, which is more descriptive and better fits the naming conventions of other tools.

gottesmm · 2020-05-08T19:51:21Z

@mikeash what does dt stand for? Maybe it would make sense to expand out the name? My thought: swift_memory_dumping_tool?

gottesmm · 2020-05-08T20:09:33Z

include/swift/Demangling/ManglingUtils.h

@@ -191,8 +191,8 @@ void mangleIdentifier(Mangler &M, StringRef ident) {
      // of the identifier - that's why we added the dummy-word).
      // The first thing: we add the encoded sub-string length.
      M.Buffer << (Repl.StringPos - Pos);
-      assert(!isDigit(ident[Pos]) &&
-             "first char of sub-string may not be a digit");
+//       assert(!isDigit(ident[Pos]) &&


Missed this.

Thanks. I'm fixing this up along with some other stuff. Not quite done with it yet!

mikeash · 2020-05-08T20:11:49Z

@gottesmm It stands for Swift Debug Tool. There's some internal precedent with other blahdt tools that it's named after, but I'm not completely opposed to changing it.

gottesmm · 2020-05-08T20:15:10Z

@mikeash does it debug everything about a swift program? To me debug tool is very generic to the point of being confusing = /.

MadCoder · 2020-05-08T20:17:44Z

it stands for debug/dump tool

gparker42 · 2020-05-08T20:32:04Z

"swiftrtdt" for Swift runtime debug tool?

gparker42 · 2020-05-08T20:40:11Z

tools/swiftdt/main.swift

+}
+
+if let commandName = commandName {
+  print("error: \(executableName): unknown command \(commandName)", to: &Std.err)


Can these both be argFail() calls?

gparker42 · 2020-05-08T20:42:10Z

tools/swiftdt/main.swift

+}
+
+let commandName = argv.popFirst()
+for command in commands {


Check that argv is now empty here, or else allow multiple commands to be run in one execution.

gottesmm · 2020-05-08T21:06:55Z

@MadCoder I figured as much. I was just pointing out the ambiguity of the name.

gottesmm · 2020-05-08T21:35:49Z

@gparker42 I think swift runtime debug tool would be a better name. At least it is clear what you are debugging.

gottesmm · 2020-05-08T21:39:50Z

tools/swiftdt/CMakeLists.txt

@@ -0,0 +1,14 @@
+add_swift_target_executable(swiftdt


This is the wrong way to do this. We do not want add_swift_target_executable to be used outside of the stdlib directory. This is in preparation for splitting the stdlib from the main host build.

If you need to be in swift's cmake build, I would suggest putting this tool into ./stdlib/tools.

Otherwise, if you can compile against a just built toolchain, I would create a build-script tool that builds against the just built toolchain.

I imagine that you can do the latter since you are just using remote mirrors.

gparker42 · 2020-05-08T23:02:05Z

tools/swiftdt/main.swift

+}
+
+func printUsage(args: inout ArraySlice<String>) {
+  print("Usage: \(executableName) <command>", to: &Std.err)


print("Usage: \(executableName) <pid | process name> <command>", to: &Std.err)

@gottesmm helped me transform this into a proper package, so I went ahead and redid it to use ArgumentParser instead of rolling my own. Now we get all the right behavior for free!

gottesmm · 2020-05-08T23:36:16Z

@mikeash one cool thing is that if you make this a build-script tool, you can use the swift package manager and get access to the argument parser/other things.

gottesmm · 2020-05-08T23:36:34Z

*use the just built swiftpm with your project.

gottesmm · 2020-05-09T04:10:08Z

@mikeash #31667. I tested that locally. It works. If one wants to invoke the build_script_helper.py directly, one does it like this:

+./build_script_helper.py --toolchain $BUILD_DIR/toolchain-macosx-x86_64/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain \
                                          --build-path $PWD/build \
                                          --package-path $PWD

Or to do it using build-script normally:

./swift/utils/build-script --release-debuginfo --swiftdt --skip-build-benchmarks \
     --force-optimized-typechecker --swiftpm --llbuild   --install-swift \
     --install-swiftpm --install-llbuild

natecook1000 · 2020-05-18T17:22:03Z

tools/swiftdt/Sources/swiftdt/main.swift

+  nameOrPid: String
+) -> (Inspector, SwiftReflectionContextRef) {
+  guard let pid = pidFromHint(nameOrPid) else {
+    argFail("Cannot find pid/process \(nameOrPid)")


If you don’t mind making this a throwing method, you can use ValidationError here.

Suggested change

argFail("Cannot find pid/process \(nameOrPid)")

throw ValidationError("Cannot find pid/process \(nameOrPid)")

…e in another process. rdar://problem/55481578

…symbols from libswiftCore. rdar://problem/55481578

…mbols and read the contents of Conformances. rdar://problem/55481578

rdar://problem/55481578

…running. rdar://problem/55481578

rdar://problem/55481578

mikeash · 2020-05-29T16:54:39Z

@swift-ci please test

swift-ci · 2020-05-29T16:57:32Z

Build failed
Swift Test Linux Platform
Git Sha - 27df9e3

swift-ci · 2020-05-29T16:57:41Z

Build failed
Swift Test OS X Platform
Git Sha - 27df9e3

mikeash · 2020-05-29T17:26:06Z

@swift-ci please test os x platform

swift-ci · 2020-05-29T17:37:51Z

Build failed
Swift Test OS X Platform
Git Sha - 23ebf78

mikeash · 2020-05-29T17:38:48Z

@swift-ci please test os x platform

swift-ci · 2020-05-29T17:49:00Z

Build failed
Swift Test OS X Platform
Git Sha - 23ebf78

airspeedswift · 2020-05-29T17:56:58Z

@swift-ci please test

swift-ci · 2020-05-29T17:57:12Z

Build failed
Swift Test OS X Platform
Git Sha - 23ebf78

swift-ci · 2020-05-29T18:00:04Z

Build failed
Swift Test Linux Platform
Git Sha - 23ebf78

mikeash · 2020-05-29T19:30:16Z

@swift-ci please test

swift-ci · 2020-05-29T19:56:02Z

Build failed
Swift Test Linux Platform
Git Sha - b3a5cb7

swift-ci · 2020-05-29T20:35:43Z

Build failed
Swift Test OS X Platform
Git Sha - b3a5cb7

mikeash · 2020-05-29T20:43:30Z

@swift-ci please test

swift-ci · 2020-05-29T20:46:04Z

Build failed
Swift Test OS X Platform
Git Sha - b3a5cb7

swift-ci · 2020-05-29T20:46:22Z

Build failed
Swift Test Linux Platform
Git Sha - b3a5cb7

mikeash · 2020-05-30T18:04:48Z

@swift-ci please test

swift-ci · 2020-05-30T18:05:11Z

Build failed
Swift Test Linux Platform
Git Sha - 6c8fa93

swift-ci · 2020-05-30T18:07:50Z

Build failed
Swift Test OS X Platform
Git Sha - 6c8fa93

mikeash · 2020-05-31T01:43:18Z

@swift-ci please test

swift-ci · 2020-05-31T03:49:30Z

Build failed
Swift Test Linux Platform
Git Sha - 2a55b9d

mikeash · 2020-05-31T03:54:54Z

@swift-ci please test Linux platform

airspeedswift

gottesmm reviewed May 8, 2020

View reviewed changes

gparker42 reviewed May 8, 2020

View reviewed changes

gottesmm reviewed May 8, 2020

View reviewed changes

gparker42 reviewed May 8, 2020

View reviewed changes

BasThomas mentioned this pull request May 9, 2020

[160] Issue #160 - May 21, 2020 SwiftWeekly/swiftweekly.github.io#515

Closed

mikeash force-pushed the swiftdt branch 2 times, most recently from d516251 to ca1add9 Compare May 15, 2020 20:24

mikeash marked this pull request as ready for review May 15, 2020 20:25

mikeash requested review from tbkka and airspeedswift May 15, 2020 20:25

This comment has been minimized.

Sign in to view

natecook1000 reviewed May 18, 2020

View reviewed changes

mikeash added 7 commits May 29, 2020 08:31

[Tools] Start implementing swifdt, a tool to inspect the Swift runtim…

985cc3d

…e in another process. rdar://problem/55481578

[Tools] Get swiftdt calling into Symbolication and have it print all …

f5377cb

…symbols from libswiftCore. rdar://problem/55481578

[Tools] Start hooking up swiftdt to Symbolication, print out found sy…

661209b

…mbols and read the contents of Conformances. rdar://problem/55481578

[Tools] Start hooking up swiftdt to Remote Mirror.

7103ee8

rdar://problem/55481578

[Tools] Fill out Remote Mirror callbacks for swiftdt.

f5464e0

rdar://problem/55481578

[Tools] Get a first pass of swiftdt conformance cache dumping up and …

98c2a35

…running. rdar://problem/55481578

[Tools] Teach swiftdt how to retrieve ObjC class names.

0ddbd84

rdar://problem/55481578

airspeedswift force-pushed the swiftdt branch from 60c05b6 to 49cbf97 Compare May 29, 2020 15:31

mikeash changed the title ~~[Tools] Create swiftdt, a debugging tool for dumping Swift runtime data from another process~~ [Tools] Create swift-inspect, a debugging tool for dumping Swift runtime data from another process May 29, 2020

Add copywrite headers

23ebf78

Separate raw metadata and generic metadata dump commands

b3a5cb7

Add swift-inspect default values to expected_options.py.

6c8fa93

Fix python lint errors.

2a55b9d

airspeedswift approved these changes May 31, 2020

View reviewed changes

mikeash merged commit ad0a8d3 into swiftlang:master May 31, 2020

	argFail("Cannot find pid/process \(nameOrPid)")
	throw ValidationError("Cannot find pid/process \(nameOrPid)")

[Tools] Create swift-inspect, a debugging tool for dumping Swift runtime data from another process #31468

[Tools] Create swift-inspect, a debugging tool for dumping Swift runtime data from another process #31468

Uh oh!

Conversation

mikeash commented May 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gottesmm commented May 8, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mikeash commented May 8, 2020

Uh oh!

gottesmm commented May 8, 2020

Uh oh!

MadCoder commented May 8, 2020

Uh oh!

gparker42 commented May 8, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gottesmm commented May 8, 2020

Uh oh!

gottesmm commented May 8, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gottesmm commented May 8, 2020

Uh oh!

gottesmm commented May 8, 2020

Uh oh!

gottesmm commented May 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

Choose a reason for hiding this comment

Uh oh!

mikeash commented May 29, 2020

Uh oh!

swift-ci commented May 29, 2020

Uh oh!

swift-ci commented May 29, 2020

Uh oh!

mikeash commented May 29, 2020

Uh oh!

swift-ci commented May 29, 2020

Uh oh!

mikeash commented May 29, 2020

Uh oh!

swift-ci commented May 29, 2020

Uh oh!

airspeedswift commented May 29, 2020

Uh oh!

swift-ci commented May 29, 2020

Uh oh!

swift-ci commented May 29, 2020

Uh oh!

mikeash commented May 29, 2020

Uh oh!

swift-ci commented May 29, 2020

Uh oh!

swift-ci commented May 29, 2020

Uh oh!

mikeash commented May 29, 2020

Uh oh!

swift-ci commented May 29, 2020

Uh oh!

swift-ci commented May 29, 2020

Uh oh!

mikeash commented May 30, 2020

mikeash commented May 1, 2020 •

edited

Loading

gottesmm commented May 9, 2020 •

edited

Loading