Skip to content

[swift-inspect] implement Android support including remote heap iteration #78275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
Jan 16, 2025

Conversation

andrurogerz
Copy link
Contributor

@andrurogerz andrurogerz commented Dec 18, 2024

Purpose

Implement swift-inspect Android support, building on the Linux implementation added in #77938. The bulk of this PR is focused on implementing remote heap iteration using Android's malloc_iterate, which is not supported on Linux.

Overview

  • Extend the LinuxRemoteProcess class with a new AndroidRemoteProcess class, overriding and implementing the iterateHeap method
  • Add Swift wrappers for ptrace calls to the SwiftInspectLinux module for use in heap iteration
  • Use existing memory map parsing for /proc/<pid>/maps to identify heap regions in the target process
  • Implement a mechanism to call remote functions in the remote process using ptrace
  • Allocate memory for heap-iteration callback and metadata in the remote process by remotely calling mmap (and munmap) using ptrace
  • Implement heap-iteration by remotely calling malloc_iterate using ptrace with a custom callback copied to the the process using process_vm_writev
  • Call malloc_disable and malloc_enable before and after calls to malloc_iterate in the remote process to ensure a consistent snapshot of heap allocations
  • Use simple signal-based IPC and process_vm_readv to fetch heap-iteration metadata from the remote process
  • Fix a few minor Android-specific build quirks
  • Update README.md to reflect Android support

Background

While there is no standard heap iteration interface on Linux, Android supports malloc_iterate to walk all heap allocations in the current process. This API is supported on the newer scudo heap as well as older jemalloc heap (which may still be used on low memory devices). The API is not part of the public NDK, but is used by the AOSP libmemunreachable tool.

Validation

  • Built release and debug for Android x86_64 and arm64 with swift build on Windows 11 using recent Android Swift SDK and NDK r26b
swift build --triple %ANDROID_ARCH%-unknown-linux-android%ANDROID_API_LEVEL% ^
    --sdk %ANDROID_NDK_ROOT%\toolchains\llvm\prebuilt\windows-x86_64\sysroot ^
    -Xswiftc -sdk -Xswiftc %SDKROOT_ANDROID% ^
    -Xswiftc -sysroot -Xswiftc %ANDROID_NDK_ROOT%\toolchains\llvm\prebuilt\windows-x86_64\sysroot ^
    -Xswiftc -I -Xswiftc %SDKROOT_ANDROID%\usr\include ^
    -Xlinker -L%ANDROID_NDK_ROOT%\toolchains\llvm\prebuilt\windows-x86_64\lib\clang\17.0.2\lib\linux\%ANDROID_ARCH% ^
    -Xcc -I%SDKROOT_ANDROID%\usr\include\swift\SwiftRemoteMirror ^
    -Xlinker %SDKROOT_ANDROID%\usr\lib\swift\android\%ANDROID_ARCH%\libswiftRemoteMirror.so ^
  • Built release and debug for Android x86_64 and arm64 with CMake on Windows 11 using recent Android Swift SDK and NDK r26b
cmake -B build -S . -G Ninja ^
    -D CMAKE_BUILD_WITH_INSTALL_RPATH=YES ^
    -D CMAKE_SYSTEM_NAME=Android ^
    -D CMAKE_ANDROID_ARCH_ABI=%ANDROID_ARCH_ABI% ^
    -D CMAKE_SYSTEM_VERSION=%ANDROID_API_LEVEL% ^
    -D CMAKE_Swift_COMPILER_TARGET=%ANDROID_ARCH%-unknown-linux-android%ANDROID_API_LEVEL% ^
    -D CMAKE_Swift_FLAGS="-sdk %SDKROOT_ANDROID% -L%ANDROID_NDK_ROOT%\toolchains\llvm\prebuilt\windows-x86_64\lib\clang\17.0.2\lib\linux\%ANDROID_ARCH% -Xcc -I%SDKROOT_ANDROID%\usr\include -I%SDKROOT_ANDROID%\usr\include\swift\SwiftRemoteMirror" ^
    -D ArgumentParser_DIR=S:\swift-argument-parser\build\cmake\modules
  • Manually tested all commands against a trivial command-line Swift app run with SWIFT_DEBUG_ENABLE_METADATA_ALLOCATION_ITERATION=1 and SWIFT_DEBUG_ENABLE_METADATA_BACKTRACE_LOGGING=1 on an x86_64 emulator (Android 15) and arm64 device (Pixel 6a + Android 13)
  • Manually tested all commands against the android-swift-support (modified to set SWIFT_DEBUG_ env vars from Application.onCreate) with swift-inspect copied to its sandbox on an x86_64 emulator (Android 15) and arm64 device (Pixel 6a + Android 13)
adb -d push S:\swift\tools\swift-inspect\.build\aarch64-unknown-linux-android29\release\swift-inspect /data/local/tmp
adb -d push %SDKROOT%\usr\lib\swift\android\aarch64\libswiftRemoteMirror.so /data/local/tmp
adb -d shell
bluejay:/ $ run-as com.thebrowsercompany.SwiftSupportTestApp
bluejay:/data/user/0/com.thebrowsercompany.SwiftSupportTestApp $ cp /data/local/tmp/swift-inspect ./swift-inspect
bluejay:/data/user/0/com.thebrowsercompany.SwiftSupportTestApp $ chmod +x ./swift-inspect
bluejay:/data/user/0/com.thebrowsercompany.SwiftSupportTestApp $ export LD_LIBRARY_PATH=$(dirname $(pm path com.thebrowsercompany.SwiftSupportTestApp | sed 's/package://'))/lib/arm64
bluejay:/data/user/0/com.thebrowsercompany.SwiftSupportTestApp $ export SWIFT_DEBUG_ENABLE_METADATA_ALLOCATION_ITERATION=1
bluejay:/data/user/0/com.thebrowsercompany.SwiftSupportTestApp $ export SWIFT_DEBUG_ENABLE_METADATA_BACKTRACE_LOGGING=1
bluejay:/data/user/0/com.thebrowsercompany.SwiftSupportTestApp $ ./swift-inspect dump-arrays <pid>
bluejay:/data/user/0/com.thebrowsercompany.SwiftSupportTestApp $ ./swift-inspect dump-cache-nodes <pid>
bluejay:/data/user/0/com.thebrowsercompany.SwiftSupportTestApp $ ./swift-inspect dump-conformance-cache <pid>
  • Confirmed backtraces are printed with the --backtrace and --backtrace-long arguments
bluejay:/data/user/0/com.thebrowsercompany.SwiftSupportTestApp $ ./swift-inspect dump-raw-metadata --backtrace-long <pid>
bluejay:/data/user/0/com.thebrowsercompany.SwiftSupportTestApp $ ./swift-inspect dump-generic-metadata --backtrace-long <pid>
  • Confirmed swift-inspect still builds for Linux with CMake and swift build

@andrurogerz andrurogerz requested a review from compnerd January 6, 2025 21:38
Copy link
Member

@compnerd compnerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good to me!

@compnerd
Copy link
Member

compnerd commented Jan 7, 2025

@swift-ci please test

@compnerd
Copy link
Member

compnerd commented Jan 7, 2025

@swift-ci please test

@andrurogerz
Copy link
Contributor Author

I made a couple of updates since this PR is still pending:

  1. Added CMake build support and documentation for Android.
  2. Re-worked how the remote heap function size is calculated. I was not able to get .local linkage symbols to work between C and asm in the CMake build, and I opted to make a more robust solution by just moving the function to its own section and adding a linker-defined global to mark the end of the section in a custom linker script (see section-text-remote.ld). This solution avoids inline assembly and allows us to easily and correctly calculate the size of the callback function.

@compnerd please have a look when you get a chance and request test run if you're good with the changes. Thanks!

@compnerd
Copy link
Member

@swift-ci please test

@compnerd
Copy link
Member

@swift-ci please smoke test

@compnerd
Copy link
Member

@swift-ci please test Windows platform

@compnerd compnerd merged commit 5f0bd9b into swiftlang:main Jan 16, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants