Skip to content

Codesign w/ debugging entitlement for backtraces on macOS #7010

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Oct 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ Swift Next

In packages that specify resources using a future tools version, the generated resource bundle accessor will import `Foundation.Bundle` for its own implementation only. _Clients_ of such packages therefore no longer silently import `Foundation`, preventing inadvertent use of Foundation extensions to standard library APIs, which helps to avoid unexpected code size increases.

* [#7010]

On macOS, `swift build` and `swift run` now produce binaries that allow backtraces in debug builds. Pass `SWIFT_BACKTRACE=enable=yes` environment variable to enable backtraces on such binaries when running them.

Swift 5.9
-----------
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -367,6 +367,10 @@ public final class ProductBuildDescription: SPMBuildCore.ProductBuildDescription

return flags
}

func codeSigningArguments(plistPath: AbsolutePath, binaryPath: AbsolutePath) -> [String] {
["codesign", "--force", "--sign", "-", "--entitlements", plistPath.pathString, binaryPath.pathString]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be a good idea to have a command line option to let you specify the code signing identity (here we're using -).

Copy link
Contributor Author

@MaxDesiatov MaxDesiatov Oct 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO that should come as a separate PR to make this facility a bit more general.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I only mention it because people who are aware of how this works will pretty quickly notice that we're signing things and ask how to sign with a different identity.

Copy link
Contributor Author

@MaxDesiatov MaxDesiatov Oct 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair, but signing with a different identity or with a different entitlement requires a public API design and likely an evolution proposal. Scoping this to one specific case means we don't have to block backtraces with this potential future design work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. No objection from me.

}
}

extension SortedArray where Element == AbsolutePath {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,19 @@ extension LLBuildManifestBuilder {
testInputs = []
}

// Create a phony node to represent the entire target.
let targetName = try buildProduct.product.getLLBuildTargetName(config: self.buildConfig)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please check if this has performance implications, I vaguely remeber @neonichu ran into something like that with this API, tho no 100% sure

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This phony node is not new, it's only moved from previous L62 where it was declared too late for us. I had to move it up to this line.

let output: Node = .virtual(targetName)

let finalProductNode: Node
switch buildProduct.product.type {
case .library(.static):
finalProductNode = try .file(buildProduct.binaryPath)
try self.manifest.addShellCmd(
name: cmdName,
description: "Archiving \(buildProduct.binaryPath.prettyPath())",
inputs: (buildProduct.objects + [buildProduct.linkFileListPath]).map(Node.file),
outputs: [.file(buildProduct.binaryPath)],
outputs: [finalProductNode],
arguments: try buildProduct.archiveArguments()
)

Expand All @@ -49,23 +55,55 @@ extension LLBuildManifestBuilder {
+ [buildProduct.linkFileListPath]
+ testInputs

let shouldCodeSign: Bool
let linkedBinaryNode: Node
let linkedBinaryPath = try buildProduct.binaryPath
if case .executable = buildProduct.product.type,
buildParameters.targetTriple.isMacOSX,
buildParameters.debuggingParameters.shouldEnableDebuggingEntitlement {
shouldCodeSign = true
linkedBinaryNode = try .file(buildProduct.binaryPath, isMutated: true)
} else {
shouldCodeSign = false
linkedBinaryNode = try .file(buildProduct.binaryPath)
}

try self.manifest.addShellCmd(
name: cmdName,
description: "Linking \(buildProduct.binaryPath.prettyPath())",
inputs: inputs.map(Node.file),
outputs: [.file(buildProduct.binaryPath)],
outputs: [linkedBinaryNode],
arguments: try buildProduct.linkArguments()
)
}

// Create a phony node to represent the entire target.
let targetName = try buildProduct.product.getLLBuildTargetName(config: self.buildConfig)
let output: Node = .virtual(targetName)
if shouldCodeSign {
let basename = try buildProduct.binaryPath.basename
let plistPath = try buildProduct.binaryPath.parentDirectory
.appending(component: "\(basename)-entitlement.plist")
self.manifest.addEntitlementPlistCommand(
entitlement: "com.apple.security.get-task-allow",
outputPath: plistPath
)

let cmdName = try buildProduct.product.getCommandName(config: self.buildConfig)
let codeSigningOutput = Node.virtual(targetName + "-CodeSigning")
try self.manifest.addShellCmd(
name: "\(cmdName)-entitlements",
description: "Applying debug entitlements to \(buildProduct.binaryPath.prettyPath())",
inputs: [linkedBinaryNode, .file(plistPath)],
outputs: [codeSigningOutput],
arguments: buildProduct.codeSigningArguments(plistPath: plistPath, binaryPath: linkedBinaryPath)
)
finalProductNode = codeSigningOutput
} else {
finalProductNode = linkedBinaryNode
}
}

self.manifest.addNode(output, toTarget: targetName)
try self.manifest.addPhonyCmd(
self.manifest.addPhonyCmd(
name: output.name,
inputs: [.file(buildProduct.binaryPath)],
inputs: [finalProductNode],
outputs: [output]
)

Expand Down
5 changes: 3 additions & 2 deletions Sources/Build/BuildManifest/LLBuildManifestBuilder.swift
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import struct TSCBasic.ByteString
import enum TSCBasic.ProcessEnv
import func TSCBasic.topologicalSort

/// High-level interface to ``LLBuildManifest`` and ``LLBuildManifestWriter``.
public class LLBuildManifestBuilder {
public enum TargetKind {
case main
Expand All @@ -42,7 +43,7 @@ public class LLBuildManifestBuilder {
public let disableSandboxForPluginCommands: Bool

/// File system reference.
let fileSystem: FileSystem
let fileSystem: any FileSystem

/// ObservabilityScope with which to emit diagnostics
public let observabilityScope: ObservabilityScope
Expand All @@ -60,7 +61,7 @@ public class LLBuildManifestBuilder {
public init(
_ plan: BuildPlan,
disableSandboxForPluginCommands: Bool = false,
fileSystem: FileSystem,
fileSystem: any FileSystem,
observabilityScope: ObservabilityScope
) {
self.plan = plan
Expand Down
6 changes: 6 additions & 0 deletions Sources/CoreCommands/Options.swift
Original file line number Diff line number Diff line change
Expand Up @@ -477,6 +477,12 @@ public struct BuildOptions: ParsableArguments {
)
public var linkTimeOptimizationMode: LinkTimeOptimizationMode?

@Flag(help: .hidden)
public var enableGetTaskAllowEntitlement: Bool = false

@Flag(help: .hidden)
public var disableGetTaskAllowEntitlement: Bool = false
Copy link
Contributor

@tomerd tomerd Oct 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can probably marge this into one optional Bool setting

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't look @Flag supports optional types, unless I'm something? Maybe @natecook1000 or @rauhul can clarify?

Copy link
Member

@rauhul rauhul Oct 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the difference between true/false/nil behaviors?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC I had to provide a default value, but the default value depended on values of other options. I assume false would mean --disable..., true would mean --enable..., and nil would mean neither were passed on CLI?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm so you want to infer true/false based on other flags unless explicitly specified... that is reasonable. I don't think we support Optional Flags yet, but I wonder if you could use an EnumerableFlag instead.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can actually have a boolean flag as long as there's an inversion, which I think is exactly what you want in this case. The declaration would look like:

@Flag(inversion: .prefixedEnableDisable)
var getTaskAllowEntitlement: Bool?

It looks like we might not allow an = nil in there, which may have been what you ran into. That's also an issue we should be able to fix pretty easily.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooh good catch!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this is what I had in mind. @MaxDesiatov

Copy link
Contributor Author

@MaxDesiatov MaxDesiatov Oct 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, it actually does support an optional default value, at least with .prefixedEnableDisable it works great. I'll submit a follow-up PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in #7028


// @Flag works best when there is a default value present
// if true, false aren't enough and a third state is needed
// nil should not be the goto. Instead create an enum
Expand Down
19 changes: 18 additions & 1 deletion Sources/CoreCommands/SwiftTool.swift
Original file line number Diff line number Diff line change
Expand Up @@ -664,6 +664,11 @@ public final class SwiftTool {
return buildSystem
}

static let entitlementsMacOSWarning = """
`--disable-get-task-allow-entitlement` and `--disable-get-task-allow-entitlement` only have an effect \
when building on macOS.
"""

private func _buildParams(toolchain: UserToolchain) throws -> BuildParameters {
let hostTriple = try self.getHostToolchain().targetTriple
let targetTriple = toolchain.targetTriple
Expand All @@ -672,6 +677,12 @@ public final class SwiftTool {
component: targetTriple.platformBuildPathComponent(buildSystem: options.build.buildSystem)
)

if !targetTriple.isMacOSX && (
options.build.disableGetTaskAllowEntitlement || options.build.enableGetTaskAllowEntitlement
) {
observabilityScope.emit(warning: Self.entitlementsMacOSWarning)
}

return try BuildParameters(
dataPath: dataPath,
configuration: options.build.configuration,
Expand All @@ -685,7 +696,13 @@ public final class SwiftTool {
sanitizers: options.build.enabledSanitizers,
indexStoreMode: options.build.indexStoreMode.buildParameter,
isXcodeBuildSystemEnabled: options.build.buildSystem == .xcode,
debugInfoFormat: options.build.debugInfoFormat.buildParameter,
debuggingParameters: .init(
debugInfoFormat: options.build.debugInfoFormat.buildParameter,
targetTriple: targetTriple,
shouldEnableDebuggingEntitlement:
(options.build.configuration == .debug && !options.build.disableGetTaskAllowEntitlement) ||
(options.build.enableGetTaskAllowEntitlement && !options.build.disableGetTaskAllowEntitlement)
),
driverParameters: .init(
canRenameEntrypointFunctionName: driverSupport.checkSupportedFrontendFlags(
flags: ["entry-point-function-name"],
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift open source project
//
// Copyright (c) 2020-2023 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See http://swift.org/LICENSE.txt for license information
// See http://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
//
//===----------------------------------------------------------------------===//

import struct Basics.Triple
import enum PackageModel.BuildConfiguration

extension BuildParameters {
public struct Debugging: Encodable {
public init(
debugInfoFormat: DebugInfoFormat = .dwarf,
targetTriple: Triple,
shouldEnableDebuggingEntitlement: Bool
) {
self.debugInfoFormat = debugInfoFormat

// Per rdar://112065568 for backtraces to work on macOS a special entitlement needs to be granted on the final
// executable.
self.shouldEnableDebuggingEntitlement = targetTriple.isMacOSX && shouldEnableDebuggingEntitlement
}

public var debugInfoFormat: DebugInfoFormat

/// Whether the produced executable should be codesigned with the debugging entitlement, enabling enhanced
/// backtraces on macOS.
public var shouldEnableDebuggingEntitlement: Bool
}

/// Represents the debugging strategy.
///
/// Swift binaries requires the swiftmodule files in order for lldb to work.
/// On Darwin, linker can directly take the swiftmodule file path using the
/// -add_ast_path flag. On other platforms, we convert the swiftmodule into
/// an object file using Swift's modulewrap tool.
public enum DebuggingStrategy {
case swiftAST
case modulewrap
}

/// The debugging strategy according to the current build parameters.
public var debuggingStrategy: DebuggingStrategy? {
guard configuration == .debug else {
return nil
}

if targetTriple.isApple() {
return .swiftAST
}
return .modulewrap
}

/// Represents the debug information format.
///
/// The debug information format controls the format of the debug information
/// that the compiler generates. Some platforms support debug information
// formats other than DWARF.
public enum DebugInfoFormat: String, Encodable {
/// DWARF debug information format, the default format used by Swift.
case dwarf
/// CodeView debug information format, used on Windows.
case codeview
/// No debug information to be emitted.
case none
}

}
53 changes: 9 additions & 44 deletions Sources/SPMBuildCore/BuildParameters/BuildParameters.swift
Original file line number Diff line number Diff line change
Expand Up @@ -26,31 +26,6 @@ public struct BuildParameters: Encodable {
case auto
}

/// Represents the debug information format.
///
/// The debug information format controls the format of the debug information
/// that the compiler generates. Some platforms support debug information
// formats other than DWARF.
public enum DebugInfoFormat: String, Encodable {
/// DWARF debug information format, the default format used by Swift.
case dwarf
/// CodeView debug information format, used on Windows.
case codeview
/// No debug information to be emitted.
case none
}

/// Represents the debugging strategy.
///
/// Swift binaries requires the swiftmodule files in order for lldb to work.
/// On Darwin, linker can directly take the swiftmodule file path using the
/// -add_ast_path flag. On other platforms, we convert the swiftmodule into
/// an object file using Swift's modulewrap tool.
public enum DebuggingStrategy {
case swiftAST
case modulewrap
}

/// The path to the data directory.
public var dataPath: AbsolutePath

Expand Down Expand Up @@ -125,10 +100,11 @@ public struct BuildParameters: Encodable {
/// Whether the Xcode build system is used.
public var isXcodeBuildSystemEnabled: Bool

public var debugInfoFormat: DebugInfoFormat

public var shouldSkipBuilding: Bool

/// Build parameters related to debugging.
public var debuggingParameters: Debugging

/// Build parameters related to Swift Driver.
public var driverParameters: Driver

Expand All @@ -155,21 +131,25 @@ public struct BuildParameters: Encodable {
sanitizers: EnabledSanitizers = EnabledSanitizers(),
indexStoreMode: IndexStoreMode = .auto,
isXcodeBuildSystemEnabled: Bool = false,
debugInfoFormat: DebugInfoFormat = .dwarf,
shouldSkipBuilding: Bool = false,
debuggingParameters: Debugging? = nil,
driverParameters: Driver = .init(),
linkingParameters: Linking = .init(),
outputParameters: Output = .init(),
testingParameters: Testing? = nil
) throws {
let targetTriple = try targetTriple ?? .getHostTriple(usingSwiftCompiler: toolchain.swiftCompilerPath)
self.debuggingParameters = debuggingParameters ?? .init(
targetTriple: targetTriple,
shouldEnableDebuggingEntitlement: configuration == .debug
)

self.dataPath = dataPath
self.configuration = configuration
self._toolchain = _Toolchain(toolchain: toolchain)
self.hostTriple = try hostTriple ?? .getHostTriple(usingSwiftCompiler: toolchain.swiftCompilerPath)
self.targetTriple = targetTriple
switch debugInfoFormat {
switch self.debuggingParameters.debugInfoFormat {
case .dwarf:
var flags = flags
// DWARF requires lld as link.exe expects CodeView debug info.
Expand Down Expand Up @@ -205,13 +185,11 @@ public struct BuildParameters: Encodable {
self.sanitizers = sanitizers
self.indexStoreMode = indexStoreMode
self.isXcodeBuildSystemEnabled = isXcodeBuildSystemEnabled
self.debugInfoFormat = debugInfoFormat
self.shouldSkipBuilding = shouldSkipBuilding
self.driverParameters = driverParameters
self.linkingParameters = linkingParameters
self.outputParameters = outputParameters
self.testingParameters = testingParameters ?? .init(configuration: configuration, targetTriple: targetTriple)

}

public func forTriple(_ targetTriple: Triple) throws -> BuildParameters {
Expand Down Expand Up @@ -280,19 +258,6 @@ public struct BuildParameters: Encodable {
public var testOutputPath: AbsolutePath {
return buildPath.appending(component: "testOutput.txt")
}

/// The debugging strategy according to the current build parameters.
public var debuggingStrategy: DebuggingStrategy? {
guard configuration == .debug else {
return nil
}

if targetTriple.isApple() {
return .swiftAST
}
return .modulewrap
}

/// Returns the path to the binary of a product for the current build parameters.
public func binaryPath(for product: ResolvedProduct) throws -> AbsolutePath {
return try buildPath.appending(binaryRelativePath(for: product))
Expand Down
1 change: 1 addition & 0 deletions Sources/SPMBuildCore/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
add_library(SPMBuildCore
BinaryTarget+Extensions.swift
BuildParameters/BuildParameters.swift
BuildParameters/BuildParameters+Debugging.swift
BuildParameters/BuildParameters+Driver.swift
BuildParameters/BuildParameters+Linking.swift
BuildParameters/BuildParameters+Output.swift
Expand Down
Loading