Skip to content

[Macros] Add attached extension macros. #1859

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 29, 2023
1 change: 1 addition & 0 deletions Examples/Sources/ExamplePlugin/ExamplePlugin.swift
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ struct ThePlugin: CompilerPlugin {
PeerValueWithSuffixNameMacro.self,
MemberDeprecatedMacro.self,
EquatableConformanceMacro.self,
SendableExtensionMacro.self,
DidSetPrintMacro.self,
PrintAnyMacro.self,
]
Expand Down
20 changes: 20 additions & 0 deletions Examples/Sources/ExamplePlugin/Macros.swift
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,26 @@ struct EquatableConformanceMacro: ConformanceMacro {
}
}

public struct SendableExtensionMacro: ExtensionMacro {
public static func expansion(
of node: AttributeSyntax,
attachedTo: some DeclGroupSyntax,
providingExtensionsOf type: some TypeSyntaxProtocol,
in context: some MacroExpansionContext
) throws -> [ExtensionDeclSyntax] {
let sendableExtension: DeclSyntax =
"""
extension \(type.trimmed): Sendable {}
"""

guard let extensionDecl = sendableExtension.as(ExtensionDeclSyntax.self) else {
return []
}

return [extensionDecl]
}
}

/// Add 'didSet' printing the new value.
struct DidSetPrintMacro: AccessorMacro {
static func expansion(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,14 +122,15 @@ extension CompilerPluginMessageHandler {
expandingSyntax: expandingSyntax
)

case .expandAttachedMacro(let macro, let macroRole, let discriminator, let attributeSyntax, let declSyntax, let parentDeclSyntax):
case .expandAttachedMacro(let macro, let macroRole, let discriminator, let attributeSyntax, let declSyntax, let parentDeclSyntax, let extendedTypeSyntax):
try expandAttachedMacro(
macro: macro,
macroRole: macroRole,
discriminator: discriminator,
attributeSyntax: attributeSyntax,
declSyntax: declSyntax,
parentDeclSyntax: parentDeclSyntax
parentDeclSyntax: parentDeclSyntax,
extendedTypeSyntax: extendedTypeSyntax
)

case .loadPluginLibrary(let libraryPath, let moduleName):
Expand Down
8 changes: 7 additions & 1 deletion Sources/SwiftCompilerPluginMessageHandling/Macros.swift
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@ extension CompilerPluginMessageHandler {
discriminator: String,
attributeSyntax: PluginMessage.Syntax,
declSyntax: PluginMessage.Syntax,
parentDeclSyntax: PluginMessage.Syntax?
parentDeclSyntax: PluginMessage.Syntax?,
extendedTypeSyntax: PluginMessage.Syntax?
) throws {
let sourceManager = SourceManager()
let context = PluginMacroExpansionContext(
Expand All @@ -100,6 +101,9 @@ extension CompilerPluginMessageHandler {
).cast(AttributeSyntax.self)
let declarationNode = sourceManager.add(declSyntax).cast(DeclSyntax.self)
let parentDeclNode = parentDeclSyntax.map { sourceManager.add($0).cast(DeclSyntax.self) }
let extendedType = extendedTypeSyntax.map {
sourceManager.add($0).cast(TypeSyntax.self)
}

// TODO: Make this a 'String?' and remove non-'hasExpandMacroResult' branches.
let expandedSources: [String]?
Expand All @@ -115,6 +119,7 @@ extension CompilerPluginMessageHandler {
attributeNode: attributeNode,
declarationNode: declarationNode,
parentDeclNode: parentDeclNode,
extendedType: extendedType,
in: context
)
if let expansions, hostCapability.hasExpandMacroResult {
Expand Down Expand Up @@ -159,6 +164,7 @@ private extension MacroRole {
case .peer: self = .peer
case .conformance: self = .conformance
case .codeItem: self = .codeItem
case .extension: self = .extension
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@ internal enum HostToPluginMessage: Codable {
discriminator: String,
attributeSyntax: PluginMessage.Syntax,
declSyntax: PluginMessage.Syntax,
parentDeclSyntax: PluginMessage.Syntax?
parentDeclSyntax: PluginMessage.Syntax?,
extendedTypeSyntax: PluginMessage.Syntax?
)

/// Optionally implemented message to load a dynamic link library.
Expand Down Expand Up @@ -76,7 +77,7 @@ internal enum PluginToHostMessage: Codable {
}

/*namespace*/ internal enum PluginMessage {
static var PROTOCOL_VERSION_NUMBER: Int { 5 } // Added 'expandMacroResult'.
static var PROTOCOL_VERSION_NUMBER: Int { 6 } // Added 'expandMacroResult'.

struct HostCapability: Codable {
var protocolVersion: Int
Expand Down Expand Up @@ -107,6 +108,7 @@ internal enum PluginToHostMessage: Codable {
case peer
case conformance
case codeItem
case `extension`
}

struct SourceLocation: Codable {
Expand Down
32 changes: 31 additions & 1 deletion Sources/SwiftParser/Attributes.swift
Original file line number Diff line number Diff line change
Expand Up @@ -332,14 +332,44 @@ extension Parser {
)
)
case nil:
let isAttached = self.peek().isAttachedKeyword
return parseAttribute(argumentMode: .customAttribute) { parser in
let arguments = parser.parseArgumentListElements(pattern: .none)
let arguments: [RawTupleExprElementSyntax]
if isAttached {
arguments = parser.parseAttachedArguments()
} else {
arguments = parser.parseArgumentListElements(pattern: .none)
}

Comment on lines +335 to +343
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of matching the string contents of the current token here, it’s much clearer to add attached as a contextual keyword. I’m fixing that in #1881.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

return .argumentList(RawTupleExprElementListSyntax(elements: arguments, arena: parser.arena))
}
}
}
}

extension Parser {
mutating func parseAttachedArguments() -> [RawTupleExprElementSyntax] {
let (unexpectedBeforeRole, role) = self.expect(.identifier, TokenSpec(.extension, remapping: .identifier), default: .identifier)
let roleTrailingComma = self.consume(if: .comma)
let roleElement = RawTupleExprElementSyntax(
label: nil,
colon: nil,
expression: RawExprSyntax(
RawIdentifierExprSyntax(
unexpectedBeforeRole,
identifier: role,
declNameArguments: nil,
arena: self.arena
)
),
trailingComma: roleTrailingComma,
arena: self.arena
)
let additionalArgs = self.parseArgumentListElements(pattern: .none)
return [roleElement] + additionalArgs
}
}

extension Parser {
mutating func parseDifferentiableAttribute() -> RawAttributeSyntax {
let (unexpectedBeforeAtSign, atSign) = self.expect(.atSign)
Expand Down
4 changes: 4 additions & 0 deletions Sources/SwiftParser/Types.swift
Original file line number Diff line number Diff line change
Expand Up @@ -1076,6 +1076,10 @@ extension Lexer.Lexeme {
|| self.rawTokenKind == .prefixOperator
}

var isAttachedKeyword: Bool {
return self.rawTokenKind == .identifier && self.tokenText == "attached"
}

var isEllipsis: Bool {
return self.isAnyOperator && self.tokenText == "..."
}
Expand Down
44 changes: 43 additions & 1 deletion Sources/SwiftSyntaxMacroExpansion/MacroExpansion.swift
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ public enum MacroRole {
case peer
case conformance
case codeItem
case `extension`
}

extension MacroRole {
Expand All @@ -35,6 +36,7 @@ extension MacroRole {
case .peer: return "PeerMacro"
case .conformance: return "ConformanceMacro"
case .codeItem: return "CodeItemMacro"
case .extension: return "ExtensionMacro"
}
}
}
Expand All @@ -45,6 +47,7 @@ private enum MacroExpansionError: Error, CustomStringConvertible {
case parentDeclGroupNil
case declarationNotDeclGroup
case declarationNotIdentified
case noExtendedTypeSyntax
case noFreestandingMacroRoles(Macro.Type)

var description: String {
Expand All @@ -61,6 +64,9 @@ private enum MacroExpansionError: Error, CustomStringConvertible {
case .declarationNotIdentified:
return "declaration is not a 'Identified' syntax"

case .noExtendedTypeSyntax:
return "no extended type for extension macro"

case .noFreestandingMacroRoles(let type):
return "macro implementation type '\(type)' does not conform to any freestanding macro protocol"

Expand Down Expand Up @@ -113,7 +119,7 @@ public func expandFreestandingMacro(
let rewritten = try codeItemMacroDef.expansion(of: node, in: context)
expandedSyntax = Syntax(CodeBlockItemListSyntax(rewritten))

case (.accessor, _), (.memberAttribute, _), (.member, _), (.peer, _), (.conformance, _), (.expression, _), (.declaration, _),
case (.accessor, _), (.memberAttribute, _), (.member, _), (.peer, _), (.conformance, _), (.extension, _), (.expression, _), (.declaration, _),
(.codeItem, _):
throw MacroExpansionError.unmatchedMacroRole(definition, macroRole)
}
Expand Down Expand Up @@ -178,6 +184,7 @@ public func expandAttachedMacroWithoutCollapsing<Context: MacroExpansionContext>
attributeNode: AttributeSyntax,
declarationNode: DeclSyntax,
parentDeclNode: DeclSyntax?,
extendedType: TypeSyntax?,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed the new parameter introduced doesn't have documentation in the Parameters doc-comment section above. It might be a good idea to add some info there for clarity. We could use something similar to what we have for parentDeclNode, since it's also used exclusively for MacroRole.extension. If we agree on this, we might want to add the same info for expandAttachedMacro(...) function as well. Just a thought!

btw: I was wondering, instead of adding a new parameter for a specific case from MacroRole, what if we created an enum with associated values? Here's an idea I had:

enum MacroRoleContext {
  case expression
  case declaration
  case accessor
  case memberAttribute(parentDeclNode: DeclSyntax)
  case member
  case peer
  case conformance
  case codeItem
  case `extension`(extendedType: TypeSyntax)
}

public func expandAttachedMacroWithoutCollapsing<Context: MacroExpansionContext>(
  definition: Macro.Type,
  macroRole: MacroRoleContext,
  attributeNode: AttributeSyntax,
  declarationNode: DeclSyntax,
  in context: Context
) -> [String]? { (...)

I'd love to hear your thoughts on this! This is just something that popped into my mind, and I'm not sure if it makes any sense - just a thought. 🤔😊

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually had the same thought while I was implementing this! But @rintaro pointed out to me that we might also need to make the extendedType an associated value of PluginMessage.MacroRole which would complicate the Codable conformance, so it seemed more straightforward to pass extendedType as an argument to expandAttachedMacro. I'm still interested in exploring this approach further separately though, because it seems plausible that we might want more role-specific arguments, and those conceptually make sense as enum associated values.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah, I see.

in context: Context
) -> [String]? {
do {
Expand Down Expand Up @@ -295,6 +302,39 @@ public func expandAttachedMacroWithoutCollapsing<Context: MacroExpansionContext>
return "extension \(typeName) : \(protocolName) \(whereClause) {}"
}

case (let attachedMacro as ExtensionMacro.Type, .extension):
guard let declGroup = declarationNode.asProtocol(DeclGroupSyntax.self) else {
// Compiler error: type mismatch.
throw MacroExpansionError.declarationNotDeclGroup
}

guard let extendedType = extendedType else {
throw MacroExpansionError.noExtendedTypeSyntax
}

// Local function to expand an extension macro once we've opened up
// the existential.
func expandExtensionMacro(
_ node: some DeclGroupSyntax
) throws -> [ExtensionDeclSyntax] {
return try attachedMacro.expansion(
of: attributeNode,
attachedTo: node,
providingExtensionsOf: extendedType,
in: context
)
}

let extensions = try _openExistential(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect we don't need to do _openExistential tricks nowadays, but it's fine this way

declGroup,
do: expandExtensionMacro
)

// Form a buffer of peer declarations to return to the caller.
return extensions.map {
$0.formattedExpansion(definition.formatMode)
}

default:
throw MacroExpansionError.unmatchedMacroRole(definition, macroRole)
}
Expand Down Expand Up @@ -323,6 +363,7 @@ public func expandAttachedMacro<Context: MacroExpansionContext>(
attributeNode: AttributeSyntax,
declarationNode: DeclSyntax,
parentDeclNode: DeclSyntax?,
extendedType: TypeSyntax?,
in context: Context
) -> String? {
let expandedSources = expandAttachedMacroWithoutCollapsing(
Expand All @@ -331,6 +372,7 @@ public func expandAttachedMacro<Context: MacroExpansionContext>(
attributeNode: attributeNode,
declarationNode: declarationNode,
parentDeclNode: parentDeclNode,
extendedType: extendedType,
in: context
)
return expandedSources.map {
Expand Down
1 change: 1 addition & 0 deletions Sources/SwiftSyntaxMacros/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ add_swift_host_library(SwiftSyntaxMacros
MacroProtocols/ConformanceMacro.swift
MacroProtocols/DeclarationMacro.swift
MacroProtocols/ExpressionMacro.swift
MacroProtocols/ExtensionMacro.swift
MacroProtocols/FreestandingMacro.swift
MacroProtocols/Macro.swift
MacroProtocols/Macro+Format.swift
Expand Down
35 changes: 35 additions & 0 deletions Sources/SwiftSyntaxMacros/MacroProtocols/ExtensionMacro.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift.org open source project
//
// Copyright (c) 2014 - 2023 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
//
//===----------------------------------------------------------------------===//

import SwiftSyntax

/// Describes a macro that can add extensions to the declaration it's
/// attached to.
public protocol ExtensionMacro: AttachedMacro {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had an idea! We could make ConformanceMacro refine ExtensionMacro, with a default implementation of the extension entry point based on the conformance entry point. That way, any macro implementations that implement ConformanceMacro today are automatically "upgraded" to ExtensionMacro implementations, and they provide the appropriate @attached(extension...) alongside the conformance one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh! That is a really good idea

/// Expand an attached extension macro to produce a set of extensions.
///
/// - Parameters:
/// - node: The custom attribute describing the attached macro.
/// - declaration: The declaration the macro attribute is attached to.
/// - type: The type to provide extensions of.
/// - context: The context in which to perform the macro expansion.
///
/// - Returns: the set of extension declarations introduced by the macro,
/// which are always inserted at top-level scope. Each extension must extend
/// the `type` parameter.
static func expansion(
of node: AttributeSyntax,
attachedTo declaration: some DeclGroupSyntax,
providingExtensionsOf type: some TypeSyntaxProtocol,
in context: some MacroExpansionContext
) throws -> [ExtensionDeclSyntax]
}
20 changes: 19 additions & 1 deletion Sources/SwiftSyntaxMacros/MacroSystem.swift
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,8 @@ class MacroApplication<Context: MacroExpansionContext>: SyntaxRewriter {
|| macro is MemberMacro.Type
|| macro is AccessorMacro.Type
|| macro is MemberAttributeMacro.Type
|| macro is ConformanceMacro.Type)
|| macro is ConformanceMacro.Type
|| macro is ExtensionMacro.Type)
}

if newAttributes.isEmpty {
Expand Down Expand Up @@ -438,6 +439,23 @@ extension MacroApplication {
}
}

let extensionMacroAttrs = getMacroAttributes(attachedTo: decl.as(DeclSyntax.self)!, ofType: ExtensionMacro.Type.self)
let extendedTypeSyntax = TypeSyntax("\(extendedType.trimmed)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the proposal it's mentioned:

Each ExtensionDeclSyntax in the resulting array must use the providingExtensionsOf parameter as the extended > type, which is a qualified type name. For example, for the following code:

struct Outer {
  @MyProtocol
  struct Inner {}
}

The type syntax passed to ExtensionMacro.expansion for providingExtensionsOf is Outer.Inner.

I'm not sure if I've understood this part of the proposal correctly. Does it mean that if I create this macro:

public struct SomeExtensionMacro: ExtensionMacro {
  public static func expansion(
    of node: AttributeSyntax,
    attachedTo declaration: some DeclGroupSyntax,
    providingExtensionsOf type: some TypeSyntaxProtocol,
    in context: some MacroExpansionContext
  ) throws -> [ExtensionDeclSyntax] {
    [ExtensionDeclSyntax(extendedType: type, memberBlock: MemberDeclBlockSyntax {})]
  }
}

and I attach it like so

struct Outer {
  @SomeExtensionMacro
  struct Inner {}
}

are we expecting that extendedType.description here will look something like this: Outer.Inner? If I understand the assumptions correctly, I think this would happen because the code above:

let extendedType: Syntax
if let identified = decl.asProtocol(IdentifiedDeclSyntax.self) {
  extendedType = Syntax(identified.identifier.trimmed)
} else if let ext = decl.as(ExtensionDeclSyntax.self) {
  extendedType = Syntax(ext.extendedType.trimmed)
} else {
  return []
}

can only extract the identifier from the root type.

Here is a test case covering this example using @AddSendableExtension macro:

func testExtensionExpansionOfNestedType() {
  assertMacroExpansion(
    """
    struct Outer {
      @AddSendableExtension
      struct Inner {
      }
    }
    """,
    expandedSource:
      """

      struct Outer {
        struct Inner {
        }
      }
      extension Outer.Inner: Sendable {
      }
      """,
    macros: testMacros,
    indentationWidth: indentationWidth
  )
}

right now it's failing with message:

failed - Actual output (+) differed from expected output (-):
 struct Outer {
   struct Inner {
   }
 }
–extension Outer.Inner: Sendable {
–}

Actual expanded source:

struct Outer {
  struct Inner {
  }
}
extension Outer.Inner: Sendable {
}

Am I understanding it correctly that this shouldn't fail?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great question, and there's a little bit of nuance here because the expansion performed by the MacroSystem in SwiftSyntax is only used for unit testing macros, but the real expansion is integrated into the compiler (and implemented in swiftlang/swift#66967). It's not possible for MacroSystem to form the same fully qualified type that the compiler can form in the general case, e.g. the compiler can look through type aliases, while the macro system here cannot. I believe there are also some known issues with where the expansion is inserted in MacroSystem for conformance macros, because it doesn't insert them at the top-level like the compiler does. I definitely think we can improve what MacroSystem does in many cases (e.g. we could form a qualified name from the lexical nesting structure), but it can't perfectly match what the compiler does during extension macro expansion.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't fully aware of that, I really appreciate the explanation.

for (attribute, extensionMacro) in extensionMacroAttrs {
do {
let newExtensions = try extensionMacro.expansion(
of: attribute,
attachedTo: decl,
providingExtensionsOf: extendedTypeSyntax,
in: context
)

extensions.append(contentsOf: newExtensions.map(DeclSyntax.init))
} catch {
context.addDiagnostics(from: error, node: attribute)
}
}

Comment on lines +442 to +458
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure what the assumptions are about how we're structuring code in this project, but I wonder whether it might be better to extract this code into a separate method. The name and doc-comment of this method suggest that the responsibility of this method is solely to expand conformances - I guess expanding extensions could be seen as a bit different responsibility. Just a thought! 💜

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conformance macros are effectively a special case of extension macros. I'm going to take Doug's suggestion here in a follow-up PR and make ConformanceMacro refine ExtensionMacro so that they can be implemented in terms of extension macros. As part of that, I will clean up this code to only operate on ExtensionMacro, since ConformanceMacro will be a refinement of it!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds really cool! Can't wait to see the result!

return extensions
}

Expand Down
16 changes: 16 additions & 0 deletions Tests/SwiftParserTest/AttributeTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -629,4 +629,20 @@ final class AttributeTests: XCTestCase {
"""
)
}

func testAttachedExtensionAttribute() {
assertParse(
"""
@attached(extension)
macro m()
"""
)

assertParse(
"""
@attached(extension, names: named(test))
macro m()
"""
)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conformances: as well? Shouldn't really matter since it's just parsed as a custom attribute anyway, but good to have the test case IMO.

}
Loading