Allow combining code block item lists

[anreitersimon]

Hi!

I was trying to write a SwiftPackage to allow generating some code.

As part of my setup i tried to introduce a abstraction of a CodeGenStep.

The idea is that i would define multiple CodeGenStep, each of them would generate some (top-level) code
And i wanted to then loop over all of them and combine their outputs.

I used CodeBlockItemListBuilder since that felt most appropriate to generate code in that scope.

protocol CodeGenStep {

    @CodeBlockItemListBuilder
    var code: CodeBlockItemList { get }
}

This worked great!
My problem then was combining these CodeBlockItemLists and generating the final result.

let steps: [CodeGenStep] = [...]

let sourceFile = SourceFile(
    eofToken: .eof,
    statementsBuilder: {
      for step in steps {
        step.code // this currently emits a error
      }
    }
)

Also manually merging (outside of the result builder) did also not work, since CodeBlockItemList.elements has internal accessLevel so i cannot manually merge them

I can imagine this being a common use case so i thought it open up a PR.

My Solution

This basically adds a buildExpression to the CodeBlockItemListBuilder which makes the code snippet above legal.
Additionally it provides a initializer to allow combining multiple CodeBlockItemLists into one.

[anreitersimon]

Thinking about this some more it might make sense to extend this approach to more constructs.

Basically everything ending in List (for instance MemberDeclList, DeclNameArgumentList)

The pattern would be really similar to the one in the PR, which makes me think it could potentially be added to the code-generation.

Following this template

extension {{Base}}ListBuilder {

     public static func buildExpression(_ expression: {{Base}}List) -> Component {
         return expression.elements
     }
 }

 extension{{Base}}List {

     public init(combining lists: [ExpressibleAs{{Base}}List]) {
         self.elements = lists.flatMap {
             $0.create{{Base}}().elements
         }
     }

 }

[ahoppen]

Thanks for your PR @anreitersimon, this sounds like a wonderful idea. I also agree that this is not only useful for CodeBlockItemListBuilder but for all syntax collections that can be initialized by result builders.

As you might have already seen, we use .swift.gyb files to code-generate most of SwiftSyntaxBuilder, so you should be able to add exactly your pattern to these files and make them apply to all syntax collections. The result builder extension would need to go into ResultBuilders.swift.gyb and your combining initializer into BuildableCollectionNodes.swift.gyb. If you need any help writing the gyb code, please let me know.

[anreitersimon]

@ahoppen Thanks for the pointers, i think i figured it out.

[ahoppen]

Thanks. That looks great. I’ve added a few comments inline regarding doc-comments inline. Could you also add a test case for this new behavior?

[anreitersimon]

Thanks. That looks great. I’ve added a few comments inline regarding doc-comments inline. Could you also add a test case for this new behavior?

Do you have any suggestions around the tests?
Im specifically wondering if the tests should cover all of the builders, or if its ok to just cover a few to test that it works in general.

If yes my approach would be to actually go with the example from the description and extend it to handle all the different constructs.

protocol CodeGenerator {

  @CodeBlockItemListBuilder
  var codeBlockItemList: CodeBlockItemList { get }

  @TupleExprElementListBuilder
  var tupleExprElementList: ExpressibleAsTupleExprElementList { get }
  
  @ArrayElementListBuilder
  var arrayElementList: ExpressibleAsArrayElementList { get }

  ....
}

And then test the composition with something like this

struct SimpleCodeGenerator {
   var codeBlockItemList: CodeBlockItemList { ... }
   var tupleExprElementList: ExpressibleAsTupleExprElementList { ... }

}


struct CodeGeneratorPipeline: CodeGenerator {
  let steps: [CodeGenerator]

  var codeBlockItemList: CodeBlockItemList {
    for step in steps {
      step.codeBlockItemList
    }
  } 
  var tupleExprElementList: ExpressibleAsTupleExprElementList {
    for step in steps {
      step.tupleExprElementList
    }
  } 
}

This approach is really boilerplate heavy though, so maybe this could also use gyb to generate some of it.

I think it might actually be useful to have a "catalog" of sample data for each of the constructs, which this would provide (at least for the collection-nodes)

@ahoppen Do you have any thoughts/suggestions on this?

[ahoppen]

Im specifically wondering if the tests should cover all of the builders, or if it's ok to just cover a few to test that it works in general.

Just testing one builder would be totally fine. I don’t think there’s much value in testing two of them since they are all generated using the same technique.

After finishing your changes, could you squash all of your commits to keep the Git history clean?

[anreitersimon]

@ahoppen I think all the Feedback should be addressed.

• added CollectionNodeFlatteningTests.swift with two tests (one for the result-builder and another one for the constructor)
• Removed the generic doc-comments on the resultbuilder method, keeping only the comment about flattening
• squashed the commits into a single commit

[ahoppen]

This looks good. Thanks for your contribution 🙏

[ahoppen]

@swift-ci Please test

[ahoppen]

@swift-ci Please test