Skip to content

Improve modeling of Child in CodeGeneration #1289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

ahoppen
Copy link
Member

@ahoppen ahoppen commented Jan 28, 2023

Instead of having tokenChoices, textChoices, nodeChoices and collectionElementName in Child, which are all pretty much mutually exclusive with each other, model the kind of a child as an enum that be either of

  • A single node kind
  • A list of node choices
  • A syntax collection
  • A list of possible tokens

The syntax in the Python definition files is a little awkward with this because you specify keywords e.g. as Keyword|async but since I anticipate that they will soon be gone anyway once we've moved all code generation to use SwiftSyntaxBuilder, I don't think we need to put too much effort into making the Python files look nice.


This is a prerequisite for #1180, which will assert that tokens in a syntax tree match the specified token kind and text choices.

@ahoppen ahoppen requested review from bnbarham and kimdv January 28, 2023 08:15
@ahoppen
Copy link
Member Author

ahoppen commented Jan 28, 2023

@swift-ci Please test

case .token(choices: let choices):
if choices.count == 1 {
switch choices.first! {
case .keyword(text: _): return "KeywordToken"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could text also be replaced with _

@ahoppen ahoppen force-pushed the ahoppen/assert-token-choices-preparation branch from 6208ab4 to 2467d12 Compare January 28, 2023 16:29
@ahoppen
Copy link
Member Author

ahoppen commented Jan 28, 2023

@swift-ci Please test

@@ -81,14 +66,11 @@ public let AVAILABILITY_NODES: [Node] = [
kind: "Syntax",
children: [
Child(name: "Platform",
kind: "IdentifierToken",
kind: .token(choices: [.token(tokenKind: "IdentifierToken")]),
description: "The name of the OS on which the availability should berestricted or 'swift' if the availability should berestricted based on a Swift version.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not actually sure what these are used for. But it appears we're not adding a space when we drop the newlines and indentation (separate to this PR, just noticed it here).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What exactly do you mean?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"The name of the OS on which the availability should berestricted or 'swift' if the availability should berestricted based on a Swift version."

Note the two "berestricted". This is coming from the python where it's defined as a multiline string. But the lines are being joined as is (well, with indentation removed), without a space separator.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see, this wasn’t related to the line I changed.

Interesting that I haven’t noticed this before. I fixed it now.

# This will force validation logic to check the text passed into the
# token against the choices.
self.text_choices = text_choices or []
self.token_choices.append((token, choice_text))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a case where token != KeywordToken and kind != KeywordToken? Can't we just check/use kind instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What exactly are you suggesting? Also, this is going to be removed in a month or so anyway when all gyb files are migrated to CodeGeneration so maybe it doesn’t matter too much anyway.

…Generation

# Conflicts:
#	CodeGeneration/Sources/SyntaxSupport/gyb_generated/AttributeNodes.swift
#	CodeGeneration/Sources/SyntaxSupport/gyb_generated/AvailabilityNodes.swift
#	CodeGeneration/Sources/SyntaxSupport/gyb_generated/DeclNodes.swift
@ahoppen
Copy link
Member Author

ahoppen commented Jan 31, 2023

@swift-ci Please test

@kimdv
Copy link
Contributor

kimdv commented Feb 1, 2023

@swift-ci please test macOS

@ahoppen
Copy link
Member Author

ahoppen commented Feb 1, 2023

@swift-ci Please test macOS

@ahoppen ahoppen merged commit 03f748b into swiftlang:main Feb 2, 2023
@ahoppen ahoppen deleted the ahoppen/assert-token-choices-preparation branch February 2, 2023 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants