[DO NOT MERGE] main -> 5.7 diffs #362

milseman · 2022-04-27T15:32:26Z

This is not intended for merging, it's for showing what the remaining diffs and un-cherry-picked changes are between main and 5.7

…or support.

A quick pass to flip `/.../` out of the alternatives and into the main syntax. Still needs a bunch of work. Also add some commentary on a regex with `]` as the starting character.

…n. Split out Proposed solution from Detailed design. Parallelize the structure a bit better.

This isn't actually used, as we convert to a DSL custom character class, and then use that consumer logic.

Convert AST escape sequences that represent a scalar value (e.g `\f`, `n`, `\a`) into scalars in the DSL tree. This allows the matching engine to match against them.

The "motivation tests" still aren't working on macOS CI

* Number captures consistently * Proper indentation

Co-authored-by: Nate Cook <[email protected]>

milseman · 2022-04-27T15:39:50Z

@swift-ci please test

milseman · 2022-04-27T16:21:54Z

Package.swift

@@ -76,7 +76,7 @@ let package = Package(
                .unsafeFlags(["-Xfrontend", "-enable-experimental-pairwise-build-block"]),
                .unsafeFlags(["-Xfrontend", "-disable-availability-checking"])
            ]),
-        .target(
+        .testTarget(


@rxwei can you get this (3293905) cherry-picked on to 5.7?

milseman · 2022-04-27T16:22:38Z

Sources/RegexBuilder/Algorithms.swift

+//
+//===----------------------------------------------------------------------===//
+
+import _StringProcessing


@itingliu can you get this (5af1427) cherry-picked on to 5.7?

milseman · 2022-04-27T16:23:10Z

Sources/RegexBuilder/Anchor.swift

-      case .endOfLine: fatalError("Not yet supported")
-      case .wordBoundary: return .notWordBoundary
-      }
+  var baseAssertion: DSLTree._AST.AssertionKind {


@natecook1000 can you get this (f8e1dc2) and others cherry-picked onto 5.7?

milseman · 2022-04-27T16:24:12Z

Sources/VariadicsGenerator/VariadicsGenerator.swift

+    "C\($0+1)\(opt)"
+  }.joined(separator: ", ")
+}
+


@milseman can you get this (435090d) and others cherry-picked onto 5.7?

milseman · 2022-04-27T16:24:55Z

Sources/_CUnicode/UnicodeNormalization.c

@@ -1,116 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// This source file is part of the Swift.org open source project


@Azoy can you make sure to get everything cherry-picked onto 5.7?

milseman · 2022-04-27T16:27:17Z

Sources/_StringProcessing/Regex/AnyRegexOutput.swift

+  ///   Returns `nil` otherwise.
+  public func `as`<Output>(_ type: Output.Type) -> Regex<Output>? {
+    fatalError("FIXME: Not implemented")
+  }


@milseman can you get this (e0922ec) and others cherry-picked onto 5.7?

milseman · 2022-04-27T16:28:42Z

Tests/RegexBuilderTests/CustomTests.swift

@@ -133,10 +133,55 @@ func customTest<Match: Equatable>(
  }
 }

+// Test support
+struct Concat : Equatable {


@itingliu can you get this (73a5ccf) and others cherry-picked onto 5.7?

milseman · 2022-04-27T16:30:06Z

Tests/RegexBuilderTests/RegexDSLTests.swift

+        Optionally {
+          Capture(as: value) {
+            OneOrMore(.digit)
+          } transform: { Int($0)! }


@natecook1000 can you get this (d0598b7) and others cherry-picked onto 5.7?

milseman · 2022-04-27T16:31:00Z

Tests/RegexTests/AlgorithmsTests.swift

+      "a|a|||a|a|||a|a|||".split(separator: "|||", maxSplits: 2),
+      ["a|a", "a|a", "a|a|||"])
+
+    XCTAssertEqual(


@natecook1000 can you get this (d6a01e7) and others cherry-picked onto 5.7?

milseman · 2022-04-27T16:31:33Z

Tests/RegexTests/AlgorithmsTests.swift

+      XCTFail()
+    case try! Regex("abc"):
+      XCTFail()
+


@rxwei can you make sure the ~= overloads and tests get onto 5.7?

Confirmed (using the command below) that this branch now contains only changes to doc comments, changes to // comments where content was moved into or out of a doc comment, and one spelling correction to code. % git diff origin/main... | grep -v '^+ *///\|- *///' | grep -v '^@@' | grep -v '^ ' | grep -v '^+++\|^---' | grep -v '^diff --git' | grep -v '^index ' + // Individual public API functions are in the generated Variadics.swift file. +// For now, we use String as the source while prototyping... + + // NOTE: Currently, this means we have raw quotes. + // Better would be to have real Swift string delimiter parsing logic. + + // NOTE: traditional comments are not nested. Currently, we are neither. + // Traditional comments can't have `)`, not even escaped in them either, we + // can. Traditional comments can have `*/` in them, we can't without + // escaping. We don't currently do escaping. + - // Finish, flush, and clear. Returns the rendered output + // Finish, flush, and clear. + // + // - Returns: The rendered output. + // Note: The `Script` enum includes the "meta" script type "Katakana_Or_Hiragana", which + // isn't defined by https://www.unicode.org/Public/UCD/latest/ucd/Scripts.txt, + // but is defined by https://www.unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt. + // We may want to split it out, as it's the only case that is a union of + // other script types. + +// TODO: These should become aliases for the Block (blk) Unicode character +// property. + - // Ensures `.0` always refers to the whole match. - // Allows `.0` when `Match` is not a tuple. - func withInversion(_ invertion: Bool) -> Self { + func withInversion(_ inversion: Bool) -> Self { - if invertion { + if inversion {

Revise doc comments for API reference style.

Add the SPI interface `swiftCompilerLexRegexLiteral` and `swiftCompilerParseRegexLiteral` for the Swift compiler to call into. This allows us to avoid depending on other library API on the compiler side, while letting it keep the bridging gunk. While we're here, add an extra `String` return for the parsing function that could allow us to change the regex emission format in the future. This still needs to be plumbed through on the complier side though.

Update PatternConverter

…wiftlang#367) Work around the compiler bug rdar://92459215 by using generic parameters instead of opaque result types for a couple of methods.

Introduce CaptureList, make CaptureStructure semi-vestigial

This is defined in UAX#44 as being equivalent to `Lu | Ll | Lt`.

This is a PCRE spelling for a cased letter.

* Add tests for AnyRegexOutput Fix a few missing areas of functionality while we're at it

hamishknight and others added 30 commits March 18, 2022 17:57

Add DelimiterSyntax.md

381a3c7

Expand on parsing issues with / as delimited. Add a note about edit…

c79f457

…or support.

Rewrite that Editor Considerations paragraph.

731292e

Change single quote constructs to be invalid

1049276

Elaborate on starting character limitations

c7d556c

Elaborate on comma case

7074cfb

grammar

945219e

Add comma disambiguation

d1d0d57

Update comma disambiguation

bcebfc6

Flip pitch to /.../ as the main syntax

1c2b7ad

A quick pass to flip `/.../` out of the alternatives and into the main syntax. Still needs a bunch of work. Also add some commentary on a regex with `]` as the starting character.

Update alternatives considered

e841184

Fix headings

be7a802

Tweak phrasing

06c2b28

Small tweaks

91a93a8

Tweak

c0e3bef

Flesh things out a bit more. Initial bits for the Intro and Motivatio…

298092c

…n. Split out Proposed solution from Detailed design. Parallelize the structure a bit better.

Expand out disclosure triangles, and other tweaks

1ebefa1

Generalize discussion on language mode

811bfcb

Expand some prose

70be006

Update to also pitch #/.../#

4bb25b3

Add DSL example

35d9132

Rejig motivation/solution

f99dadb

Expand on typed captures

eed1b24

Generalize language mode

7ad4037

Add multi-line mode

f4ef0c2

Remove Normalization and Grapheme data for SPI

b25a0b8

Update pitch

addfbfd

Clarify upgrade path

bb819f6

Remove AST CustomCharacterClass consumer generation

36f7160

This isn't actually used, as we convert to a DSL custom character class, and then use that consumer logic.

Convert scalar escape sequences to DSL scalars

ed9f72c

Convert AST escape sequences that represent a scalar value (e.g `\f`, `n`, `\a`) into scalars in the DSL tree. This allows the matching engine to match against them.

milseman and others added 8 commits April 26, 2022 07:38

Mention language level pattern matching (swiftlang#354)

bef2092

Fix empty.split w/ empty separator (swiftlang#353)

9ff87db

Add a section describing 'find empty' behavior (swiftlang#352)

2401a58

Add example from RegexBuilder proposal as a test (swiftlang#344)

d0598b7

The "motivation tests" still aren't working on macOS CI

Refactor generator script (swiftlang#356)

435090d

* Number captures consistently * Proper indentation

Fix cut-off sentence.

96df14d

Co-authored-by: Nate Cook <[email protected]>

Update PatternConverter

12ef33b

Update Sources/_StringProcessing/Regex/Match.swift

0e59bdf

milseman force-pushed the 5_7_mega_glob branch from 435090d to dbd1e8c Compare April 27, 2022 15:33

milseman commented Apr 27, 2022

View reviewed changes

milseman mentioned this pull request Apr 27, 2022

[DO NOT MERGE] String Processing Executor 1 (main) swiftlang/swift#58461

Closed

amartini51 and others added 8 commits April 27, 2022 11:11

Merge pull request swiftlang#350 from apple/amartini/docs_91301229

6c5c082

Revise doc comments for API reference style.

Merge pull request swiftlang#364 from hamishknight/compiler-interface

b7a03c9

Add some comments

2c386ef

Merge pull request swiftlang#359 from Azoy/update-patternconverter

40c177f

Update PatternConverter

Replace opaque results with generic parameters in Algorithms.swift (s…

b7fb965

…wiftlang#367) Work around the compiler bug rdar://92459215 by using generic parameters instead of opaque result types for a couple of methods.

Simplify capture representations (swiftlang#360)

838bdfe

Introduce CaptureList, make CaptureStructure semi-vestigial

milseman force-pushed the 5_7_mega_glob branch from dbd1e8c to 7f6e6a5 Compare April 29, 2022 15:50

natecook1000 and others added 6 commits May 2, 2022 15:05

Add NegativeLookahead and Anchor comments (swiftlang#372)

e748aea

Add matching support for \p{Lc}

13342eb

This is defined in UAX#44 as being equivalent to `Lu | Ll | Lt`.

Add parser support for \p{L&}

925f51b

This is a PCRE spelling for a cased letter.

Merge pull request swiftlang#373 from hamishknight/case-in-prop

ade8f01

Update ProposalOverview.md

c44efeb

Add tests for AnyRegexOutput (swiftlang#371)

9801855

* Add tests for AnyRegexOutput Fix a few missing areas of functionality while we're at it

milseman force-pushed the 5_7_mega_glob branch from 7f6e6a5 to 9801855 Compare May 3, 2022 18:57

milseman closed this May 10, 2022

milseman deleted the 5_7_mega_glob branch May 10, 2022 16:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[DO NOT MERGE] main -> 5.7 diffs #362

[DO NOT MERGE] main -> 5.7 diffs #362

Uh oh!

milseman commented Apr 27, 2022

Uh oh!

milseman commented Apr 27, 2022

Uh oh!

milseman Apr 27, 2022

Uh oh!

milseman Apr 27, 2022

Uh oh!

milseman Apr 27, 2022

Uh oh!

milseman Apr 27, 2022

Uh oh!

milseman Apr 27, 2022

Uh oh!

milseman Apr 27, 2022

Uh oh!

milseman Apr 27, 2022

Uh oh!

milseman Apr 27, 2022

Uh oh!

milseman Apr 27, 2022

Uh oh!

milseman Apr 27, 2022

Uh oh!

milseman Apr 27, 2022

Uh oh!

Uh oh!

[DO NOT MERGE] main -> 5.7 diffs #362

[DO NOT MERGE] main -> 5.7 diffs #362

Uh oh!

Conversation

milseman commented Apr 27, 2022

Uh oh!

milseman commented Apr 27, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!