Skip to content

Reorganize our modules #59

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Dec 8, 2021
Merged

Reorganize our modules #59

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
860e1da
Make AST subdirectory
milseman Dec 7, 2021
22c507e
Scale back variadic arity to 7 for compilation time
milseman Dec 7, 2021
dc1e329
Remove Util dependency from Regex and MatchingEngine
milseman Dec 7, 2021
9bdc604
Remove Util dependency from Regex and MatchingEngine
milseman Dec 7, 2021
1a28746
Remove Util module
milseman Dec 7, 2021
c5877d1
Rename core module to _MatchingEngine
milseman Dec 7, 2021
3207fa8
Move parser and ast to compile-time module
milseman Dec 7, 2021
09fce17
Rename Regex to _StringProcessing
milseman Dec 8, 2021
8d45196
Move RegexDSL to be under _StringProcessing
milseman Dec 8, 2021
a81ab05
Reorganize some files
milseman Dec 8, 2021
18090d7
Update Sources/_StringProcessing/CharacterClass.swift
milseman Dec 8, 2021
1391554
Restrict access control some
milseman Dec 8, 2021
b038258
Internalize more
milseman Dec 8, 2021
6d255ab
Minor cleanups
milseman Dec 8, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 18 additions & 24 deletions Package.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ let package = Package(
products: [
// Products define the executables and libraries produced by a package, and make them visible to other packages.
.library(
name: "Regex",
targets: ["Regex"]),
name: "_StringProcessing",
targets: ["_StringProcessing"]),
.library(
name: "PEG",
targets: ["PEG"]),
.library(
name: "MatchingEngine",
targets: ["MatchingEngine"]),
name: "_MatchingEngine",
targets: ["_MatchingEngine"]),
.executable(
name: "VariadicsGenerator",
targets: ["VariadicsGenerator"])
Expand All @@ -26,45 +26,39 @@ let package = Package(
targets: [
// Targets are the basic building blocks of a package. A target can define a module or a test suite.
// Targets can depend on other targets in this package, and on products in packages which this package depends on.
.target(
name: "Util",
dependencies: []),
.testTarget(
name: "UtilTests",
dependencies: ["Util"]),
dependencies: ["_StringProcessing"]),
.target(
name: "MatchingEngine",
dependencies: ["Util"]),
name: "_MatchingEngine",
dependencies: []),
.testTarget(
name: "MatchingEngineTests",
dependencies: ["MatchingEngine"]),
dependencies: ["_MatchingEngine"]),
.target(
name: "Regex",
dependencies: ["Util", "MatchingEngine"]),
name: "_StringProcessing",
dependencies: ["_MatchingEngine"]),
.testTarget(
name: "RegexTests",
dependencies: ["Regex"]),
.target(
name: "RegexDSL",
dependencies: ["Regex"]),
dependencies: ["_StringProcessing"]),
.testTarget(
name: "RegexDSLTests",
dependencies: ["RegexDSL"]),
dependencies: ["_StringProcessing"]),
.target(
name: "PEG",
dependencies: ["Util", "MatchingEngine"]),
dependencies: ["_MatchingEngine"]),
.testTarget(
name: "PEGTests",
dependencies: ["PEG", "Util"]),
dependencies: ["PEG"]),
.target(
name: "PTCaRet",
dependencies: ["Util", "MatchingEngine"]),
dependencies: ["_MatchingEngine"]),
.testTarget(
name: "PTCaRetTests",
dependencies: ["PTCaRet", "Util"]),
dependencies: ["PTCaRet"]),
.target(
name: "Algorithms",
dependencies: ["Regex", "PEG"]),
dependencies: ["_StringProcessing", "PEG"]),
.testTarget(
name: "AlgorithmsTests",
dependencies: ["Algorithms"]),
Expand All @@ -79,7 +73,7 @@ let package = Package(
// MARK: Exercises
.target(
name: "Exercises",
dependencies: ["MatchingEngine", "PEG", "PTCaRet", "Regex", "RegexDSL"]),
dependencies: ["_MatchingEngine", "PEG", "PTCaRet", "_StringProcessing"]),
.testTarget(
name: "ExercisesTests",
dependencies: ["Exercises"]),
Expand Down
17 changes: 4 additions & 13 deletions Sources/Algorithms/Consumers/RegexConsumer.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
import Regex
import _MatchingEngine

import _StringProcessing

public struct Regex {
let string: String
Expand All @@ -13,14 +15,9 @@ public struct Regex {
public struct RegexConsumer: CollectionConsumer {
// NOTE: existential
let vm: Executor
let referenceVM: TortoiseVM

public init(regex: Regex) {
let ast = try! parse(regex.string, .traditional)
let program = Compiler(ast: ast).emit()
self.vm = Executor(program: program)
let legacyProgram = try! compile(ast, options: regex.options)
self.referenceVM = TortoiseVM(program: legacyProgram)
self.vm = _compileRegex(regex.string)
Copy link
Contributor

@rxwei rxwei Dec 8, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can RegexConsume use the public API (e.g. firstMatch) instead? That way we wouldn't need to expose Compiler or Executor.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently it is RegexProtocol.match(in:). You can add a range parameter to it.

Copy link
Member Author

@milseman milseman Dec 8, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would I use for types? It seems like DynamicCaptures will let me parse the string, but will trap at runtime:

  let regex: _StringProcessing.Regex<DynamicCaptures>
Could not cast value of type '()' (0x7fff815b12b8) to '_StringProcessing.DynamicCaptures' (0x10f67b618).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using MockRegexLiteral<()> instead lets me get past that, but then I hit test failures. If it's going to take too long, I'd rather get this more fundamental PR merged and tweak access control after

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this needs the following change:

--- a/Sources/_StringProcessing/RegexDSL/Core.swift
+++ b/Sources/_StringProcessing/RegexDSL/Core.swift
@@ -85,7 +85,13 @@ extension RegexProtocol {
     guard let result = executor.execute(input: input) else {
       return nil
     }
-    return RegexMatch(range: result.range, captures: () as! Capture)
+    let convertedCapture: Capture
+    if Capture.self == DynamicCaptures.self {
+      convertedCapture = DynamicCaptures.tuple([]) as! Capture
+    } else {
+      convertedCapture = () as! Capture
+    }
+    return RegexMatch(range: result.range, captures: convertedCapture)
   }
 }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address this later sounds good too

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scratch that, I later also get a

Could not cast value of type 'Swift.Array<Swift.Substring>' (0x7fff815ac3d8) to '()' (0x7fff815b12b8).

}

public func consume(
Expand All @@ -30,12 +27,6 @@ public struct RegexConsumer: CollectionConsumer {
input: consumed.base,
in: index..<consumed.endIndex,
mode: .partialFromFront)
assert(
result?.range == referenceVM.execute(
input: consumed.base,
in: index..<consumed.endIndex,
mode: .partialFromFront
)?.range)
return result?.range.upperBound
}
}
Expand Down
2 changes: 1 addition & 1 deletion Sources/Algorithms/Searchers/Searcher.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import MatchingEngine
import _MatchingEngine

public struct DefaultState<Searched: Collection> {
enum _State {
Expand Down
4 changes: 1 addition & 3 deletions Sources/Combinators/Combinators.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
import MatchingEngine
import _MatchingEngine

/*

Inspired by "Staged Parser Combinators for Efficient Data Processing" by Jonnalagedda et al.

*/

import Util

// Stages are represented as nested namespaces that bind generic
// types
public enum Combinators {
Expand Down
3 changes: 1 addition & 2 deletions Sources/Exercises/Participants/RegexParticipant.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import Regex
import RegexDSL
import _StringProcessing

/*

Expand Down
2 changes: 1 addition & 1 deletion Sources/PEG/PEGCode.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import Util
import _MatchingEngine

extension PEG.VM {
struct Code {
Expand Down
2 changes: 1 addition & 1 deletion Sources/PEG/PEGCompile.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import Util
import _MatchingEngine

extension PEG.VM {
typealias InIndex = Input.Index
Expand Down
2 changes: 1 addition & 1 deletion Sources/PEG/PEGCore.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import Util
import _MatchingEngine
let emitComments = true

struct PEGCore<
Expand Down
5 changes: 2 additions & 3 deletions Sources/PEG/PEGTranspile.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
import MatchingEngine
import Util
import _MatchingEngine

extension PEG.VM {
typealias MEProgram = MatchingEngine.Program<Input>
typealias MEProgram = _MatchingEngine.Program<Input>
func transpile() -> MEProgram {
typealias Builder = MEProgram.Builder
var builder = MEProgram.Builder()
Expand Down
2 changes: 1 addition & 1 deletion Sources/PEG/PEGVM.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import Util
import _MatchingEngine

extension PEG {

Expand Down
2 changes: 1 addition & 1 deletion Sources/PEG/Printing.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import Util
import _MatchingEngine

extension PEGCore.Instruction: InstructionProtocol {
var operandPC: InstructionAddress? { self.pc }
Expand Down
10 changes: 0 additions & 10 deletions Sources/Regex/ASTTransform.swift
View file
Open in desktop

This file was deleted.

18 changes: 0 additions & 18 deletions Sources/Regex/Anchor.swift
View file
Open in desktop

This file was deleted.

67 changes: 0 additions & 67 deletions Sources/Regex/CustomCharClass.swift
View file
Open in desktop

This file was deleted.

Loading