Add scala 3 contributing guide - MVP #2194
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| --- | ||
| title: Contexts | ||
| type: section | ||
| description: This page describes symbols in the Scala 3 compiler. | ||
| num: 18 | ||
| previous-page: arch-symbols | ||
| next-page: | ||
| --- | ||
|
|
||
| > (The following is work in progress), adapted from dotty.epfl.ch | ||
|
|
||
| The `Context` contains the state of the compiler, for example | ||
|
|
||
| * `settings` | ||
| * `freshNames` (`FreshNameCreator`) | ||
| * `period` (run and phase id) | ||
| * `compilationUnit` | ||
| * `phase` | ||
| * `tree` (current tree) | ||
| * `typer` (current typer) | ||
| * `mode` (type checking mode) | ||
| * `typerState` (for example undetermined type variables) | ||
| * ... | ||
|
|
||
| ### Contexts in the typer | ||
| The type checker passes contexts through all methods and adapts fields where | ||
| necessary, e.g. | ||
|
|
||
| ```scala | ||
| case tree: untpd.Block => typedBlock(desugar.block(tree), pt)(ctx.fresh.withNewScope) | ||
| ``` | ||
|
|
||
| A number of fields in the context are typer-specific (`mode`, `typerState`). | ||
bishabosha marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### In other phases | ||
| Other phases need a context for many things, for example to access the | ||
| denotation of a symbols (depends on the period). However they typically don't | ||
| need to modify / extend the context while traversing the AST. For these phases | ||
| the context can be simply an implicit class parameter that is then available in | ||
| all members. | ||
|
|
||
| **Careful**: beware of memory leaks. Don't hold on to contexts in long lived | ||
| objects. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| --- | ||
| title: High Level Architecture | ||
| type: chapter | ||
| description: This page introduces the high level architecture of the Scala 3 compiler. | ||
| num: 12 | ||
| previous-page: procedures-checklist | ||
| next-page: arch-lifecycle | ||
| --- | ||
|
|
||
| This chapter of the guide describes the architecture and concepts of `dotc`, | ||
| the Scala 3 compiler, including answers to questions such as: | ||
| - "What are the transformations that happen to my code?" | ||
| - "How do I run a compiler programatically?" | ||
| - "What are symbols, denotations, names and types?" | ||
| - "What is a compiler phase?" | ||
| - "What is the compiler Context?" | ||
|
|
||
| and many more. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| --- | ||
| title: Compiler Overview | ||
| type: section | ||
| description: This page describes the lifecycle for the Scala 3 compiler. | ||
| num: 13 | ||
| previous-page: arch-intro | ||
| next-page: arch-phases | ||
| --- | ||
|
|
||
| As mentioned in [what is a compiler?][1] `dotc` produces programs that can run on your machine, | ||
| first by verifying its input is a valid Scala program, and then transforming it into a | ||
| representation that can run on one of Scala's target platforms. | ||
|
|
||
| ## Introducing the Compiler's Lifecycle | ||
|
|
||
| #### Core | ||
| At a high level, `dotc` centers its work around a [Compiler][4], which maintains an ordered | ||
| list of [phases][3], and is responsible for creating new [runs][2]. | ||
| A run is a complete iteration of the compiler's phases over a list of input sources. | ||
| A compiler is designed to be reusable and can create many runs over its lifetime. | ||
|
|
||
| #### Runs | ||
| During a run, the input sources are converted to [compilation units][5] (i.e. the abstraction of | ||
| compiler state associated with each input source); then iteratively: a single phase is applied to | ||
| every compilation unit before progressing to the next phase. | ||
|
|
||
| #### Phases | ||
| A phase is an abstract transformation over a compilation unit, it is usually responsible | ||
| for transforming the trees and types representing the code of a source file. The primary phases of | ||
| the compiler are the `parser`, which converts text that matches Scala's | ||
| [syntax][10] into abstract syntax trees, ASTs, and the `typer`, which checks that | ||
| trees conform to expected types, and performs other operations such as implicit search. | ||
bishabosha marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| [You can read more about phases here][9]. | ||
|
|
||
| #### Drivers | ||
|
|
||
| The core compiler also requires a lot of state to be initialised before use, such as [settings][8] | ||
| and the [Context]. For convenience, the [Driver] class contains high level functions for | ||
| configuring the compiler and invoking it programatically. The object [Main] inherits from `Driver` | ||
| and is invoked by the `scalac` script. | ||
|
|
||
| <!-- #### Further Reading --> | ||
| > [Read more about a compiler's lifecyle][6]. | ||
|
|
||
| ## Code Structure | ||
|
|
||
| The code of the compiler is found in the package [dotty.tools][7], | ||
| containing the following sub-packages: | ||
| ```scala | ||
| tools // contains helpers and the `scala` generic runner | ||
| ├── backend // Compiler backends (currently JVM and JS) | ||
| ├── dotc // The main compiler, with subpackages: | ||
| │ ├── ast // Abstract syntax trees | ||
| │ ├── classpath | ||
| │ ├── config // Compiler configuration, settings, platform specific definitions. | ||
| │ ├── core // Core data structures and operations, with specific subpackages for: | ||
|
There was a problem hiding this comment. Earlier you mentioned a few phases (typer, parser) and @sjrd mentioned erasure... |
||
| │ │ ├── classfile // Reading of Java classfiles into core data structures | ||
| │ │ ├── tasty // Reading and writing of TASTY files to/from core data structures | ||
| │ │ └── unpickleScala2 // Reading of Scala2 symbol information into core data structures | ||
| │ ├── decompiler // pretty printing TASTY as code | ||
| │ ├── fromtasty // driver for recompilation from TASTY | ||
| │ ├── interactive // presentation compiler and code completions | ||
| │ ├── parsing // Scanner and parser | ||
| │ ├── plugins // compile plugin definitions | ||
| │ ├── printing // Pretty-printing trees, types and other data | ||
| │ ├── profile // internals for profiling the compiler | ||
| │ ├── quoted // internals for quoted reflection | ||
| │ ├── reporting // Reporting of error messages, warnings and other info. | ||
| │ ├── rewrites // Helpers for rewriting Scala 2's constructs into Scala 3's. | ||
| │ ├── sbt // Helpers for communicating with the Zinc compiler. | ||
| │ ├── semanticdb // Helpers for exporting semanticdb from trees. | ||
| │ ├── transform // Miniphases and helpers for tree transformations. | ||
| │ ├── typer // Type-checking | ||
| │ └── util // General purpose utility classes and modules. | ||
| ├── io // Helper modules for file access and classpath handling. | ||
| ├── repl // REPL driver and interaction with the terminal | ||
| ├── runner // helpers for the `scala` generic runner script | ||
| └── scripting // scala runner for the -script argument | ||
| ``` | ||
|
|
||
| [1]: {% link _overviews/scala3-contribution/contribution-intro.md %}/#what-is-a-compiler | ||
| [2]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Run.scala | ||
| [3]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Phases.scala | ||
| [4]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Compiler.scala | ||
| [5]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/CompilationUnit.scala | ||
| [6]: {% link _overviews/scala3-contribution/arch-time.md %} | ||
| [7]: https://github.com/lampepfl/dotty/tree/master/compiler/src/dotty/tools | ||
| [8]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/config/ScalaSettings.scala | ||
| [9]: {% link _overviews/scala3-contribution/arch-phases.md %} | ||
| [10]: {% link _scala3-reference/syntax.md %} | ||
| [Main]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Main.scala | ||
| [Driver]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Driver.scala | ||
| [Compiler]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Compiler.scala | ||
| [Run]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Run.scala | ||
| [Context]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Contexts.scala | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| --- | ||
| title: Compiler Phases | ||
| type: section | ||
| description: This page describes the phases for the Scala 3 compiler. | ||
| num: 14 | ||
| previous-page: arch-lifecycle | ||
| next-page: arch-types | ||
| --- | ||
|
|
||
| As described in the [compiler overview][1], `dotc` is divided into a list of [phases][Phase], | ||
| specified in the [Compiler] class. | ||
|
|
||
| #### Printing the phases of the Compiler | ||
|
|
||
| a flattened list of all the phases can be displayed by invoking | ||
| the compiler with the `-Xshow-phases` flag: | ||
| ``` | ||
| $ scalac -Xshow-phases | ||
| ``` | ||
|
|
||
| ## Phase Groups | ||
|
|
||
| In class [Compiler] we can access the list of phases with the method `phases`: | ||
|
|
||
| ```scala | ||
| def phases: List[List[Phase]] = | ||
| frontendPhases ::: picklerPhases ::: transformPhases ::: backendPhases | ||
| ``` | ||
|
|
||
| We see that phases are actually grouped into sublists, given by the signature | ||
| `List[List[Phase]]`; that is, each sublist forms a phase group that is then *fused* into a | ||
| single tree traversal when a [Run] is executed. | ||
|
|
||
| Phase fusion allows each phase of a group to be small and modular, | ||
| (each performing a single function), while reducing the number of tree traversals | ||
| and increasing performance. | ||
|
|
||
| Phases are able to be grouped together if they inherit from [MiniPhase]. | ||
|
|
||
| ## Phase Categories | ||
|
|
||
| Phases fall into four categories, allowing customisation by sub-classes of [Compiler]: | ||
|
|
||
| ### `frontendPhases` | ||
| In the main compiler these include [parser], [typer], [posttyper], | ||
| [prepjsinterop] and phases for producing SemanticDB and communicating with the | ||
| incremental compiler Zinc. | ||
| The [parser] reads source programs and generates untyped abstract syntax trees, which | ||
| in [typer] are then typechecked and transformed into typed abstract syntax trees. | ||
| Following is [posttyper], performing checks and cleanups that require a fully typed program. | ||
| In particular, it | ||
| - creates super accessors representing `super` calls in traits | ||
| - creates implementations of compiler-implemented methods, | ||
| such as `equals` and `hashCode` for case classes. | ||
| - marks [compilation units][2] that require inline expansion, or quote pickling | ||
| - simplifies trees of erased definitions | ||
| - checks variance of type parameters | ||
| - mark parameters passed unchanged from subclass to superclass for later pruning. | ||
|
|
||
| ### `picklerPhases` | ||
| These phases start with [pickler], which serializes typed trees | ||
| produced by the `frontendPhases` into TASTy format. Following is [inlining], | ||
| which expand calls to inline methods, and [postInlining] providing implementations | ||
| of the Mirror framework for inlined calls. | ||
bishabosha marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| Finally are [staging], which ensures that quotes conform to the | ||
| Phase Consistency Principle (PCP), and [pickleQuotes] which converts quoted | ||
bishabosha marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| trees to embedded TASTy strings. | ||
|
|
||
| ### `transformPhases` | ||
| These phases are concerned with tranformation into lower-level forms | ||
| suitable for the runtime system, with two sub-groupings: | ||
| - High-level transformations: All phases from [firstTransform] to [erasure]. | ||
| Most of these phases transform syntax trees, expanding high-level constructs | ||
| to more primitive ones. | ||
| - Some phases perform further checks on more primitive trees, | ||
| e.g. [refchecks] verifies that no abstract methods exist in concrete classes, | ||
| and [initChecker] checks that fields are not used before initialisation. | ||
| - The last phase in the group, [erasure] translates all | ||
| types into types supported directly by the JVM. To do this, it performs | ||
| another type checking pass, but using the rules of the JVM's type system | ||
| instead of Scala's. | ||
| - Low-level transformations: All phases from `ElimErasedValueType` to | ||
| `CollectSuperCalls`. These further transform trees until they are essentially a | ||
| structured version of Java bytecode. | ||
|
|
||
| ### `backendPhases` | ||
| These map the transformed trees to Java classfiles or SJSIR files. | ||
|
|
||
| [1]: {% link _overviews/scala3-contribution/arch-lifecycle.md %}/#phases | ||
| [2]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/CompilationUnit.scala | ||
| [Compiler]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Compiler.scala | ||
| [Phase]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Phases.scala | ||
| [MiniPhase]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/MegaPhase.scala | ||
| [Run]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Run.scala | ||
| [parser]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/parsing/ParserPhase.scala | ||
| [typer]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/typer/TyperPhase.scala | ||
| [posttyper]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/PostTyper.scala | ||
| [prepjsinterop]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/sjs/PrepJSInterop.scala | ||
| [pickler]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/Pickler.scala | ||
| [inlining]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/Inlining.scala | ||
| [postInlining]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/PostInlining.scala | ||
| [staging]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/Staging.scala | ||
| [pickleQuotes]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/PickleQuotes.scala | ||
| [refchecks]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/typer/RefChecks.scala | ||
| [initChecker]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/init/Checker.scala | ||
| [firstTransform]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/FirstTransform.scala | ||
| [erasure]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/Erasure.scala | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.