update rustc driver chapters

Author: mark-i-m

src/SUMMARY.md

@@ -26,7 +26,7 @@
 
 - [Part 2: How rustc works](./part-2-intro.md)
 - [High-level overview of the compiler source](./high-level-overview.md)
-- [The Rustc Driver](./rustc-driver.md)
+- [The Rustc Driver and Interface](./rustc-driver.md)
     - [Rustdoc](./rustdoc.md)
 - [Queries: demand-driven compilation](./query.md)
     - [The Query Evaluation Model in Detail](./queries/query-evaluation-model-in-detail.md)

src/appendix/code-index.md

@@ -7,7 +7,7 @@ compiler.
 Item            |  Kind    | Short description           | Chapter            | Declaration
 ----------------|----------|-----------------------------|--------------------|-------------------
 `BodyId` | struct | One of four types of HIR node identifiers | [Identifiers in the HIR] | [src/librustc/hir/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/struct.BodyId.html)
-`CompileState` | struct | State that is passed to a callback at each compiler pass | [The Rustc Driver] | [src/librustc_driver/driver.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/driver/struct.CompileState.html)
+`Compiler` | struct | Represents a compiler session and can be used to drive a compilation. | [The Rustc Driver and Interface] | [src/librustc_interface/interface.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_interface/interface/struct.Compiler.html)
 `ast::Crate` | struct | A syntax-level representation of a parsed crate | [The parser] | [src/librustc/hir/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/struct.Crate.html)
 `hir::Crate` | struct | A more abstract, compiler-friendly form of a crate's AST | [The Hir] | [src/librustc/hir/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/struct.Crate.html)
 `DefId` | struct | One of four types of HIR node identifiers | [Identifiers in the HIR] | [src/librustc/hir/def_id.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/def_id/struct.DefId.html)
@@ -18,8 +18,9 @@ Item            |  Kind    | Short description           | Chapter            |
 `P` | struct | An owned immutable smart pointer. By contrast, `&T` is not owned, and `Box<T>` is not immutable. | None | [src/syntax/ptr.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ptr/struct.P.html)
 `ParamEnv` | struct | Information about generic parameters or `Self`, useful for working with associated or generic items | [Parameter Environment] | [src/librustc/ty/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/struct.ParamEnv.html)
 `ParseSess` | struct | This struct contains information about a parsing session | [The parser] | [src/libsyntax/parse/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/parse/struct.ParseSess.html)
+`Query` | struct | Represents the result of query to the `Compiler` interface and allows stealing, borrowing, and returning the results of compiler passes. | [The Rustc Driver and Interface] | [src/librustc_interface/queries.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_interface/queries/struct.Query.html)
 `Rib` | struct | Represents a single scope of names | [Name resolution] | [src/librustc_resolve/lib.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_resolve/struct.Rib.html)
-`Session` | struct | The data associated with a compilation session | [The parser], [The Rustc Driver] | [src/librustc/session/mod.html](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/session/struct.Session.html)
+`Session` | struct | The data associated with a compilation session | [The parser], [The Rustc Driver and Interface] | [src/librustc/session/mod.html](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/session/struct.Session.html)
 `SourceFile` | struct | Part of the `SourceMap`. Maps AST nodes to their source code for a single source file. Was previously called FileMap | [The parser] | [src/libsyntax_pos/lib.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/source_map/struct.SourceFile.html)
 `SourceMap` | struct | Maps AST nodes to their source code. It is composed of `SourceFile`s. Was previously called CodeMap | [The parser] | [src/libsyntax/source_map.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/source_map/struct.SourceMap.html)
 `Span` | struct  | A location in the user's source code, used for error reporting primarily | [Emitting Diagnostics] | [src/libsyntax_pos/span_encoding.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax_pos/struct.Span.html)
@@ -33,7 +34,7 @@ Item            |  Kind    | Short description           | Chapter            |
 [The HIR]: ../hir.html
 [Identifiers in the HIR]: ../hir.html#hir-id
 [The parser]: ../the-parser.html
-[The Rustc Driver]: ../rustc-driver.html
+[The Rustc Driver and Interface]: ../rustc-driver.html
 [Type checking]: ../type-checking.html
 [The `ty` modules]: ../ty.html
 [Rustdoc]: ../rustdoc.html

src/appendix/stupid-stats.md

@@ -1,9 +1,12 @@
 # Appendix A: A tutorial on creating a drop-in replacement for rustc
 
 > **Note:** This is a copy of `@nrc`'s amazing [stupid-stats]. You should find
-> a copy of the code on the GitHub repository although due to the compiler's
-> constantly evolving nature, there is no guarantee it'll compile on the first
-> go.
+> a copy of the code on the GitHub repository.
+>
+> Due to the compiler's constantly evolving nature, the `rustc_driver`
+> mechanisms described in this chapter have been replaced by a new
+> [`rustc_interface`] crate. See [The Rustc Driver and Interface] for more
+> information.
 
 Many tools benefit from being a drop-in replacement for a compiler. By this, I
 mean that any user of the tool can use `mytool` in all the ways they would
@@ -92,7 +95,7 @@ translation).
 > and [`librustc_codegen_utils`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_utils/index.html).
 
 All these phases are coordinated by the driver. To see the exact sequence, look
-at [the `compile_input` function in `librustc_driver`][compile-input].
+at the `compile_input` function in `librustc_driver`.
 The driver handles all the highest level coordination of compilation -
     1. handling command-line arguments
     2. maintaining compilation state (primarily in the `Session`)
@@ -101,9 +104,6 @@ The driver handles all the highest level coordination of compilation -
 To create a drop-in compiler replacement or a compiler replacement,
 we leave most of compilation alone and customise the driver using its APIs.
 
-[compile-input]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/driver/fn.compile_input.html
-
-
 ## The driver customisation APIs
 
 There are two primary ways to customise compilation - high level control of the
@@ -410,3 +410,5 @@ internally (I already changed save-analysis to use `CompilerController`). I've
 been experimenting with a prototype rustfmt which also uses these APIs.
 
 [stupid-stats]: https://github.com/nrc/stupid-stats
+[`rustc_interface`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_interface/index.html
+[The Rustc Driver and Interface]: ../rustc-driver.html

src/rustc-driver.md

@@ -1,46 +1,25 @@
-# The Rustc Driver
+# The Rustc Driver and Interface
 
 The [`rustc_driver`] is essentially `rustc`'s `main()` function. It acts as
 the glue for running the various phases of the compiler in the correct order,
-managing state such as the [`SourceMap`] \(maps AST nodes to source code),
-[`Session`] \(general build context and error messaging) and the [`TyCtxt`]
-\(the "typing context", allowing you to query the type system and other cool
-stuff). The `rustc_driver` crate also provides external users with a method
+using the interface defined in the [`rustc_interface`] crate.
+
+The `rustc_interface` crate provides external users with an (unstable) API
 for running code at particular times during the compilation process, allowing
 third parties to effectively use `rustc`'s internals as a library for
-analysing a crate or emulating the compiler in-process (e.g. the RLS).
-
-For those using `rustc` as a library, the `run_compiler()` function is the main
-entrypoint to the compiler. Its main parameters are a list of command-line
-arguments and a reference to something which implements the `CompilerCalls`
-trait. A `CompilerCalls` creates the overall `CompileController`, letting it
-govern which compiler passes are run and attach callbacks to be fired at the end
-of each phase.
-
-From `rustc_driver`'s perspective, the main phases of the compiler are:
-
-1. *Parse Input:* Initial crate parsing
-2. *Configure and Expand:* Resolve `#[cfg]` attributes, name resolution, and
-   expand macros
-3. *Run Analysis Passes:* Run trait resolution, typechecking, region checking
-   and other miscellaneous analysis passes on the crate
-4. *Translate to LLVM:* Translate to the in-memory form of LLVM IR and turn it
-   into an executable/object files
+analysing a crate or emulating the compiler in-process (e.g. the RLS or rustdoc).
 
-The `CompileController` then gives users the ability to inspect the ongoing
-compilation process
+For those using `rustc` as a library, the `interface::run_compiler()` function is the main
+entrypoint to the compiler. It takes a configuration for the compiler and a closure that
+takes a [`Compiler`]. `run_compiler` creates a `Compiler` from the configuration and passes
+it to the closure. Inside the closure, you can use the `Compiler` to drive queries to compile
+a crate and get the results. This is what the `rustc_driver` does too.
 
-- after parsing
-- after AST expansion
-- after HIR lowering
-- after analysis, and
-- when compilation is done
-
-The `CompileState`'s various `state_after_*()` constructors can be inspected to
-determine what bits of information are available to which callback.
-
-For a more detailed explanation on using `rustc_driver`, check out the
-[stupid-stats] guide by `@nrc` (attached as [Appendix A]).
+You can see what queries are currently available through the rustdocs for [`Compiler`].
+You can see an example of how to use them by looking at the `rustc_driver` implementation,
+specifically the [`rustc_driver::run_compiler` function][rd_rc] (not to be confused with
+`interface::run_compiler`). The `rustc_driver::run_compiler` function takes a bunch of
+command-line args and some other configurations and drives the compilation to completion.
 
 > **Warning:** By its very nature, the internal compiler APIs are always going
 > to be unstable. That said, we do try not to break things unnecessarily.
@@ -54,21 +33,15 @@ manifests itself in the way people can plug into the compiler, preferring a
 "push"-style API (callbacks) instead of the more Rust-ic "pull" style (think
 the `Iterator` trait).
 
-For example the [`CompileState`], the state passed to callbacks after each
-phase, is essentially just a box of optional references to pieces inside the
-compiler. The lifetime bound on the `CompilerCalls` trait then helps to ensure
-compiler internals don't "escape" the compiler (e.g. if you tried to keep a
-reference to the AST after the compiler is finished), while still letting users
-record *some* state for use after the `run_compiler()` function finishes.
-
 Thread-local storage and interning are used a lot through the compiler to reduce
 duplication while also preventing a lot of the ergonomic issues due to many
 pervasive lifetimes. The `rustc::ty::tls` module is used to access these
 thread-locals, although you should rarely need to touch it.
 
-
+[rd_rc]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/fn.run_compiler.html
+[`rustc_interface`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_interface/index.html
 [`rustc_driver`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/
-[`CompileState`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/driver/struct.CompileState.html
+[`Compiler`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_interface/interface/struct.Compiler.html
 [`Session`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/session/struct.Session.html
 [`TyCtxt`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/struct.TyCtxt.html
 [`SourceMap`]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/source_map/struct.SourceMap.html