Rollup merge of rust-lang#22635 - kmcallister:macros-chapter, r=steveklabnik

 r? @steveklabnik

Author: Manishearth

src/doc/reference.md

@@ -731,15 +731,20 @@ Rust syntax is restricted in two ways:
    pairs when they occur at the beginning of, or immediately after, a `$(...)*`;
    requiring a distinctive token in front can solve the problem.
 
-## Syntax extensions useful for the macro author
+## Syntax extensions useful in macros
 
-* `log_syntax!` : print out the arguments at compile time
-* `trace_macros!` : supply `true` or `false` to enable or disable macro expansion logging
 * `stringify!` : turn the identifier argument into a string literal
 * `concat!` : concatenates a comma-separated list of literals
-* `concat_idents!` : create a new identifier by concatenating the arguments
 
-The following attributes are used for quasiquoting in procedural macros:
+## Syntax extensions for macro debugging
+
+* `log_syntax!` : print out the arguments at compile time
+* `trace_macros!` : supply `true` or `false` to enable or disable macro expansion logging
+
+## Quasiquoting
+
+The following syntax extensions are used for quasiquoting Rust syntax trees,
+usually in [procedural macros](book/plugins.html#syntax-extensions):
 
 * `quote_expr!`
 * `quote_item!`
@@ -748,6 +753,8 @@ The following attributes are used for quasiquoting in procedural macros:
 * `quote_tokens!`
 * `quote_ty!`
 
+Documentation is very limited at the moment.
+
 # Crates and source files
 
 Rust is a *compiled* language. Its semantics obey a *phase distinction*

src/doc/trpl/advanced-macros.md

@@ -192,19 +192,58 @@ To keep this system simple and correct, `#[macro_use] extern crate ...` may
 only appear at the root of your crate, not inside `mod`. This ensures that
 `$crate` is a single identifier.
 
-# A final note
+# The deep end
 
-Macros, as currently implemented, are not for the faint of heart. Even
-ordinary syntax errors can be more difficult to debug when they occur inside a
-macro, and errors caused by parse problems in generated code can be very
-tricky. Invoking the `log_syntax!` macro can help elucidate intermediate
-states, invoking `trace_macros!(true)` will automatically print those
-intermediate states out, and passing the flag `--pretty expanded` as a
-command-line argument to the compiler will show the result of expansion.
+The introductory chapter mentioned recursive macros, but it did not give the
+full story. Recursive macros are useful for another reason: Each recursive
+invocation gives you another opportunity to pattern-match the macro's
+arguments.
+
+As an extreme example, it is possible, though hardly advisable, to implement
+the [Bitwise Cyclic Tag](http://esolangs.org/wiki/Bitwise_Cyclic_Tag) automaton
+within Rust's macro system.
+
+```rust
+#![feature(trace_macros)]
+
+macro_rules! bct {
+    // cmd 0:  d ... => ...
+    (0, $($ps:tt),* ; $_d:tt)
+        => (bct!($($ps),*, 0 ; ));
+    (0, $($ps:tt),* ; $_d:tt, $($ds:tt),*)
+        => (bct!($($ps),*, 0 ; $($ds),*));
+
+    // cmd 1p:  1 ... => 1 ... p
+    (1, $p:tt, $($ps:tt),* ; 1)
+        => (bct!($($ps),*, 1, $p ; 1, $p));
+    (1, $p:tt, $($ps:tt),* ; 1, $($ds:tt),*)
+        => (bct!($($ps),*, 1, $p ; 1, $($ds),*, $p));
+
+    // cmd 1p:  0 ... => 0 ...
+    (1, $p:tt, $($ps:tt),* ; $($ds:tt),*)
+        => (bct!($($ps),*, 1, $p ; $($ds),*));
+
+    // halt on empty data string
+    ( $($ps:tt),* ; )
+        => (());
+}
+
+fn main() {
+    trace_macros!(true);
+# /* just check the definition
+    bct!(0, 0, 1, 1, 1 ; 1, 0, 1);
+# */
+}
+```
+
+Exercise: use macros to reduce duplication in the above definition of the
+`bct!` macro.
+
+# Procedural macros
 
 If Rust's macro system can't do what you need, you may want to write a
 [compiler plugin](plugins.html) instead. Compared to `macro_rules!`
 macros, this is significantly more work, the interfaces are much less stable,
-and the warnings about debugging apply ten-fold. In exchange you get the
+and bugs can be much harder to track down. In exchange you get the
 flexibility of running arbitrary Rust code within the compiler. Syntax
 extension plugins are sometimes called *procedural macros* for this reason.

src/doc/trpl/macros.md

@@ -189,14 +189,12 @@ shorthand for a data type could be valid as either an expression or a pattern.
 
 ## Repetition
 
-The repetition behavior can seem somewhat magical, especially when multiple
-names are bound at multiple nested levels of repetition. The two rules to keep
-in mind are:
+The repetition operator follows two principal rules:
 
-1. the behavior of `$(...)*` is to walk through one "layer" of repetitions, for
-all of the `$name`s it contains, in lockstep, and
+1. `$(...)*` walks through one "layer" of repetitions, for all of the `$name`s
+   it contains, in lockstep, and
 2. each `$name` must be under at least as many `$(...)*`s as it was matched
-against. If it is under more, it'll be duplicated, as appropriate.
+   against. If it is under more, it'll be duplicated, as appropriate.
 
 This baroque macro illustrates the duplication of variables from outer
 repetition levels.
@@ -226,6 +224,10 @@ That's most of the matcher syntax. These examples use `$(...)*`, which is a
 more" match. Both forms optionally include a separator, which can be any token
 except `+` or `*`.
 
+This system is based on
+"[Macro-by-Example](http://www.cs.indiana.edu/ftp/techreports/TR206.pdf)"
+(PDF link).
+
 # Hygiene
 
 Some languages implement macros using simple text substitution, which leads to
@@ -273,19 +275,26 @@ macro, using [a GNU C extension] to emulate Rust's expression blocks.
 })
 ```
 
-This looks reasonable, but watch what happens in this example:
+Here's a simple use case that goes terribly wrong:
 
 ```text
 const char *state = "reticulating splines";
-LOG(state);
+LOG(state)
 ```
 
-The program will likely segfault, after it tries to execute
+This expands to
 
 ```text
-printf("log(%d): %s\n", state, state);
+const char *state = "reticulating splines";
+int state = get_log_state();
+if (state > 0) {
+    printf("log(%d): %s\n", state, state);
+}
 ```
 
+The second variable named `state` shadows the first one.  This is a problem
+because the print statement should refer to both of them.
+
 The equivalent Rust macro has the desired behavior.
 
 ```rust
@@ -357,6 +366,64 @@ fn main() {
 
 [items]: ../reference.html#items
 
+# Recursive macros
+
+A macro's expansion can include more macro invocations, including invocations
+of the very same macro being expanded.  These recursive macros are useful for
+processing tree-structured input, as illustrated by this (simplistic) HTML
+shorthand:
+
+```rust
+# #![allow(unused_must_use)]
+macro_rules! write_html {
+    ($w:expr, ) => (());
+
+    ($w:expr, $e:tt) => (write!($w, "{}", $e));
+
+    ($w:expr, $tag:ident [ $($inner:tt)* ] $($rest:tt)*) => {{
+        write!($w, "<{}>", stringify!($tag));
+        write_html!($w, $($inner)*);
+        write!($w, "</{}>", stringify!($tag));
+        write_html!($w, $($rest)*);
+    }};
+}
+
+fn main() {
+#   // FIXME(#21826)
+    use std::fmt::Write;
+    let mut out = String::new();
+
+    write_html!(&mut out,
+        html[
+            head[title["Macros guide"]]
+            body[h1["Macros are the best!"]]
+        ]);
+
+    assert_eq!(out,
+        "<html><head><title>Macros guide</title></head>\
+         <body><h1>Macros are the best!</h1></body></html>");
+}
+```
+
+# Debugging macro code
+
+To see the results of expanding macros, run `rustc --pretty expanded`. The
+output represents a whole crate, so you can also feed it back in to `rustc`,
+which will sometimes produce better error messages than the original
+compilation. Note that the `--pretty expanded` output may have a different
+meaning if multiple variables of the same name (but different syntax contexts)
+are in play in the same scope. In this case `--pretty expanded,hygiene` will
+tell you about the syntax contexts.
+
+`rustc` provides two syntax extensions that help with macro debugging. For now,
+they are unstable and require feature gates.
+
+* `log_syntax!(...)` will print its arguments to standard output, at compile
+  time, and "expand" to nothing.
+
+* `trace_macros!(true)` will enable a compiler message every time a macro is
+  expanded. Use `trace_macros!(false)` later in expansion to turn it off.
+
 # Further reading
 
 The [advanced macros chapter][] goes into more detail about macro syntax. It

src/doc/trpl/plugins.md

@@ -146,14 +146,7 @@ a more involved macro example, see
 
 ## Tips and tricks
 
-To see the results of expanding syntax extensions, run
-`rustc --pretty expanded`. The output represents a whole crate, so you
-can also feed it back in to `rustc`, which will sometimes produce better
-error messages than the original compilation. Note that the
-`--pretty expanded` output may have a different meaning if multiple
-variables of the same name (but different syntax contexts) are in play
-in the same scope. In this case `--pretty expanded,hygiene` will tell
-you about the syntax contexts.
+Some of the [macro debugging tips](macros.html#debugging-macro-code) are applicable.
 
 You can use [`syntax::parse`](../syntax/parse/index.html) to turn token trees into
 higher-level syntax elements like expressions:
@@ -184,6 +177,11 @@ and return
 [`DummyResult`](../syntax/ext/base/struct.DummyResult.html),
 so that the compiler can continue and find further errors.
 
+To print syntax fragments for debugging, you can use
+[`span_note`](../syntax/ext/base/struct.ExtCtxt.html#method.span_note) together
+with
+[`syntax::print::pprust::*_to_string`](http://doc.rust-lang.org/syntax/print/pprust/index.html#functions).
+
 The example above produced an integer literal using
 [`AstBuilder::expr_uint`](../syntax/ext/build/trait.AstBuilder.html#tymethod.expr_uint).
 As an alternative to the `AstBuilder` trait, `libsyntax` provides a set of

src/libcollections/slice.rs

@@ -790,7 +790,7 @@ pub trait SliceExt {
     fn ends_with(&self, needle: &[Self::Item]) -> bool where Self::Item: PartialEq;
 
     /// Convert `self` into a vector without clones or allocation.
-    #[unstable(feature = "collections")]
+    #[stable(feature = "rust1", since = "1.0.0")]
     fn into_vec(self: Box<Self>) -> Vec<Self::Item>;
 }
 

src/libcore/iter.rs

@@ -981,7 +981,7 @@ pub trait IteratorExt: Iterator + Sized {
     #[unstable(feature = "core", reason = "recent addition")]
     fn cloned(self) -> Cloned<Self> where
         Self::Item: Deref,
-        <Self::Item as Deref>::Output: Clone,
+        <Self::Item as Deref>::Target: Clone,
     {
         Cloned { it: self }
     }

src/librustc/lint/builtin.rs

@@ -405,8 +405,8 @@ struct ImproperCTypesVisitor<'a, 'tcx: 'a> {
 }
 
 impl<'a, 'tcx> ImproperCTypesVisitor<'a, 'tcx> {
-    fn check_def(&mut self, sp: Span, ty_id: ast::NodeId, path_id: ast::NodeId) {
-        match self.cx.tcx.def_map.borrow()[path_id].clone() {
+    fn check_def(&mut self, sp: Span, id: ast::NodeId) {
+        match self.cx.tcx.def_map.borrow()[id].full_def() {
             def::DefPrimTy(ast::TyInt(ast::TyIs(_))) => {
                 self.cx.span_lint(IMPROPER_CTYPES, sp,
                                   "found rust type `isize` in foreign module, while \
@@ -418,7 +418,7 @@ impl<'a, 'tcx> ImproperCTypesVisitor<'a, 'tcx> {
                                    libc::c_uint or libc::c_ulong should be used");
             }
             def::DefTy(..) => {
-                let tty = match self.cx.tcx.ast_ty_to_ty_cache.borrow().get(&ty_id) {
+                let tty = match self.cx.tcx.ast_ty_to_ty_cache.borrow().get(&id) {
                     Some(&ty::atttce_resolved(t)) => t,
                     _ => panic!("ast_ty_to_ty_cache was incomplete after typeck!")
                 };
@@ -437,9 +437,8 @@ impl<'a, 'tcx> ImproperCTypesVisitor<'a, 'tcx> {
 
 impl<'a, 'tcx, 'v> Visitor<'v> for ImproperCTypesVisitor<'a, 'tcx> {
     fn visit_ty(&mut self, ty: &ast::Ty) {
-        match ty.node {
-            ast::TyPath(_, id) => self.check_def(ty.span, ty.id, id),
-            _ => (),
+        if let ast::TyPath(..) = ty.node {
+            self.check_def(ty.span, ty.id);
         }
         visit::walk_ty(self, ty);
     }
@@ -683,8 +682,8 @@ impl LintPass for PathStatements {
         match s.node {
             ast::StmtSemi(ref expr, _) => {
                 match expr.node {
-                    ast::ExprPath(_) => cx.span_lint(PATH_STATEMENTS, s.span,
-                                                     "path statement with no effect"),
+                    ast::ExprPath(..) => cx.span_lint(PATH_STATEMENTS, s.span,
+                                                      "path statement with no effect"),
                     _ => ()
                 }
             }
@@ -1001,7 +1000,8 @@ impl LintPass for NonSnakeCase {
 
     fn check_pat(&mut self, cx: &Context, p: &ast::Pat) {
         if let &ast::PatIdent(_, ref path1, _) = &p.node {
-            if let Some(&def::DefLocal(_)) = cx.tcx.def_map.borrow().get(&p.id) {
+            let def = cx.tcx.def_map.borrow().get(&p.id).map(|d| d.full_def());
+            if let Some(def::DefLocal(_)) = def {
                 self.check_snake_case(cx, "variable", path1.node, p.span);
             }
         }
@@ -1066,8 +1066,8 @@ impl LintPass for NonUpperCaseGlobals {
 
     fn check_pat(&mut self, cx: &Context, p: &ast::Pat) {
         // Lint for constants that look like binding identifiers (#7526)
-        match (&p.node, cx.tcx.def_map.borrow().get(&p.id)) {
-            (&ast::PatIdent(_, ref path1, _), Some(&def::DefConst(..))) => {
+        match (&p.node, cx.tcx.def_map.borrow().get(&p.id).map(|d| d.full_def())) {
+            (&ast::PatIdent(_, ref path1, _), Some(def::DefConst(..))) => {
                 NonUpperCaseGlobals::check_upper_case(cx, "constant in pattern",
                                                       path1.node, p.span);
             }
@@ -1227,10 +1227,13 @@ impl LintPass for NonShorthandFieldPatterns {
     fn check_pat(&mut self, cx: &Context, pat: &ast::Pat) {
         let def_map = cx.tcx.def_map.borrow();
         if let ast::PatStruct(_, ref v, _) = pat.node {
-            for fieldpat in v.iter()
-                             .filter(|fieldpat| !fieldpat.node.is_shorthand)
-                             .filter(|fieldpat| def_map.get(&fieldpat.node.pat.id)
-                                                == Some(&def::DefLocal(fieldpat.node.pat.id))) {
+            let field_pats = v.iter()
+                              .filter(|fieldpat| !fieldpat.node.is_shorthand)
+                              .filter(|fieldpat| {
+                let def = def_map.get(&fieldpat.node.pat.id).map(|d| d.full_def());
+                def == Some(def::DefLocal(fieldpat.node.pat.id))
+            });
+            for fieldpat in field_pats {
                 if let ast::PatIdent(_, ident, None) = fieldpat.node.pat.node {
                     if ident.node.as_str() == fieldpat.node.ident.as_str() {
                         cx.span_lint(NON_SHORTHAND_FIELD_PATTERNS, fieldpat.span,
@@ -1899,10 +1902,7 @@ impl LintPass for UnconditionalRecursion {
                                       _: ast::Ident,
                                       id: ast::NodeId) -> bool {
             tcx.def_map.borrow().get(&id)
-                .map_or(false, |def| {
-                    let did = def.def_id();
-                    ast_util::is_local(did) && did.node == fn_id
-                })
+                .map_or(false, |def| def.def_id() == ast_util::local_def(fn_id))
         }
 
         // check if the method call `id` refers to method `method_id`

src/librustc/metadata/csearch.rs

@@ -150,12 +150,9 @@ pub fn get_trait_name(cstore: &cstore::CStore, def: ast::DefId) -> ast::Name {
                             def.node)
 }
 
-pub fn get_trait_item_name_and_kind(cstore: &cstore::CStore, def: ast::DefId)
-                                    -> (ast::Name, def::TraitItemKind) {
+pub fn is_static_method(cstore: &cstore::CStore, def: ast::DefId) -> bool {
     let cdata = cstore.get_crate_data(def.krate);
-    decoder::get_trait_item_name_and_kind(cstore.intr.clone(),
-                                          &*cdata,
-                                          def.node)
+    decoder::is_static_method(&*cdata, def.node)
 }
 
 pub fn get_trait_item_def_ids(cstore: &cstore::CStore, def: ast::DefId)