Add witness as a modifier

Author: odersky

docs/docs/reference/witnesses/discussion.md

@@ -0,0 +1,64 @@
+---
+layout: doc-page
+title: "Discussion"
+---
+
+## Summary
+
+The witness proposal consists of three main parts:
+
+ - Define a new [high-level syntax for witnesses](./witnesses.html) that works out better the intent underlying implicit definitions.
+ - Split the two meanings of implicit parameters into [separate concepts](./witness-params.html). One concept specifies that a parameter is implicitly applied, the other makes the parameter available as a witness.
+  - Allow `witness` as a [modifier](./witness-modifier.html) to replace remaining use cases for implicit definitions and to provide a lower level syntax into which witness definitions (with `witness` as a subject) can be translated.
+
+## Other Uses of `implicit`
+
+The only use cases that are not yet covered by the proposal are implicit classes and implicit conversions. We do not propose to use `witness` in place of `implicit` for these, since that would bring back the uncomfortable similarity between implicit conversions and parameterized implicit aliases. However, there is a way to drop implicit conversions entirely. Scala 3 already [defines](https://github.com/lampepfl/dotty/pull/2065) a class `ImplicitConverter` whose instances are available as implicit conversions.
+```scala
+  abstract class ImplicitConverter[-T, +U] extends Function1[T, U]
+```
+One could define all implicit conversions as witnesses of this class. E.g.
+```scala
+witness StringToToken for ImplicitConverter[String, Token] {
+  def apply(str: String): Token = new KeyWord(str)
+}
+```
+The fact that this syntax is more verbose than simple implicit defs could be a welcome side effect since it might dampen any over-enthusiasm for defining implicit conversions.
+
+That leaves implicit classes. Most use cases of implicit classes are probably already covered by extension methods. For the others, one could always fall back to a pair of a regular class and an `ImplicitConverter` witness. It would be good to do a survey to find out how many classes would be affected.
+
+## Migration
+
+New and old syntax would co-exist initially. Rewrite rules could rewrite old syntax to new automatically. This is trivial in the case of implicit parameters and implicit function types. It is a bit more involved in the case of implicit definitions, since more extensive pattern matching is required to recognize a definition that can be rewritten to a witness.
+
+## Discussion
+
+This is a rather sweeping proposal, which will affect most Scala code. Here are some supporting arguments and a summary of alternatives that were considered.
+
+The witness definition syntax makes the definition of implicit instances clearer and more concise. People have repeatedly asked for specific "typeclass syntax" in Scala. I believe that witnesses together with extension methods address this ask quite well.
+
+Probably the most far reaching and contentious changes affect implicit parameters. There might be resistance to change, because the existing scheme seems to work "well enough". However, I believe there is a price to pay for the status quo. The need to write `.apply` occasionally to force implicit arguments is already bad. Worse is the fact that implicits always have to come last, which makes useful program patterns much more cumbersome than before and makes the language less regular. Also problematic is the
+lack of scope control for implicit parameters which caused the invention of macro-based libraries such as MacWire for what looks like an ideal task for implicits. Without the
+changes proposed here, dependency injection in the style of MacWire will no longer be possible in Scala 3, since whitebox macros are going away.
+
+Several alternatives to the proposed syntax changes for implicit parameters were considered:
+
+ 1. Leave `implicit` parameters as they are. This suffers from the problems stated
+   in the [motivation section](./motivation.md).
+ 2. Leave the syntax of `implicit` parameters but institute two changes: First, applications
+   of implicit parameters must be via the pseudo method `.explicitly(...)`. Second, there can be more than one implicit parameter list and implicit parameters may precede explicit ones. This fixes most of the discussed problems, but at the expense of a bulky explicit application syntax. Bulk can be a problem, for instance when the programmer tries to
+   construct an extensive explicit argument tree to figure out what went wrong with a missing
+   implicit. Another issue is that migration from old to new scheme would be tricky and
+   would likely take multiple language versions.
+ 3. The design presented here, but with `implicit` instead of `witness` as the modifier for
+   parameters. This is closer to the status quo, but migration presents problems: At what point will an `implicit` modifier stop having the old meaning and acquire the new one? Using `witness` instead of `implicit` solves that problem because old and new styles can coexist and it is always clear which is which.
+ 4. Don't split the meanings of implicitly passed parameters and witness parameters. Use prefix ‘.’ as a syntax
+   for both meanings together. This is more concise and relieves the programmer from needing to choose
+   which combination of functionality is desired. On the other hand, this does not support some
+   use patterns such as MacWire style dependency injection. Also, the prefix ‘.’ syntax is maybe
+   a bit too inconspicuous at the parameter definition site, even though it works very well at the
+   function application site.
+ 5. As in 4., but using `witness` or `implicit` as the syntax that marks an implicitly passed witness parameter.
+   This breaks the correspondence between function abstraction and application syntax.
+
+Once we have high-level witness definitions and witness parameters it's a small step to convert most the remaining uses of `implicit` to `witness` as a modifier. There are no hard technical reasons for doing so, but it looks more consistent with the other changes.

docs/docs/reference/witnesses/motivation.md

@@ -7,25 +7,27 @@ title: "Motivation"
 
 Scala's implicits are its most distinguished feature. They are _the_ fundamental way to abstract over context. They represent a single concept with an extremely varied number of use cases, among them: implementing type classes, establishing context, dependency injection, expressing capabilities, computing new types and proving relationships between them.
 
-At the same time, implicits are also a controversal feature. I believe there are several reasons for this.
+At the same time, implicits are also a controversial feature. I believe there are several reasons for this.
 
-First, being very powerful, implicits are easily over-used and mis-used. This observation holds in almomst all cases when we talk about _implicit conversions_, which, even though conceptually different, share the same syntax with other implicit definitions. For instance,
+First, being very powerful, implicits are easily over-used and mis-used. This observation holds in almost all cases when we talk about _implicit conversions_, which, even though conceptually different, share the same syntax with other implicit definitions. For instance,
 regarding the two definitions
 
     implicit def i1(implicit x: T): C[T] = ...
     implicit def i2(x: T): C[T] = ...
 
 the first of these is a conditional implicit _value_, the second an implicit _conversion_. Conditional implicit values are a cornerstone for expressing type classes, whereas most applications of implicit conversions have turned out to be of dubious value. The problem is that many newcomers to the language start with defining implicit conversions since they are easy to understand and seem powerful and convenient. Scala 3 will put under a language flag both definitions and applications of "undisciplined" implicit conversions between types defined elsewhere. This is a useful step to push back against overuse of implicit conversions. But the problem remains that syntactically, conversions and values just look too similar for comfort.
 
-Second, implicits pose challenges for tooling. The set of available implicits depends on context, so command completion has to take context into account. This is feasible in an IDE but docs like ScalaDoc that are based static web pages can only provide an approximation. Another problem is that failed implicit searches often give very unspecific error messages, in particular if some deep recursion of a failed implicit search has failed. The dotty compiler implements some improvements in this case, but further progress would be desirable.
+Second, implicits pose challenges for tooling. The set of available implicits depends on context, so command completion has to take context into account. This is feasible in an IDE but docs like ScalaDoc that are based static web pages can only provide an approximation. Another problem is that failed implicit searches often give very unspecific error messages, in particular if some deeply recursive implicit search has failed. The dotty compiler implements some improvements in this case, but further progress would be desirable.
 
-Third, the syntax of implicit definitions is maybe a bit too minimal. It consists of a single modifier, `implicit`, that can be attached to a large number of language constructs. A problem with this for newcomers is that it often conveys mechanism better than intent. For instance, a typeclass instance is an implicit object or val if unconditional and an implicit def with implicit parameters if conditional. This describes precisely what the implicit definitions translate to -- just drop the `implicit` modifier, and that's it! But the cues that define intent are rather indirect and can be easily misread, as demonstrated by the definitions of `i1` and `i2` above.
+Third, the syntax of implicit definitions might be a bit too minimal. It consists of a single modifier, `implicit`, that can be attached to a large number of language constructs. A problem with this for newcomers is that it often conveys mechanism better than intent. For instance, a typeclass instance is an implicit object or val if unconditional and an implicit def with implicit parameters if conditional. This describes precisely what the implicit definitions translate to -- just drop the `implicit` modifier, and that's it! But the cues that define intent are rather indirect and can be easily misread, as demonstrated by the definitions of `i1` and `i2` above.
 
 Fourth, the syntax of implicit parameters has also some shortcomings. It starts with the position of `implicit` as a pseudo-modifier that applies to a whole parameter section instead of a single parameter. This represents an irregular case wrt to the rest of Scala's syntax. Furthermore, while implicit _parameters_ are designated specifically, arguments are not. Passing an argument to an implicit parameter looks like a regular application `f(arg)`. This is problematic because it means there can be confusion regarding what parameter gets instantiated in a call. For instance, in
 ```scala
 def currentMap(implicit ctx: Context): Map[String, Int]
 ```
-one cannot write `currentMap("abc")` since the string "abc" is taken as explicit argument to the implicit `ctx` parameter. One has to write `currentMap.apply("abc")` instead, which is awkward and irregular. For the same reason, a method definition can only have one implicit parameter section and it must always come last. This restriction not only reduces orthogonality, but also prevents some useful program constructs, such as a method with a regular parameter type that depends on an implicit value. Finally, it's also a bit annoying that implicit parameters must have a name, even though in most cases that name is never referenced.
+one cannot write `currentMap("abc")` since the string "abc" is taken as explicit argument to the implicit `ctx` parameter. One has to write `currentMap.apply("abc")` instead, which is awkward and irregular. For the same reason, a method definition can only have one implicit parameter section and it must always come last. This restriction not only reduces orthogonality, but also prevents some useful program constructs, such as a method with a regular parameter whose type depends on an implicit value. Finally, it's also a bit annoying that implicit parameters must have a name, even though in most cases that name is never referenced.
+
+Fifth, there is a lack of mechanism to contain the scope of implicits. Specifically, implicit parameters are always themselves implicit values in the body of the class or method in which they are defined. There are situations where one would like the parameter to a method or a class to be passed implicitly without that parameter becoming a candidate for further implicits in the body. The popular dependency injection framework MacWire exists precisely because such fine grained control of implicit scope is currently not available.
 
 None of the shortcomings is fatal, after all implicits are very widely used, and many libraries and applications rely on them. But together, they make code using implicits more cumbersome and less clear than it could be.
 
@@ -37,8 +39,10 @@ Can implicit function types help? Implicit function types allow to abstract over
 
 I believe a good name for is concept is _witness_. A term is a witness for a type by defining an implicit instance of this type. It's secondary whether this instance takes the form of a `val` or `object` or whether it is a method. It would be better to have a uniform syntax for all of these kinds of instances.
 
-The next sections elaborate such an alternative design. It consists of two proposals which are independent of each other:
+The next sections elaborate such an alternative design. It consists of three proposals which are independent of each other:
 
  - A proposal to replace implicit _definitions_ by [witness definitions](./witnesses.html)
  .
  - A proposal for a [new syntax](./witness-params.html) of implicit _parameters_ and their _arguments_.
+
+ - A proposal to replace most or all remaining uses of the `implicit` modifier by a new `witness` [modifier](./witness-modifier.html).

docs/docs/reference/witnesses/witness-modifier.md

@@ -0,0 +1,58 @@
+---
+layout: doc-page
+title: "Witness as a Modifier"
+---
+
+There are some uses of implicits that cannot be expressed directly by witness definitions. In particular, we are lacking analogues of abstract implicit definitions and implicit aliases. The gap can be filled by allowing `witness` to be used as a modifier for objects, vals and defs, with the same meaning as the `implicit` modifier. Arguably, `witness` is a more suitable name for these kinds of definitions than `implicit`.
+
+## Abstract and Alias Witnesses
+
+As an example for an abstract witness consider the following fragment that's derived from Scala's Tasty extractor framework:
+```scala
+trait TastyAPI {
+  type Symbol
+  trait SymDeco {
+    def name(this sym: Symbol): Name
+    def tpe(this sym: Symbol): Type
+  }
+  witness def symDeco: SymDeco
+}
+```
+An example of a witness alias would be an implementation of `symDeco` in terms of some internal compiler structure:
+```scala
+trait TastyImpl extends TastyAPI {
+  witness val symDeco: SymDeco = compilerSymOps
+}
+```
+As is the case for `implicit` value definitions, the result type of a witness value is mandatory unless the definition occurs as a statement in a block.
+
+## Translation Scheme
+
+Witness-as-a-modifier can be seen as a more low-level and general syntax layer than witness definitions. The latter can be translated into the former.
+
+The translation rule for monomorphic unconditional witnesses is straightforward:
+```
+        witness <id> for <parents> <body>
+  -->
+        witness object <id> extends <parents> <body>
+```
+The translation rule for parameterized or conditional witnesses is more involved:
+```
+        witness <id> [<tparams>] with <params> for <parents> <body>
+  -->
+        class <id'> [<tparams>].( <witness-params> ) extends <parents> <body>
+        witness def id [<tparams>].( <witness-params> ): <result-type> =
+          new <id'> [<tparam-refs>]
+  where
+        <id'>              is a fresh, compiler generated name
+        <witness-params>   is <params> with every parameter prefixed by `witness`
+        <result-type>      is derived from <parents> by dropping all value parameters
+        <tparam-refs>      is a list of type identifiers referring to each type parameter in <tparams>
+```
+
+## Syntax
+
+```
+LocalModifier   ::=  ...
+                  |  witness
+```