Add document describing layout of KeyPath objects.

docs/ABI/KeyPaths.md

@@ -0,0 +1,306 @@
+# Key Path Memory Layout
+
+**Key path objects** are laid out at runtime as a heap object with a
+variable-sized payload containing a sequence of encoded components describing
+how the key path traverses a value. When the compiler sees a key path literal,
+it generates a **key path pattern** that can be efficiently interpreted by
+the runtime to instantiate a key path object when needed. This document
+describes the layout of both. The key path pattern layout is designed in such a
+way that it can be transformed in-place into a key path object with a one-time
+initialization in the common case where the entire path is fully specialized
+and crosses no resilience boundaries.
+
+## ABI Concerns For Key Paths 
+
+For completeness, this document describes the layout of both key path objects
+and patterns; note however that the instantiated runtime layout of key path
+objects is an implementation detail of the Swift runtime, and *only key path
+patterns* are strictly ABI, since they are emitted by the compiler. The
+runtime has the freedom to change the runtime layout of key path objects, but
+will have to maintain the ability to instantiate from key path patterns emitted
+by previous ABI-stable versions of the Swift complier.
+
+## Key Path Objects
+
+### Buffer Header
+
+Key path objects begin with the standard Swift heap object header, followed by a
+key path object header. Relative to the start of the heap object header:
+
+Offset  | Description
+------- | ----------------------------------------------
+`0`       | Pointer to KVC compatibility C string, or null
+`1*sizeof(Int)` | Key path buffer header (32 bits)
+
+If the key path is Cocoa KVC-compatible, the first word will be a pointer to
+the equivalent KVC string as a null-terminated UTF-8 C string. It will be null
+otherwise.  The **key path buffer header** in the second word contains the
+following bit fields:
+
+Bits (LSB zero) | Description
+--------------- | -----------
+0...23          | **Buffer size** in bytes
+24...29         | Reserved. Must be zero in Swift 4 runtime
+30              | 1 = Has **reference prefix**, 0 = No reference prefix
+31              | 1 = Is **trivial**, 0 = Has destructor
+
+The *buffer size* indicates the total size in bytes of the components following
+the key path buffer header. A `ReferenceWritableKeyPath` may have a *reference
+prefix* of read-only components that can be projected before initiating
+mutation; bit 30 is set if one is present. A key path may capture values that
+require cleanup when the key path object is deallocated, but a key path that
+does not capture any values with cleanups will have the *trivial* bit 31 set to
+fast-path deallocation.
+
+Components are always pointer-aligned, so the first component always starts at
+offset `2*sizeof(Int)`. On 64-bit platforms, this leaves four bytes of padding.
+
+### Components
+
+After the buffer header, one or more **key path components** appear in memory
+in sequence. Each component begins with a 32-bit **key path component header**
+describing the following component.
+
+Bits (LSB zero) | Description
+--------------- | -----------
+0...28          | **Payload** (meaning is dependent on component kind)
+29...30         | **Component kind**
+31              | 1 = **End of reference prefix**, 0 = Not end of reference prefix
+
+If the key path has a *reference prefix*, then exactly one component must have
+the *end of reference prefix* bit set in its component header. This indicates
+that the component after the end of the reference prefix will initiate mutation.
+
+The following *component kinds* are recognized:
+
+Value in bit 30&29 | Description
+------------------ | -----------
+0                  | Struct/tuple/self stored property
+1                  | Computed
+2                  | Class stored property
+3                  | Optional chaining/forcing/wrapping
+
+- A **struct stored property** component, when given
+  a value of the base type in memory, can project the component value in-place
+  at a fixed offset within the base value. This applies for struct stored
+  properties, tuple fields, and the `.self` identity component (which trivially
+  projects at offset zero). The
+  *payload* contains the offset in bytes of the projected field in the
+  aggregate, or the special value `0x1FFF_FFFF`, which indicates that the
+  offset is too large to pack into the payload and is stored in the next 32 bits
+  after the header.
+- A **class stored property** component, when given a reference to a class
+  instance, can project the component value inside the class instance at
+  a fixed offset. The *payload*
+  *payload* contains the offset in bytes of the projected field from the
+  address point of the object, or the special value `0x1FFF_FFFF`, which
+  indicates that the offset is too large to pack into the payload and is stored
+  in the next 32 bits after the header.
+- An **optional** component performs an operation involving `Optional` values.
+  The `payload` contains one of the following values:
+
+    Value in payload | Description
+    ---------------- | -----------
+    0                | **Optional chaining**
+    1                | **Optional wrapping**
+    2                | **Optional force-unwrapping**
+
+    A *chaining* component behaves like the postfix `?` operator, immediately
+    ending the key path application and returning nil when the base value is nil,
+    or unwrapping the base value and continuing projection on the non-optional
+    payload when non-nil. If an optional chain ends in a non-optional value,
+    an implicit *wrapping* component is inserted to wrap it up in an
+    optional value. A *force-unwrapping* operator behaves like the postfix
+    `!` operator, trapping if the base value is nil, or unwrapping the value
+    inside the optional if not.
+
+- A **computed** component uses the conservative access pattern of `get`/`set`
+  /`materializeForSet` to project from the base value. This is used as a
+  general fallback component for any key path component without a more
+  specialized representation, including not only computed properties but
+  also subscripts, stored properties that require reabstraction, properties
+  with behaviors or custom key path components (when we get those), and weak or
+  unowned properties. The payload contains additional bitfields describing the
+  component:
+
+    Bits (LSB zero) | Description
+    --------------- | -----------
+    24              | 1 = **Has captured arguments**, 0 = no captures
+    25...26         | **Identifier kind**
+    27              | 1 = **Settable**, 0 = **Get-Only**
+    28              | 1 = **Mutating** (implies settable), 0 = Nonmutating
+
+    The component can *capture* context which is stored after the component in
+    the key path object, such as generic arguments from its original context,
+    subscript index arguments, and so on. Bit 24 is set if there are any such
+    captures. Bits 25 and 26 discriminate the *identifier* which is used to
+    determine equality of key paths referring to the same components. If
+    bit 27 is set, then the key path is **settable** and can be written through,
+    and bit 28 indicates whether the set operation **is mutating** to the base
+    value, that is, whether setting through the component changes the base value
+    like a value-semantics property or modifies state indirectly like a class
+    property or `UnsafePointer.pointee`.
+
+    After the header, the component contains the following word-aligned fields:
+
+    Offset from header | Description
+    ------------------ | -----------
+    `1*sizeof(Int)`    | The **identifier** of the component.
+    `2*sizeof(Int)`    | The **getter function** for the component.
+    `3*sizeof(Int)`    | (if settable) The **setter function** for the component
+
+    The combination of the identifier kind bits and the identifier word are
+    compared by the `==` operation on two key paths to determine whether they
+    are equivalent. Neither the kind bits nor the identifier word
+    have any stable semantic meaning other than as unique identifiers.
+    In practice, the compiler picks a stable unique artifact of the
+    underlying declaration, such as the naturally-abstracted getter entry point
+    for a computed property, the offset of a reabstracted stored property, or
+    an Objective-C selector for an imported ObjC property, to identify the
+    component. The identifier kind bits are used to discriminate
+    possibly-overlapping domains.
+
+    The getter function is a pointer to a Swift function with the signature
+    `@convention(thin) (@in Base, UnsafeRawPointer) -> @out Value`. When
+    the component is applied, the getter is invoked with a copy of the base
+    value and is passed a pointer to the captured arguments of the
+    component. If the component has no captures, the second argument is
+    undefined.
+
+    The setter function is also a pointer to a Swift function. This field is
+    only present if the *settable* bit of the header is set. If the
+    component is nonmutating, then the function has signature
+    `@convention(thin) (@in Base, @in Value, UnsafeRawPointer) -> ()`,
+    or if it is mutating, then the function has signature
+    `@convention(thin) (@inout Base, @in Value, UnsafeRawPointer) -> ()`.
+    When a mutating application of the key path is completed, the setter is
+    invoked with a copy of the base value (if nonmutating) or a reference to
+    the base value (if mutating), along with a copy of the updated component
+    value, and a pointer to the captured arguments of the component. If
+    the component has no captures, the third argument is undefined.
+
+    TODO: Make getter/nonmutating setter take base borrowed,
+    yield borrowed result (materializeForGet); use materializeForSet
+
+    If the component has captures, the capture area appears after the other
+    fields, at offset `3*sizeof(Int)` for a get-only component or
+    `4*sizeof(Int)` for a settable component. The area begins with a two-word
+    header:
+
+    Offset from start | Description
+    ----------------- | -----------
+    `0`               | Size of captures in bytes
+    `1*sizeof(Int)`   | Pointer to **argument witness table**
+
+    followed by the captures themselves. The *argument witness table* contains
+    pointers to functions needed for maintaining the captures:
+
+    Offset           | Description
+    ---------------- | -----------
+    `0`              | **Destroy**, or null if trivial
+    `1*sizeof(Int)`  | **Copy**
+    `2*sizeof(Int)`  | **Is Equal**
+    `3*sizeof(Int)`  | **Hash**
+
+    The *destroy* function, if not null, has signature 
+    `@convention(thin) (UnsafeMutableRawPointer) -> ()` and is invoked to
+    destroy the captures when the key path object is deallocated.
+
+    The *copy* function has signature
+    `@convention(thin) (_ src: UnsafeRawPointer,
+                        _ dest: UnsafeMutableRawPointer) -> ()`
+    and is invoked when the captures need to be copied into a new key path
+    object, for example when two key paths are appended.
+
+    The *is equal* function has signature
+    `@convention(thin) (UnsafeRawPointer, UnsafeRawPointer) -> Bool`
+    and is invoked when the component is compared for equality with another
+    computed component with the same identifier.
+
+    The *hash* function has signature
+    `@convention(thin) (UnsafeRawPointer, UnsafeRawPointer) -> Int`
+    and is invoked when the key path containing the component is hashed.
+    The implementation understands a return value of zero to mean that the
+    captures should have no effect on the hash value of the key path.
+
+After every component except for the final component, a pointer-aligned
+pointer to the metadata for the type of the projected component is stored.
+(The type of the final component can be found from the `Value` generic
+argument of the `KeyPath<Root, Value>` type.)
+
+### Examples
+
+Given:
+
+```swift
+struct A {
+  var padding: (128 x UInt8)
+  var b: B
+}
+
+class B {
+  var padding: (240 x UInt8)
+  var c: C
+}
+
+struct C {
+  var padding: (384 x UInt8)
+  var d: D
+}
+```
+
+On a 64-bit platform, a key path object representing `\A.b.c.d` might look like
+this in memory:
+
+Word | Contents
+---- | --------
+0    | isa pointer to `ReferenceWritableKeyPath<A, D>`
+1    | reference counts
+`-`  | `-`
+2    | buffer header 0xC000_0028 - trivial, reference prefix, buffer size 40
+`-`  | `-`
+3    | component header 0x8000_0080 - struct component, offset 128, end of prefix
+4    | type metadata pointer for `B`
+`-`  | `-`
+5    | component header 0x4000_0100 - class component, offset 256
+6    | type metadata pointer for `C`
+`-`  | `-`
+7    | component header 0x0000_0180 - struct component, offset 384
+
+If we add:
+
+```
+struct D {
+  var computed: E { get set }
+}
+
+struct E {
+  subscript(b: B) -> F { get }
+}
+```
+
+then `\D.e[B()]` would look like:
+
+Word | Contents
+---- | --------
+0    | isa pointer to `WritableKeyPath<D, E>`
+1    | reference counts
+`-`  | `-`
+2    | buffer header 0x0000_0058 - buffer size 88
+`-`  | `-`
+3    | component header 0x3800_0000 - computed, settable, mutating
+4    | identifier pointer
+5    | getter
+6    | setter
+7    | type metadata pointer for `F`
+`-`  | `-`
+8    | component header 0x2100_0000 - computed, has captures
+9    | identifier pointer
+10   | getter
+11   | argument size 8
+12   | pointer to argument witnesses for releasing/retaining/equating/hashing `B`
+13   | value of `B()`
+
+## Key Path Patterns
+
+(to be written)