|
| 1 | +# Add CustomDebugStringConvertible conformance to AnyKeyPath |
| 2 | + |
| 3 | +* Proposal: [SE-0369](0369-add-customdebugdescription-conformance-to-anykeypath.md) |
| 4 | +* Author: [Ben Pious](https://github.com/benpious) |
| 5 | +* Review Manager: [Xiaodi Wu](https://github.com/xwu) |
| 6 | +* Status: **Active Review (August 16...August 29, 2022)** |
| 7 | +* Implementation: [apple/swift#60133](https://github.com/apple/swift/pull/60133) |
| 8 | +* Review: ([pitch](https://forums.swift.org/t/pitch-add-customdebugdescription-conformance-to-anykeypath/58705)) |
| 9 | + |
| 10 | +## Introduction |
| 11 | + |
| 12 | +This proposal is to add conformance to the protocol `CustomDebugStringConvertible` to `AnyKeyPath`. |
| 13 | + |
| 14 | +## Motivation |
| 15 | + |
| 16 | +Currently, passing a keypath to `print()`, or to the `po` command in LLDB, yields the standard output for a Swift class. This is not very useful. For example, given |
| 17 | +```swift |
| 18 | +struct Theme { |
| 19 | + |
| 20 | + var backgroundColor: Color |
| 21 | + var foregroundColor: Color |
| 22 | + |
| 23 | + var overlay: Color { |
| 24 | + backgroundColor.withAlpha(0.8) |
| 25 | + } |
| 26 | +} |
| 27 | +``` |
| 28 | +`print(\Theme.backgroundColor)` would have an output of roughly |
| 29 | +``` |
| 30 | +Swift.KeyPath<Theme, Color> |
| 31 | +``` |
| 32 | +which doesn't allow `foregroundColor` to be distinguished from any other property on `Theme`. |
| 33 | + |
| 34 | +Ideally, the output would be |
| 35 | +``` |
| 36 | +\Theme.backgroundColor |
| 37 | +``` |
| 38 | +exactly as it was written in the program. |
| 39 | + |
| 40 | +## Proposed solution |
| 41 | + |
| 42 | +Take advantage of whatever information is available in the binary to implement the `debugDescription` requirement of `CustomDebugStringConvertible`. In the best case, roughly the output above will be produced, in the worst cases, other, potentially useful information will be output instead. |
| 43 | + |
| 44 | +## Detailed design |
| 45 | + |
| 46 | +### Implementation of `CustomDebugStringConvertible` |
| 47 | + |
| 48 | +Much like the `_project` functions currently implemented in `KeyPath.swift`, this function would loop through the keypath's buffer, handling each segment as follows: |
| 49 | + |
| 50 | +For offset segments, the implementation is simple: use `_getRecursiveChildCount`, `_getChildOffset`, and `_getChildMetadata` to get the string name of the property. I believe these are the same mechanisms used by `Mirror` today. |
| 51 | + |
| 52 | +For optional chain, force-unwrap, etc. the function appends a hard coded "?" or "!", as appropriate. |
| 53 | + |
| 54 | +For computed segments, call `swift::lookupSymbol()` on the result of `getter()` in the `ComputedAccessorsPtr`. Demangle the result to get the property name. |
| 55 | + |
| 56 | +### Changes to the Swift Runtime |
| 57 | + |
| 58 | +To implement descriptions for computed segments, it is necessary to make two changes to the runtime: |
| 59 | + |
| 60 | +1. Expose a Swift calling-convention function to call `swift::lookupSymbol()`. |
| 61 | +2. Implement and expose a function to demangle keypath functions without the ornamentation the existing demangling functions produce. |
| 62 | + |
| 63 | +### Dealing with missing data |
| 64 | + |
| 65 | +There are two known cases where data might not be available: |
| 66 | + |
| 67 | +1. type metadata has not been emitted because the target was built using the `swift-disable-reflection-metadata` flag |
| 68 | +2. the linker has stripped the symbol names we're trying to look up |
| 69 | + |
| 70 | +In these cases, we would print the following: |
| 71 | + |
| 72 | +#### Offset case |
| 73 | +`<offset [x] ([typename])>` where `x` is the memory offset we read from the reflection metadata, and `typename` is the type that will be returned. |
| 74 | +So |
| 75 | +``` |
| 76 | +print(\Theme.backgroundColor) // outputs "\Theme.<offset 0 (Color)>" |
| 77 | +``` |
| 78 | + |
| 79 | +#### `lookupSymbol` failure case |
| 80 | + |
| 81 | +In this case we'll print the address-in-memory as hex, plus the type name: |
| 82 | +``` |
| 83 | +print(\Theme.overlay) // outputs \Theme.<computed 0xABCDEFG (Color)> |
| 84 | +``` |
| 85 | + |
| 86 | +As it might be difficult to correlate a memory address with the name of the function, the type name may be useful here to provide extra context. |
| 87 | + |
| 88 | +## Source compatibility |
| 89 | + |
| 90 | +Programs that extend `AnyKeyPath` to implement `CustomDebugStringConvertible` themselves will no longer compile and the authors of such code will have to delete the conformance. Based on a search of Github, there are currently no publicly available Swift projects that do this. |
| 91 | + |
| 92 | +Calling `print` on a KeyPath will, of course, produce different results than before. |
| 93 | + |
| 94 | +It is unlikely that any existing Swift program is depending on the existing behavior in a production context. While it is likely that someone, somewhere has written code in unit tests that depends on the output of this function, any issues that result will be easy for the authors of such code to identify and fix, and will likely represent an improvement in the readability of those tests. |
| 95 | + |
| 96 | +## Effect on ABI stability |
| 97 | + |
| 98 | +This proposal will add a new var & protocol conformance to the Standard Library's ABI. It will be availability guarded appropriately. |
| 99 | + |
| 100 | +The new debugging output will not be backdeployed, so Swift programs running on older ABI stable versions of the OS won't be able to rely on the new output. |
| 101 | + |
| 102 | +## Effect on API resilience |
| 103 | + |
| 104 | +The implementation of `debugDescription` might change after the initial work to implement this proposal is done. In particular, the output format will not be guaranteed to be stable. Here are a few different changes we might anticipate making: |
| 105 | + |
| 106 | +- As new features are added to the compiler, there may be new metadata available in the binary to draw from. One example would be lookup tables of KeyPath segment to human-readable-name or some other unique, stable identifier |
| 107 | +- Whenever a new feature is added to `KeyPath`, it will need to be reflected in the output of this function. For example, the `KeyPath`s produced by [\_forEachFieldWithKeyPath](https://github.com/apple/swift/blob/main/stdlib/public/core/ReflectionMirror.swift#L324) are incomplete, in the sense that they merely set a value at in offset in memory and do not call `didSet` observers. If this function were ever publicly exposed, it would be useful if this was surfaced in the debugging information. |
| 108 | +- The behavior of subscript printing might be changed: for example, we might always print out the value of the argument to the subscript, or we might do so only in cases where the output is short. We might also change from `.subscript()` to `[]` |
| 109 | +- The Swift language workgroup may create new policies around debug descriptions and the output of this function might need to be updated to conform to them |
| 110 | + |
| 111 | +## Alternatives considered |
| 112 | + |
| 113 | +### Print fully qualified names or otherwise add more information to the output |
| 114 | + |
| 115 | +ex. `\ModuleName.MyType.myField`, `<KeyPath<MyType, MyFieldType>> \ModuleName.MyType.myField` `(writable) \Theme.backgroundColor` |
| 116 | + |
| 117 | +As this is just for debugging, it seems likely that the information currently being provided would be enough to resolve any ambiguities. If ambiguities arose during a debugging session, in most cases the user could figure out exactly which keypath they were dealing with simply by running `po myKeyPath == \MyType.myField` till they found the right one. |
| 118 | + |
| 119 | +### Modify KeyPaths to include a string description |
| 120 | + |
| 121 | +This is an obvious solution to this problem, and would likely be very easy to implement, as the compiler already produces `_kvcString`. |
| 122 | + |
| 123 | +It has the additional advantage of being 100% reliable, to the point where it arguably could be the basis for implementing `description` rather than `debugDescription`. |
| 124 | + |
| 125 | +However, it would add to the code size of the compiled code, perhaps unacceptably so. Furthermore, it precludes the possibility of someday printing out the arguments of subscript based keypaths, as |
| 126 | +these can be created dynamically. It would also add overhead to appending keypaths, as the string would also have to be appended. |
| 127 | + |
| 128 | +An alternative implementation of this idea would be the output of additional metadata: a lookup table of function -> name. However, this would require a lot of additional work on the compiler for a relatively small benefit. |
| 129 | + |
| 130 | +I think that most users who might want this _really_ want to use it to build something else, like encodable keypaths. Those features should be provided opt-in on a per-keypath or per-type basis, which will make it much more useful (and in the context of encodable keypaths specifically, eliminate major potential security issues). Such a feature should also include the option to let the user configure this string, so that it can remain backwards compatible with older versions of the program. |
| 131 | + |
| 132 | +### Make Keypath functions global to prevent the linker from stripping them |
| 133 | + |
| 134 | +This would also potentially make it feasible to change this proposal from implementing `debugDescription` to implementing `description`. |
| 135 | + |
| 136 | +This would also potentially bloat the binary and would increase linker times. It could also be a security issue, as `dlysm` would now be able to find these functions. |
| 137 | + |
| 138 | +I am not very knowledgeable about linkers or how typical Swift builds strip symbols, but I think it might be useful to have this as an option in some IDEs that build Swift programs. But that is beyond the scope of this proposal. |
| 139 | + |
| 140 | +## Future Directions |
| 141 | + |
| 142 | +### Add LLDB formatters/summaries |
| 143 | + |
| 144 | +This would be a good augmentation to this proposal, and might improve the developer experience, as there [might be debug metadata available to the debugger](https://forums.swift.org/t/pitch-add-customdebugdescription-conformance-to-anykeypath/58705/2) that is not available in the binary itself. |
| 145 | + |
| 146 | +However, I think it might be very difficult to implement this. I see two options: |
| 147 | + |
| 148 | +1. Implement a public reflection API for KeyPaths in the Swift Standard Library that the formatter can interact with from Python. |
| 149 | +2. The formatter parses the raw memory of the KeyPath, essentially duplicating the code in `debugDescription`. |
| 150 | + |
| 151 | +I think (1) is overkill, especially considering the limited potential applications of this API beyond its use by the formatter. If it's possible to implement this as an `internal` function in the Swift stdlib then this is a much more attractive option. |
| 152 | +From personal experience trying to parse KeyPath memory from outside the Standard Library, I think (2) would be extremely difficult to implement, and unsustainable to maintain considering that the [memory layout of KeyPaths](https://github.com/apple/swift/blob/main/docs/ABI/KeyPaths.md) is not ABI stable. |
| 153 | + |
| 154 | +### Make Keypath functions global in DEBUG builds only |
| 155 | + |
| 156 | +This may be necessary to allow `swift::lookupSymbol` to function correctly on Windows, Linux and other platforms that use COFF or ELF-like formats. |
| 157 | + |
| 158 | +## Acknowledgments |
| 159 | + |
| 160 | +Thanks to Joe Groff for answering several questions on the initial pitch document, and Slava Pestov for answering some questions about the logistics of pitching this. |
0 commit comments