Skip to content

Commit 6984ba3

Browse files
glessardglessard
committed
pitch span-providing properties in the standard library
1 parent e273da0 commit 6984ba3

File tree

1 file changed

+261
-0
lines changed

1 file changed

+261
-0
lines changed

‎proposals/AAAA-stdlib-span-properties.md

Lines changed: 261 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,261 @@
1+
# Provide Span Properties on Standard Library Types
2+
3+
* Proposal: (link tbd)
4+
* Author: [Guillaume Lessard](https://github.com/glessard)
5+
* Review Manager: (tbd)
6+
* Status: **Pitch**
7+
* Roadmap: [BufferView Roadmap](https://forums.swift.org/t/66211)
8+
* Bug: rdar://137710901
9+
* Implementation: (tbd)
10+
* Review:
11+
12+
[SE-0446]: https://github.com/swiftlang/swift-evolution/blob/main/proposals/0446-non-escapable.md
13+
[SE-0447]: https://github.com/swiftlang/swift-evolution/blob/main/proposals/0447-span-access-shared-contiguous-storage.md
14+
[PR-2305]: https://github.com/swiftlang/swift-evolution/pull/2305
15+
[SE-0453]: https://github.com/swiftlang/swift-evolution/blob/main/proposals/0453-vector.md
16+
17+
## Introduction
18+
19+
We recently [introduced][SE-0447] the `Span` and `RawSpan` types, but did not provide ways to obtain instances of either from existing types. This proposal adds properties that vend a lifetime-dependent `Span` from a variety of standard library types, as well as vend a lifetime-dependent `RawSpan` when the underlying element type supports it.
20+
21+
## Motivation
22+
23+
Many standard library container types can provide direct access to their internal representation. Up to now, it has only been possible to do so in an unsafe way. The standard library provides this unsafe functionality with closure-taking functions such as `withUnsafeBufferPointer()`, `withContiguousStorageIfAvailable()` and `withUnsafeBytes()`. These functions have a few different drawbacks, most prominently their reliance on unsafe types, which makes them unpalatable in security-conscious environments. Closure-taking API can also be difficult to compose with new features and with one another. These issues are addressed head-on with non-escapable types in general, and `Span` in particular. With this proposal, compatible standard library types will provide access to their internal representation via computed properties of type `Span` and `RawSpan`.
24+
25+
## Proposed solution
26+
27+
Computed properties returning [non-escapable][SE-0446] copyable values represent a particular case of lifetime relationships between two bindings. While initializing a non-escapable value in general requires [lifetime annotations][PR-2305] in order to correctly describe the lifetime relationship, the specific case of computed properties returning non-escapable copyable values can only represent one type of relationship between the parent binding and the non-escapable instance it provides: a borrowing relationship.
28+
29+
For example, in the example below we have an instance of type `A`, with a well-defined lifetime because it is non-copyable. An instance of `A` can provide access to a type `B` which borrows the instance `A`:
30+
31+
```swift
32+
struct A: ~Copyable, Escapable {}
33+
struct B: ~Escapable, Copyable {
34+
init(_ a: borrowing A) {}
35+
}
36+
extension A {
37+
var b: B { B(self) }
38+
}
39+
40+
func function() {
41+
var a = A()
42+
var b = a.b // access to `a` begins here
43+
read(b)
44+
// `b` has ended here, ending access to `a`
45+
modify(&a) // `modify()` can have exclusive access to `a`
46+
}
47+
```
48+
If we were to attempt using `b` again after the call to `modify(&a)`, the compiler would report an overlapping access error, due to attempting to mutate `a` (with `modify(&a)`) while it is already being accessed through `b`'s borrow. Note that the copyability of `B` means that it cannot represent a mutation of `A`; it therefore represents a non-exclusive borrowing relationship.
49+
50+
Given this, we propose to enable the definition of a borrowing relationship via a computed property. With this feature we then propose to add `storage` computed properties to standard library types that can share their internal typed storage, as well as `bytes` computed properties to those standard library types that can safely share their internal storage as untyped memory.
51+
52+
## Detailed Design
53+
54+
A computed property getter of an `Escapable` type returning a non-escapable and copyable type (`~Escapable & Copyable`) establishes a borrowing lifetime relationship of the returned value on the callee's binding. As long as the returned value exists (including local copies,) then the callee's binding is being borrowed. In terms of the law of exclusivity, a borrow is a read-only access. Multiple borrows are allowed to overlap, but cannot overlap with any mutation.
55+
56+
By allowing the language to define lifetime dependencies in this limited way, we can add `Span`-providing properties to standard library types.
57+
58+
#### Extensions to Standard Library types
59+
60+
The standard library and Foundation will provide `storage` and `bytes` computed properties. These computed properties are the safe and composable replacements for the existing `withUnsafeBufferPointer` and `withUnsafeBytes` closure-taking functions.
61+
62+
```swift
63+
extension Array {
64+
/// Share this `Array`'s elements as a `Span`
65+
var storage: Span<Element> { get }
66+
}
67+
68+
extension Array where Element: BitwiseCopyable {
69+
/// Share the bytes of this `Array`'s elements as a `RawSpan`
70+
var bytes: RawSpan { get }
71+
}
72+
73+
extension ArraySlice {
74+
/// Share this `Array`'s elements as a `Span`
75+
var storage: Span<Element> { get }
76+
}
77+
78+
extension ArraySlice where Element: BitwiseCopyable {
79+
/// Share the bytes of this `Array`'s elements as a `RawSpan`
80+
var bytes: RawSpan { get }
81+
}
82+
83+
extension ContiguousArray {
84+
/// Share this `Array`'s elements as a `Span`
85+
var storage: Span<Element> { get }
86+
}
87+
88+
extension ContiguousArray where Element: BitwiseCopyable {
89+
/// Share the bytes of this `Array`'s elements as a `RawSpan`
90+
var bytes: RawSpan { get }
91+
}
92+
93+
extension String.UTF8View {
94+
/// Share this `UTF8View`'s code units as a `Span`
95+
var storage: Span<Unicode.UTF8.CodeUnit> { get }
96+
97+
/// Share this `UTF8View`'s code units as a `RawSpan`
98+
var bytes: RawSpan { get }
99+
}
100+
101+
extension Substring.UTF8View {
102+
/// Share this `UTF8View`'s code units as a `Span`
103+
var storage: Span<Unicode.UTF8.CodeUnit> { get }
104+
105+
/// Share this `UTF8View`'s code units as a `RawSpan`
106+
var bytes: RawSpan { get }
107+
}
108+
109+
extension CollectionOfOne {
110+
/// Share this `Collection`'s element as a `Span`
111+
var storage: Span<Element> { get }
112+
}
113+
114+
extension CollectionOfOne where Element: BitwiseCopyable {
115+
/// Share the bytes of this `Collection`'s element as a `RawSpan`
116+
var bytes: RawSpan { get }
117+
}
118+
119+
extension SIMD where Scalar: BitwiseCopyable {
120+
/// Share this vector's elements as a `Span`
121+
var storage: Span<Scalar> { get }
122+
123+
/// Share this vector's underlying bytes as a `RawSpan`
124+
var bytes: RawSpan { get }
125+
}
126+
127+
extension KeyValuePairs {
128+
/// Share this `Collection`'s elements as a `Span`
129+
var storage: Span<(Key, Value)> { get }
130+
}
131+
132+
extension KeyValuePairs where Element: BitwiseCopyable {
133+
/// Share the underlying bytes of this `Collection`'s elements as a `RawSpan`
134+
var bytes: RawSpan { get }
135+
}
136+
```
137+
138+
Conditionally to the acceptance of [`Vector`][SE-0453], we will also add the following:
139+
140+
```swift
141+
extension Vector where Element: ~Copyable {
142+
/// Share this vector's elements as a `Span`
143+
var storage: Span<Element> { get }
144+
}
145+
146+
extension Vector where Element: BitwiseCopyable {
147+
/// Share the underlying bytes of vector's elements as a `RawSpan`
148+
var bytes: RawSpan { get }
149+
}
150+
```
151+
152+
#### Extensions to unsafe buffer types
153+
154+
We hope that `Span` and `RawSpan` will become the standard ways to access shared contiguous memory in Swift, but current API provide `UnsafeBufferPointer` and `UnsafeRawBufferPointer` instances to do this. We will provide ways to unsafely obtain `Span` and `RawSpan` instances from them, in order to bridge `UnsafeBufferPointer` to contexts that use `Span`, or `UnsafeRawBufferPointer` to contexts that use `RawSpan`.
155+
156+
```swift
157+
extension UnsafeBufferPointer {
158+
/// Unsafely view this buffer as a `Span`
159+
var storage: Span<Element> { get }
160+
}
161+
162+
extension UnsafeMutableBufferPointer {
163+
/// Unsafely view this buffer as a `Span`
164+
var storage: Span<Element> { get }
165+
}
166+
167+
extension UnsafeBufferPointer where Element: BitwiseCopyable {
168+
/// Unsafely view this buffer as a `RawSpan`
169+
var bytes: RawSpan { get }
170+
}
171+
172+
extension UnsafeMutableBufferPointer where Element: BitwiseCopyable {
173+
/// Unsafely view this buffer as a `RawSpan`
174+
var bytes: RawSpan { get }
175+
}
176+
177+
extension UnsafeRawBufferPointer {
178+
/// Unsafely view this buffer as a `Span`
179+
var storage: Span<Element> { get }
180+
181+
/// Unsafely view this raw buffer as a `RawSpan`
182+
var bytes: RawSpan { get }
183+
}
184+
185+
extension UnsafeMutableRawBufferPointer {
186+
/// Unsafely view this buffer as a `Span`
187+
var storage: Span<Element> { get }
188+
189+
/// Unsafely view this raw buffer as a `RawSpan`
190+
var bytes: RawSpan { get }
191+
}
192+
```
193+
194+
All of these unsafe conversions return a value whose lifetime is dependent on the _binding_ of the UnsafeBufferPointer. Note that this does not keep the underlying memory alive, as usual where the `UnsafePointer` family of types is involved. The programmer must ensure that the underlying memory is valid for as long as the `Span` or `RawSpan` are valid.
195+
196+
#### Extensions to `Foundation.Data`
197+
198+
While the `swift-foundation` package and the `Foundation` framework are not governed by the Swift evolution process, `Data` is similar in use to standard library types, and the project acknowledges that it is desirable for it to have similar API when appropriate. Accordingly, we would add the following properties to `Foundation.Data`:
199+
200+
```swift
201+
extension Foundation.Data {
202+
// Share this `Data`'s bytes as a `Span`
203+
var storage: Span<UInt8> { get }
204+
205+
// Share this `Data`'s bytes as a `RawSpan`
206+
var bytes: RawSpan { get }
207+
}
208+
```
209+
210+
## Source compatibility
211+
212+
This proposal is additive and source-compatible with existing code.
213+
214+
## ABI compatibility
215+
216+
This proposal is additive and ABI-compatible with existing code.
217+
218+
## Implications on adoption
219+
220+
The additions described in this proposal require a new version of the Swift standard library and runtime.
221+
222+
## Alternatives considered
223+
224+
#### Adding `withSpan()` and `withBytes()` closure-taking functions
225+
226+
The `storage` and `bytes` properties aim to be safe replacements for the `withUnsafeBufferPointer()` and `withUnsafeBytes()` closure-taking functions. We could consider `withSpan()` and `withBytes()` closure-taking functions that would provide an quicker migration away from the older unsafe functions. We do not believe the closure-taking functions are desirable in the long run. In the short run, there may be a desire to clearly mark the scope where a `Span` instance is used. The default method would be to explicitly consume a `Span` instance:
227+
```swift
228+
var a = ContiguousArray(0..<8)
229+
var span = a.storage
230+
read(span)
231+
_ = consume span
232+
a.append(8)
233+
```
234+
235+
In order to visually distinguish this lifetime, we could simply use a `do` block:
236+
```swift
237+
var a = ContiguousArray(0..<8)
238+
do {
239+
let span = a.storage
240+
read(span)
241+
}
242+
a.append(8)
243+
```
244+
245+
A more targeted solution may be a consuming function that takes a non-escaping closure:
246+
```swift
247+
var a = ContiguousArray(0..<8)
248+
var span = a.storage
249+
consuming(span) { span in
250+
read(span)
251+
}
252+
a.append(8)
253+
```
254+
255+
During the evolution of Swift, we have learned that closure-based API are difficult to compose, especially with one another. They can also require alterations to support new language features. For example, the generalization of closure-taking API for non-copyable values as well as typed throws is ongoing; adding more closure-taking API may make future feature evolution more labor-intensive. By instead relying on returned values, whether from computed properties or functions, we build for greater composability. Use cases where this approach falls short should be reported as enhancement requests or bugs.
256+
257+
#### Giving the properties different names
258+
We chose the names `storage` and `bytes` because those reflect _what_ they represent. Another option would be to name the properties after _how_ they represent what they do, which would be `span` and `rawSpan`. It is possible the name `storage` would be deemed to clash too much with existing properties of types that would like to provide views of their internal storage with `Span`-providing properties. For example, the Standard Library's concrete `SIMD`-conforming types have a property `var _storage`. The current proposal means that making this property of `SIMD` types into public API would entail a name change more significant than simply removing its leading underscore.
259+
260+
#### Allowing the definition of non-escapable properties of non-escapable types
261+
The particular case of the lifetime dependence created by a property of a non-escapable type is not as simple as when the parent type is escapable. There are two possible ways to define the lifetime of the new instance: it can either depend on the lifetime of the original instance, or it can acquire the lifetime of the original instance and be otherwise independent. We believe that both these cases can be useful, and therefore defer allowing either until there is a language annotation to differentiate between them.

0 commit comments

Comments
 (0)