Skip to content

Commit 3d73009

Browse files
grynspangrynspan
committed
[Proposal] Temporary uninitialized buffers
1 parent 9f1bb17 commit 3d73009

File tree

1 file changed

+287
-0
lines changed

1 file changed

+287
-0
lines changed

‎proposals/NNNN-temporary-buffers.md

Lines changed: 287 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,287 @@
1+
# Temporary uninitialized buffers
2+
3+
* Proposal: [SE-NNNN](NNNN-filename.md)
4+
* Authors: [Jonathan Grynspan](https://github.com/grynspan)
5+
* Review Manager: TBD
6+
* Status: **Awaiting implementation**
7+
* Implementation: [apple/swift#37666](https://github.com/apple/swift/pull/37666)
8+
9+
## Introduction
10+
11+
This proposal introduces new Standard Library functions for manipulating temporary buffers that are preferentially allocated on the stack instead of the heap.
12+
13+
Swift-evolution thread: [[Pitch] Temporary uninitialized buffers](https://forums.swift.org/t/pitch-temporary-uninitialized-buffers/48954)
14+
15+
## Motivation
16+
17+
Library-level code often needs to deal with C functions, and C functions have a wide variety of memory-management approaches. A common way to handle buffers of value types (i.e. structures) in C is to have a caller allocate a buffer of sufficient size and pass it to a callee to initialize; the caller is then responsible for deinitializing the buffer (if non-trivial) and then deallocating it. In C or Objective-C, it's easy enough to stack-allocate such a buffer, and the logic to switch to the heap for a larger allocation is pretty simple too. This sort of pattern is pervasive:
18+
19+
```c
20+
size_t tacoCount = ...;
21+
taco_fillings_t fillings = ...;
22+
23+
taco_t *tacos = NULL;
24+
taco_t stackBuffer[SOME_LIMIT];
25+
if (tacoCount < SOME_LIMIT) {
26+
tacos = stackBuffer;
27+
} else {
28+
tacos = calloc(tacoCount, sizeof(taco_t));
29+
}
30+
31+
taco_init(tacos, tacoCount, &fillings);
32+
33+
// do some work here
34+
ssize_t tacosEatenCount = tacos_consume(tacos, tacoCount);
35+
if (tacosEatenCount < 0) {
36+
int errorCode = errno;
37+
fprintf(stderr, "Error %i eating %zu tacos: %s", errorCode, tacoCount, strerror(errorCode));
38+
exit(EXIT_FAILURE);
39+
}
40+
41+
// Tear everything down.
42+
taco_destroy(tacos, tacoCount);
43+
if (buffer != stackBuffer) {
44+
free(buffer);
45+
}
46+
```
47+
48+
In C++, we can make judicious use of `std::array` and `std::vector` to achieve the same purpose while generally preserving memory safety.
49+
50+
But there's not really any way to express this sort of transient buffer usage in Swift. All values must be initialized in Swift before they can be used, but C typically claims responsibility for initializing the values passed to it, so a Swift caller ends up initializing the values _twice_. The caller can call `UnsafeMutableBufferPointer<taco_t>.allocate(capacity:)` to get uninitialized memory, of course. `allocate(capacity:)` places buffers on the heap by default, and the optimizer can only stack-promote them after performing escape analysis.
51+
52+
Since Swift requires values be initialized before being used, and since escape analysis is [undecidable](https://duckduckgo.com/?q=escape+analysis+undecidable) for non-trivial programs, it's not possible to get efficient (i.e. stack-allocated and uninitialized) temporary storage in Swift.
53+
54+
It is therefore quite hard for developers to provide Swift overlays for lower-level C libraries that use these sorts of memory-management techniques. This means that an idiomatic Swift program using a C library's Swift overlay will perform less optimally than the same program would if written in C.
55+
56+
## Proposed solution
57+
58+
I propose adding a new transparent function to the Swift Standard Library that would allocate a buffer of a specified type and capacity, provide that buffer to a caller-supplied closure, and then deallocate the buffer before returning. The buffer would be passed to said closure in an uninitialized state and treated as uninitialized on closure return—that is, the closure would be responsible for initializing and deinitializing the elements in the buffer.
59+
60+
A typical use case might look like this:
61+
62+
```swift
63+
64+
// Eat a temporary taco buffer. A buffet, if you will.
65+
try Taco.consume(count: tacoCount, filledWith: ...)
66+
67+
// MARK: - Swift Overlay Implementation
68+
69+
extension taco_t {
70+
public static func consume(count: Int, filledWith fillings: taco_fillings_t) throws {
71+
withUnsafeUninitializedMutableBufferPointer(to: taco_t.self, capacity: count) { buffer in
72+
withUnsafePointer(to: fillings) { fillings in
73+
taco_init(buffer.baseAddress!, buffer.count, &fillings)
74+
}
75+
defer {
76+
taco_destroy(buffer.baseAddress!, buffer.count)
77+
}
78+
79+
let eatenCount = tacos_consume(buffer.baseAddress!, buffer.count)
80+
guard eatenCount >= 0 else {
81+
let errorCode = POSIXErrorCode(rawValue: errno) ?? .ENOTHUNGRY
82+
throw POSIXError(errorCode)
83+
}
84+
}
85+
}
86+
}
87+
```
88+
89+
The proposed function allows developers to effectively assert to the compiler that the buffer pointer used in the closure cannot escape the closure's context (even if calls are made to non-transparent functions that might otherwise defeat escape analysis.) Because the compiler then "knows" that the pointer does not escape, it can optimize much more aggressively.
90+
91+
## Detailed design
92+
93+
A new free function would be introduced in the Standard Library:
94+
95+
```swift
96+
/// Provides scoped access to a buffer pointer to memory of the specified type
97+
/// and with the specified capacity.
98+
///
99+
/// - Parameters:
100+
/// - type: The type of the buffer pointer being temporarily allocated.
101+
/// - capacity: The capacity of the buffer pointer being temporarily
102+
/// allocated.
103+
/// - body: A closure to invoke and to which the allocated buffer pointer
104+
/// should be passed.
105+
///
106+
/// - Returns: Whatever is returned by `body`.
107+
///
108+
/// - Throws: Whatever is thrown by `body`.
109+
///
110+
/// This function is useful for cheaply allocating storage for a sequence of
111+
/// values for a brief duration. Storage may be allocated on the heap or on the
112+
/// stack, depending on the required size and alignment.
113+
///
114+
/// When `body` is called, the contents of the buffer pointer passed to it are
115+
/// in an unspecified, uninitialized state. `body` is responsible for
116+
/// initializing the buffer pointer before it is used _and_ for deinitializing
117+
/// it before returning. `body` does not need to deallocate the buffer pointer.
118+
///
119+
/// The implementation may allocate a larger buffer pointer than is strictly
120+
/// necessary to contain `capacity` values of type `type`. The behavior of a
121+
/// program that attempts to access any such additional storage is undefined.
122+
///
123+
/// The buffer pointer passed to `body` (as well as any pointers to elements in
124+
/// the buffer) must not escape—it will be deallocated when `body` returns and
125+
/// cannot be used afterward.
126+
@_transparent
127+
public func withUnsafeUninitializedMutableBufferPointer<T, R>(to type: T.Type, capacity: Int, _ body: (UnsafeMutableBufferPointer<T>) throws -> R) rethrows -> R
128+
```
129+
130+
We could optionally provide additional free functions for dealing with a raw buffer or a pointer to a single value. All of the proposed functions can be layered atop each other, so only one underlying implementation is ultimately needed:
131+
132+
```swift
133+
/// Provides scoped access to a raw buffer pointer with the specified byte count
134+
/// and alignment.
135+
///
136+
/// - Parameters:
137+
/// - byteCount: The number of bytes to temporarily allocate. `byteCount` must
138+
/// not be negative.
139+
/// - alignment: The alignment of the new, temporary region of allocated
140+
/// memory, in bytes.
141+
/// - body: A closure to invoke and to which the allocated buffer pointer
142+
/// should be passed.
143+
///
144+
/// - Returns: Whatever is returned by `body`.
145+
///
146+
/// - Throws: Whatever is thrown by `body`.
147+
///
148+
/// This function is useful for cheaply allocating raw storage for a brief
149+
/// duration. Storage may be allocated on the heap or on the stack, depending on
150+
/// the required size and alignment.
151+
///
152+
/// When `body` is called, the contents of the buffer pointer passed to it are
153+
/// in an unspecified, uninitialized state. `body` is responsible for
154+
/// initializing the buffer pointer before it is used _and_ for deinitializing
155+
/// it before returning. `body` does not need to deallocate the buffer pointer.
156+
///
157+
/// The implementation may allocate a larger buffer pointer than is strictly
158+
/// necessary to contain `byteCount` bytes. The behavior of a program that
159+
/// attempts to access any such additional storage is undefined.
160+
///
161+
/// The buffer pointer passed to `body` (as well as any pointers to elements in
162+
/// the buffer) must not escape—it will be deallocated when `body` returns and
163+
/// cannot be used afterward.
164+
@_transparent
165+
public func withUnsafeUninitializedMutableRawBufferPointer<R>(byteCount: Int, alignment: Int, _ body: (UnsafeMutableRawBufferPointer) throws -> R) rethrows -> R
166+
167+
/// Provides scoped access to a pointer to memory of the specified type.
168+
///
169+
/// - Parameters:
170+
/// - type: The type of the pointer to allocate.
171+
/// - body: A closure to invoke and to which the allocated pointer should be
172+
/// passed.
173+
///
174+
/// - Returns: Whatever is returned by `body`.
175+
///
176+
/// - Throws: Whatever is thrown by `body`.
177+
///
178+
/// This function is useful for cheaply allocating storage for a single value
179+
/// for a brief duration. Storage may be allocated on the heap or on the stack,
180+
/// depending on the required size and alignment.
181+
///
182+
/// When `body` is called, the contents of the pointer passed to it are in an
183+
/// unspecified, uninitialized state. `body` is responsible for initializing the
184+
/// pointer before it is used _and_ for deinitializing it before returning.
185+
/// `body` does not need to deallocate the pointer.
186+
///
187+
/// The pointer passed to `body` must not escape—it will be deallocated when
188+
/// `body` returns and cannot be used afterward.
189+
@_transparent
190+
public func withUnsafeUninitializedMutablePointer<T, R>(to type: T.Type, _ body: (UnsafeMutablePointer<T>) throws -> R) rethrows -> R
191+
```
192+
193+
Note the functions are marked `@_transparent` to ensure they are emitted into the calling frame. This is consistent with the annotations on most other pointer manipulation functions.
194+
195+
### New builtin
196+
197+
The proposed functions will need to invoke a new builtin function equivalent to C's `alloca()`, which I have named `Builtin.stackAlloc()`. Its effective declaration is:
198+
199+
```swift
200+
extension Builtin {
201+
func stackAlloc(_ byteCount: Builtin.Word, _ alignment: Builtin.Word) -> Builtin.RawPointer
202+
}
203+
```
204+
205+
If the alignment and size are known at compile-time, the compiler can convert a call to `stackAlloc()` into a single LLVM `alloca` instruction. If either needs to be computed at runtime, a dynamic stack allocation can instead be emitted by the compiler.
206+
207+
### Location of allocated buffers
208+
209+
The proposed functions do _not_ guarantee that their buffers will be stack-allocated. This omission is intentional: guaranteed stack-allocation would make this feature equivalent to C99's variable-length arrays—a feature that is extremely easy to misuse and which is the cause of many [real-world security vulnerabilities](https://duckduckgo.com/?q=cve+variable-length+array). Instead, the proposed functions should stack-promote aggressively, but heap-allocate (just as `UnsafeMutableBufferPointer.allocate(capacity:)` does today) when passed overly large sizes.
210+
211+
This fallback heuristic is an implementation detail and may be architecture- or system-dependent. A common C approach is to say "anything larger than _n_ bytes uses `calloc()`. The Standard Library could refine this approach by checking information available to it at runtime, e.g. the current thread's available stack space. Because the Standard Library would own this heuristic, all adopters would benefit from it and, subject to a recompile, from any enhancements made in future Swift revisions.
212+
213+
## Source compatibility
214+
215+
This is new API, so there are no source compatibility considerations.
216+
217+
## Effect on ABI stability
218+
219+
This is new API, so there are no ABI stability considerations. The proposed functions should always be inlined into their calling frames, so they should be back-deployable to older Swift targets.
220+
221+
## Effect on API resilience
222+
223+
The addition of the proposed functions does not affect API resilience. If they were removed in a future release, it would be a source-breaking change but not an ABI-breaking change, because the proposed functions should always be inlined into their calling frames.
224+
225+
## Alternatives considered
226+
227+
In the pitch thread for this proposal, a number of alternatives were discussed:
228+
229+
### Doing nothing
230+
231+
* A number of developers both at Apple and in the broader Swift community have indicated that the performance costs cited above are measurably affecting the performance of their libraries and applications.
232+
* The proposed functions would let developers build higher-order algorithms, structures, and interfaces that they cannot build properly today.
233+
234+
### Naming the functions something different
235+
236+
* One commenter suggested making the proposed functions static members of `UnsafeMutableBufferPointer` (etc.) instead of free functions. I don't feel strongly here, but the Standard Library has precedent for producing transient resources via free function, e.g. `withUnsafePointer(to:)` and `withUnsafeThrowingContinuation(_:)`. I am not immediately aware of counter-examples in the Standard Library.
237+
* Several commenters proposed less verbose names: `withEphemeral(...)`, `withUnsafeLocalStorage(...)`, and `withUnsafeUninitializedBuffer(...)` were all suggested. I don't have strong opinions here and will defer to reviewers' wisdom here.
238+
239+
### Exposing some subset of the three proposed functions
240+
241+
* One commenter wanted to expose _only_ `withUnsafeUninitializedMutableRawBufferPointer(byteCount:alignment:_:)` in order to add friction and reduce the risk that someone would adopt the function without understanding its behaviour. Since most adopters would immediately need to call `bindMemory(to:)` to get a typed buffer, my suspicion is that developers would quickly learn to do so anyway.
242+
* Another commenter did not want to expose `withUnsafeUninitializedMutablePointer(to:_)` on the premise that it is trivial to get an `UnsafeMutablePointer` out of an `UnsafeMutableBufferPointer` with a `count` of `1`. It is indeed easy to do so, however the two types have different sets of member functions and I'm not sure that the added friction _improves_ adopting code. On the other hand, if anyone needs a _single_ stack-allocated value, they can use `Optional` today to get one.
243+
244+
### Letting the caller specify a size limit for stack promotion
245+
246+
* It is unlikely that the caller will have sufficient additional information about the state of the program such that it can make better decisions about stack promotion than the compiler and/or Standard Library.
247+
248+
### Exposing this functionality as a type rather than as a scoped function
249+
250+
* It is likely the capacity of such a type would need to be known at compile time. Swift already has a mechanism for declaring types with a fixed-size sequence of values: homogeneous tuples. In fact, Michael Gottesman has [a pitch](https://forums.swift.org/t/pitch-improved-compiler-support-for-large-homogenous-tuples/49023) open at the time of this writing to add syntactic sugar to make homogeneous tuples look more like C arrays.
251+
* As a type, values thereof would need to be initialized before being used. They would impose the same initialization overhead we want to avoid.
252+
* A type, even a value type, suffers from the same stack-promotion limitations as `UnsafeMutableBufferPointer<T>` or `Array<T>`, namely that the optimizer must act conservatively and may still need to heap-allocate. Value types also get copied around quite a bit (although the Swift compiler is quite good at copy elision.)
253+
* One commenter suggested making this hypothetical type _only_ stack-allocatable, but no such type exists today in Swift. It would be completely new to both the Swift language and the Swift compiler. It would not generalize well, because (to my knowledge) there are no other use cases for stack-only types.
254+
255+
### Telling adopters to use `ManagedBuffer<Void, T>`
256+
257+
* `ManagedBuffer` has the same general drawbacks as any other type (see above.)
258+
* `ManagedBuffer` is a reference type, not a value type, so the compiler _defaults_ to heap-allocating it. Stack promotion is possible but is not the common case.
259+
* `ManagedBuffer` is not a great interface in and of itself.
260+
* To me, `ManagedBuffer` says "I want to allocate a refcounted object with an arbitrarily long tail-allocated buffer" thus avoiding two heap allocations when one will do. I can then use that object as I would use any other object. This sort of use case doesn't really align with the use cases for the proposed functions.
261+
262+
### Exposing an additional function to initialize a value by address without copying
263+
264+
* One commenter suggested:
265+
> A variation of the signature that gives you the initialized value you put in the memory as the return value, `makeValueAtUninitializedPointer<T>(_: (UnsafeMutablePointer<T>) -> Void) -> T`, which could be implemented with return value optimization to do the initialization in-place.
266+
267+
The proposed functions can be used for this purpose:
268+
269+
```swift
270+
let value = withUnsafeUninitializedMutablePointer(to: T.self) { ptr in
271+
...
272+
return ptr.move()
273+
}
274+
```
275+
276+
Subject to the optimizer eliminating the `move()`, which in the common case it should be able to do.
277+
278+
### Eliminating "unsafe" interfaces from Swift entirely
279+
280+
* Some commenters were concerned by the idea of adding more "unsafe" interfaces to the Standard Library. The proposed functions are "unsafe" by the usual Swift definition, but not moreso than existing functions such as `withUnsafePointer(to:_:)` or `Data.withUnsafeBytes(_:)`.
281+
* As discussed previously, in order to provide a high-level "safe" interface, the language needs some amount of lower-level unsafety. `Data` cannot be implemented without `UnsafeRawPointer` (or equivalent,) nor `Array<T>` without `UnsafeMutableBufferPointer<T>`, nor `CheckedContinuation<T, E>` without `UnsafeContinuation<T, E>`.
282+
* The need for unsafe interfaces is not limited to the Standard Library: developers working at every layer of the software stack can benefit from careful use of unsafe interfaces like the proposed functions.
283+
* Creating a dialect of Swift that bans unsafe interfaces entirely is an interesting idea and is worth discussing in more detail, but it is beyond the scope of this proposal.
284+
285+
## Acknowledgments
286+
287+
Thank you to the Swift team for your help and patience as I learn how to write Swift proposals. And thank you to everyone who commented in the pitch thread—it was great to see your feedback and your ideas!

0 commit comments

Comments
 (0)