Skip to content

Commit e409568

Browse files
BatmanAoDBatmanAoD
authored
Inside Rust post: FFI-unwind design meeting (#522)
* Add first draft of post, with meeting date TBD * Simplify description * Use HTML generated by GitHub for table * Discuss aborting on UB in debug mode * Ensure new post is chronologically last * Use March 2nd as prospective meeting date * Fix non-inline links
1 parent b94aee3 commit e409568

File tree

1 file changed

+268
-0
lines changed

1 file changed

+268
-0
lines changed

‎posts/inside-rust/2020-02-27-ffi-unwind-design-meeting.md

Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
---
2+
layout: post
3+
title: "Announcing the first FFI-unwind project design meeting"
4+
author: Kyle Strand, Niko Matsakis, and Amanieu d'Antras
5+
description: "First design meeting for the FFI-unwind project"
6+
team: the FFI-unwind project group <https://www.rust-lang.org/governance/teams/lang#wg-ffi-unwind>
7+
---
8+
9+
The FFI-unwind project group, announced in [this RFC][rfc-announcement], is
10+
working to extend the language to support unwinding that crosses FFI
11+
boundaries.
12+
13+
We have reached our first technical decision point, on a question we have been
14+
discussing internally for quite a while. This blog post lays out the arguments
15+
on each side of the issue and invites the Rust community to join us at an
16+
upcoming meeting to help finalize our decision, which will be formalized and
17+
published as our first language-change RFC. This RFC will propose an "MVP"
18+
specification for well-defined cross-language unwinding.
19+
20+
The meeting will be on [March 2nd][meeting-link].
21+
22+
## Background: what is unwinding?
23+
24+
Exceptions are a familiar control flow mechanism in many programming languages.
25+
They are particularly commonplace in managed languages such as Java, but they
26+
are also part of the C++ language, which introduced them to the world of
27+
unmanaged systems programming.
28+
29+
When an exception is thrown, the runtime _unwinds_ the stack, essentially
30+
traversing it backwards and calling clean-up or error-recovery code such as
31+
destructors or `catch` blocks.
32+
33+
Compilers may implement their own unwinding mechanisms, but in native code such
34+
as Rust binaries, the mechanism is more commonly provided by the platform ABI.
35+
36+
It is well known that Rust does not have exceptions as such. But Rust _does_
37+
support unwinding! There are two scenarios that will cause unwinding to occur:
38+
39+
* By default, Rust's `panic!()` unwinds the stack.
40+
* Using FFI, Rust can call functions in other languages (such as C++) that can
41+
unwind the stack.
42+
* There are some special cases where C libraries can actually cause
43+
unwinding. For instance, on Microsoft platforms, `longjmp` is implemented
44+
as "forced unwinding" (see below)
45+
46+
Currently, when foreign (non-Rust) code invokes Rust code, the behavior of a
47+
`panic!()` unwind that "escapes" from Rust is explicitly undefined. Similarly,
48+
when Rust calls a foreign function that unwinds, the behavior once the unwind
49+
operation encounters Rust frames is undefined. The primary reason for this is
50+
to ensure that Rust implementations may use their own unwinding mechanism,
51+
which may not be compatible with the platform-provided "native" unwinding
52+
mechanism. Currently, however, `rustc` uses the native mechanism, and there are
53+
no plans to change this.
54+
55+
### Forced unwinding
56+
57+
Platform ABIs can define a special kind of unwinding called "forced unwinding."
58+
This type of unwinding works regardless of whether the code being unwound
59+
supports unwinding or not. However, destructors may not be executed if the
60+
frames being unwound do not have unwinding support.
61+
62+
There are two common examples of forced unwinding:
63+
64+
* On Windows platforms, `longjmp` is implemented as a forced unwind.
65+
* On glibc Linux, `pthread_exit` and `pthread_cancel` are implemented as a forced unwind.
66+
* In fact, `pthread_cancel` can cause all manner of C functions to unwind,
67+
including common functions like `read` and `write`. (For a complete list,
68+
search for "cancellation points" in the [pthreads man page][man-pthreads].)
69+
70+
## Requirements for any cross-language unwinding specification
71+
72+
* Unwinding between Rust functions (and in particular unwinding of Rust panics)
73+
may not necessarily use the system unwinding mechanism
74+
* In practice, we do use the system mechanism today, but we would like to
75+
reserve the freedom to change this.
76+
* If you enable `-Cpanic=abort`, we are able to optimize the size of binaries
77+
to remove most code related to unwinding.
78+
* Even with `-Cpanic=unwind` it should be possible to optimize away code when
79+
unwinding is known to never occur.
80+
* In practice, most "C" functions are never expected to unwind (because they
81+
are written in C, for example, and not in C++).
82+
* However, because unwinding is now part of most system ABIs, even C
83+
functions can unwind &mdash; most notably cancellation points triggered
84+
by `pthread_cancel`.
85+
* Changing the behavior from `-Cpanic=unwind` to `-Cpanic=abort` should not
86+
cause Undefined Behavior.
87+
* However, this may not be tenable, or at least not without making binaries
88+
much larger. See the discussion below for more details.
89+
* It may, of course, cause programs to abort that used to execute
90+
successfully. This could occur if a panic would've been caught and
91+
recovered.
92+
* We cannot change the ABI (the `"C"` in `extern "C"`) of functions in the libc
93+
crate, because this would be a breaking change: function pointers of different
94+
ABIs have different types.
95+
* This is relevant for the libc functions which may perform forced unwinding
96+
when `pthread_cancel` is called.
97+
98+
## The primary question: introduce a new ABI, or let the `"C"` ABI permit unwinding?
99+
100+
The core question that we would like to decide is whether the `"C"` ABI, as
101+
defined by Rust, should permit unwinding.
102+
103+
This is not a question we expected to be debating. We've long declared that
104+
unwinding through Rust's `"C"` ABI is undefined behavior. In part, this is
105+
because nobody had spent the time to figure out what the correct behavior would
106+
be, or how to implement it, although (as we'll see shortly) there are other
107+
good reasons for this choice.
108+
109+
In any case, in PR #65646, @Amanieu proposed that we could, in fact, simply
110+
define the behavior of unwinding across `"C"` boundaries. In discussing this,
111+
discovered that the question of whether the `"C"` ABI should permit unwinding was
112+
less clear-cut than we had assumed.
113+
114+
If the `"C"` ABI does not permit unwinding, a new ABI, called `"C unwind"`,
115+
will be introduced specifically to support unwinding.
116+
117+
## Three specific proposals
118+
119+
The project group has narrowed the design space down to three specific
120+
proposals. Two of these introduce the new `"C unwind"` ABI, and one does not.
121+
122+
Each proposal specifies the behavior of each type of unwind (Rust `panic!`,
123+
foreign (e.g. C++), and forced (e.g. `pthread_exit`)) when it encounters an
124+
ABI boundary under either the `panic=unwind` or `panic=abort` compile-mode.
125+
126+
Note that currently, `catch_unwind` does not intercept foreign unwinding
127+
(forced or unforced), and our initial RFCs will not change that. We may decide
128+
at a later date to define a way for Rust code to intercept foreign exceptions.
129+
130+
Throughout, the unwind generated by `panic!` will be referred to as
131+
`panic`-unwind.
132+
133+
### Proposal 1: Introduce `"C unwind"`, minimal specification
134+
135+
* `"C"` ABI boundary, `panic=<any>`
136+
* `panic`-unwind: program aborts
137+
* forced unwinding, no destructors: unwind behaves normally
138+
* other foreign unwinding: undefined behavior
139+
* `"C unwind"` ABI boundary
140+
* With `panic=unwind`: all types of unwinding behave normally
141+
* With `panic=abort`: all types of unwinding abort the program
142+
143+
This proposal provides 2 ABIs, each suited for different purposes: you would
144+
generally use `extern "C"` when interacting with C APIs (making sure to avoid
145+
destructors where `longjmp` might be used), and `extern "C unwind"`
146+
when interacting with C++ APIs. The main advantage of this proposal is that
147+
switching between `panic=unwind` and `panic=abort` does not introduce UB if you
148+
have correctly marked all potential unwinding calls as `"C unwind"` (your
149+
program will abort instead).
150+
151+
### Proposal 2: Introduce `"C unwind"`, forced unwinding always permitted
152+
153+
This is the same as the previous design, except that when compiled with
154+
`panic=abort`, forced unwinding would *not* be intercepted at `"C unwind"` ABI
155+
boundaries; that is, they would behave normally (though still UB if there are
156+
any destructors), without causing the program to abort. `panic`-unwind and
157+
non-forced foreign exceptions would still cause the program to abort.
158+
159+
The advantage of treating forced unwinding differently is that it reduces
160+
portability incompatibilities. Specifically, it ensures that using `"C unwind"`
161+
cannot cause `longjmp` or `pthread_exit` to stop working (abort the program)
162+
when the target platform and/or compile flags are changed. With proposal 1,
163+
`longjmp` will be able to cross `"C unwind"` boundaries _except_ on Windows
164+
with MSVC under `panic=abort`, and `pthread_exit` will work inside `"C unwind"`
165+
functions _except_ when linked with glibc under `panic=abort`. The downside of
166+
this proposal is that the abort stubs around `"C unwind"` calls in `panic=abort`
167+
become more complicated since they need to distinguish between different types
168+
of foreign exceptions.
169+
170+
### Proposal 3: No new ABI
171+
172+
* `panic=unwind`: unwind behaves normally
173+
* `panic=abort`:
174+
* `panic`-unwind: does not exist; `panic!` aborts the program
175+
* forced unwinding, no destructors: unwind behaves normally
176+
* other foreign unwinding: undefined behavior
177+
178+
The main advantage of this proposal is its simplicity: there is only one ABI and
179+
the behavior of `panic=abort` is identical to that of `-fno-exceptions` in C++.
180+
However this comes with the downside that switching to `panic=abort` may in some
181+
cases introduce UB (though only in unsafe code) if FFI calls unwind through Rust
182+
code.
183+
184+
Another advantage is that forced unwinding from existing functions defined in
185+
the `libc` crate such as `pthread_exit` and `longjmp` will be able to unwind
186+
frames with destructors when compiled with `panic=unwind`, which is not possible
187+
with the other proposals.
188+
189+
## Comparison table for the proposed designs
190+
191+
In this table, "UB" stands for "undefined behavior". We believe that all of
192+
these instances of undefined behavior could be detected at runtime, but the
193+
code to do so would impose an undesirable code-size penalty, entirely negating
194+
the optimizations made possible by using `panic=unwind` or the non-unwinding
195+
`"C"` ABI. This code would therefore only be appropriate for debug builds.
196+
Additionally, the complexity of implementing such checks may outweight their
197+
benefits.
198+
199+
Note that unwinding through a frame that has destructors without running those
200+
destructors (e.g. because they have been optimized out by `panic=abort`) is
201+
always undefined behavior.
202+
203+
<table>
204+
<thead>
205+
<tr>
206+
<th></th>
207+
<th><code>panic</code>-unwind</th>
208+
<th>Forced unwind, no destructors</th>
209+
<th>Forced unwind with destructors</th>
210+
<th>Other foreign unwind</th>
211+
</tr>
212+
</thead>
213+
<tbody>
214+
<tr>
215+
<td>Proposals 1 &amp; 2, <code>"C"</code> boundary, <code>panic=unwind</code></td>
216+
<td>abort</td>
217+
<td>unwind</td>
218+
<td>UB</td>
219+
<td>UB</td>
220+
</tr>
221+
<tr>
222+
<td>Proposals 1 &amp; 2, <code>"C"</code> boundary, <code>panic=abort</code></td>
223+
<td><code>panic!</code> aborts (no unwinding occurs)</td>
224+
<td>unwind</td>
225+
<td>UB</td>
226+
<td>UB</td>
227+
</tr>
228+
<tr>
229+
<td>Proposals 1 &amp; 2, <code>"C unwind"</code> boundary, <code>panic=unwind</code></td>
230+
<td>unwind</td>
231+
<td>unwind</td>
232+
<td>unwind</td>
233+
<td>unwind</td>
234+
</tr>
235+
<tr>
236+
<td>Proposal 1, <code>"C unwind"</code> boundary, <code>panic=abort</code></td>
237+
<td><code>panic!</code> aborts</td>
238+
<td>abort</td>
239+
<td>abort</td>
240+
<td>abort</td>
241+
</tr>
242+
<tr>
243+
<td>Proposal 2, <code>"C unwind"</code> boundary, <code>panic=abort</code></td>
244+
<td><code>panic!</code> aborts</td>
245+
<td>unwind</td>
246+
<td>UB</td>
247+
<td>abort</td>
248+
</tr>
249+
<tr>
250+
<td>Proposal 3, <code>"C"</code> boundary, <code>panic=unwind</code></td>
251+
<td>unwind</td>
252+
<td>unwind</td>
253+
<td>unwind</td>
254+
<td>unwind</td>
255+
</tr>
256+
<tr>
257+
<td>Proposal 3, <code>"C"</code> boundary, <code>panic=abort</code></td>
258+
<td><code>panic!</code> aborts</td>
259+
<td>unwind</td>
260+
<td>UB</td>
261+
<td>UB</td>
262+
</tr>
263+
</tbody>
264+
</table>
265+
266+
[rfc-announcement]: https://github.com/rust-lang/rfcs/pull/2797
267+
[meeting-link]: https://arewemeetingyet.com/UTC/2020-03-02/17:00/Lang%20Team%20Design%20Meeting:%20FFI-unwind#eyJ1cmwiOiJodHRwczovL21vemlsbGEuem9vbS51cy9qLzc2ODIzMTc2MCJ9
268+
[man-pthreads]: http://man7.org/linux/man-pages/man7/pthreads.7.html

0 commit comments

Comments
 (0)