Skip to content

Commit 83909e4

Browse files
committed
[Proposal] Random Unification
Update implementation link Update current example Correctly gets a number within the bounds Fixes some typos
1 parent ebf3209 commit 83909e4

File tree

1 file changed

+322
-0
lines changed

1 file changed

+322
-0
lines changed

proposals/nnnn-random-unification.md

Lines changed: 322 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,322 @@
1+
# Random Unification
2+
3+
* Proposal: [SE-NNNN](NNNN-random-unification.md)
4+
* Authors: [Alejandro Alonso](https://github.com/Azoy)
5+
* Review Manager: TBD
6+
* Status: **Awaiting review**
7+
8+
## Introduction
9+
10+
This proposal's main focus is to create a unified random API for all platforms.
11+
12+
*This idea has been floating around swift-evolution for a while now, but this is the thread that started this proposal: https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170904/039605.html*
13+
14+
## Motivation
15+
16+
The current random functionality that Swift provides is through imported C APIs. This of course is dependent on what system is running the code. This means that a different random implementation is provided on a per system basis. Swift relies on Darwin to provide developers the `arc4random(3)` approach. While on Linux, many developers tend to use `random()`, from the POSIX standard, or `rand()`, from the C standard library, for a quick and dirty solution for Linux systems. A very common implementation that developers tend to use for quick random functionality is here:
17+
18+
```swift
19+
// This is confusing because some may look at this and think Foundation provides these functions,
20+
// when in reality Foundation imports Darwin and Glibc from which these are defined and implemented.
21+
// You get the same behavior when you import UIKit, or AppKit.
22+
import Foundation
23+
24+
// We can't forget to seed the Linux implementation.
25+
// This is unsecure as time is a very predictable seed.
26+
// This raises more questions to whether or not Foundation, UIKit, or AppKit provided these functions.
27+
#if os(Linux)
28+
srandom(UInt32(time(nil)))
29+
#endif
30+
31+
// We name this randomNumber because random() interferes with Glibc's random()'s namespace
32+
func randomNumber() -> Int {
33+
#if !os(Linux)
34+
return Int(arc4random()) // This is inefficient as it doesn't utilize all of Int's range
35+
#else
36+
return random() // or Int(rand())
37+
#endif
38+
}
39+
40+
// Many tend to opt in for ranges here, but for this example I opt for a start and end argument
41+
func random(from start: Int, to end: Int) -> Int {
42+
#if !os(Linux)
43+
var random = Int(arc4random_uniform(UInt32(end - start)))
44+
#else
45+
// Not recommended as it introduces modulo bias
46+
var random = random() % (end - start)
47+
#endif
48+
49+
random += start
50+
51+
return random
52+
}
53+
54+
// Alternatively, an easier solution would be:
55+
/*
56+
func random(from start: Int, to end: Int) -> Int {
57+
var random = randomNumber() % end
58+
59+
if random < start {
60+
random += start
61+
}
62+
63+
return random
64+
}
65+
*/
66+
// However this approach introduces modulo bias for all systems rather than just Linux.
67+
```
68+
69+
While although this does work, it just provides a quick workaround to a much larger outstanding problem. Below is a list outlining the problems the example contains, and the problems the example introduces.
70+
71+
1. In order to define these workarounds, developers must utilize a few platform checks. Developers should not be forced to define platform checks for such trivial requests like this one.
72+
2. Unexperienced developers who rely on workarounds like these, may be pushing unsecure code that is used by tens of hundreds or thousands of users. Starting with Darwin's `arc4random(3)`, pre macOS 10.12 (Sierra) and iOS 10, the implementation of `arc4random(3)` utilized the RC4 algorithm. This algorithm is now considered non-cryptographically secure due to RC4 weakness. Post macOS 10.12 (Sierra) and iOS 10, "...it was replaced with the NIST-approved AES cipher", as stated from the man pages in terminal (`man arc4random`). Moving on to Linux we see that using `random()` or `rand()` to generate numbers make it completely predictable as these weren't designed to be at a crypto level.
73+
3. In the example, it uses modulo to generate the number within the upper bound. Many developers may not realize it, but for a quick workaround, it introduces modulo bias in which modulo does not correctly distribute the probability in dividing the upper bound equally within the range.
74+
4. Highly inefficient as creating a new `Int` from a `UInt32` doesn't utilize the full extent of `Int`'s range.
75+
76+
## Proposed Solution
77+
78+
### Random Number Generator
79+
80+
To kick this off, we will be discussing the rngs that each operating system will utilize. We will make the default random generator cryptographically-secure. This means on each operating system, the random api for Swift will produce a secure random number. It is also worth noting that if any of the following fail, particularly reading from /dev/urandom, then we produce a fatal error and abort the application. Reasons why I went with this approach in Alternatives Considered at the bottom of this proposal.
81+
82+
#### Darwin Platform
83+
84+
1. macOS
85+
86+
| macOS < 10.12 | macOS >= 10.12 |
87+
|:-----------------------:|:-----------------:|
88+
| Read from /dev/urandom | Use arc4random(3) |
89+
90+
2. iOS
91+
92+
| iOS < 10 | iOS >= 10 |
93+
|:--------------------------------------------------:|:-----------------:|
94+
| Use `SecRandomCopyBytes` in the Security framework | Use arc4random(3) |
95+
96+
#### Linux Platform
97+
98+
1. Ubuntu
99+
100+
| Kernel Version < 3.17 | Kernel Version >= 3.17 |
101+
|:----------------------:|:----------------------:|
102+
| Read from /dev/urandom | Use getrandom(2) |
103+
104+
### Random API
105+
106+
For the core API, introduce a new protocol named `RandomGenerator`. This type is used to define rngs that can be used within the stdlib. Developers can conform to this type and use their own custom rng throughout the stdlib.
107+
Then for the stdlib's default rng implementation, introduce a new enum named `Random`. This enum contains one case called `default` which provides access to the methods of `RandomGenerator`. `Random` is created as an enum to remove initialization of this type as there is no state to be held.
108+
Finally, introduce a new protocol named `Randomizable`. This type is used to provide types static methods which generate a random representation of that type.
109+
110+
Next, we need to make `FixedWidthInteger`, `BinaryFloatingPoint`, and `Bool` conform to `Randomizable` which allows for random creation of that type.
111+
112+
`FixedWidthInteger` example:
113+
```swift
114+
// Utilizes the standard library's default random (alias for Int.random(using: Random.default))
115+
let randomInt = Int.random
116+
let randomUInt = UInt.random(using: myCustomRandomGenerator)
117+
```
118+
119+
`BinaryFloatingPoint` example:
120+
```swift
121+
// For floating points, the range is set to 0 to 1
122+
123+
// Utilizes the standard library's default random (alias for Float.random(using: Random.default))
124+
let randomFloat = Float.random
125+
let randomDouble = Double.random(using: myCustomRandomGenerator)
126+
```
127+
128+
`Bool` example:
129+
```swift
130+
// Utilizes the standard library's default random (alias for Bool.random(using: Random.default))
131+
let randomBool1 = Bool.random
132+
let randomBool2 = Bool.random(using: myCustomRandomGenerator)
133+
```
134+
135+
For `RandomAccessCollection` we add these functions as a requirement on that type itself.
136+
137+
`RandomAccessCollection` example:
138+
```swift
139+
let greetings = ["hey", "hi", "hello", "hola"]
140+
141+
// Utilizes the standard library's default random (alias for greetings.random(using: Random.default))
142+
print(greetings.random)
143+
print(greetings.random(using: myCustomRandomGenerator))
144+
```
145+
146+
### Shuffle API
147+
148+
As a result of adding the random api, it only makes sense to utilize that power to fuel the shuffle methods. We add a method requirement for `MutableCollection` to shuffle the collection itself, and add a method requirement for `Collection` to return a shuffled version of itself in a new array. Example:
149+
150+
```swift
151+
var greetings = ["hey", "hi", "hello", "hola"]
152+
153+
// Utilizes the standard library's default random (alias for greetings.shuffle(using: Random.default))
154+
greetings.shuffle()
155+
print(greetings) // A possible output could be ["hola", "hello", "hey", "hi"]
156+
157+
let numbers = 0 ..< 5
158+
print(numbers.shuffled(using: myCustomRandomGenerator)) // A possible output could be [1, 3, 0, 4, 2]
159+
```
160+
161+
## Detailed Design
162+
163+
The actual implementation can be found here: [apple/swift#NNNNN](https://github.com/apple/swift/pull/12772)
164+
165+
```swift
166+
public protocol RandomGenerator {
167+
func next<T : FixedWidthInteger>(_ type: T.Type) -> T
168+
func next<T : FixedWidthInteger>(_ type: T.Type, upperBound: T) -> T
169+
}
170+
171+
public enum Random : RandomGenerator {
172+
case `default`
173+
174+
// Conformance for `RandomGenerator`
175+
public func next<T : FixedWidthInteger>(_ type: T.Type) -> T
176+
177+
// Conformance for `RandomGenerator`
178+
public func next<T : FixedWidthInteger>(_ type: T.Type, upperBound: T) -> T
179+
}
180+
181+
public protocol RandomAccessCollection {
182+
var random: Element { get }
183+
func random(using generator: RandomGenerator) -> Element
184+
}
185+
186+
extension RandomAccessCollection {
187+
// Default implementation
188+
public var random: Element {
189+
return self.random(using: Random.default)
190+
}
191+
192+
// Default implementation
193+
public func random(using generator: RandomGenerator) -> Element
194+
}
195+
196+
public protocol Randomizable {
197+
static var random: Self { get }
198+
static func random(using generator: RandomGenerator) -> Self
199+
}
200+
201+
extension Randomizable {
202+
// Default implementation
203+
public static var random: Self {
204+
return self.random(using: Random.default)
205+
}
206+
}
207+
208+
public protocol FixedWidthInteger : Randomizable {}
209+
210+
extension FixedWidthInteger {
211+
// Conformance to `Randomizable`
212+
public static func random(using generator: RandomGenerator) -> Self
213+
}
214+
215+
public protocol BinaryFloatingPoint : Randomizable {}
216+
217+
extension BinaryFloatingPoint {
218+
// Conformance to `Randomizable`
219+
public static func random(using generator: RandomGenerator) -> Self
220+
}
221+
222+
public struct Bool : Randomizable {}
223+
224+
extension Bool {
225+
// Conformance to `Randomizable`
226+
public static func random(using generator: RandomGenerator) -> Bool
227+
}
228+
229+
public protocol Collection {
230+
func shuffled(using generator: RandomGenerator) -> [Element]
231+
}
232+
233+
extension Collection {
234+
// Default implementation
235+
public func shuffled(using generator: RandomGenerator = Random.default) -> [Element]
236+
}
237+
238+
public protocol MutableCollection {
239+
mutating func shuffle(using generator: RandomGenerator)
240+
}
241+
242+
extension MutableCollection {
243+
// Default implementation
244+
public mutating func shuffle(using generator: RandomGenerator = Random.default)
245+
}
246+
```
247+
248+
## Source compatibility
249+
250+
This change is purely additive, thus source compatibility is not affected.
251+
252+
## Effect on ABI stability
253+
254+
This change is purely additive, thus ABI stability is not affected.
255+
256+
## Effect on API resilience
257+
258+
This change is purely additive, thus API resilience is not affected.
259+
260+
## Alternatives considered
261+
262+
There were very many alternatives to be considered in this proposal.
263+
264+
### Why would the program abort if it failed to generate a random number?
265+
266+
I spent a lot of time deciding what to do if it failed. Ultimately it came down to the fact that these will almost never fail. In the cases where this can fail is where `/dev/urandom` doesn't exist, or there were too many file descriptors open on the process level or system level. In the case where `/dev/urandom` doesn't exist, either the kernel is too old to generate that file by itself on a fresh install, or a privileged user deleted it. Both of which are way out of scope for Swift in my opinion. In the case where there are too many file descriptors, with modern technology this should almost never happen. If the process has opened too many descriptors then it should be up to the developer to optimize opening and closing descriptors.
267+
268+
In a world where this did return an error to Swift, it would require types that implement `Randomizable` to return Optionals as well.
269+
270+
```swift
271+
let random = Int.random!
272+
```
273+
274+
"I just want a random number, what is this ! the compiler is telling me to add?"
275+
276+
This syntax wouldn't make sense for a custom rng that deterministically generates numbers with no fail.
277+
278+
Rust aborts when experiencing an unexpected error with any of the forms of randomness. [source](https://doc.rust-lang.org/rand/src/rand/os.rs.html)
279+
280+
It would be silly to account for these edge cases that would only happen to those who need to update their os, optimize their file descriptors, or deleted their `/dev/urandom`. Accounting for these cases sacrifices the clean api for everyone else.
281+
282+
### Shouldn't this fallback on something more secure at times of low entropy?
283+
284+
Thomas Hühn explains it very well [here](https://www.2uo.de/myths-about-urandom/). There is also a deeper discussion [here talking about python's implementation](https://www.python.org/dev/peps/pep-0524). Both articles discuss that even though /dev/urandom may not have enough entropy at a fresh install, "It doesn't matter. The underlying cryptographic building blocks are designed such that an attacker cannot predict the outcome." Using `getrandom(2)` on linux systems where the kernel version is >= 3.17, will block if it decides that the entropy pool is too small. In python's implementation, they fallback to reading `/dev/urandom` if `getrandom(2)` decides there is not enough entropy.
285+
286+
### Why not make the default rng non-secure?
287+
288+
Swift is a safe language which means that it shouldn't be encouraging non-experienced developers to be pushing unsecure code. Making the default secure removes this issue and gives developers a feeling of comfort knowing their code is secure out of the box.
289+
290+
### Rename RandomGenerator
291+
292+
It has been discussed to give this a name such as `RNG`. The reason why I went with `RandomGenerator` was because it is concise, whereas `RNG` has a level of obscurity to those who don't know the acronym.
293+
294+
### Add different types of ranges to `.random` on integer types.
295+
296+
In my opinion, these don't make sense with the current `Randomizable` protocol. Take for instance, a custom `Date` structure, what does it mean to get a random `Date` within a range? While this is possible, it requires `Date` to conform to `Comparable` and `Strideable`. One could make the argument that we could just add these to the integer protocols, but then that breaks the meaning of that specific protocol. Think adding `.random(in:)` to `FixedWidthInteger`, the purpose of this protocol is to define integers with a fixed size, not the ability to get a random representation of it. Plus this functionality is found with `(0 ..< 5).random` and `(0 ... 5).random(using:)`.
297+
298+
### Make `.random` and `.random(using:)` on `RandomAccessCollection` Optional
299+
300+
Many will debate here that `.random` behaves the same as `.first` and `.last`. While they are correct, making `.random` would provide a rather ugly and confusing api for the user api. Plus I wanted to mimic the same functionality for `RandomAccessCollection` as I did with the integers, and boolean. "I just want a random element, what is this ! that the compiler is telling me to add?"
301+
302+
```swift
303+
let nums = 0 ..< 5
304+
305+
guard let num = nums.random else {
306+
// handle empty collection
307+
}
308+
309+
print(num)
310+
```
311+
312+
The above could be also be written as the following for those unsure if the collection is empty or not, without sacrificing the user api:
313+
314+
```swift
315+
let nums = 0 ..< 5
316+
317+
guard !nums.isEmpty else {
318+
// handle empty collection
319+
}
320+
321+
print(num.random)
322+
```

0 commit comments

Comments
 (0)