[Proposal] Random Unification

Update implementation link

Update current example

Correctly gets a number within the bounds

Fixes some typos

Update implementation text

Author: Azoy

proposals/nnnn-random-unification.md

@@ -0,0 +1,322 @@
+# Random Unification
+
+* Proposal: [SE-NNNN](NNNN-random-unification.md)
+* Authors: [Alejandro Alonso](https://github.com/Azoy)
+* Review Manager: TBD
+* Status: **Awaiting review**
+
+## Introduction
+
+This proposal's main focus is to create a unified random API for all platforms.
+
+*This idea has been floating around swift-evolution for a while now, but this is the thread that started this proposal: https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170904/039605.html*
+
+## Motivation
+
+The current random functionality that Swift provides is through imported C APIs. This of course is dependent on what system is running the code. This means that a different random implementation is provided on a per system basis. Swift relies on Darwin to provide developers the `arc4random(3)` approach. While on Linux, many developers tend to use `random()`, from the POSIX standard, or `rand()`,  from the C standard library, for a quick and dirty solution for Linux systems. A very common implementation that developers tend to use for quick random functionality is here:
+
+```swift
+// This is confusing because some may look at this and think Foundation provides these functions,
+// when in reality Foundation imports Darwin and Glibc from which these are defined and implemented.
+// You get the same behavior when you import UIKit, or AppKit.
+import Foundation
+
+// We can't forget to seed the Linux implementation.
+// This is unsecure as time is a very predictable seed.
+// This raises more questions to whether or not Foundation, UIKit, or AppKit provided these functions.
+#if os(Linux)
+srandom(UInt32(time(nil)))
+#endif
+
+// We name this randomNumber because random() interferes with Glibc's random()'s namespace
+func randomNumber() -> Int {
+#if !os(Linux)
+  return Int(arc4random()) // This is inefficient as it doesn't utilize all of Int's range
+#else
+  return random() // or Int(rand())
+#endif
+}
+
+// Many tend to opt in for ranges here, but for this example I opt for a start and end argument
+func random(from start: Int, to end: Int) -> Int {
+#if !os(Linux)
+  var random = Int(arc4random_uniform(UInt32(end - start)))
+#else
+  // Not recommended as it introduces modulo bias
+  var random = random() % (end - start)
+#endif
+
+  random += start
+
+  return random
+}
+
+// Alternatively, an easier solution would be:
+/*
+  func random(from start: Int, to end: Int) -> Int {
+    var random = randomNumber() % end
+
+    if random < start {
+      random += start
+    }
+
+    return random
+  }
+*/
+// However this approach introduces modulo bias for all systems rather than just Linux.
+```
+
+While although this does work, it just provides a quick workaround to a much larger outstanding problem. Below is a list outlining the problems the example contains, and the problems the example introduces.
+
+1. In order to define these workarounds, developers must utilize a few platform checks. Developers should not be forced to define platform checks for such trivial requests like this one.
+2. Unexperienced developers who rely on workarounds like these, may be pushing unsecure code that is used by tens of hundreds or thousands of users. Starting with Darwin's `arc4random(3)`, pre macOS 10.12 (Sierra) and iOS 10, the implementation of `arc4random(3)` utilized the RC4 algorithm. This algorithm is now considered non-cryptographically secure due to RC4 weakness. Post macOS 10.12 (Sierra) and iOS 10, "...it was replaced with the NIST-approved AES cipher", as stated from the man pages in terminal (`man arc4random`). Moving on to Linux we see that using `random()` or `rand()` to generate numbers make it completely predictable as these weren't designed to be at a crypto level.
+3. In the example, it uses modulo to generate the number within the upper bound. Many developers may not realize it, but for a quick workaround, it introduces modulo bias in which modulo does not correctly distribute the probability in dividing the upper bound equally within the range.
+4. Highly inefficient as creating a new `Int` from a `UInt32` doesn't utilize the full extent of `Int`'s range.
+
+## Proposed Solution
+
+### Random Number Generator
+
+To kick this off, we will be discussing the rngs that each operating system will utilize. We will make the default random generator cryptographically-secure. This means on each operating system, the random api for Swift will produce a secure random number. It is also worth noting that if any of the following fail, particularly reading from /dev/urandom, then we produce a fatal error and abort the application. Reasons why I went with this approach in Alternatives Considered at the bottom of this proposal.
+
+#### Darwin Platform
+
+1. macOS
+
+|      macOS < 10.12      |   macOS >= 10.12  |
+|:-----------------------:|:-----------------:|
+| Read from /dev/urandom  | Use arc4random(3) |
+
+2. iOS
+
+|                      iOS < 10                      |     iOS >= 10     |
+|:--------------------------------------------------:|:-----------------:|
+| Use `SecRandomCopyBytes` in the Security framework | Use arc4random(3) |
+
+#### Linux Platform
+
+1. Ubuntu
+
+|  Kernel Version < 3.17 | Kernel Version >= 3.17 |
+|:----------------------:|:----------------------:|
+| Read from /dev/urandom |    Use getrandom(2)    |
+
+### Random API
+
+For the core API, introduce a new protocol named `RandomGenerator`. This type is used to define rngs that can be used within the stdlib. Developers can conform to this type and use their own custom rng throughout the stdlib.
+Then for the stdlib's default rng implementation, introduce a new enum named `Random`. This enum contains one case called `default` which provides access to the methods of `RandomGenerator`. `Random` is created as an enum to remove initialization of this type as there is no state to be held.
+Finally, introduce a new protocol named `Randomizable`. This type is used to provide types static methods which generate a random representation of that type.
+
+Next, we need to make `FixedWidthInteger`, `BinaryFloatingPoint`, and `Bool` conform to `Randomizable` which allows for random creation of that type.
+
+`FixedWidthInteger` example:
+```swift
+// Utilizes the standard library's default random (alias for Int.random(using: Random.default))
+let randomInt = Int.random
+let randomUInt = UInt.random(using: myCustomRandomGenerator)
+```
+
+`BinaryFloatingPoint` example:
+```swift
+// For floating points, the range is set to 0 to 1
+
+// Utilizes the standard library's default random (alias for Float.random(using: Random.default))
+let randomFloat = Float.random
+let randomDouble = Double.random(using: myCustomRandomGenerator)
+```
+
+`Bool` example:
+```swift
+// Utilizes the standard library's default random (alias for Bool.random(using: Random.default))
+let randomBool1 = Bool.random
+let randomBool2 = Bool.random(using: myCustomRandomGenerator)
+```
+
+For `RandomAccessCollection` we add these functions as a requirement on that type itself.
+
+`RandomAccessCollection` example:
+```swift
+let greetings = ["hey", "hi", "hello", "hola"]
+
+// Utilizes the standard library's default random (alias for greetings.random(using: Random.default))
+print(greetings.random)
+print(greetings.random(using: myCustomRandomGenerator))
+```
+
+### Shuffle API
+
+As a result of adding the random api, it only makes sense to utilize that power to fuel the shuffle methods. We add a method requirement for `MutableCollection` to shuffle the collection itself, and add a method requirement for `Collection` to return a shuffled version of itself in a new array. Example:
+
+```swift
+var greetings = ["hey", "hi", "hello", "hola"]
+
+// Utilizes the standard library's default random (alias for greetings.shuffle(using: Random.default))
+greetings.shuffle()
+print(greetings) // A possible output could be ["hola", "hello", "hey", "hi"]
+
+let numbers = 0 ..< 5
+print(numbers.shuffled(using: myCustomRandomGenerator)) // A possible output could be [1, 3, 0, 4, 2]
+```
+
+## Detailed Design
+
+The actual implementation can be found here: [apple/swift#12772](https://github.com/apple/swift/pull/12772)
+
+```swift
+public protocol RandomGenerator {
+  func next<T : FixedWidthInteger>(_ type: T.Type) -> T
+  func next<T : FixedWidthInteger>(_ type: T.Type, upperBound: T) -> T
+}
+
+public enum Random : RandomGenerator {
+  case `default`
+
+  // Conformance for `RandomGenerator`
+  public func next<T : FixedWidthInteger>(_ type: T.Type) -> T
+
+  // Conformance for `RandomGenerator`
+  public func next<T : FixedWidthInteger>(_ type: T.Type, upperBound: T) -> T
+}
+
+public protocol RandomAccessCollection {
+  var random: Element { get }
+  func random(using generator: RandomGenerator) -> Element
+}
+
+extension RandomAccessCollection {
+  // Default implementation
+  public var random: Element {
+    return self.random(using: Random.default)
+  }
+
+  // Default implementation
+  public func random(using generator: RandomGenerator) -> Element
+}
+
+public protocol Randomizable {
+  static var random: Self { get }
+  static func random(using generator: RandomGenerator) -> Self
+}
+
+extension Randomizable {
+  // Default implementation
+  public static var random: Self {
+    return self.random(using: Random.default)
+  }
+}
+
+public protocol FixedWidthInteger : Randomizable {}
+
+extension FixedWidthInteger {
+  // Conformance to `Randomizable`
+  public static func random(using generator: RandomGenerator) -> Self
+}
+
+public protocol BinaryFloatingPoint : Randomizable {}
+
+extension BinaryFloatingPoint {
+  // Conformance to `Randomizable`
+  public static func random(using generator: RandomGenerator) -> Self
+}
+
+public struct Bool : Randomizable {}
+
+extension Bool {
+  // Conformance to `Randomizable`
+  public static func random(using generator: RandomGenerator) -> Bool
+}
+
+public protocol Collection {
+  func shuffled(using generator: RandomGenerator) -> [Element]
+}
+
+extension Collection {
+  // Default implementation
+  public func shuffled(using generator: RandomGenerator = Random.default) -> [Element]
+}
+
+public protocol MutableCollection {
+  mutating func shuffle(using generator: RandomGenerator)
+}
+
+extension MutableCollection {
+  // Default implementation
+  public mutating func shuffle(using generator: RandomGenerator = Random.default)
+}
+```
+
+## Source compatibility
+
+This change is purely additive, thus source compatibility is not affected.
+
+## Effect on ABI stability
+
+This change is purely additive, thus ABI stability is not affected.
+
+## Effect on API resilience
+
+This change is purely additive, thus API resilience is not affected.
+
+## Alternatives considered
+
+There were very many alternatives to be considered in this proposal.
+
+### Why would the program abort if it failed to generate a random number?
+
+I spent a lot of time deciding what to do if it failed. Ultimately it came down to the fact that these will almost never fail. In the cases where this can fail is where `/dev/urandom` doesn't exist, or there were too many file descriptors open on the process level or system level. In the case where `/dev/urandom` doesn't exist, either the kernel is too old to generate that file by itself on a fresh install, or a privileged user deleted it. Both of which are way out of scope for Swift in my opinion. In the case where there are too many file descriptors, with modern technology this should almost never happen. If the process has opened too many descriptors then it should be up to the developer to optimize opening and closing descriptors.
+
+In a world where this did return an error to Swift, it would require types that implement `Randomizable` to return Optionals as well.
+
+```swift
+let random = Int.random!
+```
+
+"I just want a random number, what is this ! the compiler is telling me to add?"
+
+This syntax wouldn't make sense for a custom rng that deterministically generates numbers with no fail.
+
+Rust aborts when experiencing an unexpected error with any of the forms of randomness. [source](https://doc.rust-lang.org/rand/src/rand/os.rs.html)
+
+It would be silly to account for these edge cases that would only happen to those who need to update their os, optimize their file descriptors, or deleted their `/dev/urandom`. Accounting for these cases sacrifices the clean api for everyone else.
+
+### Shouldn't this fallback on something more secure at times of low entropy?
+
+Thomas Hühn explains it very well [here](https://www.2uo.de/myths-about-urandom/). There is also a deeper discussion [here talking about python's implementation](https://www.python.org/dev/peps/pep-0524). Both articles discuss that even though /dev/urandom may not have enough entropy at a fresh install, "It doesn't matter. The underlying cryptographic building blocks are designed such that an	attacker cannot predict the outcome." Using `getrandom(2)` on linux systems where the kernel version is >= 3.17, will block if it decides that the entropy pool is too small. In python's implementation, they fallback to reading `/dev/urandom` if `getrandom(2)` decides there is not enough entropy.
+
+### Why not make the default rng non-secure?
+
+Swift is a safe language which means that it shouldn't be encouraging non-experienced developers to be pushing unsecure code. Making the default secure removes this issue and gives developers a feeling of comfort knowing their code is secure out of the box.
+
+### Rename RandomGenerator
+
+It has been discussed to give this a name such as `RNG`. The reason why I went with `RandomGenerator` was because it is concise, whereas `RNG` has a level of obscurity to those who don't know the acronym.
+
+### Add different types of ranges to `.random` on integer types.
+
+In my opinion, these don't make sense with the current `Randomizable` protocol. Take for instance, a custom `Date` structure, what does it mean to get a random `Date` within a range? While this is possible, it requires `Date` to conform to `Comparable` and `Strideable`. One could make the argument that we could just add these to the integer protocols, but then that breaks the meaning of that specific protocol. Think adding `.random(in:)` to `FixedWidthInteger`, the purpose of this protocol is to define integers with a fixed size, not the ability to get a random representation of it. Plus this functionality is found with `(0 ..< 5).random` and `(0 ... 5).random(using:)`.
+
+### Make `.random` and `.random(using:)` on `RandomAccessCollection` Optional
+
+Many will debate here that `.random` behaves the same as `.first` and `.last`. While they are correct, making `.random` would provide a rather ugly and confusing api for the user api. Plus I wanted to mimic the same functionality for `RandomAccessCollection` as I did with the integers, and boolean. "I just want a random element, what is this ! that the compiler is telling me to add?"
+
+```swift
+let nums = 0 ..< 5
+
+guard let num = nums.random else {
+  // handle empty collection
+}
+
+print(num)
+```
+
+The above could be also be written as the following for those unsure if the collection is empty or not, without sacrificing the user api:
+
+```swift
+let nums = 0 ..< 5
+
+guard !nums.isEmpty else {
+  // handle empty collection
+}
+
+print(num.random)
+```