Expose the available devices from the _ExecutionContext.

[saeta]

In order for code to determine what sort of parallelism strategies make sense,
we need to expose the list of available devices on the host. This change makes
the list of available devices publicly readable.

This change also orders the list of test suites alphabetically.

[saeta]

Thanks for the super fast review @rxwei ! PTAL?

[eaplatanios]

Could we instead make this return an array of Device where Device is something like this?

public struct Device {
    public let job: String?
    public let replica: Int?
    public let task: Int?
    public let type: DeviceType
    public let index: Int
}

public enum DeviceType {
  case cpu, gpu, tpu
}

This would be both more Swifty and also easier to manage on the user-side so that the user does not have to handle any string parsing, etc.

[rxwei]

We definitely could in the future, but note that ‘_ExecutionContext’ is itself not a public API (it’s underscored) and all of this is experimental.

[Shashi456]

Does this api ultimately give way for functions like isCudaAvailable and shifting data from gpu to cpu and so on so forth? or am i wrong in my understanding

[bgogul]

Usually good idea to keep unrelated changes in a separate PR?