[SYCL][CUDA] Updated CUDA documentation

[Ruyk]

Various updates to CUDA backend documentation:

• CMake help strings
• Get Started Guide
  - [ ] Basic doxygen generation

[Alexander-Johnston]

Looks much better. Just some minor changes

[fwyzard]

I would suggest to add few things:

• that if one builds an application for the CUDA backend, one must use choose a CUDA device, either explictly or through the SYCL_BE variable - otherwise the runtime will likely pick and OpenCL device, and fail to run (this is something I find particularly confusing);
• that one can build an application for both the OpenCL and the CUDA back ends at the same time, using -fsycl-targets=nvptx64-*-*-sycldevice,spir64-*-*-sycldevice (or -fsycl-targets=nvptx64-unknown-unknown-sycldevice,spir64-unknown-unknown-sycldevice`, etc.);
• that one can specify the CUDA device to target using -Xsycl-target-backend=nvptx64-*-*-sycldevice '--cuda-gpu-arch=sm_35'

Though, the last may be more relevant for a slightly more in-depth documentation that the "getting started" document.

[Ruyk]

Thanks for your suggestions (and all the issue reporting!)

I would suggest to add few things:

• that if one builds an application for the CUDA backend, one must use choose a CUDA device, either explictly or through the SYCL_BE variable - otherwise the runtime will likely pick and OpenCL device, and fail to run (this is something I find particularly confusing);

That is an interesting one. How do you build an application for the CUDA backend? You can't really do that at the moment. The only thing you can do is to generate the kernel code for nvptx64 target and link against the libsycl.so, which will have both OpenCL and CUDA support. There is no connection between device images and backends in the SYCL RT. Once a plugin is loaded, all device images are passed to all backends. In the case of the OpenCL, it fails if the OpenCL platform used is not the NVidia one, but in theory, you can generate a PTX device image and use it on the NVidia OpenCL platform and it should work!
A possible solution is to add some metadata to the device images that ties them to a backend, but to make that usable we would have to somehow ban using PTX on the OpenCL backend.

This is an interesting discussion that we are having in Khronos SYCL WG as well. The SYCL generalization and the SYCL modules are trying to address this problems, but we are still working on them.

As for the documentation, maybe I can add that when building for an nvptx64 target, SYCL_BE=PI_CUDA should be used at runtime to ensure the OpenCL backend is not used?
The CUDA Device Selector that I added on the examples is another route.

• that one can build an application for both the OpenCL and the CUDA back ends at the same time, using -fsycl-targets=nvptx64-*-*-sycldevice,spir64-*-*-sycldevice (or -fsycl-targets=nvptx64-unknown-unknown-sycldevice,spir64-unknown-unknown-sycldevice`, etc.);

I am reluctant to add this until intel/llvm#1194 is solved, otherwise it may cause further confusion.

• that one can specify the CUDA device to target using -Xsycl-target-backend=nvptx64-*-*-sycldevice '--cuda-gpu-arch=sm_35'

Though, the last may be more relevant for a slightly more in-depth documentation that the "getting started" document.

I think I could add it, but I rather see a solution to intel/llvm#1170 first. @Naghasan and @Alexander-Johnston are looking into a number of clang driver issues, so this may come soon.

[fwyzard]

I see - yes, I agree that waiting until intel#1194 and intel#1170 are fixed will make those topics less confusing.

I still need to understand the first part better ...

[Ruyk]

There are other issues with doxygen upstream, so i'll do that on a separate branch