CUDA: fix FA logic for PTX 7.0 and CC >= 7.5

[JohannesGaessler]

See #12187 .

The problem is as far as I can tell the explicit check against CC 7.0. However, if on a GPU with CC >= 7.5 the highest available PTX is 7.0 then the WMMA implementation should also be used.

[LostRuins]

Is there any logic to when the __CUDA_ARCH_LIST__ macro will be defined? I've noticed it existing in some places and not in others, but I don't see it explicitly defined anywhere in llama.cpp repo code.

[JohannesGaessler]

It's an NVCC macro, it's implicitly defined by CMake via e.g. set(CMAKE_CUDA_ARCHITECTURES "50;61;70;75;80").

[LostRuins]

I see. Do you know if it will be populated if I built with the makefile using something like -arch=all-major

[JohannesGaessler]

-arch=all_major should produce correct results but the performance will be bad. If you need to define an arbitrary list of compute capabilities you should use CMake.