Skip to content

[Offload] Add support for riscv64 to host plugin #115773

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 13, 2024

Conversation

aurel32
Copy link
Contributor

@aurel32 aurel32 commented Nov 11, 2024

This adds support for the riscv64 architecture to the offload host plugin. The check to define FFI_DEFAULT_ABI is intentionally not guarded by __riscv_xlen as the value is the same for riscv32 and riscv64 (support for OpenMP on riscv32 is still under review).

Copy link

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

@llvmbot
Copy link
Member

llvmbot commented Nov 11, 2024

@llvm/pr-subscribers-offload

Author: None (aurel32)

Changes

This adds support for the riscv64 architecture to the offload host plugin. The check to define FFI_DEFAULT_ABI is intentionally not guarded by __riscv_xlen as the value is the same for riscv32 and riscv64 (support for OpenMP on riscv32 is still under review).


Full diff: https://github.com/llvm/llvm-project/pull/115773.diff

5 Files Affected:

  • (modified) offload/CMakeLists.txt (+2)
  • (modified) offload/plugins-nextgen/common/src/Utils/ELF.cpp (+2)
  • (modified) offload/plugins-nextgen/host/CMakeLists.txt (+5-1)
  • (modified) offload/plugins-nextgen/host/dynamic_ffi/ffi.h (+2-1)
  • (modified) offload/plugins-nextgen/host/src/rtl.cpp (+2)
diff --git a/offload/CMakeLists.txt b/offload/CMakeLists.txt
index 9b771d1116ee38..dfd25bad608436 100644
--- a/offload/CMakeLists.txt
+++ b/offload/CMakeLists.txt
@@ -199,6 +199,8 @@ set (LIBOMPTARGET_ALL_TARGETS "${LIBOMPTARGET_ALL_TARGETS} nvptx64-nvidia-cuda-L
 set (LIBOMPTARGET_ALL_TARGETS "${LIBOMPTARGET_ALL_TARGETS} nvptx64-nvidia-cuda-JIT-LTO")
 set (LIBOMPTARGET_ALL_TARGETS "${LIBOMPTARGET_ALL_TARGETS} s390x-ibm-linux-gnu")
 set (LIBOMPTARGET_ALL_TARGETS "${LIBOMPTARGET_ALL_TARGETS} s390x-ibm-linux-gnu-LTO")
+set (LIBOMPTARGET_ALL_TARGETS "${LIBOMPTARGET_ALL_TARGETS} riscv64-unknown-linux-gnu")
+set (LIBOMPTARGET_ALL_TARGETS "${LIBOMPTARGET_ALL_TARGETS} riscv64-unknown-linux-gnu-LTO")
 
 # Once the plugins for the different targets are validated, they will be added to
 # the list of supported targets in the current system.
diff --git a/offload/plugins-nextgen/common/src/Utils/ELF.cpp b/offload/plugins-nextgen/common/src/Utils/ELF.cpp
index 90d6950b83e5ad..88642fd5b56400 100644
--- a/offload/plugins-nextgen/common/src/Utils/ELF.cpp
+++ b/offload/plugins-nextgen/common/src/Utils/ELF.cpp
@@ -45,6 +45,8 @@ uint16_t utils::elf::getTargetMachine() {
   return EM_AARCH64;
 #elif defined(__powerpc64__)
   return EM_PPC64;
+#elif defined(__riscv)
+  return EM_RISCV;
 #else
 #warning "Unknown ELF compilation target architecture"
   return EM_NONE;
diff --git a/offload/plugins-nextgen/host/CMakeLists.txt b/offload/plugins-nextgen/host/CMakeLists.txt
index 0ab827a355379d..cbfe4b951af453 100644
--- a/offload/plugins-nextgen/host/CMakeLists.txt
+++ b/offload/plugins-nextgen/host/CMakeLists.txt
@@ -1,4 +1,4 @@
-set(supported_targets x86_64 aarch64 ppc64 ppc64le s390x)
+set(supported_targets x86_64 aarch64 ppc64 ppc64le riscv64 s390x)
 if(NOT ${CMAKE_SYSTEM_PROCESSOR} IN_LIST supported_targets)
   message(STATUS "Not building ${machine} NextGen offloading plugin")
   return()
@@ -59,4 +59,8 @@ elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "s390x$")
   list(APPEND LIBOMPTARGET_SYSTEM_TARGETS
        "s390x-ibm-linux-gnu" "s390x-ibm-linux-gnu-LTO")
   set(LIBOMPTARGET_SYSTEM_TARGETS "${LIBOMPTARGET_SYSTEM_TARGETS}" PARENT_SCOPE)
+elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "riscv64$")
+  list(APPEND LIBOMPTARGET_SYSTEM_TARGETS
+       "riscv64-unknown-linux-gnu" "riscv64-unknown-linux-gnu-LTO")
+  set(LIBOMPTARGET_SYSTEM_TARGETS "${LIBOMPTARGET_SYSTEM_TARGETS}" PARENT_SCOPE)
 endif()
diff --git a/offload/plugins-nextgen/host/dynamic_ffi/ffi.h b/offload/plugins-nextgen/host/dynamic_ffi/ffi.h
index 0ae025805e1d40..8b4e0286d65e3c 100644
--- a/offload/plugins-nextgen/host/dynamic_ffi/ffi.h
+++ b/offload/plugins-nextgen/host/dynamic_ffi/ffi.h
@@ -43,7 +43,8 @@ typedef enum {
 typedef enum ffi_abi {
 #if (defined(_M_X64) || defined(__x86_64__))
   FFI_DEFAULT_ABI = 2, // FFI_UNIX64.
-#elif defined(__aarch64__) || defined(__arm64__) || defined(_M_ARM64)
+#elif defined(__aarch64__) || defined(__arm64__) || defined(_M_ARM64) ||       \
+    defined(__riscv)
   FFI_DEFAULT_ABI = 1, // FFI_SYSV.
 #elif defined(__powerpc64__)
   FFI_DEFAULT_ABI = 8, // FFI_LINUX.
diff --git a/offload/plugins-nextgen/host/src/rtl.cpp b/offload/plugins-nextgen/host/src/rtl.cpp
index fe296b77c7d557..2395cea6a7de75 100644
--- a/offload/plugins-nextgen/host/src/rtl.cpp
+++ b/offload/plugins-nextgen/host/src/rtl.cpp
@@ -440,6 +440,8 @@ struct GenELF64PluginTy final : public GenericPluginTy {
 #else
     return llvm::Triple::ppc64;
 #endif
+#elif defined(__riscv) && (__riscv_xlen == 64)
+    return llvm::Triple::ArchType::riscv64;
 #else
     return llvm::Triple::UnknownArch;
 #endif

Copy link
Contributor

@jhuber6 jhuber6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me.

This adds support for the riscv64 architecture to the offload host
plugin. The check to define FFI_DEFAULT_ABI is intentionally not guarded
by __riscv_xlen as the value is the same for riscv32 and riscv64
(support for OpenMP on riscv32 is still under review).
@aurel32 aurel32 force-pushed the riscv64-offload-host branch from 2978479 to 8f24fd1 Compare November 13, 2024 05:30
@jhuber6 jhuber6 merged commit b6bd747 into llvm:main Nov 13, 2024
5 checks passed
Copy link

@aurel32 Congratulations on having your first Pull Request (PR) merged into the LLVM Project!

Your changes will be combined with recent changes from other authors, then tested by our build bots. If there is a problem with a build, you may receive a report in an email or a comment on this PR.

Please check whether problems have been caused by your change specifically, as the builds can include changes from many authors. It is not uncommon for your change to be included in a build that fails due to someone else's changes, or infrastructure issues.

How to do this, and the rest of the post-merge process, is covered in detail here.

If your change does cause a problem, it may be reverted, or you can revert it yourself. This is a normal part of LLVM development. You can fix your changes and open a new PR to merge them again.

If you don't get any reports, no action is required from you. Your changes are working as expected, well done!

@llvm-ci
Copy link
Collaborator

llvm-ci commented Nov 13, 2024

LLVM Buildbot has detected a new failure on builder openmp-offload-libc-amdgpu-runtime running on omp-vega20-1 while building offload at step 6 "test-openmp".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/73/builds/8504

Here is the relevant piece of the build log for the reference
Step 6 (test-openmp) failure: test (failure)
******************** TEST 'libomp :: tasking/issue-94260-2.c' FAILED ********************
Exit Code: -11

Command Output (stdout):
--
# RUN: at line 1
/home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.build/./bin/clang -fopenmp   -I /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.build/runtimes/runtimes-bins/openmp/runtime/src -I /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.src/openmp/runtime/test -L /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.build/runtimes/runtimes-bins/openmp/runtime/src  -fno-omit-frame-pointer -I /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.src/openmp/runtime/test/ompt /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.src/openmp/runtime/test/tasking/issue-94260-2.c -o /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.build/runtimes/runtimes-bins/openmp/runtime/test/tasking/Output/issue-94260-2.c.tmp -lm -latomic && /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.build/runtimes/runtimes-bins/openmp/runtime/test/tasking/Output/issue-94260-2.c.tmp
# executed command: /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.build/./bin/clang -fopenmp -I /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.build/runtimes/runtimes-bins/openmp/runtime/src -I /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.src/openmp/runtime/test -L /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.build/runtimes/runtimes-bins/openmp/runtime/src -fno-omit-frame-pointer -I /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.src/openmp/runtime/test/ompt /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.src/openmp/runtime/test/tasking/issue-94260-2.c -o /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.build/runtimes/runtimes-bins/openmp/runtime/test/tasking/Output/issue-94260-2.c.tmp -lm -latomic
# executed command: /home/ompworker/bbot/openmp-offload-libc-amdgpu-runtime/llvm.build/runtimes/runtimes-bins/openmp/runtime/test/tasking/Output/issue-94260-2.c.tmp
# note: command had no output on stdout or stderr
# error: command failed with exit status: -11

--

********************


@joshua-arch1
Copy link
Contributor

It seems that we still cannot use -fopenmp-target=riscv64-unknown-linux-gnu.

@jhuber6
Copy link
Contributor

jhuber6 commented Nov 22, 2024

It seems that we still cannot use -fopenmp-target=riscv64-unknown-linux-gnu.

What's the issue?

@joshua-arch1
Copy link
Contributor

It seems that we still cannot use -fopenmp-target=riscv64-unknown-linux-gnu.

What's the issue?

I‘m running openmp_vv suite on riscv clang. It seems that tests with target offloading cannot pass, like https://github.com/OpenMP-Validation-and-Verification/OpenMP_VV/blob/master/tests/4.5/target/test_target_defaultmap.c.

If I add -fopenmp-targets=riscv64-unknown-linux-gnu, it will show unsupported.

@jhuber6
Copy link
Contributor

jhuber6 commented Nov 22, 2024

Some debug information would help, I don't know if you built with -DLIBOMPTARGET_ENABLE_DEBUG=ON

@joshua-arch1
Copy link
Contributor

joshua-arch1 commented Nov 22, 2024

Some debug information would help, I don't know if you built with -DLIBOMPTARGET_ENABLE_DEBUG=ON

clang -I./ompvv -lm -fopenmp -fopenmp-targets=riscv64-unknown-linux-gnu -march=rv64gc -DVERBOSE_MODE=1 -DOMPTARGET_DEBUG=1 tests/4.5/target/test_target_defaultmap.c -o clang_test_target_defaultmap.c.o
error: OpenMP target is invalid: 'riscv64-unknown-linux-gnu'

Do we need -fopenmp-targets to enable target offloading on riscv?

From the error in https://github.com/OpenMP-Validation-and-Verification/OpenMP_VV/blob/master/tests/4.5/target/test_target_defaultmap.c, I think the #pragma omp target cannot work.

[OMPVV_ERROR: test_target_defaultmap.c:81] Condition scalar_char != 'a' failed
[OMPVV_ERROR: test_target_defaultmap.c:82] Condition scalar_short != 10 failed
[OMPVV_ERROR: test_target_defaultmap.c:83] Condition scalar_int != 11 failed
[OMPVV_ERROR: test_target_defaultmap.c:84] Condition scalar_float != 5.5f failed
[OMPVV_ERROR: test_target_defaultmap.c:85] Condition scalar_double != 10.45 failed
[OMPVV_ERROR: test_target_defaultmap.c:86] Condition scalar_enum != VAL1 failed
[OMPVV_ERROR: test_target_defaultmap.c:94] Condition test_defaultmap_off() failed
[OMPVV_INFO: test_target_defaultmap.c:96] The value of errors is 6.
[OMPVV_RESULT: test_target_defaultmap.c] Test failed on the host.

@jhuber6
Copy link
Contributor

jhuber6 commented Nov 22, 2024

Add it to ../clang/lib/Frontend/CompilerInvocation.cpp:4228

@joshua-arch1
Copy link
Contributor

Add it to ../clang/lib/Frontend/CompilerInvocation.cpp:4228

Can't we use "-fopenmp -fopenmp-targets=riscv64-unknown-linux-gnu"? It will show '-fopenmp-targets' must be used in conjunction with a '-fopenmp' option compatible with offloading; e.g., '-fopenmp=libomp' or '-fopenmp=libiomp5'.

For other target like powerpc, I can use "-fopenmp -fopenmp-targets=powerpc64le" without errors.

@jhuber6
Copy link
Contributor

jhuber6 commented Nov 22, 2024

Add the triple to

if (TT.getArch() == llvm::Triple::UnknownArch ||

@joshua-arch1
Copy link
Contributor

joshua-arch1 commented Nov 23, 2024

Add the triple to

if (TT.getArch() == llvm::Triple::UnknownArch ||

Thank you. It works now. But for some cases, there are still some errors.
For example:
ld: /tmp/test_target_firstprivate-12cba4.o: in function `main':
test_target_firstprivate.c:(.text+0xf6): undefined reference to '__kmpc_fork_call'

Do we need to add extra support for riscv64 in kmp related codes?

@jhuber6
Copy link
Contributor

jhuber6 commented Nov 23, 2024

Add the triple to

if (TT.getArch() == llvm::Triple::UnknownArch ||

Thank you. It works now. But for some cases, there are still some errors. For example: ld: /tmp/test_target_firstprivate-12cba4.o: in function `main': test_target_firstprivate.c:(.text+0xf6): undefined reference to '__kmpc_fork_call'

Do we need to add extra support for riscv64 in kmp related codes?

There should be some RISCV definitions in libomp but you'd probably need to check there. If you're not getting that defined it means you're not linking libomp or something. Also I think you might need to add it to

@shiltian
Copy link
Contributor

libomp doesn't support RISC-V yet. There is a PR but the author doesn't know whether it works properly or not.

@joshua-arch1
Copy link
Contributor

joshua-arch1 commented Nov 24, 2024

Line 4228 in 0e3c791

I'm sure that I have linked to libomp since I can see -lomp in the linking command. This undefined error stills exists after I added riscv64 to linkDevice.

Can libomptarget.rtl.riscv64.so be built with your patch?

@joshua-arch1
Copy link
Contributor

joshua-arch1 commented Nov 24, 2024

libomp doesn't support RISC-V yet. There is a PR but the author doesn't know whether it works properly or not.

Which PR?

Did you mean this patch (https://reviews.llvm.org/D59880#change-Hrr5PuIsxRRX)? It was early in 2019. If it doesn't work, are there any update on this support?

@joshua-arch1
Copy link
Contributor

libomp doesn't support RISC-V yet. There is a PR but the author doesn't know whether it works properly or not.

To be exact, I think it is libomptarget that does not support RISC-V yet. Since the target offloading openmp runtime libaray cannot work on RISC-V, I suppose the host plugin patch cannot work normally.

wangleiat added a commit that referenced this pull request Dec 17, 2024
This adds support for the loongarch64 architecture to the offload host
plugin.

Similar to #115773

To fix some test issues, I've had to add the LoongArch64 target to:

- CompilerInvocation::ParseLangArgs
- linkDevice in ClangLinuxWrapper.cpp
- OMPContext::OMPContext (to set the device_kind_cpu trait)

Reviewed By: jhuber6

Pull Request: #120173
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants