Skip to content

[clang-repl] Pass triple to IncrementalCompilerBuilder as explicit argument #84174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 6, 2024

Conversation

weliveindetail
Copy link
Member

With out-of-process execution the target triple can be different from the one on the host. We need an interface to configure it.

@llvmbot llvmbot added clang Clang issues not falling into any other category clang:frontend Language frontend issues, e.g. anything involving "Sema" labels Mar 6, 2024
@llvmbot
Copy link
Member

llvmbot commented Mar 6, 2024

@llvm/pr-subscribers-clang

Author: Stefan Gränitz (weliveindetail)

Changes

With out-of-process execution the target triple can be different from the one on the host. We need an interface to configure it.


Full diff: https://github.com/llvm/llvm-project/pull/84174.diff

3 Files Affected:

  • (modified) clang/include/clang/Interpreter/Interpreter.h (+8-5)
  • (modified) clang/lib/Interpreter/Interpreter.cpp (+11-11)
  • (modified) clang/tools/clang-repl/ClangRepl.cpp (+8-4)
diff --git a/clang/include/clang/Interpreter/Interpreter.h b/clang/include/clang/Interpreter/Interpreter.h
index 292fa566ae7037..8d111aa086867a 100644
--- a/clang/include/clang/Interpreter/Interpreter.h
+++ b/clang/include/clang/Interpreter/Interpreter.h
@@ -49,7 +49,7 @@ class IncrementalCompilerBuilder {
   }
 
   // General C++
-  llvm::Expected<std::unique_ptr<CompilerInstance>> CreateCpp();
+  llvm::Expected<std::unique_ptr<CompilerInstance>> CreateCpp(llvm::Triple TT);
 
   // Offload options
   void SetOffloadArch(llvm::StringRef Arch) { OffloadArch = Arch; };
@@ -57,14 +57,17 @@ class IncrementalCompilerBuilder {
   // CUDA specific
   void SetCudaSDK(llvm::StringRef path) { CudaSDKPath = path; };
 
-  llvm::Expected<std::unique_ptr<CompilerInstance>> CreateCudaHost();
-  llvm::Expected<std::unique_ptr<CompilerInstance>> CreateCudaDevice();
+  llvm::Expected<std::unique_ptr<CompilerInstance>>
+  CreateCudaHost(llvm::Triple TT);
+  llvm::Expected<std::unique_ptr<CompilerInstance>>
+  CreateCudaDevice(llvm::Triple TT);
 
 private:
   static llvm::Expected<std::unique_ptr<CompilerInstance>>
-  create(std::vector<const char *> &ClangArgv);
+  create(llvm::Triple TT, std::vector<const char *> &ClangArgv);
 
-  llvm::Expected<std::unique_ptr<CompilerInstance>> createCuda(bool device);
+  llvm::Expected<std::unique_ptr<CompilerInstance>> createCuda(llvm::Triple TT,
+                                                               bool device);
 
   std::vector<const char *> UserArgs;
 
diff --git a/clang/lib/Interpreter/Interpreter.cpp b/clang/lib/Interpreter/Interpreter.cpp
index 9f97a3c6b0be9e..429c1b65f6665d 100644
--- a/clang/lib/Interpreter/Interpreter.cpp
+++ b/clang/lib/Interpreter/Interpreter.cpp
@@ -132,7 +132,8 @@ CreateCI(const llvm::opt::ArgStringList &Argv) {
 } // anonymous namespace
 
 llvm::Expected<std::unique_ptr<CompilerInstance>>
-IncrementalCompilerBuilder::create(std::vector<const char *> &ClangArgv) {
+IncrementalCompilerBuilder::create(llvm::Triple TT,
+                                   std::vector<const char *> &ClangArgv) {
 
   // If we don't know ClangArgv0 or the address of main() at this point, try
   // to guess it anyway (it's possible on some platforms).
@@ -162,8 +163,7 @@ IncrementalCompilerBuilder::create(std::vector<const char *> &ClangArgv) {
   TextDiagnosticBuffer *DiagsBuffer = new TextDiagnosticBuffer;
   DiagnosticsEngine Diags(DiagID, &*DiagOpts, DiagsBuffer);
 
-  driver::Driver Driver(/*MainBinaryName=*/ClangArgv[0],
-                        llvm::sys::getProcessTriple(), Diags);
+  driver::Driver Driver(/*MainBinaryName=*/ClangArgv[0], TT.str(), Diags);
   Driver.setCheckInputsExist(false); // the input comes from mem buffers
   llvm::ArrayRef<const char *> RF = llvm::ArrayRef(ClangArgv);
   std::unique_ptr<driver::Compilation> Compilation(Driver.BuildCompilation(RF));
@@ -179,17 +179,17 @@ IncrementalCompilerBuilder::create(std::vector<const char *> &ClangArgv) {
 }
 
 llvm::Expected<std::unique_ptr<CompilerInstance>>
-IncrementalCompilerBuilder::CreateCpp() {
+IncrementalCompilerBuilder::CreateCpp(llvm::Triple TT) {
   std::vector<const char *> Argv;
   Argv.reserve(5 + 1 + UserArgs.size());
   Argv.push_back("-xc++");
   Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end());
 
-  return IncrementalCompilerBuilder::create(Argv);
+  return IncrementalCompilerBuilder::create(TT, Argv);
 }
 
 llvm::Expected<std::unique_ptr<CompilerInstance>>
-IncrementalCompilerBuilder::createCuda(bool device) {
+IncrementalCompilerBuilder::createCuda(llvm::Triple TT, bool device) {
   std::vector<const char *> Argv;
   Argv.reserve(5 + 4 + UserArgs.size());
 
@@ -213,17 +213,17 @@ IncrementalCompilerBuilder::createCuda(bool device) {
 
   Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end());
 
-  return IncrementalCompilerBuilder::create(Argv);
+  return IncrementalCompilerBuilder::create(TT, Argv);
 }
 
 llvm::Expected<std::unique_ptr<CompilerInstance>>
-IncrementalCompilerBuilder::CreateCudaDevice() {
-  return IncrementalCompilerBuilder::createCuda(true);
+IncrementalCompilerBuilder::CreateCudaDevice(llvm::Triple TT) {
+  return IncrementalCompilerBuilder::createCuda(TT, true);
 }
 
 llvm::Expected<std::unique_ptr<CompilerInstance>>
-IncrementalCompilerBuilder::CreateCudaHost() {
-  return IncrementalCompilerBuilder::createCuda(false);
+IncrementalCompilerBuilder::CreateCudaHost(llvm::Triple TT) {
+  return IncrementalCompilerBuilder::createCuda(TT, false);
 }
 
 Interpreter::Interpreter(std::unique_ptr<CompilerInstance> CI,
diff --git a/clang/tools/clang-repl/ClangRepl.cpp b/clang/tools/clang-repl/ClangRepl.cpp
index 5bad8145324d06..96337ada043a6f 100644
--- a/clang/tools/clang-repl/ClangRepl.cpp
+++ b/clang/tools/clang-repl/ClangRepl.cpp
@@ -24,6 +24,7 @@
 #include "llvm/Support/ManagedStatic.h" // llvm_shutdown
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/TargetSelect.h"
+#include "llvm/TargetParser/Host.h"
 #include <optional>
 
 // Disable LSan for this test.
@@ -109,7 +110,8 @@ ReplListCompleter::operator()(llvm::StringRef Buffer, size_t Pos,
   std::vector<llvm::LineEditor::Completion> Comps;
   std::vector<std::string> Results;
 
-  auto CI = CB.CreateCpp();
+  llvm::Triple TT(MainInterp.getCompilerInstance()->getTargetOpts().Triple);
+  auto CI = CB.CreateCpp(TT);
   if (auto Err = CI.takeError()) {
     ErrRes = std::move(Err);
     return {};
@@ -167,6 +169,8 @@ int main(int argc, const char **argv) {
   clang::IncrementalCompilerBuilder CB;
   CB.SetCompilerArgs(ClangArgv);
 
+  llvm::Triple TT(llvm::sys::getProcessTriple());
+
   std::unique_ptr<clang::CompilerInstance> DeviceCI;
   if (CudaEnabled) {
     if (!CudaPath.empty())
@@ -177,16 +181,16 @@ int main(int argc, const char **argv) {
     }
     CB.SetOffloadArch(OffloadArch);
 
-    DeviceCI = ExitOnErr(CB.CreateCudaDevice());
+    DeviceCI = ExitOnErr(CB.CreateCudaDevice(TT));
   }
 
   // FIXME: Investigate if we could use runToolOnCodeWithArgs from tooling. It
   // can replace the boilerplate code for creation of the compiler instance.
   std::unique_ptr<clang::CompilerInstance> CI;
   if (CudaEnabled) {
-    CI = ExitOnErr(CB.CreateCudaHost());
+    CI = ExitOnErr(CB.CreateCudaHost(TT));
   } else {
-    CI = ExitOnErr(CB.CreateCpp());
+    CI = ExitOnErr(CB.CreateCpp(TT));
   }
 
   // Set an error handler, so that any LLVM backend diagnostics go through our

…gument

With out-of-process execution the target triple can be different from the one on the host. We need an interface to configure it.
@weliveindetail weliveindetail force-pushed the clang-repl-triple-arg branch from 767fccd to 0d7f6a8 Compare March 6, 2024 15:50
@@ -213,7 +214,8 @@ IncrementalCompilerBuilder::createCuda(bool device) {

Argv.insert(Argv.end(), UserArgs.begin(), UserArgs.end());

return IncrementalCompilerBuilder::create(Argv);
std::string TT = TargetTriple ? *TargetTriple : llvm::sys::getProcessTriple();
return IncrementalCompilerBuilder::create(TT, Argv);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The duplication is due to create() being static right now, but I think it's acceptable.

@@ -48,6 +48,8 @@ class IncrementalCompilerBuilder {
UserArgs = Args;
}

void SetTargetTriple(std::string TT) { TargetTriple = TT; }
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a lot less intrusive!

Copy link
Contributor

@vgvassilev vgvassilev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! LGTM!

@weliveindetail
Copy link
Member Author

Thanks for the quick feedback and review!

@weliveindetail weliveindetail merged commit 6494f9b into llvm:main Mar 6, 2024
@weliveindetail weliveindetail deleted the clang-repl-triple-arg branch March 6, 2024 19:23
@fmayer
Copy link
Contributor

fmayer commented Mar 6, 2024

This triggers the leak detector in our HWASan build bot https://lab.llvm.org/buildbot/#/builders/236/builds/9873/steps/10/logs/stdio

Note: This is test shard 1 of 23.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from IncrementalCompilerBuilder
[ RUN      ] IncrementalCompilerBuilder.SetCompilerArgs
[       OK ] IncrementalCompilerBuilder.SetCompilerArgs (12 ms)
[----------] 1 test from IncrementalCompilerBuilder (12 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (12 ms total)
[  PASSED  ] 1 test.
=================================================================
==2996657==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 33 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac1507a9c in operator new(unsigned long) /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/hwasan/hwasan_new_delete.cpp:64:3
    #1 0xaaaac2593884 in operator new(unsigned long, (anonymous namespace)::NamedBufferAlloc const&) /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/lib/Support/MemoryBuffer.cpp:82:35
    #2 0xaaaac2593550 in llvm::MemoryBuffer::getMemBuffer(llvm::StringRef, llvm::StringRef, bool) /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/lib/Support/MemoryBuffer.cpp:124:15
    #3 0xaaaac39509d4 in CreateCI /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/Interpreter/Interpreter.cpp:111:28
    #4 0xaaaac39509d4 in clang::IncrementalCompilerBuilder::create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::vector<char const*, std::__1::allocator<char const*>>&) /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/Interpreter/Interpreter.cpp:178:10
    #5 0xaaaac3953298 in clang::IncrementalCompilerBuilder::CreateCpp() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/Interpreter/Interpreter.cpp:189:10
    #6 0xaaaac1509c10 in (anonymous namespace)::IncrementalCompilerBuilder_SetCompilerArgs_Test::TestBody() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/unittests/Interpreter/IncrementalCompilerBuilderTest.cpp:24:25
    #7 0xaaaac273ec98 in HandleExceptionsInMethodIfSupported<testing::Test, void> /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc
    #8 0xaaaac273ec98 in testing::Test::Run() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc:2687:5
    #9 0xaaaac2742074 in testing::TestInfo::Run() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc:2836:11
    #10 0xaaaac2744284 in testing::TestSuite::Run() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc:3015:30
    #11 0xaaaac276ac1c in testing::internal::UnitTestImpl::RunAllTests() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc:5920:44
    #12 0xaaaac27695e8 in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc
    #13 0xaaaac27695e8 in testing::UnitTest::Run() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc:5484:10
    #14 0xaaaac2700288 in RUN_ALL_TESTS /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/include/gtest/gtest.h:2317:73
    #15 0xaaaac2700288 in main /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/UnitTestMain/TestMain.cpp:55:10
    #16 0xffff98856dbc  (/lib/aarch64-linux-gnu/libc.so.6+0x26dbc) (BuildId: b3e2fd825ee86277a10a2c20b9fc836b101a2b7f)
    #17 0xffff98856e94 in __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x26e94) (BuildId: b3e2fd825ee86277a10a2c20b9fc836b101a2b7f)
    #18 0xaaaac14cf22c in _start (/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/tools/clang/unittests/Interpreter/ClangReplInterpreterTests+0x3fdf22c)
SUMMARY: HWAddressSanitizer: 33 byte(s) leaked in 1 allocation(s).
libc++abi: Pure virtual function called!
--
exit: -6
--
********************
Testing:  0.. 10.. 20
FAIL: Clang-Unit :: Interpreter/./ClangReplInterpreterTests/1/23 (19683 of 78329)
******************** TEST 'Clang-Unit :: Interpreter/./ClangReplInterpreterTests/1/23' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/tools/clang/unittests/Interpreter/./ClangReplInterpreterTests-Clang-Unit-2413105-1-23.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=23 GTEST_SHARD_INDEX=1 /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/tools/clang/unittests/Interpreter/./ClangReplInterpreterTests
--
Note: This is test shard 2 of 23.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from IncrementalCompilerBuilder
[ RUN      ] IncrementalCompilerBuilder.SetTargetTriple
[       OK ] IncrementalCompilerBuilder.SetTargetTriple (15 ms)
[----------] 1 test from IncrementalCompilerBuilder (15 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (16 ms total)
[  PASSED  ] 1 test.
=================================================================
==2996658==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 33 byte(s) in 1 object(s) allocated from:
    #0 0xaaaaaf737a9c in operator new(unsigned long) /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/hwasan/hwasan_new_delete.cpp:64:3
    #1 0xaaaab07c3884 in operator new(unsigned long, (anonymous namespace)::NamedBufferAlloc const&) /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/lib/Support/MemoryBuffer.cpp:82:35
    #2 0xaaaab07c3550 in llvm::MemoryBuffer::getMemBuffer(llvm::StringRef, llvm::StringRef, bool) /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/lib/Support/MemoryBuffer.cpp:124:15
    #3 0xaaaab1b809d4 in CreateCI /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/Interpreter/Interpreter.cpp:111:28
    #4 0xaaaab1b809d4 in clang::IncrementalCompilerBuilder::create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::vector<char const*, std::__1::allocator<char const*>>&) /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/Interpreter/Interpreter.cpp:178:10
    #5 0xaaaab1b8321c in clang::IncrementalCompilerBuilder::CreateCpp() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/Interpreter/Interpreter.cpp:189:10
    #6 0xaaaaaf73b01c in (anonymous namespace)::IncrementalCompilerBuilder_SetTargetTriple_Test::TestBody() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/unittests/Interpreter/IncrementalCompilerBuilderTest.cpp:31:25
    #7 0xaaaab096ec98 in HandleExceptionsInMethodIfSupported<testing::Test, void> /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc
    #8 0xaaaab096ec98 in testing::Test::Run() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc:2687:5
    #9 0xaaaab0972074 in testing::TestInfo::Run() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc:2836:11
    #10 0xaaaab0974284 in testing::TestSuite::Run() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc:3015:30
    #11 0xaaaab099ac1c in testing::internal::UnitTestImpl::RunAllTests() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc:5920:44
    #12 0xaaaab09995e8 in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc
    #13 0xaaaab09995e8 in testing::UnitTest::Run() /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/src/gtest.cc:5484:10
    #14 0xaaaab0930288 in RUN_ALL_TESTS /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/googletest/include/gtest/gtest.h:2317:73
    #15 0xaaaab0930288 in main /b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/third-party/unittest/UnitTestMain/TestMain.cpp:55:10
    #16 0xffffb75a6dbc  (/lib/aarch64-linux-gnu/libc.so.6+0x26dbc) (BuildId: b3e2fd825ee86277a10a2c20b9fc836b101a2b7f)
    #17 0xffffb75a6e94 in __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x26e94) (BuildId: b3e2fd825ee86277a10a2c20b9fc836b101a2b7f)
    #18 0xaaaaaf6ff22c in _start (/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/tools/clang/unittests/Interpreter/ClangReplInterpreterTests+0x3fdf22c)
SUMMARY: HWAddressSanitizer: 33 byte(s) leaked in 1 allocation(s).
libc++abi: Pure virtual function called!

@weliveindetail
Copy link
Member Author

Interesting, thanks for reporting! It's this code:

  llvm::MemoryBuffer *MB = llvm::MemoryBuffer::getMemBuffer("").release();
  Clang->getPreprocessorOpts().addRemappedFile("<<< inputs >>>", MB);

Apparently, it is related to releasing the MemBuffer and passing the raw pointer to addRemappedFile(), but I don't see why this fails here and not in any of the existing tests. Let me revert and investigate tomorrow.

weliveindetail added a commit that referenced this pull request Mar 7, 2024
…licit argument" (#84261)

Reverts #84174 due too sanitizer memory leak detection
@weliveindetail weliveindetail restored the clang-repl-triple-arg branch March 7, 2024 22:06
weliveindetail added a commit that referenced this pull request Mar 7, 2024
…ilder (#84174)

With out-of-process execution the target triple can be different from
the one on the host. We need an interface to configure it.

Relanding this with cleanup-fixes in the unittest.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:frontend Language frontend issues, e.g. anything involving "Sema" clang Clang issues not falling into any other category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants