-
Notifications
You must be signed in to change notification settings - Fork 14.3k
[mlir] Replace MLIR_ENABLE_CUDA_CONVERSIONS with LLVM_HAS_NVPTX_TARGET #93008
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@llvm/pr-subscribers-mlir-gpu @llvm/pr-subscribers-mlir-llvm Author: None (tyb0807) ChangesCurrently we only allow to serialize GPU module if CUDA toolkit is present, while in fact only serializing to binary format requires the latter. This change make it possible to serialize GPU module to offload or assembly formats even when CUDA toolkit is not provided. The condition to be able to serialize GPU module is NVPTX target having been specified when building LLVM. Full diff: https://github.com/llvm/llvm-project/pull/93008.diff 2 Files Affected:
diff --git a/mlir/lib/Target/LLVM/NVVM/Target.cpp b/mlir/lib/Target/LLVM/NVVM/Target.cpp
index e438ce84af1b5..02b1c1ae3a6b5 100644
--- a/mlir/lib/Target/LLVM/NVVM/Target.cpp
+++ b/mlir/lib/Target/LLVM/NVVM/Target.cpp
@@ -158,7 +158,7 @@ SerializeGPUModuleBase::loadBitcodeFiles(llvm::Module &module) {
return std::move(bcFiles);
}
-#if MLIR_ENABLE_CUDA_CONVERSIONS
+#if LLVM_HAS_NVPTX_TARGET
namespace {
class NVPTXSerializer : public SerializeGPUModuleBase {
public:
@@ -167,6 +167,15 @@ class NVPTXSerializer : public SerializeGPUModuleBase {
gpu::GPUModuleOp getOperation();
+ std::optional<SmallVector<char, 0>>
+ moduleToObject(llvm::Module &llvmModule) override;
+
+private:
+ // Target options.
+ gpu::TargetOptions targetOptions;
+
+#if MLIR_ENABLE_CUDA_CONVERSIONS
+public:
// Compile PTX to cubin using `ptxas`.
std::optional<SmallVector<char, 0>>
compileToBinary(const std::string &ptxCode);
@@ -175,9 +184,6 @@ class NVPTXSerializer : public SerializeGPUModuleBase {
std::optional<SmallVector<char, 0>>
compileToBinaryNVPTX(const std::string &ptxCode);
- std::optional<SmallVector<char, 0>>
- moduleToObject(llvm::Module &llvmModule) override;
-
private:
using TmpFile = std::pair<llvm::SmallString<128>, llvm::FileRemover>;
@@ -190,9 +196,7 @@ class NVPTXSerializer : public SerializeGPUModuleBase {
// 2. In the system PATH.
// 3. The path from `getCUDAToolkitPath()`.
std::optional<std::string> findTool(StringRef tool);
-
- // Target options.
- gpu::TargetOptions targetOptions;
+#endif // MLIR_ENABLE_CUDA_CONVERSIONS
};
} // namespace
@@ -201,6 +205,11 @@ NVPTXSerializer::NVPTXSerializer(Operation &module, NVVMTargetAttr target,
: SerializeGPUModuleBase(module, target, targetOptions),
targetOptions(targetOptions) {}
+gpu::GPUModuleOp NVPTXSerializer::getOperation() {
+ return dyn_cast<gpu::GPUModuleOp>(&SerializeGPUModuleBase::getOperation());
+}
+
+#if MLIR_ENABLE_CUDA_CONVERSIONS
std::optional<NVPTXSerializer::TmpFile>
NVPTXSerializer::createTemp(StringRef name, StringRef suffix) {
llvm::SmallString<128> filename;
@@ -214,10 +223,6 @@ NVPTXSerializer::createTemp(StringRef name, StringRef suffix) {
return TmpFile(filename, llvm::FileRemover(filename.c_str()));
}
-gpu::GPUModuleOp NVPTXSerializer::getOperation() {
- return dyn_cast<gpu::GPUModuleOp>(&SerializeGPUModuleBase::getOperation());
-}
-
std::optional<std::string> NVPTXSerializer::findTool(StringRef tool) {
// Find the `tool` path.
// 1. Check the toolkit path given in the command line.
@@ -512,10 +517,11 @@ NVPTXSerializer::compileToBinaryNVPTX(const std::string &ptxCode) {
return binary;
}
#endif // MLIR_ENABLE_NVPTXCOMPILER
+#endif // MLIR_ENABLE_CUDA_CONVERSIONS
std::optional<SmallVector<char, 0>>
NVPTXSerializer::moduleToObject(llvm::Module &llvmModule) {
- // Return LLVM IR if the compilation target is offload.
+ // Return LLVM IR if the compilation target is `offload`.
#define DEBUG_TYPE "serialize-to-llvm"
LLVM_DEBUG({
llvm::dbgs() << "LLVM IR for module: " << getOperation().getNameAttr()
@@ -549,7 +555,7 @@ NVPTXSerializer::moduleToObject(llvm::Module &llvmModule) {
});
#undef DEBUG_TYPE
- // Return PTX if the compilation target is assembly.
+ // Return PTX if the compilation target is `assembly`.
if (targetOptions.getCompilationTarget() ==
gpu::CompilationTarget::Assembly) {
// Make sure to include the null terminator.
@@ -557,6 +563,13 @@ NVPTXSerializer::moduleToObject(llvm::Module &llvmModule) {
return SmallVector<char, 0>(bin.begin(), bin.end());
}
+ // At this point, compilation target is either `binary` or `fatbinary`, which
+ // requires CUDA toolkit.
+ if (!(MLIR_ENABLE_CUDA_CONVERSIONS)) {
+ getOperation().emitError(
+ "CUDA toolkit not provided when trying to serialize GPU module.");
+ return std::nullopt;
+ }
// Compile to binary.
#if MLIR_ENABLE_NVPTXCOMPILER
return compileToBinaryNVPTX(*serializedISA);
@@ -564,7 +577,7 @@ NVPTXSerializer::moduleToObject(llvm::Module &llvmModule) {
return compileToBinary(*serializedISA);
#endif // MLIR_ENABLE_NVPTXCOMPILER
}
-#endif // MLIR_ENABLE_CUDA_CONVERSIONS
+#endif // LLVM_HAS_NVPTX_TARGET
std::optional<SmallVector<char, 0>>
NVVMTargetAttrImpl::serializeToObject(Attribute attribute, Operation *module,
@@ -576,7 +589,7 @@ NVVMTargetAttrImpl::serializeToObject(Attribute attribute, Operation *module,
module->emitError("Module must be a GPU module.");
return std::nullopt;
}
-#if MLIR_ENABLE_CUDA_CONVERSIONS
+#if LLVM_HAS_NVPTX_TARGET
NVPTXSerializer serializer(*module, cast<NVVMTargetAttr>(attribute), options);
serializer.init();
return serializer.run();
@@ -584,7 +597,7 @@ NVVMTargetAttrImpl::serializeToObject(Attribute attribute, Operation *module,
module->emitError(
"The `NVPTX` target was not built. Please enable it when building LLVM.");
return std::nullopt;
-#endif // MLIR_ENABLE_CUDA_CONVERSIONS
+#endif // LLVM_HAS_NVPTX_TARGET
}
Attribute
diff --git a/mlir/unittests/Target/LLVM/SerializeNVVMTarget.cpp b/mlir/unittests/Target/LLVM/SerializeNVVMTarget.cpp
index cea49356538f0..a8fe20d52fb2a 100644
--- a/mlir/unittests/Target/LLVM/SerializeNVVMTarget.cpp
+++ b/mlir/unittests/Target/LLVM/SerializeNVVMTarget.cpp
@@ -30,7 +30,7 @@
using namespace mlir;
// Skip the test if the NVPTX target was not built.
-#if MLIR_ENABLE_CUDA_CONVERSIONS
+#if LLVM_HAS_NVPTX_TARGET
#define SKIP_WITHOUT_NVPTX(x) x
#else
#define SKIP_WITHOUT_NVPTX(x) DISABLED_##x
|
@llvm/pr-subscribers-mlir Author: None (tyb0807) ChangesCurrently we only allow to serialize GPU module if CUDA toolkit is present, while in fact only serializing to binary format requires the latter. This change make it possible to serialize GPU module to offload or assembly formats even when CUDA toolkit is not provided. The condition to be able to serialize GPU module is NVPTX target having been specified when building LLVM. Full diff: https://github.com/llvm/llvm-project/pull/93008.diff 2 Files Affected:
diff --git a/mlir/lib/Target/LLVM/NVVM/Target.cpp b/mlir/lib/Target/LLVM/NVVM/Target.cpp
index e438ce84af1b5..02b1c1ae3a6b5 100644
--- a/mlir/lib/Target/LLVM/NVVM/Target.cpp
+++ b/mlir/lib/Target/LLVM/NVVM/Target.cpp
@@ -158,7 +158,7 @@ SerializeGPUModuleBase::loadBitcodeFiles(llvm::Module &module) {
return std::move(bcFiles);
}
-#if MLIR_ENABLE_CUDA_CONVERSIONS
+#if LLVM_HAS_NVPTX_TARGET
namespace {
class NVPTXSerializer : public SerializeGPUModuleBase {
public:
@@ -167,6 +167,15 @@ class NVPTXSerializer : public SerializeGPUModuleBase {
gpu::GPUModuleOp getOperation();
+ std::optional<SmallVector<char, 0>>
+ moduleToObject(llvm::Module &llvmModule) override;
+
+private:
+ // Target options.
+ gpu::TargetOptions targetOptions;
+
+#if MLIR_ENABLE_CUDA_CONVERSIONS
+public:
// Compile PTX to cubin using `ptxas`.
std::optional<SmallVector<char, 0>>
compileToBinary(const std::string &ptxCode);
@@ -175,9 +184,6 @@ class NVPTXSerializer : public SerializeGPUModuleBase {
std::optional<SmallVector<char, 0>>
compileToBinaryNVPTX(const std::string &ptxCode);
- std::optional<SmallVector<char, 0>>
- moduleToObject(llvm::Module &llvmModule) override;
-
private:
using TmpFile = std::pair<llvm::SmallString<128>, llvm::FileRemover>;
@@ -190,9 +196,7 @@ class NVPTXSerializer : public SerializeGPUModuleBase {
// 2. In the system PATH.
// 3. The path from `getCUDAToolkitPath()`.
std::optional<std::string> findTool(StringRef tool);
-
- // Target options.
- gpu::TargetOptions targetOptions;
+#endif // MLIR_ENABLE_CUDA_CONVERSIONS
};
} // namespace
@@ -201,6 +205,11 @@ NVPTXSerializer::NVPTXSerializer(Operation &module, NVVMTargetAttr target,
: SerializeGPUModuleBase(module, target, targetOptions),
targetOptions(targetOptions) {}
+gpu::GPUModuleOp NVPTXSerializer::getOperation() {
+ return dyn_cast<gpu::GPUModuleOp>(&SerializeGPUModuleBase::getOperation());
+}
+
+#if MLIR_ENABLE_CUDA_CONVERSIONS
std::optional<NVPTXSerializer::TmpFile>
NVPTXSerializer::createTemp(StringRef name, StringRef suffix) {
llvm::SmallString<128> filename;
@@ -214,10 +223,6 @@ NVPTXSerializer::createTemp(StringRef name, StringRef suffix) {
return TmpFile(filename, llvm::FileRemover(filename.c_str()));
}
-gpu::GPUModuleOp NVPTXSerializer::getOperation() {
- return dyn_cast<gpu::GPUModuleOp>(&SerializeGPUModuleBase::getOperation());
-}
-
std::optional<std::string> NVPTXSerializer::findTool(StringRef tool) {
// Find the `tool` path.
// 1. Check the toolkit path given in the command line.
@@ -512,10 +517,11 @@ NVPTXSerializer::compileToBinaryNVPTX(const std::string &ptxCode) {
return binary;
}
#endif // MLIR_ENABLE_NVPTXCOMPILER
+#endif // MLIR_ENABLE_CUDA_CONVERSIONS
std::optional<SmallVector<char, 0>>
NVPTXSerializer::moduleToObject(llvm::Module &llvmModule) {
- // Return LLVM IR if the compilation target is offload.
+ // Return LLVM IR if the compilation target is `offload`.
#define DEBUG_TYPE "serialize-to-llvm"
LLVM_DEBUG({
llvm::dbgs() << "LLVM IR for module: " << getOperation().getNameAttr()
@@ -549,7 +555,7 @@ NVPTXSerializer::moduleToObject(llvm::Module &llvmModule) {
});
#undef DEBUG_TYPE
- // Return PTX if the compilation target is assembly.
+ // Return PTX if the compilation target is `assembly`.
if (targetOptions.getCompilationTarget() ==
gpu::CompilationTarget::Assembly) {
// Make sure to include the null terminator.
@@ -557,6 +563,13 @@ NVPTXSerializer::moduleToObject(llvm::Module &llvmModule) {
return SmallVector<char, 0>(bin.begin(), bin.end());
}
+ // At this point, compilation target is either `binary` or `fatbinary`, which
+ // requires CUDA toolkit.
+ if (!(MLIR_ENABLE_CUDA_CONVERSIONS)) {
+ getOperation().emitError(
+ "CUDA toolkit not provided when trying to serialize GPU module.");
+ return std::nullopt;
+ }
// Compile to binary.
#if MLIR_ENABLE_NVPTXCOMPILER
return compileToBinaryNVPTX(*serializedISA);
@@ -564,7 +577,7 @@ NVPTXSerializer::moduleToObject(llvm::Module &llvmModule) {
return compileToBinary(*serializedISA);
#endif // MLIR_ENABLE_NVPTXCOMPILER
}
-#endif // MLIR_ENABLE_CUDA_CONVERSIONS
+#endif // LLVM_HAS_NVPTX_TARGET
std::optional<SmallVector<char, 0>>
NVVMTargetAttrImpl::serializeToObject(Attribute attribute, Operation *module,
@@ -576,7 +589,7 @@ NVVMTargetAttrImpl::serializeToObject(Attribute attribute, Operation *module,
module->emitError("Module must be a GPU module.");
return std::nullopt;
}
-#if MLIR_ENABLE_CUDA_CONVERSIONS
+#if LLVM_HAS_NVPTX_TARGET
NVPTXSerializer serializer(*module, cast<NVVMTargetAttr>(attribute), options);
serializer.init();
return serializer.run();
@@ -584,7 +597,7 @@ NVVMTargetAttrImpl::serializeToObject(Attribute attribute, Operation *module,
module->emitError(
"The `NVPTX` target was not built. Please enable it when building LLVM.");
return std::nullopt;
-#endif // MLIR_ENABLE_CUDA_CONVERSIONS
+#endif // LLVM_HAS_NVPTX_TARGET
}
Attribute
diff --git a/mlir/unittests/Target/LLVM/SerializeNVVMTarget.cpp b/mlir/unittests/Target/LLVM/SerializeNVVMTarget.cpp
index cea49356538f0..a8fe20d52fb2a 100644
--- a/mlir/unittests/Target/LLVM/SerializeNVVMTarget.cpp
+++ b/mlir/unittests/Target/LLVM/SerializeNVVMTarget.cpp
@@ -30,7 +30,7 @@
using namespace mlir;
// Skip the test if the NVPTX target was not built.
-#if MLIR_ENABLE_CUDA_CONVERSIONS
+#if LLVM_HAS_NVPTX_TARGET
#define SKIP_WITHOUT_NVPTX(x) x
#else
#define SKIP_WITHOUT_NVPTX(x) DISABLED_##x
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks reasonable, IIRC, the design of the new GPU pipeline was intended to avoid a compile-time dependency on CUDA libraries.
Does the AMD backend suffer from the same issue?
mlir/lib/Target/LLVM/NVVM/Target.cpp
Outdated
// requires CUDA toolkit. | ||
if (!(MLIR_ENABLE_CUDA_CONVERSIONS)) { | ||
getOperation().emitError( | ||
"CUDA toolkit not provided when trying to serialize GPU module."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"CUDA toolkit not provided when trying to serialize GPU module."); | |
"CUDA toolkit not provided when trying to serialize GPU module"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, is it possible to add a test?
mlir/lib/Target/LLVM/NVVM/Target.cpp
Outdated
// requires CUDA toolkit. | ||
if (!(MLIR_ENABLE_CUDA_CONVERSIONS)) { | ||
getOperation().emitError( | ||
"CUDA toolkit not provided when trying to serialize GPU module."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we test that path?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They're already decoupled, MLIR_ENABLE_CUDA_CONVERSIONS
is an alias for checking whether the NVPTX
target was built, see:
https://github.com/llvm/llvm-project/blob/main/mlir/CMakeLists.txt#L115C1-L119
Also, moduleToObject
will succeed if NVPTX
was built and the compilation target is offload
or ptx
, I know this because I use it frequently in a machine without the toolkit.
I'm ok with changing the name of the flag as it might be confusing.
Yes, one of the changes introduced here is just replacing Does this make sense? |
Not really. Because you're not changing the definition of If the CUDA toolkit is not installed and try to serialize to binary you'll get:
And that's by design, as it allows to ship the functionality of compilation without requiring a hard dependency on the toolkit. This is also what |
Yes I'd wanted to change the definition of
Right, I know this situation is handled, but thought it'd be better to guard the whole definition of Note that naming of llvm-project/utils/bazel/llvm-project-overlay/mlir/BUILD.bazel Lines 54 to 56 in cdcd653
So I guess we should only change |
Considering the discussion, the whole PR title and description will need a revamp I believe. |
I've replaced |
I don't see a change in mlir-config.h ? Can you grep for MLIR_ENABLE_CUDA_CONVERSIONS and ensure it does not remain any occurrence in the codebase. |
Indeed, I messed up during rebase somehow. Thanks! |
@fabianmcg is it better now? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thank you! Just make sure that:
- all integration tests pass
ninja check-mlir
succeeds with and without the NVPTX being built.- fix the description of the commit.
8e70f20
to
2ba1365
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks even cleaner, thanks! But I suspect there is something off with how the macro is interpreted, it should be an always-defined 0/1 macro.
Note that Bazel is an optional build system for LLVM, you don't necessarily have to update it.
@@ -47,7 +47,7 @@ add_mlir_dialect_library(MLIRNVVMTarget | |||
MLIRNVVMToLLVMIRTranslation | |||
) | |||
|
|||
if(MLIR_ENABLE_CUDA_CONVERSIONS) | |||
if ("NVPTX" IN_LIST LLVM_TARGETS_TO_BUILD) | |||
# Find the CUDA toolkit. | |||
find_package(CUDAToolkit) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add an INFO message in the else()
branch below saying "CUDA not found" rather than always saying "MLIR default CUDA toolkit path"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is marked resolved but I don't see the change?
Thanks, your comment pushed me to find a better solution, and I'm glad there's a nice one without having to add quotes and stuff. Nice! |
LLVM_HAS_NVPTX_TARGET is automatically set depending on whether NVPTX was enabled when building LLVM. Use this instead of manually defining MLIR_ENABLE_CUDA_CONVERSIONS (whose name is a bit misleading btw).