Skip to content

[AsyncAlloc][CUDA] Don't set pool maxSize if CUDA version < 12.2 #17733

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 31, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions unified-runtime/source/adapters/cuda/usm.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -431,13 +431,24 @@ ur_usm_pool_handle_t_::ur_usm_pool_handle_t_(ur_context_handle_t Context,
case UR_STRUCTURE_TYPE_USM_POOL_LIMITS_DESC: {
const ur_usm_pool_limits_desc_t *Limits =
reinterpret_cast<const ur_usm_pool_limits_desc_t *>(BaseDesc);
#if CUDA_VERSION >= 12020
// maxSize as a member of CUmemPoolProps was introduced in CUDA 12.2.
MemPoolProps.maxSize =
Limits->maxPoolableSize; // CUDA lazily reserves memory for pools in
// 32MB chunks. maxSize is elevated to the
// next 32MB multiple. Each 32MB chunk is
// only reserved when it's needed for the
// first time (cuMemAllocFromPoolAsync).

#else
// Only error if the user set a value >0 for the maximum size.
// Otherwise, do nothing.
if (Limits->maxPoolableSize > 0) {
setErrorMessage(
"The memory pool maximum size feature requires CUDA 12.2 or later.",
UR_RESULT_ERROR_ADAPTER_SPECIFIC);
throw UsmAllocationException(UR_RESULT_ERROR_ADAPTER_SPECIFIC);
}
#endif
maxSize = Limits->maxPoolableSize;
size_t chunkSize = 33554432; // 32MB
size_t remainder = Limits->maxPoolableSize % chunkSize;
Expand Down Expand Up @@ -485,8 +496,8 @@ urUSMPoolCreateExp(ur_context_handle_t Context, ur_device_handle_t Device,
try {
*pPool = reinterpret_cast<ur_usm_pool_handle_t>(
new ur_usm_pool_handle_t_(Context, Device, pPoolDesc));
} catch (ur_result_t err) {
return err;
} catch (const UsmAllocationException &Ex) {
return Ex.getError();
} catch (...) {
return UR_RESULT_ERROR_UNKNOWN;
}
Expand Down