Skip to content

Add sycl ext intel kernel queries extension #16834

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Mar 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
5434020
Add extension doc draft
kurapov-peter Jan 29, 2025
ae3d216
Specify UR version to inlude upcoming changes
kurapov-peter Jan 29, 2025
94dea19
Add a draft implementation for the extension
kurapov-peter Jan 29, 2025
cd0b7be
Update specification
gmlueck Jan 31, 2025
20e364c
Change names to spell out memory
kurapov-peter Feb 3, 2025
519e2b1
Change the return type to size_t
kurapov-peter Feb 3, 2025
1d2f7bc
Move spill_memory_size to kernel_device_specific
kurapov-peter Feb 3, 2025
2029e2b
Update UR tag
kurapov-peter Feb 4, 2025
51b9c0c
fixup! Update UR tag
kurapov-peter Feb 4, 2025
4f4e4db
fix formatting
kurapov-peter Feb 4, 2025
ea46e7c
Remove the bogus metadata test
kurapov-peter Feb 6, 2025
03ba27d
Check for the aspect presence before testing the kernel info
kurapov-peter Feb 6, 2025
58d392d
Fix undef variable
kurapov-peter Feb 12, 2025
b0f2b27
Add get_info declaration for the new device-specific traits
kurapov-peter Feb 12, 2025
36cf3bd
Add the exported symbol to abi checks
kurapov-peter Feb 13, 2025
3cf5c72
Add the exported symbol on windows
kurapov-peter Feb 24, 2025
79f35f7
[UR] Return spills for all ZE kernels in UR kernel via v1 level_zero …
kurapov-peter Feb 24, 2025
f47ba21
fix formatting
kurapov-peter Feb 24, 2025
7e7a422
Merge branch 'sycl' into sycl_ext_intel_kernel_queries
kurapov-peter Feb 26, 2025
07a3b7c
Fix merge conflict error
kurapov-peter Feb 26, 2025
597e5f0
Remove an empty line
kurapov-peter Feb 28, 2025
106b1e2
Inline UR_KERNEL_INFO_SPILL_MEM_SIZE
kurapov-peter Feb 28, 2025
34f24f3
Rename Result to Device2SpillMap
kurapov-peter Feb 28, 2025
13ff634
Update sycl/source/detail/kernel_info.hpp
kurapov-peter Feb 28, 2025
fd68f4d
Format the updated comment
kurapov-peter Feb 28, 2025
3443815
Format the renaming
kurapov-peter Feb 28, 2025
7402507
Merge remote-tracking branch 'upstream/sycl' into sycl_ext_intel_kern…
kurapov-peter Mar 5, 2025
d918044
Merge remote-tracking branch 'upstream/sycl' into sycl_ext_intel_kern…
kurapov-peter Mar 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion llvm/include/llvm/SYCLLowerIR/DeviceConfigFile.td
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ def AspectExt_oneapi_cuda_cluster_group : Aspect<"ext_oneapi_cuda_cluster_group"
def AspectExt_intel_fpga_task_sequence : Aspect<"ext_intel_fpga_task_sequence">;
def AspectExt_oneapi_atomic16 : Aspect<"ext_oneapi_atomic16">;
def AspectExt_oneapi_virtual_functions : Aspect<"ext_oneapi_virtual_functions">;
def AspectExt_intel_spill_memory_size : Aspect<"ext_intel_spill_memory_size">;
// Deprecated aspects
def AspectInt64_base_atomics : Aspect<"int64_base_atomics">;
def AspectInt64_extended_atomics : Aspect<"int64_extended_atomics">;
Expand Down Expand Up @@ -150,7 +151,8 @@ def : TargetInfo<"__TestAspectList",
AspectExt_oneapi_queue_profiling_tag, AspectExt_oneapi_virtual_mem, AspectExt_oneapi_cuda_cluster_group,
AspectExt_intel_fpga_task_sequence,
AspectExt_oneapi_atomic16,
AspectExt_oneapi_virtual_functions],
AspectExt_oneapi_virtual_functions,
AspectExt_intel_spill_memory_size],
[]>;
// This definition serves the only purpose of testing whether the deprecated aspect list defined in here and in SYCL RT
// match.
Expand Down
139 changes: 139 additions & 0 deletions sycl/doc/extensions/supported/sycl_ext_intel_kernel_queries.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
= sycl_ext_intel_kernel_queries

:source-highlighter: coderay
:coderay-linenums-mode: table

// This section needs to be after the document title.
:doctype: book
:toc2:
:toc: left
:encoding: utf-8
:lang: en
:dpcpp: pass:[DPC++]
:endnote: &#8212;{nbsp}end{nbsp}note

// Set the default source code type in this document to C++,
// for syntax highlighting purposes. This is needed because
// docbook uses c++ and html5 uses cpp.
:language: {basebackend@docbook:c++:cpp}


== Notice

[%hardbreaks]
Copyright (C) 2025 Intel Corporation. All rights reserved.

Khronos(R) is a registered trademark and SYCL(TM) and SPIR(TM) are trademarks
of The Khronos Group Inc. OpenCL(TM) is a trademark of Apple Inc. used by
permission by Khronos.


== Contact

To report problems with this extension, please open a new issue at:

https://github.com/intel/llvm/issues


== Dependencies

This extension is written against the SYCL 2020 revision 9 specification.
All references below to the "core SYCL specification" or to section numbers in
the SYCL specification refer to that revision.


== Status

This extension is implemented and fully supported by {dpcpp}.


== Overview

This extension contains a collection of queries that provide low-level
information about kernels.
These queries generally forward directly to the backend and expose concepts that
are specific to a particular implementation.
As a result, these queries may not be supported for all devices.
Each query has an associate device aspect, which tells whether the query is
supported on that device.


== Specification

=== Feature test macro

This extension provides a feature-test macro as described in the core SYCL
specification.
An implementation supporting this extension must predefine the macro
`SYCL_EXT_INTEL_KERNEL_QUERIES` to one of the values defined in the table below.
Applications can test for the existence of this macro to determine if the
implementation supports this feature, or applications can test the macro's value
to determine which of the extension's features the implementation supports.

[%header,cols="1,5"]
|===
|Value
|Description

|1
|Initial version of this extension.
|===

=== Spill memory size

This query returns the kernel's spill memory size that is allocated by the
compiler, as reported by Level Zero.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is L0 important here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally, we would not define a SYCL API in terms of a particular backend like this. However, it seemed to me that "spill memory size" is too vaguely defined. The reality is that SYCL is just returning whatever value comes from Level Zero. If people have questions about what it means, I'd rather direct them to Level Zero.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair enough


==== New device aspect

This extension adds the following new device aspect.

[source,c++]
----
namespace sycl {

enum class aspect {
ext_intel_spill_memory_size

// ...
};

}
----

'''

`*ext_intel_spill_memory_size*`

Indicates that the `spill_memory_size` kernel information descriptor may be used
to query kernels for this device.

'''

==== New device specific kernel information descriptor

This extension adds the following information descriptor that can be used with
`kernel::get_info(const device&)`.

'''

`*ext::intel::info::kernel_device_specific::spill_memory_size*`

[source,c++]
----
namespace sycl::ext::intel::info::kernel_device_specific {
struct spill_memory_size {
using return_type = size_t;
};
} // namespace sycl::ext::intel::info::kernel_device_specific
----

_Remarks:_ Template parameter to `kernel::get_info(const device&)`.

_Returns:_ The spill memory size that is allocated by the compiler for this
kernel for the given device.

_Throws:_ An `exception` with the `errc::feature_not_supported` error code if
the device does not have `aspect::ext_intel_spill_memory_size`.

'''
9 changes: 9 additions & 0 deletions sycl/include/sycl/detail/info_desc_helpers.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,15 @@ struct IsKernelInfo<info::kernel_device_specific::ext_codeplay_num_regs>
#include <sycl/info/kernel_device_specific_traits.def>
#undef __SYCL_PARAM_TRAITS_SPEC

#define __SYCL_PARAM_TRAITS_SPEC(Namespace, DescType, Desc, ReturnT, UrCode) \
template <> \
struct is_##DescType##_info_desc<Namespace::info::DescType::Desc> \
: std::true_type { \
using return_type = Namespace::info::DescType::Desc::return_type; \
};
#include <sycl/info/ext_intel_kernel_info_traits.def>
#undef __SYCL_PARAM_TRAITS_SPEC

#define __SYCL_PARAM_TRAITS_SPEC(DescType, Desc, ReturnT, UrCode) \
template <> \
struct is_##DescType##_info_desc<info::DescType::Desc> : std::true_type { \
Expand Down
1 change: 1 addition & 0 deletions sycl/include/sycl/info/aspects.def
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,4 @@ __SYCL_ASPECT(ext_oneapi_bindless_images_sample_1d_usm, 78)
__SYCL_ASPECT(ext_oneapi_bindless_images_sample_2d_usm, 79)
__SYCL_ASPECT(ext_oneapi_atomic16, 80)
__SYCL_ASPECT(ext_oneapi_virtual_functions, 81)
__SYCL_ASPECT(ext_intel_spill_memory_size, 82)
1 change: 1 addition & 0 deletions sycl/include/sycl/info/ext_intel_kernel_info_traits.def
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__SYCL_PARAM_TRAITS_SPEC(ext::intel, kernel_device_specific, spill_memory_size, size_t, UR_KERNEL_INFO_SPILL_MEM_SIZE)
1 change: 1 addition & 0 deletions sycl/include/sycl/info/info_desc.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,7 @@ struct work_item_progress_capabilities;
} // namespace ext::oneapi::experimental::info::device
#include <sycl/info/ext_codeplay_device_traits.def>
#include <sycl/info/ext_intel_device_traits.def>
#include <sycl/info/ext_intel_kernel_info_traits.def>
#include <sycl/info/ext_oneapi_device_traits.def>
#include <sycl/info/ext_oneapi_kernel_queue_specific_traits.def>

Expand Down
5 changes: 5 additions & 0 deletions sycl/source/detail/device_impl.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -786,6 +786,11 @@ bool device_impl::has(aspect Aspect) const {
BE == sycl::backend::opencl;
return (is_cpu() || is_gpu()) && isCompatibleBE;
}
case aspect::ext_intel_spill_memory_size: {
backend BE = getBackend();
bool isCompatibleBE = BE == sycl::backend::ext_oneapi_level_zero;
return is_gpu() && isCompatibleBE;
}
}

return false; // This device aspect has not been implemented yet.
Expand Down
17 changes: 17 additions & 0 deletions sycl/source/detail/kernel_impl.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -416,6 +416,23 @@ inline typename syclex::info::kernel_queue_specific::max_num_work_groups::
DynamicLocalMemorySize);
}

template <>
inline typename ext::intel::info::kernel_device_specific::spill_memory_size::
return_type
kernel_impl::get_info<
ext::intel::info::kernel_device_specific::spill_memory_size>(
const device &Device) const {
if (!Device.has(aspect::ext_intel_spill_memory_size))
throw exception(
make_error_code(errc::feature_not_supported),
"This device does not have the ext_intel_spill_memory_size aspect");

return get_kernel_device_specific_info<
ext::intel::info::kernel_device_specific::spill_memory_size>(
this->getHandleRef(), getSyclObjImpl(Device)->getHandleRef(),
getAdapter());
}

template <>
inline typename syclex::info::kernel_queue_specific::max_work_group_size::
return_type
Expand Down
52 changes: 52 additions & 0 deletions sycl/source/detail/kernel_info.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,58 @@ uint32_t get_kernel_device_specific_info_with_input(ur_kernel_handle_t Kernel,

return Result;
}

template <>
inline ext::intel::info::kernel_device_specific::spill_memory_size::return_type
get_kernel_device_specific_info<
ext::intel::info::kernel_device_specific::spill_memory_size>(
ur_kernel_handle_t Kernel, ur_device_handle_t Device,
const AdapterPtr &Adapter) {
size_t ResultSize = 0;

// First call to get the number of device images
Adapter->call<UrApiKind::urKernelGetInfo>(
Kernel, UR_KERNEL_INFO_SPILL_MEM_SIZE, 0, nullptr, &ResultSize);

size_t DeviceCount = ResultSize / sizeof(uint32_t);

// Second call to retrieve the data
std::vector<uint32_t> Device2SpillMap(DeviceCount);
Adapter->call<UrApiKind::urKernelGetInfo>(
Kernel, UR_KERNEL_INFO_SPILL_MEM_SIZE, ResultSize, Device2SpillMap.data(),
nullptr);

ur_program_handle_t Program;
Adapter->call<UrApiKind::urKernelGetInfo>(Kernel, UR_KERNEL_INFO_PROGRAM,
sizeof(ur_program_handle_t),
&Program, nullptr);
// Retrieve the associated device list
size_t URDevicesSize = 0;
Adapter->call<UrApiKind::urProgramGetInfo>(Program, UR_PROGRAM_INFO_DEVICES,
0, nullptr, &URDevicesSize);

std::vector<ur_device_handle_t> URDevices(URDevicesSize /
sizeof(ur_device_handle_t));
Adapter->call<UrApiKind::urProgramGetInfo>(Program, UR_PROGRAM_INFO_DEVICES,
URDevicesSize, URDevices.data(),
nullptr);
assert(Device2SpillMap.size() == URDevices.size());

// Map the result back to the program devices. UR provides the following
// guarantee:
// The order of the devices is guaranteed (i.e., the same as queried by
// urDeviceGet) by the UR within a single application even if the runtime is
// reinitialized.
for (size_t idx = 0; idx < URDevices.size(); ++idx) {
if (URDevices[idx] == Device)
return size_t{Device2SpillMap[idx]};
}
throw exception(
make_error_code(errc::runtime),
"ext::intel::info::kernel::spill_memory_size failed to retrieve "
"the requested value");
}

} // namespace detail
} // namespace _V1
} // namespace sycl
9 changes: 9 additions & 0 deletions sycl/source/detail/ur_info_code.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,15 @@ template <typename T> struct UrInfoCode;
#include <sycl/info/ext_oneapi_device_traits.def>
#undef __SYCL_PARAM_TRAITS_SPEC

#define __SYCL_PARAM_TRAITS_SPEC(Namespace, DescType, Desc, ReturnT, UrCode) \
template <> struct UrInfoCode<Namespace::info::DescType::Desc> { \
static constexpr ur_kernel_info_t value = \
static_cast<ur_kernel_info_t>(UrCode); \
};

#include <sycl/info/ext_intel_kernel_info_traits.def>
#undef __SYCL_PARAM_TRAITS_SPEC

} // namespace detail
} // namespace _V1
} // namespace sycl
8 changes: 8 additions & 0 deletions sycl/source/kernel.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,14 @@ kernel::get_info(const device &Device, const range<3> &WGSize) const {

#undef __SYCL_PARAM_TRAITS_SPEC

#define __SYCL_PARAM_TRAITS_SPEC(Namespace, DescType, Desc, ReturnT, UrCode) \
template __SYCL_EXPORT ReturnT \
kernel::get_info<Namespace::info::DescType::Desc>(const device &) const;

#include <sycl/info/ext_intel_kernel_info_traits.def>

#undef __SYCL_PARAM_TRAITS_SPEC

template __SYCL_EXPORT uint32_t
kernel::get_info<info::kernel_device_specific::max_sub_group_size>(
const device &, const sycl::range<3> &) const;
Expand Down
3 changes: 3 additions & 0 deletions sycl/test-e2e/Basic/aspects.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,9 @@ int main() {
if (plt.has(aspect::ext_oneapi_virtual_functions)) {
std::cout << " ext_oneapi_virtual_functions" << std::endl;
}
if (plt.has(aspect::ext_intel_spill_memory_size)) {
std::cout << " ext_intel_spill_memory_size" << std::endl;
}
}
std::cout << "Passed." << std::endl;
return 0;
Expand Down
13 changes: 13 additions & 0 deletions sycl/test-e2e/Basic/kernel_info.cpp
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There might be a better way to test the thing. This would just return zero. Any suggestions?

Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,12 @@ int main() {
krn.get_info<info::kernel_device_specific::compile_num_sub_groups>(dev);
assert(compileNumSg <= maxNumSg);

size_t spillMemSz = 0;
if (dev.has(aspect::ext_intel_spill_memory_size)) {
spillMemSz = krn.get_info<
ext::intel::info::kernel_device_specific::spill_memory_size>(dev);
}

// Use ext_oneapi_get_kernel_info extension and check that answers match.
const size_t wgSizeExt = syclex::get_kernel_info<
SingleTask, info::kernel_device_specific::work_group_size>(ctx, dev);
Expand All @@ -125,6 +131,13 @@ int main() {
dev);
assert(compileNumSgExt == compileNumSg);

if (dev.has(aspect::ext_intel_spill_memory_size)) {
const size_t spillMemSizeExt = syclex::get_kernel_info<
SingleTask,
ext::intel::info::kernel_device_specific::spill_memory_size>(ctx, dev);
assert(spillMemSizeExt == spillMemSz);
}

// Use ext_oneapi_get_kernel_info extension with queue parameter and check the
// result.
const size_t wgSizeExtQ =
Expand Down
1 change: 1 addition & 0 deletions sycl/test/abi/sycl_symbols_linux.dump
Original file line number Diff line number Diff line change
Expand Up @@ -3976,6 +3976,7 @@ _ZNK4sycl3_V16kernel19ext_oneapi_get_infoINS0_3ext6oneapi12experimental4info21ke
_ZNK4sycl3_V16kernel19ext_oneapi_get_infoINS0_3ext6oneapi12experimental4info21kernel_queue_specific14num_sub_groupsEEENS0_6detail34is_kernel_queue_specific_info_descIT_E11return_typeENS0_5queueERKNS0_5rangeILi2EEE
_ZNK4sycl3_V16kernel19ext_oneapi_get_infoINS0_3ext6oneapi12experimental4info21kernel_queue_specific18max_sub_group_sizeEEENS0_6detail34is_kernel_queue_specific_info_descIT_E11return_typeENS0_5queueERKNS0_5rangeILi3EEE
_ZNK4sycl3_V16kernel3getEv
_ZNK4sycl3_V16kernel8get_infoINS0_3ext5intel4info22kernel_device_specific17spill_memory_sizeEEENS0_6detail35is_kernel_device_specific_info_descIT_E11return_typeERKNS0_6deviceE
_ZNK4sycl3_V16kernel8get_infoINS0_4info22kernel_device_specific15work_group_sizeEEENS0_6detail35is_kernel_device_specific_info_descIT_E11return_typeERKNS0_6deviceE
_ZNK4sycl3_V16kernel8get_infoINS0_4info22kernel_device_specific16global_work_sizeEEENS0_6detail35is_kernel_device_specific_info_descIT_E11return_typeERKNS0_6deviceE
_ZNK4sycl3_V16kernel8get_infoINS0_4info22kernel_device_specific16private_mem_sizeEEENS0_6detail35is_kernel_device_specific_info_descIT_E11return_typeERKNS0_6deviceE
Expand Down
1 change: 1 addition & 0 deletions sycl/test/abi/sycl_symbols_windows.dump
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@
??$get_info@Ureference_count@context@info@_V1@sycl@@@context@_V1@sycl@@QEBAIXZ
??$get_info@Ureference_count@event@info@_V1@sycl@@@event@_V1@sycl@@QEBAIXZ
??$get_info@Ureference_count@queue@info@_V1@sycl@@@queue@_V1@sycl@@QEBAIXZ
??$get_info@Uspill_memory_size@kernel_device_specific@info@intel@ext@_V1@sycl@@@kernel@_V1@sycl@@QEBA_KAEBVdevice@12@@Z
??$get_info@Usupports_fusion@device@info@experimental@codeplay@ext@_V1@sycl@@@device_impl@detail@_V1@sycl@@QEBA_NXZ
??$get_info@Uuuid@device@info@intel@ext@_V1@sycl@@@device_impl@detail@_V1@sycl@@QEBA?AV?$array@E$0BA@@std@@XZ
??$get_info@Uwork_group_size@kernel_device_specific@info@_V1@sycl@@@kernel@_V1@sycl@@QEBA_KAEBVdevice@12@@Z
Expand Down
1 change: 1 addition & 0 deletions sycl/test/include_deps/sycl_accessor.hpp.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@
// CHECK-NEXT: info/event_profiling_traits.def
// CHECK-NEXT: info/ext_codeplay_device_traits.def
// CHECK-NEXT: info/ext_intel_device_traits.def
// CHECK-NEXT: info/ext_intel_kernel_info_traits.def
// CHECK-NEXT: info/ext_oneapi_device_traits.def
// CHECK-NEXT: info/ext_oneapi_kernel_queue_specific_traits.def
// CHECK-NEXT: info/sycl_backend_traits.def
Expand Down
1 change: 1 addition & 0 deletions sycl/test/include_deps/sycl_detail_core.hpp.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@
// CHECK-NEXT: info/event_profiling_traits.def
// CHECK-NEXT: info/ext_codeplay_device_traits.def
// CHECK-NEXT: info/ext_intel_device_traits.def
// CHECK-NEXT: info/ext_intel_kernel_info_traits.def
// CHECK-NEXT: info/ext_oneapi_device_traits.def
// CHECK-NEXT: info/ext_oneapi_kernel_queue_specific_traits.def
// CHECK-NEXT: info/sycl_backend_traits.def
Expand Down
Loading