[SYCL] Add Level-Zero interop with specification of ownership for Queue. (#4066)

The L0 backend interop functions take the ownership of the passed L0 handles. This is problematic for some cases as the users can not use the original handle when they go out of scope and destroyed.
This PR provides a way for users of SYCL - L0 interop to retain the ownership of the L0 handles for the Queue.

Signed-off-by: rehana begam <[email protected]>

Author: rbegam

sycl/doc/extensions/LevelZeroBackend/LevelZeroBackend.md

@@ -88,7 +88,7 @@ a SYCL object that encapsulates a corresponding Level-Zero object:
 |``` make<platform>(ze_driver_handle_t);```|Constructs a SYCL platform instance from a Level-Zero ```ze_driver_handle_t```.|
 |``` make<device>(const platform &, ze_device_handle_t);```|Constructs a SYCL device instance from a Level-Zero ```ze_device_handle_t```. The platform argument gives a SYCL platform, encapsulating a Level-Zero driver supporting the passed Level-Zero device.|
 |``` make<context>(const vector_class<device> &, ze_context_handle_t, ownership = transfer);```| Constructs a SYCL context instance from a Level-Zero ```ze_context_handle_t```. The context is created against the devices passed in. There must be at least one device given and all the devices must be from the same SYCL platform and thus from the same Level-Zero driver. The ```ownership``` argument specifies if the SYCL runtime should take ownership of the passed native handle. The default behavior is to transfer the ownership to the SYCL runtime. See section 4.4 for details.|
-|``` make<queue>(const context &, ze_command_queue_handle_t);```| Constructs a SYCL queue instance from a Level-Zero ```ze_command_queue_handle_t```. The context argument must be a valid SYCL context encapsulating a Level-Zero context. The queue is attached to the first device in the passed SYCL context.|
+|``` make<queue>(const context &, ze_command_queue_handle_t, ownership = transfer);```| Constructs a SYCL queue instance from a Level-Zero ```ze_command_queue_handle_t```. The context argument must be a valid SYCL context encapsulating a Level-Zero context. The queue is attached to the first device in the passed SYCL context. The ```ownership``` argument specifies if the SYCL runtime should take ownership of the passed native handle. The default behavior is to transfer the ownership to the SYCL runtime. See section 4.4 for details.|
 |``` make<program>(const context &, ze_module_handle_t);```| Constructs a SYCL program instance from a Level-Zero ```ze_module_handle_t```. The context argument must be a valid SYCL context encapsulating a Level-Zero context. The Level-Zero module must be fully linked (i.e. not require further linking through [```zeModuleDynamicLink```](https://spec.oneapi.com/level-zero/latest/core/api.html?highlight=zemoduledynamiclink#_CPPv419zeModuleDynamicLink8uint32_tP18ze_module_handle_tP28ze_module_build_log_handle_t)), and thus the SYCL program is created in the "linked" state.|
 
 NOTE: We shall consider adding other interoperability as needed, if possible.
@@ -189,4 +189,4 @@ struct free_memory {
 |1|2021-01-26|Sergey Maslov|Initial public working draft
 |2|2021-02-22|Sergey Maslov|Introduced explicit ownership for context
 |3|2021-04-13|James Brodman|Free Memory Query
-
+|4|2021-07-06|Rehana Begam|Introduced explicit ownership for queue

sycl/include/CL/sycl/backend.hpp

@@ -90,6 +90,9 @@ __SYCL_EXPORT device make_device(pi_native_handle NativeHandle,
 __SYCL_EXPORT context make_context(pi_native_handle NativeHandle,
                                    const async_handler &Handler,
                                    backend Backend);
+__SYCL_EXPORT queue make_queue(pi_native_handle NativeHandle,
+                               const context &TargetContext, bool KeepOwnership,
+                               const async_handler &Handler, backend Backend);
 __SYCL_EXPORT queue make_queue(pi_native_handle NativeHandle,
                                const context &TargetContext,
                                const async_handler &Handler, backend Backend);
@@ -139,9 +142,10 @@ typename std::enable_if<
     detail::InteropFeatureSupportMap<Backend>::MakeQueue == true, queue>::type
 make_queue(const typename backend_traits<Backend>::template input_type<queue>
                &BackendObject,
-           const context &TargetContext, const async_handler Handler = {}) {
+           const context &TargetContext, bool KeepOwnership,
+           const async_handler Handler = {}) {
   return detail::make_queue(detail::pi::cast<pi_native_handle>(BackendObject),
-                            TargetContext, Handler, Backend);
+                            TargetContext, KeepOwnership, Handler, Backend);
 }
 
 template <backend Backend>

sycl/include/CL/sycl/backend/level_zero.hpp

@@ -92,7 +92,8 @@ __SYCL_EXPORT context make_context(const std::vector<device> &DeviceList,
 __SYCL_EXPORT program make_program(const context &Context,
                                    pi_native_handle NativeHandle);
 __SYCL_EXPORT queue make_queue(const context &Context,
-                               pi_native_handle InteropHandle);
+                               pi_native_handle InteropHandle,
+                               bool keep_ownership = false);
 
 // Construction of SYCL platform.
 template <typename T, typename detail::enable_if_t<
@@ -139,8 +140,10 @@ T make(const context &Context,
 template <typename T, typename detail::enable_if_t<
                           std::is_same<T, queue>::value> * = nullptr>
 T make(const context &Context,
-       typename interop<backend::level_zero, T>::type Interop) {
-  return make_queue(Context, reinterpret_cast<pi_native_handle>(Interop));
+       typename interop<backend::level_zero, T>::type Interop,
+       ownership Ownership = ownership::transfer) {
+  return make_queue(Context, reinterpret_cast<pi_native_handle>(Interop),
+                    Ownership == ownership::keep);
 }
 
 } // namespace level_zero

sycl/include/CL/sycl/detail/pi.h

@@ -36,10 +36,12 @@
 // 2. A number of types needed to define pi_device_binary_property_set added.
 // 3. Added new ownership argument to piextContextCreateWithNativeHandle.
 // 4. Add interoperability interfaces for kernel.
+// 4.6 Added new ownership argument to piextQueueCreateWithNativeHandle which
+// changes the API version from 3.5 to 4.6.
 //
 #include "CL/cl.h"
-#define _PI_H_VERSION_MAJOR 3
-#define _PI_H_VERSION_MINOR 5
+#define _PI_H_VERSION_MAJOR 4
+#define _PI_H_VERSION_MINOR 6
 
 #define _PI_STRING_HELPER(a) #a
 #define _PI_CONCAT(a, b) _PI_STRING_HELPER(a.b)
@@ -1053,8 +1055,11 @@ piextQueueGetNativeHandle(pi_queue queue, pi_native_handle *nativeHandle);
 /// \param nativeHandle is the native handle to create PI queue from.
 /// \param context is the PI context of the queue.
 /// \param queue is the PI queue created from the native handle.
+/// \param ownNativeHandle tells if SYCL RT should assume the ownership of
+///        the native handle, if it can.
 __SYCL_EXPORT pi_result piextQueueCreateWithNativeHandle(
-    pi_native_handle nativeHandle, pi_context context, pi_queue *queue);
+    pi_native_handle nativeHandle, pi_context context, pi_queue *queue,
+    bool ownNativeHandle);
 
 //
 // Memory

sycl/plugins/cuda/pi_cuda.cpp

@@ -2193,10 +2193,14 @@ pi_result cuda_piextQueueGetNativeHandle(pi_queue queue,
 /// \param[in] nativeHandle The native handle to create PI queue object from.
 /// \param[in] context is the PI context of the queue.
 /// \param[out] queue Set to the PI queue object created from native handle.
+/// \param ownNativeHandle tells if SYCL RT should assume the ownership of
+///        the native handle, if it can.
 ///
 /// \return TBD
 pi_result cuda_piextQueueCreateWithNativeHandle(pi_native_handle, pi_context,
-                                                pi_queue *) {
+                                                pi_queue *,
+                                                bool ownNativeHandle) {
+  (void)ownNativeHandle;
   cl::sycl::detail::pi::die(
       "Creation of PI queue from native handle not implemented");
   return {};

sycl/plugins/esimd_cpu/pi_esimd_cpu.cpp

@@ -675,7 +675,7 @@ pi_result piextQueueGetNativeHandle(pi_queue, pi_native_handle *) {
 }
 
 pi_result piextQueueCreateWithNativeHandle(pi_native_handle, pi_context,
-                                           pi_queue *) {
+                                           pi_queue *, bool) {
   DIE_NO_IMPLEMENTATION;
   return PI_SUCCESS;
 }

sycl/plugins/level_zero/pi_level_zero.cpp

@@ -2423,7 +2423,7 @@ pi_result piQueueCreate(pi_context Context, pi_device Device,
 
   try {
     *Queue = new _pi_queue(ZeComputeCommandQueue, ZeCopyCommandQueue, Context,
-                           Device, ZeCommandListBatchSize, Properties);
+                           Device, ZeCommandListBatchSize, true, Properties);
   } catch (const std::bad_alloc &) {
     return PI_OUT_OF_HOST_MEMORY;
   } catch (...) {
@@ -2506,10 +2506,16 @@ pi_result piQueueRelease(pi_queue Queue) {
         ZE_CALL(zeFenceDestroy, (MapEntry.second.ZeFence));
       }
       Queue->ZeCommandListFenceMap.clear();
-      ZE_CALL(zeCommandQueueDestroy, (Queue->ZeComputeCommandQueue));
+
+      if (Queue->OwnZeCommandQueue) {
+        ZE_CALL(zeCommandQueueDestroy, (Queue->ZeComputeCommandQueue));
+        if (Queue->ZeCopyCommandQueue) {
+          ZE_CALL(zeCommandQueueDestroy, (Queue->ZeCopyCommandQueue));
+        }
+      }
+
       Queue->ZeComputeCommandQueue = nullptr;
       if (Queue->ZeCopyCommandQueue) {
-        ZE_CALL(zeCommandQueueDestroy, (Queue->ZeCopyCommandQueue));
         Queue->ZeCopyCommandQueue = nullptr;
       }
 
@@ -2557,8 +2563,8 @@ pi_result piextQueueGetNativeHandle(pi_queue Queue,
 }
 
 pi_result piextQueueCreateWithNativeHandle(pi_native_handle NativeHandle,
-                                           pi_context Context,
-                                           pi_queue *Queue) {
+                                           pi_context Context, pi_queue *Queue,
+                                           bool OwnNativeHandle) {
   PI_ASSERT(Context, PI_INVALID_CONTEXT);
   PI_ASSERT(NativeHandle, PI_INVALID_VALUE);
   PI_ASSERT(Queue, PI_INVALID_QUEUE);
@@ -2570,8 +2576,8 @@ pi_result piextQueueCreateWithNativeHandle(pi_native_handle NativeHandle,
   pi_device Device = Context->Devices[0];
   // TODO: see what we can do to correctly initialize PI queue for
   // compute vs. copy Level-Zero queue.
-  *Queue =
-      new _pi_queue(ZeQueue, nullptr, Context, Device, ZeCommandListBatchSize);
+  *Queue = new _pi_queue(ZeQueue, nullptr, Context, Device,
+                         ZeCommandListBatchSize, OwnNativeHandle);
   return PI_SUCCESS;
 }
 

sycl/plugins/level_zero/pi_level_zero.hpp

@@ -482,12 +482,13 @@ const pi_uint32 DynamicBatchStartSize = 4;
 struct _pi_queue : _pi_object {
   _pi_queue(ze_command_queue_handle_t Queue,
             ze_command_queue_handle_t CopyQueue, pi_context Context,
-            pi_device Device, pi_uint32 BatchSize,
+            pi_device Device, pi_uint32 BatchSize, bool OwnZeCommandQueue,
             pi_queue_properties PiQueueProperties = 0)
       : ZeComputeCommandQueue{Queue},
         ZeCopyCommandQueue{CopyQueue}, Context{Context}, Device{Device},
         QueueBatchSize{BatchSize > 0 ? BatchSize : DynamicBatchStartSize},
-        UseDynamicBatching{BatchSize == 0},
+        OwnZeCommandQueue{OwnZeCommandQueue}, UseDynamicBatching{BatchSize ==
+                                                                 0},
         PiQueueProperties(PiQueueProperties) {}
 
   // Level Zero compute command queue handle.
@@ -540,6 +541,10 @@ struct _pi_queue : _pi_object {
   // is thread safe because of the locking of the queue that occurs.
   pi_uint32 QueueBatchSize = {0};
 
+  // Indicates if we own the ZeCommandQueue or it came from interop that
+  // asked to not transfer the ownership to SYCL RT.
+  bool OwnZeCommandQueue;
+
   // specifies whether this queue will be using dynamic batch size adjustment
   // or not.  This is set only at queue creation time, and is therefore
   // const for the life of the queue.

sycl/plugins/opencl/pi_opencl.cpp

@@ -354,7 +354,9 @@ pi_result piQueueCreate(pi_context context, pi_device device,
 }
 
 pi_result piextQueueCreateWithNativeHandle(pi_native_handle nativeHandle,
-                                           pi_context, pi_queue *piQueue) {
+                                           pi_context, pi_queue *piQueue,
+                                           bool ownNativeHandle) {
+  (void)ownNativeHandle;
   assert(piQueue != nullptr);
   *piQueue = reinterpret_cast<pi_queue>(nativeHandle);
   return PI_SUCCESS;

sycl/source/backend.cpp

@@ -80,12 +80,18 @@ __SYCL_EXPORT context make_context(pi_native_handle NativeHandle,
 __SYCL_EXPORT queue make_queue(pi_native_handle NativeHandle,
                                const context &Context,
                                const async_handler &Handler, backend Backend) {
+  return make_queue(NativeHandle, Context, false, Handler, Backend);
+}
+
+__SYCL_EXPORT queue make_queue(pi_native_handle NativeHandle,
+                               const context &Context, bool KeepOwnership,
+                               const async_handler &Handler, backend Backend) {
   const auto &Plugin = getPlugin(Backend);
   const auto &ContextImpl = getSyclObjImpl(Context);
   // Create PI queue first.
   pi::PiQueue PiQueue = nullptr;
   Plugin.call<PiApiKind::piextQueueCreateWithNativeHandle>(
-      NativeHandle, ContextImpl->getHandleRef(), &PiQueue);
+      NativeHandle, ContextImpl->getHandleRef(), &PiQueue, !KeepOwnership);
   // Construct the SYCL queue from PI queue.
   return detail::createSyclObjFromImpl<queue>(
       std::make_shared<queue_impl>(PiQueue, ContextImpl, Handler));

sycl/source/backend/level_zero.cpp

@@ -79,13 +79,20 @@ __SYCL_EXPORT program make_program(const context &Context,
 //----------------------------------------------------------------------------
 // Implementation of level_zero::make<queue>
 __SYCL_EXPORT queue make_queue(const context &Context,
-                               pi_native_handle NativeHandle) {
+                               pi_native_handle NativeHandle,
+                               bool KeepOwnership) {
   const auto &ContextImpl = getSyclObjImpl(Context);
-  return detail::make_queue(NativeHandle, Context,
+  return detail::make_queue(NativeHandle, Context, KeepOwnership,
                             ContextImpl->get_async_handler(),
                             backend::level_zero);
 }
 
+// TODO: remove this version (without ownership) when allowed to break ABI.
+__SYCL_EXPORT queue make_queue(const context &Context,
+                               pi_native_handle NativeHandle) {
+  return make_queue(Context, NativeHandle, false);
+}
+
 } // namespace level_zero
 } // namespace sycl
 } // __SYCL_INLINE_NAMESPACE(cl)

sycl/source/backend/opencl.cpp

@@ -52,7 +52,7 @@ __SYCL_EXPORT program make_program(const context &Context,
 __SYCL_EXPORT queue make_queue(const context &Context,
                                pi_native_handle NativeHandle) {
   const auto &ContextImpl = getSyclObjImpl(Context);
-  return detail::make_queue(NativeHandle, Context,
+  return detail::make_queue(NativeHandle, Context, false,
                             ContextImpl->get_async_handler(), backend::opencl);
 }
 } // namespace opencl

sycl/source/detail/queue_impl.hpp

@@ -115,7 +115,7 @@ class queue_impl {
         DeviceImplPtr(new device_impl(Device, Context->getPlatformImpl()));
 
     // TODO catch an exception and put it to list of asynchronous exceptions
-    Plugin.call<PiApiKind::piQueueRetain>(MQueues[0]);
+    getPlugin().call<PiApiKind::piQueueRetain>(MQueues[0]);
   }
 
   ~queue_impl() {

sycl/test/abi/sycl_symbols_linux.dump

@@ -3591,6 +3591,7 @@ _ZN2cl10__host_std9u_sub_satEjj
 _ZN2cl10__host_std9u_sub_satEmm
 _ZN2cl10__host_std9u_sub_satEtt
 _ZN2cl4sycl10level_zero10make_queueERKNS0_7contextEm
+_ZN2cl4sycl10level_zero10make_queueERKNS0_7contextEmb
 _ZN2cl4sycl10level_zero11make_deviceERKNS0_8platformEm
 _ZN2cl4sycl10level_zero12make_contextERKSt6vectorINS0_6deviceESaIS3_EEm
 _ZN2cl4sycl10level_zero12make_contextERKSt6vectorINS0_6deviceESaIS3_EEmb
@@ -3744,6 +3745,7 @@ _ZN2cl4sycl6detail10image_implILi3EED1Ev
 _ZN2cl4sycl6detail10image_implILi3EED2Ev
 _ZN2cl4sycl6detail10make_eventEmRKNS0_7contextENS0_7backendE
 _ZN2cl4sycl6detail10make_queueEmRKNS0_7contextERKSt8functionIFvNS0_14exception_listEEENS0_7backendE
+_ZN2cl4sycl6detail10make_queueEmRKNS0_7contextEbRKSt8functionIFvNS0_14exception_listEEENS0_7backendE
 _ZN2cl4sycl6detail10waitEventsESt6vectorINS0_5eventESaIS3_EE
 _ZN2cl4sycl6detail11SYCLMemObjT10releaseMemESt10shared_ptrINS1_12context_implEEPv
 _ZN2cl4sycl6detail11SYCLMemObjT16determineHostPtrERKSt10shared_ptrINS1_12context_implEEbRPvRb