[SYCL] Move device handling to Unified Runtime #8088

smaslov-intel · 2023-01-24T02:10:08Z

The changes although being large, are just move of the code with modifications necessary to being able to share it with L0 Plugin.
Signed-off-by: Sergey V Maslov [email protected]

smaslov-intel · 2023-01-24T19:56:37Z

@bso-intel , @igchor , @jandres742 : please review

steffenlarsen

Couple of comments, but otherwise looks good.

sycl/plugins/unified_runtime/ur/adapters/level_zero/ur_level_zero.cpp

sycl/plugins/unified_runtime/ur/ur.hpp

bso-intel

LGTM

sycl/plugins/unified_runtime/ur/ur.hpp

sycl/plugins/unified_runtime/ur/adapters/level_zero/ur_level_zero.cpp

jandres742 · 2023-01-25T01:12:37Z

+1

smaslov-intel · 2023-01-25T01:37:22Z

sycl/plugins/unified_runtime/ur/ur.hpp

+// TODO: promote all of the below extensions to the Unified Runtime
+//       and get rid of these ZER_EXT constants.
+const int ZER_EXT_DEVICE_INFO_END = ZER_DEVICE_INFO_FORCE_UINT32;
+const int ZER_EXT_DEVICE_INFO_BUILD_ON_SUBDEVICE = ZER_EXT_DEVICE_INFO_END - 1;
+const int ZER_EXT_DEVICE_INFO_MAX_WORK_GROUPS_3D = ZER_EXT_DEVICE_INFO_END - 2;
+const int ZER_EXT_DEVICE_INFO_ATOMIC_MEMORY_SCOPE_CAPABILITIES =
+    ZER_EXT_DEVICE_INFO_END - 3;
+const int ZER_EXT_DEVICE_INFO_BFLOAT16_MATH_FUNCTIONS =
+    ZER_EXT_DEVICE_INFO_END - 4;
+const int ZER_EXT_DEVICE_PARTITION_BY_CSLICE = ZER_EXT_DEVICE_INFO_END - 5;
+const int ZER_EXT_DEVICE_INFO_MAX_MEM_BANDWIDTH = ZER_EXT_DEVICE_INFO_END - 6;
+const int ZER_EXT_DEVICE_INFO_GPU_HW_THREADS_PER_EU =
+    ZER_EXT_DEVICE_INFO_END - 7;
+const int ZER_EXT_DEVICE_INFO_GPU_EU_COUNT_PER_SUBSLICE =
+    ZER_EXT_DEVICE_INFO_END - 8;
+const int ZER_EXT_DEVICE_INFO_GPU_SLICES = ZER_EXT_DEVICE_INFO_END - 9;
+const int ZER_EXT_DEVICE_INFO_MAX_COMPUTE_QUEUE_INDICES =
+    ZER_EXT_DEVICE_INFO_END - 10;
+const int ZER_EXT_DEVICE_INFO_MEMORY_BUS_WIDTH = ZER_EXT_DEVICE_INFO_END - 11;
+const int ZER_EXT_DEVICE_INFO_MEMORY_CLOCK_RATE = ZER_EXT_DEVICE_INFO_END - 12;
+const int ZER_EXT_DEVICE_INFO_FREE_MEMORY = ZER_EXT_DEVICE_INFO_END - 13;
+const int ZER_EXT_DEVICE_INFO_DEVICE_ID = ZER_EXT_DEVICE_INFO_END - 14;
+const int ZER_EXT_DEVICE_INFO_IMAGE_MAX_ARRAY_SIZE =
+    ZER_DEVICE_INFO_IMAGE_MAX_ARRAR_SIZE;
+
+const int ZER_EXT_RESULT_END = 0x1000;
+const zer_result_t ZER_EXT_RESULT_ADAPTER_SPECIFIC_ERROR =
+    zer_result_t(ZER_EXT_RESULT_END - 1);
+
+const int ZER_EXT_USM_CAPS_ACCESS = 1 << 0;
+const int ZER_EXT_USM_CAPS_ATOMIC_ACCESS = 1 << 1;
+const int ZER_EXT_USM_CAPS_CONCURRENT_ACCESS = 1 << 2;
+const int ZER_EXT_USM_CAPS_CONCURRENT_ATOMIC_ACCESS = 1 << 3;
+
+const zer_device_partition_property_flag_t
+    ZER_EXT_DEVICE_PARTITION_PROPERTY_FLAG_BY_CSLICE =
+        zer_device_partition_property_flag_t(
+            ZER_DEVICE_PARTITION_PROPERTY_FLAG_FORCE_UINT32 - 1);
+


@pbalcer: please note these UR extensions and make sure there are issues created to follow up

igchor · 2023-01-25T01:47:32Z

LGTM. Just FYI, the were some changes in the UR API (mostly ZER prefix was changed to UR) since this code was created: https://github.com/oneapi-src/unified-runtime/blob/main/include/ur_api.h

Is the plan to update the code in llvm to the newest UR version after this PR is merged?

smaslov-intel · 2023-01-25T04:02:05Z

LGTM. Just FYI, the were some changes in the UR API (mostly ZER prefix was changed to UR) since this code was created: https://github.com/oneapi-src/unified-runtime/blob/main/include/ur_api.h

Is the plan to update the code in llvm to the newest UR version after this PR is merged?

Right. We have to update to the latest UR.
@igchor : would you take an AR to follow up on that after this PR is merged?

jandres742 · 2023-01-25T06:55:54Z

FYI, the were some changes in the UR API (mostly ZER prefix was changed to UR) since this code was created: https://github.com/oneapi-src/unified-runtime/blob/main/include/ur_api.h

Is the plan to update the code in llvm to the newest UR version after this PR is merged?

we are checking out at fd711c920acc4434cb52ff18b078c082d9d7f44d from the UR repo,

llvm/sycl/plugins/unified_runtime/CMakeLists.txt

Line 7 in 981981f

set(UNIFIED_RUNTIME_TAG fd711c920acc4434cb52ff18b078c082d9d7f44d)

LGTM. Just FYI, the were some changes in the UR API (mostly ZER prefix was changed to UR) since this code was created: https://github.com/oneapi-src/unified-runtime/blob/main/include/ur_api.h
Is the plan to update the code in llvm to the newest UR version after this PR is merged?

Right. We have to update to the latest UR. @igchor : would you take an AR to follow up on that after this PR is merged?

@smaslov-intel I'm working on integrating with the UR loader (https://github.com/intel/llvm/blob/sycl/sycl/plugins/unified_runtime/CMakeLists.txt#L32). I can do it on the same patch if you'd want me to.

sycl/plugins/unified_runtime/pi2ur.hpp

sycl/plugins/unified_runtime/ur/adapters/level_zero/ur_level_zero.hpp

smaslov-intel · 2023-01-25T17:22:39Z

Is the plan to update the code in llvm to the newest UR version after this PR is merged?

Right. We have to update to the latest UR. @igchor : would you take an AR to follow up on that after this PR is merged?

@smaslov-intel I'm working on integrating with the UR loader (https://github.com/intel/llvm/blob/sycl/sycl/plugins/unified_runtime/CMakeLists.txt#L32). I can do it on the same patch if you'd want me to.

Thanks for volunteering!

smaslov-intel · 2023-01-26T04:57:05Z

/verify with intel/llvm-test-suite#1544

smaslov-intel · 2023-01-26T04:57:59Z

The remaining test failure

Failed Tests (1):
SYCL :: Regression/device_num.cpp

Is covered with intel/llvm-test-suite#1544

jandres742

+1

jandres742 · 2023-02-08T01:51:29Z

thanks @smaslov-intel . LGTM

Question: if all that we were doing here was to move code from one PI to UR, why were tests failing? just some misses in the porting, or something else major?

smaslov-intel · 2023-02-08T04:08:38Z

thanks @smaslov-intel . LGTM

Question: if all that we were doing here was to move code from one PI to UR, why were tests failing? just some misses in the porting, or something else major?

@jandres742 : The move from Pi to UR is straightforward.
The complications were related to UR <-> PI translation mostly here here: https://github.com/intel/llvm/pull/8088/files#diff-f1103e5442bf4f71ac0fb3cd9aa969042ee3e864e7b3e8c6f3255b7162ffadfe

smaslov-intel · 2023-02-08T16:51:31Z

thanks @smaslov-intel . LGTM
Question: if all that we were doing here was to move code from one PI to UR, why were tests failing? just some misses in the porting, or something else major?

@jandres742 : The move from Pi to UR is straightforward. The complications were related to UR <-> PI translation mostly here here: https://github.com/intel/llvm/pull/8088/files#diff-f1103e5442bf4f71ac0fb3cd9aa969042ee3e864e7b3e8c6f3255b7162ffadfe

Note also that the remaining PI interfaces are mostly the same between UR and PI and so will require minimum to no translation.

Signed-off-by: Sergey V Maslov <[email protected]>

smaslov-intel · 2023-02-09T17:14:42Z

@intel/llvm-gatekeepers : please merge this

jandres742 · 2023-02-10T09:35:56Z

thanks @smaslov-intel . LGTM
Question: if all that we were doing here was to move code from one PI to UR, why were tests failing? just some misses in the porting, or something else major?

@jandres742 : The move from Pi to UR is straightforward. The complications were related to UR <-> PI translation mostly here here: https://github.com/intel/llvm/pull/8088/files#diff-f1103e5442bf4f71ac0fb3cd9aa969042ee3e864e7b3e8c6f3255b7162ffadfe

Note also that the remaining PI interfaces are mostly the same between UR and PI and so will require minimum to no translation.

Working now on updating CMakeLists to user UR loader.

The changes although being large, are just move of the code with modifications necessary to being able to share it with L0 Plugin. Signed-off-by: Sergey V Maslov <[email protected]> --------- Signed-off-by: Sergey V Maslov <[email protected]>

smaslov-intel requested review from a team as code owners January 24, 2023 02:10

smaslov-intel requested review from steffenlarsen and againull January 24, 2023 02:10

smaslov-intel temporarily deployed to aws January 24, 2023 02:51 — with GitHub Actions Inactive

smaslov-intel temporarily deployed to aws January 24, 2023 03:22 — with GitHub Actions Inactive

smaslov-intel requested a review from bso-intel January 24, 2023 19:56

steffenlarsen approved these changes Jan 24, 2023

View reviewed changes

sycl/plugins/unified_runtime/ur/adapters/level_zero/ur_level_zero.cpp Outdated Show resolved Hide resolved

sycl/plugins/unified_runtime/ur/ur.hpp Outdated Show resolved Hide resolved

bso-intel approved these changes Jan 24, 2023

View reviewed changes

jandres742 reviewed Jan 24, 2023

View reviewed changes

smaslov-intel commented Jan 25, 2023

View reviewed changes

smaslov-intel temporarily deployed to aws January 25, 2023 02:01 — with GitHub Actions Inactive

smaslov-intel temporarily deployed to aws January 25, 2023 02:34 — with GitHub Actions Inactive

smaslov-intel temporarily deployed to aws January 25, 2023 05:01 — with GitHub Actions Inactive

keryell reviewed Jan 25, 2023

View reviewed changes

sycl/plugins/unified_runtime/pi2ur.hpp Outdated Show resolved Hide resolved

sycl/plugins/unified_runtime/ur/adapters/level_zero/ur_level_zero.hpp Show resolved Hide resolved

smaslov-intel had a problem deploying to aws January 25, 2023 17:39 — with GitHub Actions Failure

smaslov-intel temporarily deployed to aws January 25, 2023 18:05 — with GitHub Actions Inactive

smaslov-intel temporarily deployed to aws January 25, 2023 18:36 — with GitHub Actions Inactive

smaslov-intel temporarily deployed to aws January 26, 2023 00:53 — with GitHub Actions Inactive

smaslov-intel temporarily deployed to aws January 26, 2023 01:24 — with GitHub Actions Inactive

smaslov-intel temporarily deployed to aws January 26, 2023 02:41 — with GitHub Actions Inactive

smaslov-intel temporarily deployed to aws January 26, 2023 03:12 — with GitHub Actions Inactive

smaslov-intel temporarily deployed to aws February 7, 2023 21:33 — with GitHub Actions Inactive

smaslov-intel mentioned this pull request Feb 7, 2023

[SYCL] Test that the UUID is read correctly from Level Zero intel/llvm-test-suite#1581

Merged

jandres742 reviewed Feb 8, 2023

View reviewed changes

smaslov-intel temporarily deployed to aws February 8, 2023 23:35 — with GitHub Actions Inactive

smaslov-intel added 13 commits February 8, 2023 18:51

[SYCL] Move device handling to Unified Runtime

23a85b0

Signed-off-by: Sergey V Maslov <[email protected]>

clang-format

b83f5f9

Signed-off-by: Sergey V Maslov <[email protected]>

[SYCL] convert UR info values to PI info values

8651824

Signed-off-by: Sergey V Maslov <[email protected]>

address test failures

3093135

Signed-off-by: Sergey V Maslov <[email protected]>

address review comments

aa81547

Signed-off-by: Sergey V Maslov <[email protected]>

fix UUID return

027e0a2

Signed-off-by: Sergey V Maslov <[email protected]>

more fixes

79722ef

Signed-off-by: Sergey V Maslov <[email protected]>

fix Win build

1fb32b3

Signed-off-by: Sergey V Maslov <[email protected]>

fix no_partition device info translation

b3565fa

Signed-off-by: Sergey V Maslov <[email protected]>

convert arrays

8cd83d8

Signed-off-by: Sergey V Maslov <[email protected]>

handle 0-terminated array

96f274b

Signed-off-by: Sergey V Maslov <[email protected]>

fix partition flags handling

8ef7970

Signed-off-by: Sergey V Maslov <[email protected]>

remove printf

050936b

Signed-off-by: Sergey V Maslov <[email protected]>

smaslov-intel force-pushed the urdevice branch from 487f982 to 050936b Compare February 9, 2023 02:53

format

2dfd56e

Signed-off-by: Sergey V Maslov <[email protected]>

smaslov-intel temporarily deployed to aws February 9, 2023 03:25 — with GitHub Actions Inactive

smaslov-intel temporarily deployed to aws February 9, 2023 12:48 — with GitHub Actions Inactive

againull approved these changes Feb 9, 2023

View reviewed changes

againull merged commit 33be961 into intel:sycl Feb 9, 2023

[SYCL] Move device handling to Unified Runtime #8088

[SYCL] Move device handling to Unified Runtime #8088

Uh oh!

Conversation

smaslov-intel commented Jan 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

smaslov-intel commented Jan 24, 2023

Uh oh!

steffenlarsen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bso-intel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jandres742 commented Jan 25, 2023

Uh oh!

smaslov-intel Jan 25, 2023

Choose a reason for hiding this comment

Uh oh!

igchor commented Jan 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

smaslov-intel commented Jan 25, 2023

Uh oh!

jandres742 commented Jan 25, 2023

Uh oh!

Uh oh!

Uh oh!

smaslov-intel commented Jan 25, 2023

Uh oh!

smaslov-intel commented Jan 26, 2023

Uh oh!

smaslov-intel commented Jan 26, 2023

Uh oh!

jandres742 left a comment

Choose a reason for hiding this comment

Uh oh!

jandres742 commented Feb 8, 2023

Uh oh!

smaslov-intel commented Feb 8, 2023

Uh oh!

smaslov-intel commented Feb 8, 2023

Uh oh!

smaslov-intel commented Feb 9, 2023

Uh oh!

jandres742 commented Feb 10, 2023

Uh oh!

Uh oh!

smaslov-intel commented Jan 24, 2023 •

edited

Loading

igchor commented Jan 25, 2023 •

edited

Loading