Skip to content

[SYCL][Graph] Support for sycl_ext_oneapi_enqueue_barrier extension #11418

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 10, 2023

Conversation

mfrancepillois
Copy link
Contributor

@mfrancepillois mfrancepillois commented Oct 4, 2023

Adds support to handle barrier enqueuing with Record&Replay API. Barriers are implemented as empty nodes enforcing the required dependencies.

Adds tests that check:

  1. correctness of graph structure when barriers have been enqueued,
  2. processing behavior,
  3. exception throwing if barriers are used within explicit API.

Notes:

  1. Barriers can only be used with Record&Replay API, since barriers rely on events to enforce dependencies.
  2. The scope of a barrier is limited to the Graph in which the barrier was added. As a result, if a barrier with no explicit events to wait for is part of a graph, when the graph is executed this barrier waits for all the kernels/operations in the graph that were previously submitted. However, the barrier does not wait for kernels/operations outside the graph that might have been submitted to the same execution queue.

Adds support to handle barrier enqueuing with Record&Replay API.
Barriers are implemented as empty nodes enforcing the required dependencies.

Adds tests that check 1) correctness of graph structure when barriers have been enqueued, 2) processing behavior, 3) exception throwing if barriers are used within explicit API.

Notes:
1) Multi-queues barrier is not supported since it does not make sense with asynchronous graph execution.
2) Barriers can only be used with Record&Replay API, since barriers rely on events to enforce dependencies.

Co-authored-by: Ben Tracy <[email protected]>

---------

Co-authored-by: Ben Tracy <[email protected]>
@mfrancepillois mfrancepillois requested review from a team as code owners October 4, 2023 09:34
@mfrancepillois mfrancepillois temporarily deployed to WindowsCILock October 4, 2023 09:35 — with GitHub Actions Inactive
@mfrancepillois mfrancepillois temporarily deployed to WindowsCILock October 4, 2023 10:02 — with GitHub Actions Inactive
@mfrancepillois mfrancepillois temporarily deployed to WindowsCILock October 4, 2023 11:27 — with GitHub Actions Inactive
@mfrancepillois mfrancepillois temporarily deployed to WindowsCILock October 4, 2023 11:58 — with GitHub Actions Inactive
@EwanC
Copy link
Contributor

EwanC commented Oct 9, 2023

Hi @intel/llvm-reviewers-runtime, could we have a review of this please

// (B2)
// /|\
// / | \
// (6) (7) (8) (those nodes also have as a precedessor)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: those nodes also have barrier as a predecessor?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. The sentence should have been "those nodes also have B1 as a predecessor".
We fixed it.

@al42and
Copy link
Contributor

al42and commented Oct 9, 2023

Multi-queues barrier is not supported

Does it mean that q.ext_oneapi_submit_barrier({e1, e2}); is forbidden if e1 and e2 come from different queues?

Or does it forbid any mixing of the queues, including the following code?

sycl::event e = q1.ext_oneapi_submit_barrier();
q2.ext_oneapi_submit_barrier({e});

@mfrancepillois
Copy link
Contributor Author

Multi-queues barrier is not supported

Does it mean that q.ext_oneapi_submit_barrier({e1, e2}); is forbidden if e1 and e2 come from different queues?

Or does it forbid any mixing of the queues, including the following code?

sycl::event e = q1.ext_oneapi_submit_barrier();
q2.ext_oneapi_submit_barrier({e});

The sentence was not accurate. Indeed, the implementation supports both cases you mentioned. We added tests (unitests) to verify these use cases. We also updated the PR description to better explain the limitations of our implementation.

@mfrancepillois mfrancepillois temporarily deployed to WindowsCILock October 10, 2023 11:16 — with GitHub Actions Inactive
@mfrancepillois mfrancepillois temporarily deployed to WindowsCILock October 10, 2023 11:59 — with GitHub Actions Inactive
@mfrancepillois
Copy link
Contributor Author

@intel/llvm-gatekeepers could you please merge this PR?

@steffenlarsen steffenlarsen merged commit 6a0d6ef into intel:sycl Oct 10, 2023
@mfrancepillois mfrancepillois deleted the command-buffer-support-barrier branch October 10, 2023 13:50
// \ | /
// \ | /
// (B)
// / \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like post-commit doesn't like these \ despite them being in comments. @mfrancepillois - Could you please address this issue ASAP?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've managed to reproduce this locally, will open a fix PR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out it's the trailing whitespace character that triggers the warning #11493
Can see it show in godbolt too https://godbolt.org/z/veeYvs6de if you add a whitespace character to the end of a line

aelovikov-intel pushed a commit that referenced this pull request Oct 10, 2023
Fixes post commit CI fail
https://github.com/intel/llvm/actions/runs/6470268418/job/17566091513
```
/__w/llvm/llvm/src/sycl/unittests/Extensions/CommandGraph.cpp:1459:12: error: backslash and newline separated by space [-Werror,-Wbackslash-newline-escape]
 1459 |   //     /|\
      |            ^
1 error generated.
```

This was occuring due to a trailing whitespace character in the comment
that was introduced in
#11418 (comment)
EwanC added a commit to reble/llvm that referenced this pull request Nov 10, 2023
The following features defined in the specification
as unsupported, have working implementations upstream.

* intel#11418
* intel#11505
* intel#11556
EwanC added a commit to reble/llvm that referenced this pull request Nov 15, 2023
The following features are defined in the specification as unsupported, but
have working implementations merged upstream. This PR updates the graphs
specification to reflect that and removes some trailing whitespace.

* intel#11418
* intel#11505
* intel#11556
* intel#11855
steffenlarsen pushed a commit that referenced this pull request Nov 15, 2023
The following features are defined in the specification as unsupported,
but have working implementations merged upstream. This PR updates the
graphs specification to reflect that and removes some trailing
whitespace.

* #11418
* #11505
* #11556
* #11855
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants