Skip to content

[SYCL][CUDA] Use cuEventQuery to check event completion status #3544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 16, 2021

Conversation

psalz
Copy link
Contributor

@psalz psalz commented Apr 14, 2021

Previously the completion status of a PI CUDA event (returned by a get_info query for info::event::command_execution_status) would only be changed to command_execution_status::complete once the event had been waited upon.

This changes the mechanism to query the underlying event's execution status using cuEventQuery if the event has not been waited upon. I didn't memoize the result of the query as is_completed() is marked const (and making isCompleted_ a mutable member also doesn't seem right).

I've created a corresponding PR for the SYCL CTS that tests for this.

@psalz psalz requested a review from a team as a code owner April 14, 2021 12:33
@psalz psalz requested a review from smaslov-intel April 14, 2021 12:33
@bader bader added the cuda CUDA back-end label Apr 14, 2021
@bader
Copy link
Contributor

bader commented Apr 14, 2021

+@steffenlarsen

Copy link
Contributor

@steffenlarsen steffenlarsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for fixing this and great find! You have potentially spared someone a headache (hopefully not at the cost of having one yourself.)

}
if (ret == CUDA_ERROR_NOT_READY) {
return false;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we can safely set isCompleted_ to true here. Will allow it to take a slightly faster path in case this check is made again.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! I've changed both _pi_event::is_completed and _pi_event::get_execution_status to non-const.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, now I get the comment you made in the summary. It should be fine as non-const, but if it goes against your better judgement I am okay with you reverting it and keeping is_completed as const. I will leave that up to you.

Either way, LGTM! 😄 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to go on a limb and say once the status is complete , users aren't likely to query it again, so optimizing this shouldn't be necessary. Reverted ;-).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would leave the "_pi_event" object in a malformed state where the isCompleted_ is false for a completed event. Light +1 for making this non-const.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option is to rename it to hasBeenWaitedOn_ or the like. To my knowledge, isCompleted_ is only used for this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smaslov-intel You are right, that is no good either. I'm still feeing a bit iffy about making an is_* function non-const though.

I like the idea of hasBeenWaitedOn_ as well. If there are no objections, I will move forward with this in a couple of hours!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@psalz psalz force-pushed the fix-event-cmd-status-query branch 3 times, most recently from 30a0087 to 39485a3 Compare April 15, 2021 16:45
Previously the completion status of a PI CUDA event (returned by a
`get_info` query for `info::event::command_execution_status`) would only
be changed to `command_execution_status::complete` once the event had
been waited upon.

This changes the mechanism to query the underlying event's execution
status using `cuEventQuery` if the event has not been waited upon.
@psalz psalz force-pushed the fix-event-cmd-status-query branch from 39485a3 to eb7b20d Compare April 15, 2021 16:46
Copy link
Contributor

@smaslov-intel smaslov-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bader
Copy link
Contributor

bader commented Apr 15, 2021

/summary:run

@bader bader merged commit be7c1cb into intel:sycl Apr 16, 2021
@psalz psalz deleted the fix-event-cmd-status-query branch April 16, 2021 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cuda CUDA back-end
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants