Skip to content

[SYCL][L0] Improve L0 timestamps #6006

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 13, 2022
Merged

[SYCL][L0] Improve L0 timestamps #6006

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 17 additions & 10 deletions sycl/plugins/level_zero/pi_level_zero.cpp
100755 → 100644
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5147,8 +5147,15 @@ pi_result piEventGetProfilingInfo(pi_event Event, pi_profiling_info ParamName,
Event->Queue
? Event->Queue->Device->ZeDeviceProperties->timerResolution
: Event->Context->Devices[0]->ZeDeviceProperties->timerResolution;
// Get timestamp frequency
const double ZeTimerFreq = 1E09 / ZeTimerResolution;

const uint64_t TimestampMaxValue =
Event->Queue
? ((1ULL << Event->Queue->Device->ZeDeviceProperties
->kernelTimestampValidBits) -
1ULL)
: ((1ULL << Event->Context->Devices[0]
->ZeDeviceProperties->kernelTimestampValidBits) -
1ULL);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please compute device first and the use it 5146-5158 to make it cleaner

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smaslov-intel : Please see here #6010


ReturnHelper ReturnValue(ParamValueSize, ParamValue, ParamValueSizeRet);

Expand All @@ -5157,27 +5164,27 @@ pi_result piEventGetProfilingInfo(pi_event Event, pi_profiling_info ParamName,
switch (ParamName) {
case PI_PROFILING_INFO_COMMAND_START: {
ZE_CALL(zeEventQueryKernelTimestamp, (Event->ZeEvent, &tsResult));
uint64_t ContextStartTime = tsResult.context.kernelStart * ZeTimerFreq;
uint64_t ContextStartTime =
(tsResult.context.kernelStart & TimestampMaxValue) * ZeTimerResolution;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it mean that the Level-Zero spec guidance needs to be adjusted?

https://spec.oneapi.io/level-zero/latest/core/PROG.html#kernel-timestamp-events

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smaslov-intel if you are referring to

const double timestampFreq = NS_IN_SEC / device_properties.timerResolution;

this is valid only if you use ZE_STRUCTURE_TYPE_DEVICE_PROPERTIES_1_2

uint64_t timerResolution
[out] Returns the resolution of device timer used for profiling, timestamps, etc.
When stype==ZE_STRUCTURE_TYPE_DEVICE_PROPERTIES the units are in nanoseconds.
When stype==ZE_STRUCTURE_TYPE_DEVICE_PROPERTIES_1_2 units are in cycles/sec

is that what you are referring to or something else in the spec?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, thanks

return ReturnValue(ContextStartTime);
}
case PI_PROFILING_INFO_COMMAND_END: {
ZE_CALL(zeEventQueryKernelTimestamp, (Event->ZeEvent, &tsResult));

uint64_t ContextStartTime = tsResult.context.kernelStart;
uint64_t ContextEndTime = tsResult.context.kernelEnd;
uint64_t ContextStartTime =
(tsResult.context.kernelStart & TimestampMaxValue);
uint64_t ContextEndTime = (tsResult.context.kernelEnd & TimestampMaxValue);

//
// Handle a possible wrap-around (the underlying HW counter is < 64-bit).
// Note, it will not report correct time if there were multiple wrap
// arounds, and the longer term plan is to enlarge the capacity of the
// HW timestamps.
//
if (ContextEndTime <= ContextStartTime) {
pi_device Device = Event->Context->Devices[0];
const uint64_t TimestampMaxValue =
(1LL << Device->ZeDeviceProperties->kernelTimestampValidBits) - 1;
ContextEndTime += TimestampMaxValue - ContextStartTime;
ContextEndTime += TimestampMaxValue;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch!

}
ContextEndTime *= ZeTimerFreq;
ContextEndTime *= ZeTimerResolution;
return ReturnValue(ContextEndTime);
}
case PI_PROFILING_INFO_COMMAND_QUEUED:
Expand Down