Skip to content

Commit 81d9eb9

Browse files
krishung5mc-nv
authored andcommitted
Add support for instance group of type 'MODEL' (#107)
* Add support for instance group of type 'MODEL' * Format * Handle multi GPU cases when recording timestamps * Address comment * Use callback function to record timestamp for 'MODEL' kind * Add missing #ifdef * Update comment and if condition for input tensor memory alloc_perference * Fix for cuda stream. Use separate cuda callback to capture timestamp * Add comment to mention the possible timestamp issue * For 'KIND_MODEL', use cuda events for compute_input_duration and use callback for compute_infer_duration * Move the cudaLaunchHostFunc from RecordBackendTimestamp function * Fix up naming * Fix up * Fix up atomic initialization * Capture the timestamp after synchronization
1 parent 550cf62 commit 81d9eb9

File tree

1 file changed

+242
-65
lines changed

1 file changed

+242
-65
lines changed

0 commit comments

Comments
 (0)