-
Notifications
You must be signed in to change notification settings - Fork 788
[CI][Benchmark] Merge new static html dashboard implementation #17659
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
108 commits
Select commit
Hold shift + click to select a range
e6ca992
Move UR devops scripts to devops folder
ianayl 3d42db2
Restrict number of cores used
ianayl fc70520
Merge branch 'sycl' of https://github.com/intel/llvm into unify-bench…
ianayl 4f08dd6
Restore ur-benchmark*.yml
ianayl 497dcce
[benchmarks] improve HTML and Markdown output
pbalcer 3cbed5e
Test UR benchmarking suite
ianayl 1936207
Merge branch 'unify-benchmark-ci' of https://github.com/intel/llvm in…
ianayl f79bbbf
Bump tolerance to 7%
ianayl ffc8139
Revert "Bump tolerance to 7%"
ianayl 0a34e0d
[benchmarks] fix failing benchmarks, improve html output
pbalcer 3f42420
[benchmarks] fix python formatting with black
pbalcer 1c7b189
update driver version
pbalcer ad13e93
simplify preset implementation and fix normal preset
pbalcer 68ed0c4
Add PVC and BMG as runners
ianayl 18fff93
Merge branch 'unify-benchmark-ci' of https://github.com/intel/llvm in…
ianayl 3a65b98
Install dependencies before running UR script
ianayl 220121a
Use venv for python packages
ianayl 37d361c
Install venv before using venv
ianayl 07f1e10
[benchmarks] allow specifying custom results directories
pbalcer 64cf79c
[benchmarks] sort runs by date for html output
pbalcer 6c28d33
simplify presets, remove suites if all set
pbalcer e15b94f
[benchmarks] use python venv for scripts
pbalcer 78fd037
Run apt with sudo
ianayl 0ed1599
Merge branch 'unify-benchmark-ci' of https://github.com/intel/llvm in…
ianayl 82b6e55
Ignore "missing" apt packages in workflow
ianayl 162cba0
Change pip to install to user
ianayl 848f741
Ignore system controlled python env
ianayl 918604e
[CI] use realpaths when referring to SYCL
ianayl 72d8730
[CI] use minimal preset when running benchmarks
ianayl 066f5a6
[CI] Allow 2 bench scripts locations (#17394)
lukaszstolarczuk 18e5291
add ulls compute benchmarks
pbalcer 237750e
[CI][Benchmark] Decouple results from existing file structure, fetch …
ianayl ba1297f
[benchmark] Disabling UR test suites
ianayl cd6097f
update compute benchmarks and fix requirements
pbalcer c4e92c6
fix url updates
pbalcer ed8eecc
use timestamps in result file names
pbalcer 130212d
add hostname to benchmark run
pbalcer a884df8
Merge branch 'sycl' of https://github.com/intel/llvm into unify-bench…
ianayl 5323386
add SubmitGraph benchmark
pbalcer 5bd1d56
Restore sycl-linux-run-tests benchmarking action
ianayl e9b1375
Restore old SYCL benchmarking CI
ianayl a3edf7a
Add benchmarking results to sycl-docs.yml
ianayl 6620e4a
[CI] Bump compute bench (#17431)
lukaszstolarczuk f4a2e39
Initial implementation of unified benchmark workflow
ianayl 5d3b0d9
Merge branch 'unify-benchmark-ci' of https://github.com/intel/llvm in…
ianayl 38394bb
[CI] Use commit hash instead, fix issues with run
ianayl f232b93
add benchmark metadata
pbalcer 30cd308
apply formatting
pbalcer 5e0539a
fix multiple descriptions/notes
pbalcer 137407a
fix benchmark descriptions
pbalcer e0f5ca6
fix remote html output
pbalcer 1041db6
fix metadata collection with dry run
pbalcer fae04f4
cleanup compute bench, fix readme, use newer sycl-bench
pbalcer cfa4a9c
[CI] configure upload results
ianayl ca963e6
[CI] Change config to update during workflow run instead
ianayl 45a02e1
[CI] Change save name depending on build
ianayl 98f9d38
bump to 2024-2025
ianayl ef88ea0
[CI] Enforce commit hash to be string regardless
ianayl b7acba2
cleanup options in js scripts and fix ordering on bar charts
pbalcer e330a50
use day on x axis for timeseries
pbalcer cde744c
Merge branch 'sycl' of https://github.com/intel/llvm into unify-bench…
ianayl cae7049
[benchmarks] Undo merging in prior tests
ianayl 6bff3d6
add an option to limit build parallelism
pbalcer 3662b43
tiny tweaks for benchmark tags
pbalcer d2610c3
add support for benchmark tags
pbalcer ffc60bf
support for tags in html
pbalcer 75dd229
better and more tags
pbalcer cec8f05
formatting
pbalcer a0d8370
fix fetching tags from remote json
pbalcer c7f8d10
fix results /w descriptions and add url/commit of benchmarks
pbalcer 1dad513
fix git repo/hash for benchmarks
pbalcer 8437b89
Merge branch 'sycl' of https://github.com/intel/llvm into unify-bench…
ianayl 2dbf350
Revert changes to workflow files
ianayl a4a9907
Revert changes to composite actions
ianayl bdef08b
Revert changes to get_system_info.sh
ianayl 9e51c86
Revert changes not related to metadata
ianayl 6fa722b
Merge branch 'sycl' of https://github.com/intel/llvm into benchmark-m…
ianayl 5cc02c5
Revert changes to html
ianayl b49ff88
Revert presets.py
ianayl 9357df2
Revert benchmark.yml
ianayl 115dd5e
Merge branch 'benchmark-metadata' of https://github.com/ianayl/sycl i…
ianayl 03bfd15
Update imports to reflect result.py move
ianayl 0ff0142
Add benchmark history updates
ianayl 1a20f98
Merge html changes
ianayl 1ef9251
Correct bad conflict resolution over cudnn/cublas flags
ianayl 31c6695
Remove use of typing to stay consistent across files
ianayl 136f64e
Remove debug comments
ianayl ccb2a9c
Remove trailing spaces
ianayl f8ccc30
Specify that git metadata is modifiable
ianayl c54cd76
Remove unused metadata variable for now
ianayl 6e0690b
Merge branch 'benchmark-scripts-metadata' of https://github.com/intel…
ianayl 715f6b6
Fix automerge artifacts
ianayl 5e4dcf6
Merge branch 'new-html-dashboard' of https://github.com/ianayl/sycl i…
ianayl 2006770
Fix typo in argparse help message
ianayl 1686051
enforce default values in output-dir
ianayl 27850f4
Update/add more comments
ianayl f41f800
Clarify unstable scenarios
ianayl 71a3927
Clarify unstable scenarios
ianayl da8e60f
Reduce wit
ianayl ceedb61
layer comparison charts
pbalcer 51b259c
fix js tags and make colors more vibrant
pbalcer de0dc29
omit reversing run data, and instead simply store the first result
pbalcer 988d54e
stop default toggles from showing up in url
pbalcer 105fe0b
Merge branch 'sycl' of https://github.com/intel/llvm into benchmark-s…
ianayl 2cf61a4
Fixed english
ianayl 5bf00d4
Remove merge artifacts
ianayl 8a90039
Catch html errors in loadData
ianayl 282fb5c
apply python formatting
pbalcer File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
// Specify the following variables to read data remotely from `remoteDataUrl`: | ||
|
||
//remoteDataUrl = 'https://example.com/data.json'; | ||
//defaultCompareNames = ['baseline']; | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
// This file serves as a placeholder for loading data locally: If | ||
// `remoteDataUrl` (etc.) is not defined in config.js, the dashboard will | ||
// attempt to load data from variables defined here instead. | ||
// | ||
// These variables are empty by default, and are populated by main.py if | ||
// `--output-html local` is specified. | ||
|
||
benchmarkRuns = []; | ||
|
||
defaultCompareNames = []; | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,82 @@ | ||
<!-- | ||
Copyright (C) 2024-2025 Intel Corporation | ||
Part of the Unified-Runtime Project, under the Apache License v2.0 with LLVM Exceptions. | ||
See LICENSE.TXT | ||
SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception | ||
--> | ||
<!DOCTYPE html> | ||
<html> | ||
<head> | ||
<meta charset="utf-8"> | ||
<meta name="viewport" content="width=device-width, initial-scale=1"> | ||
<title>Benchmark Results</title> | ||
<script src="https://cdn.jsdelivr.net/npm/chart.js"></script> | ||
<script src="https://cdn.jsdelivr.net/npm/chartjs-adapter-date-fns"></script> | ||
<script src="data.js"></script> | ||
<script src="config.js"></script> | ||
<script src="scripts.js"></script> | ||
<link rel="stylesheet" href="styles.css"> | ||
</head> | ||
<body> | ||
<div class="container"> | ||
<h1>Benchmark Results</h1> | ||
<div id="loading-indicator" class="loading-indicator" style="display: none;"> | ||
Loading data, please wait... | ||
</div> | ||
<div class="filter-container"> | ||
<input type="text" id="bench-filter" placeholder="Regex..."> | ||
</div> | ||
<div class="run-selector"> | ||
<select id="run-select"> | ||
<option value="">Select a run to compare...</option> | ||
</select> | ||
<button onclick="addSelectedRun()">Add</button> | ||
<div id="selected-runs" class="selected-runs"></div> | ||
</div> | ||
<details class="options-container"> | ||
<summary>Options</summary> | ||
<div class="options-content"> | ||
<div class="filter-section"> | ||
<h3>Display Options</h3> | ||
<div class="display-options"> | ||
<label> | ||
<input type="checkbox" id="show-notes" checked> | ||
Director's commentary | ||
</label> | ||
<label> | ||
<input type="checkbox" id="show-unstable"> | ||
Show unstable scenarios | ||
</label> | ||
</div> | ||
</div> | ||
|
||
<div class="filter-section"> | ||
<h3>Suites</h3> | ||
<div id="suite-filters"> | ||
<!-- Suite checkboxes will be generated by JavaScript --> | ||
</div> | ||
</div> | ||
|
||
<div class="filter-section"> | ||
<h3>Tags <button class="tag-action-button" onclick="toggleAllTags(false)">Clear All</button></h3> | ||
<div id="tag-filters"> | ||
<!-- Tag checkboxes will be generated by JavaScript --> | ||
</div> | ||
</div> | ||
</div> | ||
</details> | ||
<details class="timeseries"> | ||
<summary>Historical Results</summary> | ||
<div class="charts"></div> | ||
</details> | ||
<details class="layer-comparisons"> | ||
<summary>Historical Layer Comparisons</summary> | ||
<div class="charts"></div> | ||
</details> | ||
<details class="bar-charts"> | ||
<summary>Comparisons</summary> | ||
<div class="charts"></div> | ||
</details> | ||
</div> | ||
</body> | ||
</html> |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.