This repository was archived by the owner on Mar 28, 2023. It is now read-only.
forked from llvm/llvm-test-suite
-
Notifications
You must be signed in to change notification settings - Fork 130
[DOC] Major documentation update #200
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
48c6c01
[DOC] Major documenation rewrite
36065b9
Update SYCL/ExtraTests/README.md
vladimirlaz c658676
Addressing review comments
a26d230
Marking External and ExtraTests as experimental
13b09d4
Fixing formatting
bd59eaa
One more time
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,82 +1,80 @@ | ||
# Table of contents | ||
* [Overview](#overview) | ||
* [Execution](#execution) | ||
* [Directory structure](#directory-structure) | ||
* [CMake parameters](#cmake-parameters) | ||
* [LIT parameters accepted by LIT executor](#lit-parameters-accepted-by-lit-executor) | ||
* [LIT features which can be used to configure test execution](#lit-features-which-can-be-used-to-configure-test-execution) | ||
> **NOTE** | ||
> This is experimental extension of test suite. Could be dropped or extended | ||
> further at some later point. | ||
|
||
# Overview | ||
This directory contains SYCL-related infrastructure for running tests which | ||
sources are stored outside of this repository. Tests are distributed in | ||
sub-directories. One directory per application/test repository. | ||
|
||
# Execution | ||
``` | ||
git clone <GIT_REPO> # e.g. https://github.com/intel/llvm-test-suite | ||
cd llvm-test-suite | ||
mkdir build | ||
cd build | ||
# configuring test execution (selecting compiler, target BE and target device, | ||
#downloading and building RSBench sources from default location) | ||
cmake -G Ninja \ | ||
-DTEST_SUITE_SUBDIRS=SYCL \ | ||
-DTEST_SUITE_LIT=<PATH_TO_llvm-lit> \ | ||
-DCHECK_SYCL_ALL="opencl:cpu,acc,gpu,host;level_zero:gpu,host" \ | ||
-DCMAKE_CXX_COMPILER=<PATH_TO_sycl_compiler> \ | ||
-DSYCL_EXTERNAL_TESTS="RSBench" \ | ||
.. | ||
# Build and run full list of SYCL tests | ||
ninja check | ||
# Build all tests dependencies | ||
ninja | ||
# Run all SYCL tests | ||
llvm-lit . | ||
# Get list of available tests | ||
llvm-lit . --show-tests | ||
# Run specific test (e.g. RSBench only) | ||
llvm-lit SYCL/External/RSBench | ||
# Run tests with parameters | ||
llvm-lit --param target_devices=host,gpu --param sycl_be=level_zero \ | ||
--param dpcpp_compiler=path/to/clang++ --param dump_ir=True SYCL/External/RSBench | ||
``` | ||
"External" tests directory has necessary infrastructure for running tests where | ||
sources of tests are expected to be located outside of llvm-test-suite | ||
repository. Each subdirectory represents external repository for corresponding | ||
application/test. | ||
|
||
Notes: | ||
- it is assumed that LIT framework, FileCheck and other LIT dependencies are | ||
available in the same directory with llvm-lit. | ||
# Adding external test | ||
|
||
# Directory structure | ||
Every sub-directory (e.g. RSBench) contains the following content: | ||
See [RSBench](RSBench) as example, it contains the following: | ||
* **CMakeLists.txt** - CMake configuration file which is used to obtain | ||
application binary and data files. There are several variables which are | ||
used by it (see [Main parameters](#main-parameters) section for details) | ||
* **\*.test** - test configuration files, containing command lines, test | ||
status lines and verification patterns following [LLVM test infrastructure](https://llvm.org/docs/TestingGuide.html). | ||
application binary and data files. | ||
* **lit.local.cfg** - application specific LIT infrastructure configuration | ||
file. In case of RSBench benchmarks it sets LIT test file suffixes to `.test` | ||
file. In case of RSBench benchmark it sets LIT test file suffixes to `.test` | ||
and mark all tests in directory unsupported if corresponding directory is | ||
not added to `SYCL_EXTERNAL_TESTS` list (benchmark is not included in | ||
testing). | ||
not added to `SYCL_EXTERNAL_TESTS` list. See more at: | ||
[LLVM Testing Guide](https://llvm.org/docs/TestingGuide.html#platform-specific-tests) | ||
* **\*.test** - test configuration files, containing command lines, test | ||
status lines and verification patterns per each platform. | ||
|
||
# CMake parameters | ||
All parameters described in [Readme.md](../README.md#cmake-parameters) are | ||
applicable. Also extra CMake parameters are introduced to configure specific | ||
|
||
All parameters described in [README.md](../README.md#cmake-parameters) are | ||
applicable. | ||
|
||
***SYCL_EXTERNAL_TESTS*** semicolon-separated names of external SYCL | ||
applications which are built and run as part of the testing. Name is | ||
subdirectory name in "External" directory. Example: | ||
``` | ||
-DSYCL_EXTERNAL_TESTS=RSBench | ||
``` | ||
Source code of external application can be downloaded from external repository | ||
as part of the build or provided in CMake variable <APPNAME>_SRC | ||
(e.g. RSBench_SRC). | ||
|
||
Also extra CMake parameters are introduced to configure specific | ||
application: | ||
* **APPName_BIN** (e.g. `RSBench_BIN`) - point to the directory containing | ||
prebuilt binaries of the application. | ||
* **APPName_SRC** (e.g. `RSBench_SRC`) - point to the directory containing | ||
sources of the application. | ||
* **APPName_BIN** (e.g. `RSBench_BIN`) - directory containing prebuilt | ||
binaries of the application. | ||
* **APPName_SRC** (e.g. `RSBench_SRC`) - directory containing sources of the | ||
application. | ||
* **APPName_URL** (e.g. `RSBench_URL`) - URL to the GIT repository containing | ||
sources of the application (default value for RSBench benchmark is | ||
`https://github.com/ANL-CESAR/RSBench.git`). | ||
sources of the application. | ||
* **APPName_TAG** (e.g. `RSBench_TAG`) - GIT tag or hash or branch name used | ||
to download source from GIT repository (default value for RSBench | ||
benchmark is `origin/master`). | ||
to download source from GIT repository. | ||
|
||
Configuration parameters are priorities from top to down. If **APPName_BIN** | ||
is specified binaries will be used directly ignoring other parameters. | ||
|
||
# LIT parameters accepted by LIT executor: | ||
All parameters described in [Readme.md](../README.md#lit-parameters-accepted-by-lit-executor) are applicable. | ||
# Build and run tests | ||
|
||
Get sources | ||
|
||
``` | ||
git clone https://github.com/intel/llvm-test-suite | ||
cd llvm-test-suite | ||
mkdir build | ||
cd build | ||
``` | ||
|
||
With compiler tools available in the PATH: | ||
|
||
``` | ||
# Configure | ||
cmake \ | ||
-DCMAKE_CXX_COMPILER=clang++ \ | ||
-DTEST_SUITE_SUBDIRS=SYCL \ | ||
-DCHECK_SYCL_ALL="level_zero:gpu" \ | ||
-DSYCL_EXTERNAL_TESTS="RSBench" \ | ||
.. | ||
|
||
# Build and Run | ||
make check-sycl-all | ||
|
||
``` | ||
|
||
# LIT features which can be used to configure test execution: | ||
All features described in [Readme.md](../README.md#lit-features-which-can-be-used-to-configure-test-execution) are applicable. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
> **NOTE** | ||
> This is experimental extension of test suite. Could be dropped or extended | ||
> further at some later point. | ||
|
||
# Overview | ||
|
||
"ExtraTests" directory contains infrastructure for picking up LIT style tests | ||
from another external directory passed in SYCL_EXTRA_TESTS_SRC. | ||
|
||
# CMake parameters | ||
|
||
All parameters described in [Readme.md](../README.md#cmake-parameters) are | ||
applicable. Additional parameters for this test category: | ||
|
||
***SYCL_EXTRA_TESTS_SRC*** path to directory which contains additional LIT | ||
tests. | ||
|
||
# Example | ||
|
||
``` | ||
# Configure | ||
cmake -G Ninja \ | ||
-DTEST_SUITE_SUBDIRS=SYCL \ | ||
-DTEST_SUITE_LIT=<path/to/llvm-lit> \ | ||
-DCHECK_SYCL_ALL="opencl:cpu,acc,gpu,host;level_zero:gpu,host" \ | ||
-DCMAKE_CXX_COMPILER=<path/to/clang++> \ | ||
-DSYCL_EXTRA_TESTS_SRC=<path/to/more/lit/tests/sources> | ||
.. | ||
|
||
# Build and run full list of SYCL tests | ||
ninja ninja check-sycl-all | ||
``` |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.