-
Notifications
You must be signed in to change notification settings - Fork 178
Add documentation for unit testing #677
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
2e9c131
add documentation for unit testing
lorjala 91127fb
update unit testing documentation for mbed cli
lorjala c59970d
move unit testing documentation for mbed cli to cli-test-debug.md
lorjala 922e351
Copy edit cli-test-debug.md
bac0962
Copy edit testing_intro.md
d9ee5c9
Copy edit unit_testing.md
90203d2
removed a link to a non-existing page
lorjala 63d9bbe
Merge branch 'development' into unittests
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
## Unit testing | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure this should be its own page. I think it would make more sense in the CLI test and debug section. |
||
|
||
Use the `mbed unittest` command to build and run unit tests, or to generate files for new unit tests. | ||
|
||
Build and run unit tests with `mbed unittest`. The arguments are: | ||
|
||
* `--skip-build` to skip building unit tests. | ||
* `--skip-run` to skip running unit tests. | ||
* `--clean` to clean previous build data. | ||
* `-d` or `--debug` to prepare debug build. | ||
* `--coverage <TYPE>` to generate code coverage report where TYPE can be "html", "xml" or "both". | ||
* `-m <NAME>` or `--make-program <NAME>` to select which make build tool to use where NAME can be "make", "gmake", "mingw32-make" or "ninja". | ||
* `-g <NAME>` or `--generator <NAME>` to select which CMake generator to use where NAME can be "Unix Makefiles", "MinGW Makefiles" or "Ninja". | ||
* `-r <EXPRESSION>` or `--regex <EXPRESSION>` to run tests matching the regular expression. | ||
* `--build-path <PATH>` to specify build path. | ||
* `-v` or `--verbose` for verbose diagnostic output. | ||
|
||
Generate files for a new unit test with `mbed unittest --new <FILE>`. | ||
|
||
### Building and running unit tests | ||
|
||
You can specify to only **build** the unit tests by using the `--skip-run` option: | ||
|
||
``` | ||
$ mbed unittest --skip-run | ||
``` | ||
|
||
You can specify to only **run** the unit tests by using the `--skip-build` option: | ||
|
||
``` | ||
$ mbed unittest --skip-build | ||
``` | ||
|
||
If you do not specify any of these, `mbed unittest` will build all available unit tests and run them. | ||
|
||
### Running a subset of tests | ||
|
||
You can run a **limited set** of unit tests by using the `-r` or `--regex` option. This takes a regular expression, which it compares against the test names. For example to run all cellular unit tests you can specify: | ||
|
||
``` | ||
$ mbed unittest -r cellular | ||
``` | ||
|
||
### Getting code coverage | ||
|
||
You can generate a code coverage report by using the `--coverage` option. For example to create an html report you can specify: | ||
|
||
``` | ||
$ mbed unittest --coverage html | ||
``` | ||
|
||
### Creating new unit tests | ||
|
||
All unit tests are under `mbed-os/UNITTESTS` directory. You can **generate** the necessary files for a unit test by using the `--new` option. For example to create the files for `rtos/Semaphore.cpp` you can specify: | ||
|
||
``` | ||
$ mbed unittest --new rtos/Semaphore.cpp | ||
``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
## Unit testing | ||
|
||
### Introduction | ||
|
||
Traditional software testing is defined into three main levels: unit testing, integration testing and system testing. These levels are often pictured as a pyramid to indicate the amount of testing per level. | ||
|
||
``` | ||
^ Testing level | ||
| | ||
| /\ | ||
| / \ System testing | ||
| /----\ | ||
| / \ Integration testing | ||
| /--------\ | ||
| / \ Unit testing | ||
| /------------\ | ||
| | ||
*-------------------> Amount of tests | ||
``` | ||
|
||
Integration and system testing are performed in an environment where the tests are run with full Mbed OS. Other testing tools for Mbed OS require specific hardware and whole Mbed OS to be built, which means traditional unit testing is not possible. | ||
|
||
Unit testing takes place in a build environment where each C/C++ class or module is tested in isolation. This means test suites are built into separate test binaries and all access outside is stubbed to remove dependency of any specific hardware or software combination. This allows the testing to be done quickly using native compilers on the build machine. | ||
|
||
### Using unit tests | ||
|
||
#### Test code structure | ||
|
||
Unit tests are located in Mbed OS repository under `UNITTESTS`. Each unit test should use identical directory tree structure to the file to be tested. This makes it easier to find unit tests for a particular class or a module. For example if the file to be tested is `rtos/Semaphore.cpp` then all the test files should be in `UNITTESTS/rtos/Semaphore` directory. | ||
|
||
##### Test discovery | ||
|
||
Registering unit tests for running is automatic and handled by the test runner, but test files are not automatically assigned to be built. Unit tests are built using a separate build system, which will search for unit tests under `UNITTESTS` directory. | ||
|
||
For the build system to find and build any test suite automatically, a unit test configuration file named `unittest.cmake` is required to be included with each unit test. This configuration file contains a name for the test and other source files required for the build. | ||
|
||
##### Test names | ||
|
||
Each test suite requires a name to be configured in the `unittest.cmake` file. This name is used for generated files and when running a subset of tests. | ||
|
||
#### Building, running and writing tests | ||
|
||
Unit testing requires external tools which need to be installed. See the developer [documentation](https://github.com/ARMmbed/mbed-os/blob/master/UNITTESTS/README.md) in GitHub for installation guide. | ||
|
||
You can build and run unit tests through Arm Mbed CLI. The tool can also be used to generate new test files. For information on using Mbed CLI, please see the [CLI documentation](/docs/development/tools/arm-mbed-cli.html). | ||
|
||
For more information about the framework, the build process or how to write unit tests, see the GitHub documentation. |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we've removed this page.