Skip to content

Add documentation about icetea testcases #8073

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 22, 2018
Merged

Add documentation about icetea testcases #8073

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions TEST_APPS/testcases/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
Icetea tests
============

This folder contains all the test cases done with Icetea residing in `mbed-os`.
The tests are divided in to subfolders and each subfolder contains a set of testcases.
The subfolder has a description of all the testcases it contains.

Testcases
---------

Current testcases:

- [netsocket](https://github.com/ARMmbed/mbed-os/blob/master/TEST_APPS/testcases/netsocket)
- [example](https://github.com/ARMmbed/mbed-os/blob/master/TEST_APPS/testcases/example)
24 changes: 24 additions & 0 deletions TEST_APPS/testcases/example/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
Example tests
=============

This folder contains example tests for Icetea
The test located under this folder is dependent of the application [exampleapp](https://github.com/ARMmbed/mbed-os/blob/master/TEST_APPS/device/exampleapp)
The exampleapp is disabled by default, to be able to run the test_cmdline with the exampleapp, either remove the preprocessor macro from exampleapp.cpp or add `-DICETEA_EXAMPLE_ENABLED` to the mbed test command

Testcases
---------

### test_cmdline

**Description:**
Send command line commands to target over serial interface.
This test introduces the Icetea testcase structure to the user.

**Test steps:**
Send "echo hello world" to the target.
Target sends "hello world" back to the host machine.
Send help to the target.
Target prints out the command line commands the application supports.

**Expected result:**
The test exits without timeouts.
7 changes: 7 additions & 0 deletions TEST_APPS/testcases/netsocket/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Netsocket tests
===============

This folder contains netsocket tests for Icetea
The tests located under this folder are dependent of the application [socket_app](https://github.com/ARMmbed/mbed-os/blob/master/TEST_APPS/device/socket_app)

The test cases under this folder are defined in [Network Socket test plan](https://github.com/ARMmbed/mbed-os/blob/master/TESTS/netsocket/README.md)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SeppoTakalo , can you help us to review this please?
The "Network Socket test plan" does not have references to Icetea, so I don't see the relationship with doc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally it did not refer to any testing tool, but since then I have added instructions of how to build existing test cases.
Icetea release was pending, so I could not add reference to this.
However, testcases are exactly same, so this sentence here is correct.
Only the test plan needs updating, but that is ongoing maintaining work for us to keep it up to date.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a user, I still don't understand how to run the Icetea test cases with the network sockets.
@jarlamsa @SeppoTakalo can we add a line explaining how to run the test cases?

We have this today (greentea):
mbed test --compile -t <toolchain> -m <target> -n mbed-os-tests-network-*,mbed-os-tests-netsocket*

I assume this is the corresponding command to use icetea but I'm not 100% sure.

mbed test --compile -t <toolchain> -m <target> -n mbed-os-tests-network-*,mbed-os-tests-netsocket* --icetea

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is already a readme about running the icetea tests in the root of TEST_APPS -folder
https://github.com/ARMmbed/mbed-os/blob/master/TEST_APPS/readme.md

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that's good.
@SeppoTakalo , I assume every test case (mbed-os-tests-network-*,mbed-os-tests-netsocket*) can be used with Icetea. If that's the case, I'd be happy to close this item.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only test cases done with Icetea can be used with Icetea, those are located under TEST_APPS/testcases, test cases done with Greentea can be used with Greentea, those are located under TESTS. The test plan needs information about used test framework for each test.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.
Some testcases exist only in Greentea, some exist only in Icetea.
All testcases exist in the test plan, and the selection of tool is based on requirement of the testcase.

For demonstration purposes some testcases exist on both, so that the differences can be studied.