Add CPU stats for greentea tests #7294

jeromecoutant · 2018-06-21T13:01:19Z

Description

Here is a proposition to add CPU info at the end of each greentea tests.

[1529585766.11][CONN][RXD] {{__cpu_info up time;1779541}}
[1529585766.15][CONN][RXD] {{__cpu_info sleep time;4516}}
[1529585766.19][CONN][RXD] {{__cpu_info deepsleep time;931397}}
[1529585766.22][CONN][RXD] {{__cpu_info % sleep / deep;0;49}}

Pull request type

[ ] Fix
[ ] Refactor
[ ] New target
[x] Feature
[ ] Breaking change

screamerbg · 2018-06-21T13:08:10Z

I like this! Do we have some way to measure tick counts and use the data as performance measurement? @0xc0170

cmonr

I -really- like this.

cmonr · 2018-06-21T13:21:26Z

@studavekar Thoughts?

cmonr · 2018-06-21T13:22:09Z

@screamerbg I think the people that could best answer that are @SenRamakri and/or @deepikabhavnani

deepikabhavnani · 2018-06-21T13:43:54Z

I like this! Do we have some way to measure tick counts and use the data as performance measurement?

Performance in terms of what?
We have way to calculate the CPU load (https://github.com/ARMmbed/mbed-os-example-cpu-usage/blob/master/main.cpp) which can be compared across different targets and application.

With the help of deep sleep / sleep comparison you can compare to see if deep sleep will help in battery saving (need to do some target specific analysis for that) - May be in future

deepikabhavnani · 2018-06-21T13:49:19Z

features/frameworks/greentea-client/source/greentea_metrics.cpp

+    greentea_send_kv("__cpu_info        up time", mbed_uptime());
+    greentea_send_kv("__cpu_info     sleep time", mbed_time_sleep());
+    greentea_send_kv("__cpu_info deepsleep time", mbed_time_deepsleep());
+    greentea_send_kv("__cpu_info % sleep / deep", (mbed_time_sleep() * 100) / mbed_uptime(), (mbed_time_deepsleep() * 100) / mbed_uptime());


(mbed_time_deepsleep() * 100) / mbed_uptime())

Not sure how useful this stat is, it says how much % time system spent in sleep since it was up. But will be interesting to see its result for different test application.

studavekar · 2018-06-21T16:42:55Z

This would be a good addition to provide additional cpu information, It would useful for teams to check device sleep status during the test execution. 👍

cmonr · 2018-06-21T16:47:31Z

/morph build

mbed-ci · 2018-06-21T18:39:01Z

Build : FAILURE

Build number : 2424
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7294/

cmonr · 2018-06-21T22:41:43Z

Darn. Looks like it's not going to be that simple and easy.

@jeromecoutant, please take a look at the build failures.

deepikabhavnani · 2018-06-21T23:28:42Z

features/frameworks/greentea-client/source/greentea_metrics.cpp

+#if defined(MBED_CPU_STATS_ENABLED)
+static void send_CPU_info()
+{
+    greentea_send_kv("__cpu_info        up time", mbed_uptime());


mbed_uptime , mbed_time_sleep , mbed_time_deepsleep are not API's. They are internal functions in Mbed OS.
Please use mbed_stats_cpu_get to get all CPU related info

@deepikabhavnani We might want to add that detail to the documentation? These functions are declared in power management header file - thus I would also use it . I do not see any internal detail mentioned there.

If they are internal, why are there declared then in the power m. header file (we got there _internal functions - intention is clear).

@0xc0170 - I was not completely clear with my comment, those functions can be used if required. But since this PR is about adding CPU stats for testing it will be good to use actual API mbed_stats_cpu_get and not the functions called by API.

Understood . Still confusing to a user - which one to use? Is it fine as it is, all those functions exposed ?

@jeromecoutant Can you update to use mbed_stats_cpu_get rather?

User can use mbed_uptime just to know since when the system is up and running (ok to have public). mbed_stats_cpu_get will give all the cpu related information. We have plans to expand this API to have additional information and parts of which may not be in power management header file.

0xc0170

We need to be clear about mbed_uptime vs mbed_stats.

@deepikabhavnani Please address the comment about those functions. Once done, this should be good to go

0xc0170

@jeromecoutant Can you update to use mbed_stats_cpu_get rather?

jeromecoutant · 2018-06-28T08:56:35Z

@0xc0170 Update is done

0xc0170 · 2018-06-28T12:15:51Z

Considering that previous set was approved (similar to the current one, just one change there), starting CI

/morph build

mbed-ci · 2018-06-28T13:41:04Z

Build : SUCCESS

Build number : 2471
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7294/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build
/morph mbed2-build

mbed-ci · 2018-06-28T14:57:49Z

Exporter Build : FAILURE

Build number : 2100
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/7294/

cmonr · 2018-06-28T16:12:41Z

/morph export-build

mbed-ci · 2018-06-28T17:52:20Z

Exporter Build : FAILURE

Build number : 2103
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/7294/

mbed-ci · 2018-06-28T19:43:54Z

Test : SUCCESS

Build number : 2248
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7294/2248

cmonr · 2018-06-29T00:49:34Z

/morph export-build

mbed-ci · 2018-06-29T02:44:24Z

Exporter Build : SUCCESS

Build number : 2109
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/7294/

0xc0170 added the needs: review label Jun 21, 2018

0xc0170 requested review from a team, OPpuolitaival and studavekar June 21, 2018 13:09

cmonr previously approved these changes Jun 21, 2018

View reviewed changes

deepikabhavnani reviewed Jun 21, 2018

View reviewed changes

studavekar previously approved these changes Jun 21, 2018

View reviewed changes

cmonr added needs: CI and removed needs: review labels Jun 21, 2018

cmonr added needs: work and removed needs: CI labels Jun 21, 2018

deepikabhavnani reviewed Jun 21, 2018

View reviewed changes

OPpuolitaival previously approved these changes Jun 26, 2018

View reviewed changes

0xc0170 requested changes Jun 27, 2018

View reviewed changes

Add CPU stats for greentea tests

35c9ddc

jeromecoutant dismissed stale reviews from OPpuolitaival, studavekar, and cmonr via 35c9ddc June 28, 2018 08:55

jeromecoutant force-pushed the PR_GREENTEA_STAT branch from ebd8655 to 35c9ddc Compare June 28, 2018 08:55

0xc0170 approved these changes Jun 28, 2018

View reviewed changes

0xc0170 added needs: review and removed needs: work labels Jun 28, 2018

deepikabhavnani approved these changes Jun 28, 2018

View reviewed changes

0xc0170 added ready for merge and removed needs: review labels Jun 29, 2018

cmonr merged commit f10bb88 into ARMmbed:master Jun 30, 2018

0xc0170 removed the ready for merge label Jun 30, 2018

cmonr added the release-version: 5.10.0-rc1 label Jun 30, 2018

jeromecoutant deleted the PR_GREENTEA_STAT branch July 2, 2018 07:24

Add CPU stats for greentea tests #7294

Add CPU stats for greentea tests #7294

Uh oh!

Conversation

jeromecoutant commented Jun 21, 2018

Description

Pull request type

Uh oh!

screamerbg commented Jun 21, 2018

Uh oh!

cmonr left a comment

Choose a reason for hiding this comment

Uh oh!

cmonr commented Jun 21, 2018

Uh oh!

cmonr commented Jun 21, 2018

Uh oh!

deepikabhavnani commented Jun 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

studavekar commented Jun 21, 2018

Uh oh!

cmonr commented Jun 21, 2018

Uh oh!

mbed-ci commented Jun 21, 2018

Uh oh!

cmonr commented Jun 21, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

0xc0170 Jun 27, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

0xc0170 left a comment

Choose a reason for hiding this comment

Uh oh!

0xc0170 left a comment

Choose a reason for hiding this comment

Uh oh!

jeromecoutant commented Jun 28, 2018

Uh oh!

0xc0170 commented Jun 28, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mbed-ci commented Jun 28, 2018

Triggering tests

Uh oh!

mbed-ci commented Jun 28, 2018

Uh oh!

cmonr commented Jun 28, 2018

Uh oh!

mbed-ci commented Jun 28, 2018

Uh oh!

mbed-ci commented Jun 28, 2018

Uh oh!

cmonr commented Jun 29, 2018

Uh oh!

mbed-ci commented Jun 29, 2018

Uh oh!

Uh oh!

deepikabhavnani commented Jun 21, 2018 •

edited

Loading

0xc0170 Jun 27, 2018 •

edited

Loading

0xc0170 commented Jun 28, 2018 •

edited

Loading