Skip to content

TESTS: Update testing of examples in mbed-os #12179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Jan 7, 2020
Merged

TESTS: Update testing of examples in mbed-os #12179

merged 5 commits into from
Jan 7, 2020

Conversation

jamesbeyond
Copy link
Contributor

Summary of changes

  • add more ble examples to the example test suite
  • enable those ble examples to be build with DISCO_L475VG_IOT01A target
  • in the examples JSON file, replace K66F with DISCO_L475VG_IOT01A in some place to balancing the test workload

Impact of changes

Migration actions required

Documentation

Pull request type 

[x] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)
[] Feature update (New feature / Functionality change / New API)
[] Major update (Breaking change E.g. Return code change / API behaviour change)

Test results 

[] No Tests required for this change (E.g docs only update)
[x] Covered by existing mbed-os tests (Greentea or Unittest)
[] Tests / results supplied as part of this PR

Reviewers

@ciarmcom ciarmcom requested review from a team January 1, 2020 16:00
@ciarmcom
Copy link
Member

ciarmcom commented Jan 1, 2020

@jamesbeyond, thank you for your changes.
@ARMmbed/mbed-os-tools @ARMmbed/mbed-os-maintainers please review.

@40Grit
Copy link

40Grit commented Jan 1, 2020

@AGlass0fMilk

@bulislaw bulislaw changed the title TESTS: Update ble examples test in mbed-os TESTS: Update testing of examples in mbed-os Jan 2, 2020
@0xc0170
Copy link
Contributor

0xc0170 commented Jan 2, 2020

CI started

@bulislaw
Copy link
Member

bulislaw commented Jan 2, 2020

CI started.

@mbed-ci
Copy link

mbed-ci commented Jan 2, 2020

Test run: FAILED

Summary: 3 of 4 test jobs failed
Build number : 1
Build artifacts

Failed test jobs:

  • jenkins-ci/mbed-os-ci_build-GCC_ARM
  • jenkins-ci/mbed-os-ci_build-ARM
  • jenkins-ci/mbed-os-ci_build-IAR

@jamesbeyond
Copy link
Contributor Author

jamesbeyond commented Jan 2, 2020 •
edited
Loading

the failures are consistent, but not all of them make sense:
NRF51_DK/NRF52_DK failling on

  • mbed-os-example-ble/BLE_Button
  • mbed-os-example-ble/BLE_GAPButton
  • mbed-os-example-ble/BLE_GattClient
    Button and GAPButton are existing examples, been tested all the time with NRF51_DK/NRF52_DK. they should not failing
    BLE_GattClient is a new example been added in the PR, I'll try to run it manually.

K66F failed on

  • nanostack-border-router
  • mbed-os-example-mesh-minimal
  • mbed-os-example-bootloader

bootloader not support K66F, I am going to change it back
mesh-minimal claim it supports K66F, but the build failed, I'll raise an issue on the example
border-router been always running OK with K66F, which is odd it failes

@jamesbeyond
Copy link
Contributor Author

Created a PR for ble example, ARMmbed/mbed-os-example-ble#275

@jamesbeyond
Copy link
Contributor Author

jamesbeyond commented Jan 2, 2020 •
edited
Loading

I updated the PR, and fixed the issue in mbed-os-example-ble. also I added BLE feature in targets.json for DISCO_L475VG_IOT01A target. So all the BLE example will be built for DISCO_L475VG_IOT01A

FYI @donatieng @pan-

@jamesbeyond
Copy link
Contributor Author

CI re-started

@mbed-ci
Copy link

mbed-ci commented Jan 2, 2020

Test run: FAILED

Summary: 3 of 4 test jobs failed
Build number : 3
Build artifacts

Failed test jobs:

  • jenkins-ci/mbed-os-ci_build-ARM
  • jenkins-ci/mbed-os-ci_build-GCC_ARM
  • jenkins-ci/mbed-os-ci_build-IAR

@jamesbeyond
Copy link
Contributor Author

seems in target.json DISCO_L475VG_IOT01A not like the BLE feature to be added, so remove the constrains from example.json

@mbed-ci
Copy link

mbed-ci commented Jan 2, 2020

Test run: SUCCESS

Summary: 11 of 11 test jobs passed
Build number : 4
Build artifacts

@0xc0170
Copy link
Contributor

0xc0170 commented Jan 3, 2020

Restarted CI for the latest update

@mbed-ci
Copy link

mbed-ci commented Jan 3, 2020

Test run: FAILED

Summary: 1 of 11 test jobs failed
Build number : 5
Build artifacts

Failed test jobs:

  • jenkins-ci/mbed-os-ci_dynamic-memory-usage

@jamesbeyond
Copy link
Contributor Author

mbed-os-ci_dynamic-memory-usage test failed! that doesn't make sense, seems arbitrary, @0xc0170

@0xc0170
Copy link
Contributor

0xc0170 commented Jan 3, 2020 •
edited
Loading

Correct. I saw one failure today but with filesystem in dynamic memory. this time it was wifi. I need to be watching this one to find out the rootcause.

I restarted it

@mbed-ci
Copy link

mbed-ci commented Jan 3, 2020

Test run: SUCCESS

Summary: 11 of 11 test jobs passed
Build number : 6
Build artifacts

mark-edgeworth
mark-edgeworth approved these changes Jan 6, 2020
View reviewed changes
Copy link
Contributor

@mark-edgeworth mark-edgeworth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok to me

@jamesbeyond
Copy link
Contributor Author

ping @0xc0170. I think this PR is good to go

@adbridge adbridge added ready for merge release-version: 6.0.0-alpha-1 First pre-release version of 6.0.0 and removed needs: CI labels Jan 6, 2020
@0xc0170 0xc0170 merged commit b943b5f into ARMmbed:master Jan 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bulislaw bulislaw bulislaw approved these changes

@mark-edgeworth mark-edgeworth mark-edgeworth approved these changes

@0xc0170 0xc0170 0xc0170 approved these changes

Assignees
No one assigned
Labels
release-version: 6.0.0-alpha-1 First pre-release version of 6.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@jamesbeyond @ciarmcom @40Grit @0xc0170 @bulislaw @mbed-ci @mark-edgeworth @adbridge