Tools changes for bare metal #9561

theotherjimmy · 2019-01-30T17:42:27Z

Description

This PR implements (and tests :D) the changes to the scanning rules
and configuration system for the requires key in mbed_app.json
and mbed_lib.json. Note: This is a 5.12 feature.

The requires key in mbed_lib.json lists the other libraries
that are required to build this library. When this library is
required by an application, the libraries it requires will be
included in the build.

The requires key in mbed_app.json lists the libraries (name
key from mbed_lib.json) required to build the application. This
causes mbed compile to filter the libraries included to those
listed in the application and those listed by the required
libraries.

Note: This pull request includes unit tests for this new feature.

Pull request type

[ ] Fix
[ ] Refactor
[ ] Target update
[x] Functionality change
[ ] Docs update
[ ] Test update
[ ] Breaking change

Reviewers

(following section required to auto-generate release notes)

Release notes

Mbed OS now supports the requires section in both
mbed_app.json and mbed_lib.json.

The requires key in mbed_lib.json lists the other libraries
that are required to build this library. When this library is
required by an application, the libraries it requires will be
included in the build.

The requires key in mbed_app.json lists the libraries (name
key from mbed_lib.json) required to build the application. This
causes mbed compile to filter the libraries included to those
listed in the application and those listed by the required
libraries.

No migration is required.

as the config system uses only abspaths

ciarmcom · 2019-01-30T18:00:37Z

@theotherjimmy, thank you for your changes.
@ARMmbed/mbed-os-maintainers @ARMmbed/mbed-os-pan @ARMmbed/mbed-os-tools @ARMmbed/mbed-os-storage please review.

bridadan

Really nice so far!

tools/test/config/config_test.py

tools/test/resources/resource_test.py

tools/config/__init__.py

cmonr · 2019-01-30T23:24:03Z

tools/resources/__init__.py

@@ -269,7 +269,7 @@ def get_file_refs(self, file_type):
                if not any(
                   path.startswith(dirname(e.path)) for e in self._excluded_libs
                ):
-        	    to_ret.append(ref)
+                    to_ret.append(ref)


I switched editors recently, until it decided to never insert spaces when I type <tab>. I have switch back because that's infuriating when working with python.

0xc0170 · 2019-02-11T10:43:00Z

@theotherjimmy Please add Release notes section (following https://os.mbed.com/docs/mbed-os/v5.11/contributing/workflow.html#functionality-change)

0xc0170

Waiting for design doc, how this is supposed to be used? Release notes section might help there.

0xc0170 · 2019-02-11T10:44:05Z

tools/test/resources/resource_test.py

@@ -0,0 +1,136 @@
+# mbed SDK
+# Copyright (c) 2019 ARM Limited
+#


Please also add SPDX identifier for new files

donatieng

I'd like to +1 the request for a design doc before we can review this efficiently!

donatieng · 2019-02-11T11:32:06Z

tools/config/definitions.json

@@ -6,6 +6,13 @@
      "type": "string"
    }
  },
+  "requires_definition": {
+    "descirption": "Required libraries",


mikisch81 · 2019-02-11T12:08:26Z

I'd like to +1 the request for a design doc before we can review this efficiently!

@donatieng
I sent offline link to confluence page about this, if possible to start look at the changes so @theotherjimmy can address any new issues (if any)?

spdx added

0xc0170 · 2019-02-11T17:02:07Z

Should this PR include design doc? docs/design-documents is proper location

donatieng · 2019-02-12T09:45:46Z

Is there a deprecation path for "features" (and components)? I feel like this requirement overlaps with this functionality. If so, it'd be worth adding an mbed_lib.json file for Bluetooth as well.

theotherjimmy · 2019-02-12T14:49:44Z

@donatieng I'm not aware of a plan to deprecate features. I can understand the overlap you see. Adding an mbed_lib.json for Bluetooth could be something that is reasonable for the future. That can certainly be a follow on PR, as I would like to get this PR in soon.

donatieng

Thanks for the clarification @theotherjimmy ! Yes I think that the complexity of all these somewhat overlapping features ("features", "components" and now this) needs to be addressed otherwise this will hurt our UX.

theotherjimmy · 2019-02-12T15:14:37Z

@donatieng I agree that no guidance on "features", "components", extra_labels, and requires in mbed_app.json and mbed_lib.json will only lead to more confusion long term. I think we could probably deprecate "features" and "components" in favor of requires.

SeppoTakalo · 2019-02-12T15:35:43Z

I'm also in favour of deprecating features and components in favour of requires as those have working dependency system.

Components and features have pushed the integration pain towards the application developer, who need to be aware of what library requires what feature.

orenc17 · 2019-02-12T18:12:30Z

@donatieng @theotherjimmy @SeppoTakalo i agree with you, but we need this for THIS release
in the one after that we could do whatever we want

bridadan

Agreed @orenc17, we need to

😄

orenc17 · 2019-02-13T07:16:10Z

can someone run CI on this please?

NirSonnenschein · 2019-02-13T07:52:52Z

CI started

mbed-ci · 2019-02-13T10:01:24Z

Test run: FAILED

Summary: 1 of 11 test jobs failed
Build number : 2
Build artifacts

Failed test jobs:

jenkins-ci/mbed-os-ci_greentea-test

orenc17 · 2019-02-13T10:03:18Z

@ARMmbed/mbed-os-maintainers why was this build aborted?

mbed-ci · 2019-02-13T10:38:40Z

Test run: FAILED

Summary: 4 of 9 test jobs failed
Build number : 5
Build artifacts

Failed test jobs:

jenkins-ci/mbed-os-ci_dynamic-memory-usage
jenkins-ci/mbed-os-ci_cloud-client-test
jenkins-ci/mbed-os-ci_exporter
jenkins-ci/mbed-os-ci_greentea-test

alekla01 · 2019-02-13T11:25:22Z

Restarted CI, was aborted & restarted due to CI related error.

mbed-ci · 2019-02-13T12:52:49Z

Test run: SUCCESS

Summary: 12 of 12 test jobs passed
Build number : 7
Build artifacts

orenc17 · 2019-02-13T18:33:17Z

Hallelujah!!!!!!!

theotherjimmy and others added 16 commits January 9, 2019 10:43

Test the Resources object

e10adff

Implement and test filtering resources by libraries

e2849e6

Add failing test that asserts mbed_app.json can require

dea8ea9

Pass requires in mbed_app.json test

243b079

Add test case for requires from a library

9f6a096

Resolve library dependencies

79eccd1

Add test that missing requires cause a config failure

4dd525d

Raise config error when missing dependencies

7a360ef

Test that libraries' source is excluded

c5a721c

Filter resources by included libraries

0b64c9c

Test that we don't exclude lib sources w/o requires

c225f2c

Correct usage of get_config

eaa4ee4

Use name definition in requires definition

4fbbac9

Filter libs with absolute paths

5cf07df

as the config system uses only abspaths

Add placeholder libraries for things we probably refer to as libraries

e201fdb

Minor fixes

06ea053

theotherjimmy changed the title ~~Tools channges for bare metal~~ Tools changes for bare metal Jan 30, 2019

ciarmcom requested review from a team January 30, 2019 18:00

ciarmcom added the needs: review label Jan 30, 2019

bridadan reviewed Jan 30, 2019

View reviewed changes

tools/test/config/config_test.py Outdated Show resolved Hide resolved

tools/test/resources/resource_test.py Outdated Show resolved Hide resolved

tools/test/resources/resource_test.py Show resolved Hide resolved

tools/config/__init__.py Outdated Show resolved Hide resolved

theotherjimmy added 2 commits January 30, 2019 15:06

Address review comments

ee5fea4

Fix indentation

54d6e86

cmonr reviewed Jan 30, 2019

View reviewed changes

0xc0170 added the release-version: 5.12.0-rc1 label Jan 31, 2019

theotherjimmy added 2 commits January 31, 2019 09:15

Check that recursive libs work correctly

eb6a972

Handle recursive lib exclude/includes

250a1de

0xc0170 previously requested changes Feb 11, 2019

View reviewed changes

donatieng suggested changes Feb 11, 2019

View reviewed changes

theotherjimmy added 2 commits February 11, 2019 08:45

Fix typo in config schema definitons

5b96cbf

Add SPDX identifier for new file

f331d9e

donatieng approved these changes Feb 12, 2019

View reviewed changes

bridadan approved these changes Feb 12, 2019

View reviewed changes

mikisch81 mentioned this pull request Feb 13, 2019

Some build issues with PSA_Target on M2351 #9699

Closed

0xc0170 approved these changes Feb 13, 2019

View reviewed changes

cmonr added ready for merge and removed needs: review labels Feb 13, 2019

cmonr approved these changes Feb 13, 2019

View reviewed changes

cmonr merged commit b820ec8 into ARMmbed:master Feb 13, 2019

cmonr removed the ready for merge label Feb 13, 2019

Tools changes for bare metal #9561

Tools changes for bare metal #9561

Uh oh!

Conversation

theotherjimmy commented Jan 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Pull request type

Reviewers

Release notes

Uh oh!

ciarmcom commented Jan 30, 2019

Uh oh!

bridadan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

0xc0170 commented Feb 11, 2019

Uh oh!

0xc0170 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

donatieng left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mikisch81 commented Feb 11, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

0xc0170 commented Feb 11, 2019

Uh oh!

donatieng commented Feb 12, 2019

Uh oh!

theotherjimmy commented Feb 12, 2019

Uh oh!

donatieng left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

theotherjimmy commented Feb 12, 2019

Uh oh!

SeppoTakalo commented Feb 12, 2019

Uh oh!

orenc17 commented Feb 12, 2019

Uh oh!

bridadan left a comment

Choose a reason for hiding this comment

Uh oh!

orenc17 commented Feb 13, 2019

Uh oh!

NirSonnenschein commented Feb 13, 2019

Uh oh!

mbed-ci commented Feb 13, 2019

Uh oh!

orenc17 commented Feb 13, 2019

Uh oh!

mbed-ci commented Feb 13, 2019

Uh oh!

alekla01 commented Feb 13, 2019

Uh oh!

mbed-ci commented Feb 13, 2019

Uh oh!

orenc17 commented Feb 13, 2019

Uh oh!

Uh oh!

theotherjimmy commented Jan 30, 2019 •

edited

Loading

mikisch81 commented Feb 11, 2019 •

edited

Loading

donatieng left a comment •

edited

Loading