Implement device management subcommand #7844

theotherjimmy · 2018-08-21T15:45:23Z

Description

This series implements the device-managment/dev-mgmt/dm mbed cli
subcommand. This subcommand is much like all of manifest tool, except it
also removes a few difficulties with using the manifest tool with an
Mbed OS project. In particular:

mbed dm init will pull create/find and download a developer
certificate for you
mbed dm update will automatially populate the payload feild with
the most recently built "application" side of a managed BL mode project

Note: mbed dm subcommand implemented by ARMmbed/mbed-cli#735

Pull request type

[ ] Fix
[ ] Refactor
[ ] New target
[x] Feature
[ ] Breaking change

cmonr · 2018-08-22T02:32:23Z

tools/device_management.py

+                app_name, output_ext
+            ))
+            options.payload = open(payload_name, "rb")
+        return func(options)


Very interesting pattern for wrapping a function call.

cmonr · 2018-08-22T02:33:28Z

@theotherjimmy Can you think of anyone else who should probably review this PR?

theotherjimmy · 2018-08-22T14:25:36Z

@bridadan Maybe would be a good candidate

bridadan

Looks good! I think a few comments sprinkled throughout some of the more obscure bits would help for future maintenance/readability, but otherwise good!

bridadan · 2018-08-22T21:19:53Z

tools/device_management.py

+from tools.options import extract_mcus
+
+
+class MbedExtendedArgs(MainArgumentParser):


So this class is basically "eating up" the unused arguments that mbed-cli will always provide? Do you think you could add a few code comments describing what this class is doing for other maintainers?

This class is not doing that. It's removing the required-ness of the "payload" parameter and adding another way to specify the payload as a build artefact.

Ok, I guess my initial feedback about the code comments still stands then. But that's just for folks who poke at this code, not for the use of the feature (aka not a blocker).

bridadan · 2018-08-22T21:21:59Z

tools/device_management.py

+        with open("mbed_cloud_dev_credentials.c", "w") as fout:
+            fout.write(dev_cert_info.header_file)
+        return func(options)
+    return inner


This is awesome! Nice 👍

Depens on ARMmbed/mbed-cli#735 and ARMmbed/mbed-os#7844

cmonr · 2018-08-24T05:55:20Z

/morph build

mbed-ci · 2018-08-24T07:21:56Z

Build : SUCCESS

Build number : 2887
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7844/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build
/morph mbed2-build

mbed-ci · 2018-08-24T10:43:00Z

Exporter Build : SUCCESS

Build number : 2512
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/7844/

cmonr · 2018-08-26T00:49:57Z

/morph test

cmonr · 2018-08-27T03:05:12Z

/morph test

0xc0170 · 2018-08-27T11:16:16Z

requirements.txt

@@ -15,3 +15,5 @@ pyelftools>=0.24
 jsonschema>=2.6
 future>=0.16.0
 six>=1.11.0
+git+https://github.com/armmbed/[email protected]
+mbed-cloud-sdk==2.0.0


This is causing test to fail to initialize. Please review the test results (find them via the log, they did not report back as it fails really early)

Collecting mbed-cloud-sdk==2.0.0 (from -r requirements.txt (line 19)) python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:369: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings SNIMissingWarning

https://pypi.org/project/mbed-cloud-sdk/

It exists, why is CI failing to install it?

@0xc0170 Fwiw, I was unable to reproduce the failure. pip install -r requirements when using this PR head worked fine.

Ah! Got the log:

pip install -r requirements.txt Collecting git+https://github.com/armmbed/[email protected] (from -r requirements.txt (line 18)) Cloning https://github.com/armmbed/manifest-tool.git (to revision v1.4.5) to /tmp/pip-req-build-x1310E Requirement already satisfied: colorama>=0.3.3 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 1)) (0.3.9) Requirement already satisfied: PySerial>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 2)) (3.4) Requirement already satisfied: PrettyTable>=0.7.2 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 3)) (0.7.2) Requirement already satisfied: Jinja2>=2.7.3 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 4)) (2.10) Requirement already satisfied: IntelHex>=1.3 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 5)) (2.1) Requirement already satisfied: junit-xml in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 6)) (1.8) Requirement already satisfied: pyYAML in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 7)) (3.12) Requirement already satisfied: requests in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 8)) (2.2.1) Requirement already satisfied: mbed-ls>=0.2.13 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 9)) (1.3.5) Requirement already satisfied: mbed-host-tests>=1.1.2 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 10)) (1.3.0) Requirement already satisfied: mbed-greentea>=0.2.24 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 11)) (1.3.2) Requirement already satisfied: beautifulsoup4>=4 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 12)) (4.6.0) Requirement already satisfied: fuzzywuzzy>=0.11 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 13)) (0.16.0) Requirement already satisfied: pyelftools>=0.24 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 14)) (0.24) Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 15)) (2.6.0) Requirement already satisfied: future>=0.16.0 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 16)) (0.16.0) Requirement already satisfied: six>=1.11.0 in ./.env/lib/python2.7/site-packages (from -r requirements.txt (line 17)) (1.11.0) Collecting mbed-cloud-sdk==2.0.0 (from -r requirements.txt (line 19)) /redacted/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:369: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings SNIMissingWarning /redacted/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecurePlatformWarning Could not find a version that satisfies the requirement mbed-cloud-sdk==2.0.0 (from -r requirements.txt (line 19)) (from versions: ) No matching distribution found for mbed-cloud-sdk==2.0.0 (from -r requirements.txt (line 19))

cmonr · 2018-08-29T20:51:54Z

Retriggering to get CI information.

/morph test

This pulls in additional packages requiured for SSL verification for Py2 Ref: https://urllib3.readthedocs.io/en/latest/user-guide.html#ssl-py2

cmonr · 2018-08-29T21:21:28Z

/morph test

cmonr · 2018-08-29T21:25:03Z

/morph test

mbed-ci · 2018-08-29T22:53:53Z

Test : FAILURE

Build number : 2704
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7844/2704

cmonr · 2018-08-29T22:57:40Z

/morph build

cmonr · 2018-08-29T23:36:30Z

Because of course we would have license issues...

/morph build

mbed-ci · 2018-08-30T00:34:18Z

Build : FAILURE

Build number : 2959
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7844/

mbed-ci · 2018-08-30T01:33:09Z

Build : SUCCESS

Build number : 2961
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7844/

Triggering tests

/morph test
/morph export-build
/morph mbed2-build

cmonr · 2018-08-30T03:22:17Z

/morph test

mbed-ci · 2018-08-30T05:06:33Z

Exporter Build : SUCCESS

Build number : 2575
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/7844/

mbed-ci · 2018-08-30T05:34:59Z

Test : FAILURE

Build number : 2708
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7844/2708

0xc0170 · 2018-08-30T07:24:34Z

CI error

/morph test

0xc0170 · 2018-08-30T08:48:41Z

Seems like Ci is not triggering the build, one more try and then I'll run it manually

/morph test

cmonr · 2018-08-30T16:50:58Z

/morph test

mbed-ci · 2018-08-31T00:02:12Z

Test : SUCCESS

Build number : 2717
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7844/2717

Jimmy Brisson and others added 5 commits August 21, 2018 10:51

Import manifest-tool

03fa626

Create a "device-management" subcommand

ebb016e

Implement device management subcommand

563ee0d

Extend init to maybe create and always DL dev cert

93309cd

Clean up formatting of device_management.py

a6163cb

theotherjimmy force-pushed the fw-update-integration branch from 3f03171 to a6163cb Compare August 21, 2018 15:51

cmonr self-requested a review August 22, 2018 01:47

cmonr added the release-version: 5.10.0-rc1 label Aug 22, 2018

cmonr reviewed Aug 22, 2018

View reviewed changes

cmonr approved these changes Aug 22, 2018

View reviewed changes

cmonr added the needs: review label Aug 22, 2018

bridadan approved these changes Aug 22, 2018

View reviewed changes

theotherjimmy added a commit to theotherjimmy/Handbook that referenced this pull request Aug 23, 2018

Add documentation about mbed dm

0dd6a93

Depens on ARMmbed/mbed-cli#735 and ARMmbed/mbed-os#7844

theotherjimmy mentioned this pull request Aug 23, 2018

Add documentation about mbed dm ARMmbed/mbed-os-5-docs#681

Merged

cmonr added risk: G needs: CI and removed needs: review labels Aug 24, 2018

0xc0170 reviewed Aug 27, 2018

View reviewed changes

0xc0170 added needs: work and removed needs: CI labels Aug 27, 2018

MarceloSalazar mentioned this pull request Aug 27, 2018

Introduce device-management/dev-mgmt/dm subcommand ARMmbed/mbed-cli#735

Merged

theotherjimmy force-pushed the fw-update-integration branch from b2bfddc to a6163cb Compare August 28, 2018 16:00

0xc0170 added the risk: A label Aug 29, 2018

0xc0170 removed the risk: G label Aug 29, 2018

Added secure urllib3 python requirement

c89147e

This pulls in additional packages requiured for SSL verification for Py2 Ref: https://urllib3.readthedocs.io/en/latest/user-guide.html#ssl-py2

Replace urllib3[secure] with discrete packages

f46bcf1

cmonr mentioned this pull request Aug 29, 2018

Add initial NFC support to Mbed OS #7822

Merged

Remove extra package used for CI debugging

ebdad75

0xc0170 added needs: CI and removed needs: work labels Aug 30, 2018

cmonr mentioned this pull request Aug 30, 2018

Export, MCUXpresso: Avoid assigning self.libraries #7937

Merged

cmonr merged commit b6f0ee2 into ARMmbed:master Aug 31, 2018

0xc0170 removed needs: CI risk: A labels Aug 31, 2018

		from tools.options import extract_mcus


		class MbedExtendedArgs(MainArgumentParser):

Implement device management subcommand #7844

Implement device management subcommand #7844

Uh oh!

Conversation

theotherjimmy commented Aug 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Pull request type

Uh oh!

cmonr Aug 22, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmonr commented Aug 22, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theotherjimmy commented Aug 22, 2018

Uh oh!

bridadan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bridadan Aug 24, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bridadan Aug 22, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmonr commented Aug 24, 2018

Uh oh!

mbed-ci commented Aug 24, 2018

Triggering tests

Uh oh!

mbed-ci commented Aug 24, 2018

Uh oh!

cmonr commented Aug 26, 2018

Uh oh!

cmonr commented Aug 27, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmonr commented Aug 29, 2018

Uh oh!

cmonr commented Aug 29, 2018

Uh oh!

cmonr commented Aug 29, 2018

Uh oh!

mbed-ci commented Aug 29, 2018

Uh oh!

cmonr commented Aug 29, 2018

Uh oh!

cmonr commented Aug 29, 2018

Uh oh!

mbed-ci commented Aug 30, 2018

Uh oh!

mbed-ci commented Aug 30, 2018

Triggering tests

Uh oh!

cmonr commented Aug 30, 2018

Uh oh!

mbed-ci commented Aug 30, 2018

Uh oh!

mbed-ci commented Aug 30, 2018

Uh oh!

0xc0170 commented Aug 30, 2018

Uh oh!

0xc0170 commented Aug 30, 2018

Uh oh!

cmonr commented Aug 30, 2018

theotherjimmy commented Aug 21, 2018 •

edited

Loading

cmonr Aug 22, 2018 •

edited

Loading

cmonr commented Aug 22, 2018 •

edited

Loading

bridadan Aug 24, 2018 •

edited

Loading

bridadan Aug 22, 2018 •

edited

Loading