Skip to content

Implement device management subcommand #7844

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Aug 31, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,5 @@ pyelftools>=0.24
jsonschema>=2.6
future>=0.16.0
six>=1.11.0
git+https://github.com/armmbed/[email protected]
mbed-cloud-sdk==2.0.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is causing test to fail to initialize. Please review the test results (find them via the log, they did not report back as it fails really early)

Collecting mbed-cloud-sdk==2.0.0 (from -r requirements.txt (line 19))
python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:369: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  SNIMissingWarning

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://pypi.org/project/mbed-cloud-sdk/

It exists, why is CI failing to install it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0xc0170 Fwiw, I was unable to reproduce the failure. pip install -r requirements when using this PR head worked fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! Got the log:

pip install -r requirements.txt
Collecting git+https://github.com/armmbed/[email protected] (from -r requirements.txt (line 18))
  Cloning https://github.com/armmbed/manifest-tool.git (to revision v1.4.5) to /tmp/pip-req-build-x1310E
Requirement already satisfied: colorama>=0.3.3 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 1)) (0.3.9)
Requirement already satisfied: PySerial>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 2)) (3.4)
Requirement already satisfied: PrettyTable>=0.7.2 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 3)) (0.7.2)
Requirement already satisfied: Jinja2>=2.7.3 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 4)) (2.10)
Requirement already satisfied: IntelHex>=1.3 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 5)) (2.1)
Requirement already satisfied: junit-xml in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 6)) (1.8)
Requirement already satisfied: pyYAML in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 7)) (3.12)
Requirement already satisfied: requests in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 8)) (2.2.1)
Requirement already satisfied: mbed-ls>=0.2.13 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 9)) (1.3.5)
Requirement already satisfied: mbed-host-tests>=1.1.2 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 10)) (1.3.0)
Requirement already satisfied: mbed-greentea>=0.2.24 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 11)) (1.3.2)
Requirement already satisfied: beautifulsoup4>=4 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 12)) (4.6.0)
Requirement already satisfied: fuzzywuzzy>=0.11 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 13)) (0.16.0)
Requirement already satisfied: pyelftools>=0.24 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 14)) (0.24)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 15)) (2.6.0)
Requirement already satisfied: future>=0.16.0 in /usr/local/lib/python2.7/dist-packages (from -r requirements.txt (line 16)) (0.16.0)
Requirement already satisfied: six>=1.11.0 in ./.env/lib/python2.7/site-packages (from -r requirements.txt (line 17)) (1.11.0)
Collecting mbed-cloud-sdk==2.0.0 (from -r requirements.txt (line 19))
/redacted/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:369: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  SNIMissingWarning
/redacted/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
  Could not find a version that satisfies the requirement mbed-cloud-sdk==2.0.0 (from -r requirements.txt (line 19)) (from versions: )
No matching distribution found for mbed-cloud-sdk==2.0.0 (from -r requirements.txt (line 19))

155 changes: 155 additions & 0 deletions tools/device_management.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
#! /usr/bin/env python2
"""
mbed SDK
Copyright (c) 2011-2013 ARM Limited

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.


device-management, dev-mgmt, and dm sub command
"""
from __future__ import print_function, absolute_import
import logging
import sys
import argparse
from os.path import join, abspath, dirname, basename
from os import getenv

from manifesttool import create, parse, verify, cert, init, update
from manifesttool.argparser import MainArgumentParser
from mbed_cloud import AccountManagementAPI, CertificatesAPI
import colorama
colorama.init()


LOG = logging.getLogger(__name__)
LOG_FORMAT = '[%(levelname)s] %(asctime)s - %(name)s - %(message)s'

# Be sure that the tools directory is in the search path
ROOT = abspath(join(dirname(__file__), ".."))
sys.path.insert(0, ROOT)

from tools.config import Config
from tools.options import extract_mcus


class MbedExtendedArgs(MainArgumentParser):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this class is basically "eating up" the unused arguments that mbed-cli will always provide? Do you think you could add a few code comments describing what this class is doing for other maintainers?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class is not doing that. It's removing the required-ness of the "payload" parameter and adding another way to specify the payload as a build artefact.

Copy link
Contributor

@bridadan bridadan Aug 24, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I guess my initial feedback about the code comments still stands then. But that's just for folks who poke at this code, not for the use of the feature (aka not a blocker).

def _addCreateArgs(self, parser, exclusions=[]):
if 'payload' not in exclusions:
parser.add_argument(
'-p', '--payload',
help='Supply a local copy of the payload file.'
'This option overrides any payload file supplied in a '
'`-i` argument.',
metavar='FILE',
type=argparse.FileType('rb')
)
parser.add_argument('-m', '--mcu')
parser.add_argument('-t', '--toolchain')
parser.add_argument('--source', nargs='+', dest='source_dir')
parser.add_argument('--build')
exclusions.append('payload')
super(MbedExtendedArgs, self)._addCreateArgs(parser, exclusions)


def wrap_payload(func):
def inner(options):
if not options.payload and options.mcu and options.build:
mcus = extract_mcus(MbedExtendedArgs(), options)
sources = options.source_dir or ['.']
config = Config(mcus[0], sources)
app_name = config.name or basename(abspath(sources[0]))
output_ext = getattr(config.target, "OUTPUT_EXT", "bin")
payload_name = join(options.build, "{}_application.{}".format(
app_name, output_ext
))
options.payload = open(payload_name, "rb")
return func(options)
Copy link
Contributor

@cmonr cmonr Aug 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very interesting pattern for wrapping a function call.

return inner


def wrap_init(func):
def inner(options):
accounts = AccountManagementAPI()
certs = CertificatesAPI()
api_key = accounts.list_api_keys(filter={
'key': getenv("MBED_CLOUD_SDK_API_KEY")
}).next()
user = accounts.get_user(api_key.owner_id)
certificates_owned = list(certs.list_certificates())
dev_cert_info = None
for certif in certificates_owned:
if certif.type == "developer" and (certif.owner_id == user.id or
certif.owner_id == api_key.id):
dev_cert_info = certs.get_certificate(certif.id)
LOG.info("Found developer certificate onwed by %s named %s",
user.full_name, dev_cert_info.name)
break
else:
LOG.warning(
"Could not find developer certificate for this account."
" Generting a new developer certificate."
)
dev_cert_info = CertificatesAPI().add_developer_certificate(
"mbed-cli-auto {}".format(user.full_name),
description="cetificate auto-generated by Mbed CLI"
)
LOG.info("Writing developer certificate %s into c file "
"mbed_cloud_dev_credentials.c", dev_cert_info.name)
with open("mbed_cloud_dev_credentials.c", "w") as fout:
fout.write(dev_cert_info.header_file)
return func(options)
return inner
Copy link
Contributor

@bridadan bridadan Aug 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome! Nice 👍



def main():
options = MbedExtendedArgs().parse_args().options

log_level = {
'debug': logging.DEBUG,
'info': logging.INFO,
'warning': logging.WARNING,
'exception': logging.CRITICAL,
}[options.log_level]
logging.basicConfig(
level=log_level,
format=LOG_FORMAT,
datefmt='%Y-%m-%d %H:%M:%S',
)
logging.addLevelName(
logging.INFO,
"\033[1;32m%s\033[1;0m" % logging.getLevelName(logging.INFO)
)
logging.addLevelName(
logging.WARNING,
"\033[1;93m%s\033[1;0m" % logging.getLevelName(logging.WARNING)
)
logging.addLevelName(
logging.CRITICAL,
"\033[1;31m%s\033[1;0m" % logging.getLevelName(logging.CRITICAL)
)
LOG.debug('CLIDriver created. Arguments parsed and logging setup.')

rc = {
"create": wrap_payload(create.main),
"parse": parse.main,
"verify": verify.main,
"cert": cert.main,
"init": wrap_init(init.main),
"update": wrap_payload(update.main),
}[options.action](options) or 0

sys.exit(rc)

if __name__ == "__main__":
main()