Skip to content

Improve domain handling and status checking of updated examples. #5332

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 30, 2017
Merged

Improve domain handling and status checking of updated examples. #5332

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bb579dc
Improve domain handling and status checking of updated examples.
adbridge Oct 17, 2017
c7e51da
Combine string constants passed to 'join' function
adbridge Oct 24, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 33 additions & 20 deletions tools/test/examples/examples.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,17 @@
},
"via-branch" : {
"help" : "-b cmd line option. Update dst branch, created from src branch",
"src-branch" : "mbed-os-5.5.0-rc1-oob",
"dst-branch" : "mbed-os-5.5.0-rc2-oob"
"src-branch" : "mbed-os-5.6.0-oob2",
"dst-branch" : "mbed-os-5.6.0-oob2"
},
"tag" : "mbed-os-5.5.0-rc2"
"tag" : "mbed-os-5.6.2"
},
"examples": [
{
"name": "mbed-os-example-blinky",
"github": "https://github.com/ARMmbed/mbed-os-example-blinky",
"mbed": [
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-blinky"
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-blinky"
],
"test-repo-source": "github",
"features" : [],
Expand All @@ -32,10 +32,10 @@
"name": "mbed-os-example-tls",
"github": "https://github.com/ARMmbed/mbed-os-example-tls",
"mbed": [
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-tls-benchmark",
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-tls-tls-client",
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-tls-hashing",
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-tls-authcrypt"
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-tls-benchmark",
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-tls-tls-client",
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-tls-hashing",
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-tls-authcrypt"
],
"test-repo-source": "mbed",
"features" : [],
Expand All @@ -50,7 +50,7 @@
"name": "mbed-os-example-mesh-minimal",
"github":"https://github.com/ARMmbed/mbed-os-example-mesh-minimal",
"mbed": [
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-mesh-minimal"
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-mesh-minimal"
],
"test-repo-source": "github",
"features" : [],
Expand All @@ -68,14 +68,14 @@
"name": "mbed-os-example-ble",
"github":"https://github.com/ARMmbed/mbed-os-example-ble",
"mbed": [
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-ble-Beacon",
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-ble-HeartRate",
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-ble-Thermometer",
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-ble-LEDBlinker",
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-ble-LED",
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-ble-GAPButton",
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-ble-Button",
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-ble-BatteryLevel"
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-ble-Beacon",
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-ble-HeartRate",
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-ble-Thermometer",
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-ble-LEDBlinker",
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-ble-LED",
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-ble-GAPButton",
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-ble-Button",
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-ble-BatteryLevel"
],
"test-repo-source": "mbed",
"features" : ["BLE"],
Expand All @@ -90,7 +90,7 @@
"name": "mbed-os-example-client",
"github":"https://github.com/ARMmbed/mbed-os-example-client",
"mbed": [
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-client"
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-client"
],
"test-repo-source": "github",
"features" : ["LWIP"],
Expand All @@ -101,6 +101,19 @@
"export": false,
"auto-update" : true
},
{
"name": "mbed-cloud-client-example-internal",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I'm sure we want to test this, I'm sure they don't want it publicly advertised.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The client team asked for it to be added.....

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay!

"github":"https://github.com/ARMmbed/mbed-cloud-client-example-internal",
"mbed": [],
"test-repo-source": "github",
"features" : ["LWIP"],
"targets" : [],
"toolchains" : [],
"exporters": [],
"compile" : false,
"export": false,
"auto-update" : true
},
{
"name": "mbed-os-example-sockets",
"github":"https://github.com/ARMmbed/mbed-os-example-sockets",
Expand Down Expand Up @@ -172,7 +185,7 @@
"name": "mbed-os-example-bootloader",
"github":"https://github.com/ARMmbed/mbed-os-example-bootloader",
"mbed": [
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-bootloader"
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-bootloader"
],
"test-repo-source": "github",
"features" : [],
Expand All @@ -187,7 +200,7 @@
"name": "mbed-os-example-fat-filesystem",
"github":"https://github.com/ARMmbed/mbed-os-example-fat-filesystem",
"mbed": [
"https://developer.mbed.org/teams/mbed-os-examples/code/mbed-os-example-fat-filesystem"
"https://os.mbed.com/teams/mbed-os-examples/code/mbed-os-example-fat-filesystem"
],
"test-repo-source": "github",
"features" : [],
Expand Down
59 changes: 55 additions & 4 deletions tools/test/examples/update.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
#
# Command usage:
#
# update.py -c <config file> - T <github_token> -l <logging level> -f -b
# update.py -c <config file> - T <github_token> -f -b -s
#
# Where:
# -c <config file> - Optional path to an examples file.
Expand All @@ -47,8 +47,10 @@
# -b - Update branched repos. This will use the "src-branch" and
# "dst-branch" parameters in the 'via-branch' section. The destination
# branch is created from the source branch (if it doesn't already exist).
# -s - Show the status of any pull requests with a tag matching that in the
# json config file
#
# The options -f and -b are mutually exlusive. Only one can be specified.
# The options -f, -b and -s are mutually exlusive. Only one can be specified.
#
#

Expand Down Expand Up @@ -431,6 +433,47 @@ def create_work_directory(path):

os.makedirs(path)

def check_update_status(examples, github, tag):
""" Check the status of previously raised update pull requests

Args:
examples - list of examples which should have had PRs raised against them.
github - github rest API instance
tag - release tag used for the update

"""

for example in examples:

repo_name = ''.join(['ARMmbed/', example['name']])
try:
repo = github.get_repo(repo_name, False)

except Exception as exc:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you not cache C-c here? Another way: Could you specify the exact exception, or exception list, that would occur? (I know it can be hard, untracked exceptions have a way of being intractable.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I know there is no specific documentation on how the pygithub module returns the github exceptions. I've had a lot of issues in the past trying to narrow down a list and TBH in this case if any exception comes back at all then it's basically fatal...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I'm generally trying to promote defensive programming because python's "unchecked exceptions" error handling is beginning to drive me crazy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, we could definitely do with more defensive programming !

text = "Cannot access: " + str(repo_name)
userlog.error(text)
userlog.exception(exc)
sys.exit(1)

# Create the full repository filter component
org_str = ''.join(['repo:ARMmbed/', example['name']])
filt = ' '.join([org_str, 'is:pr', tag])
merged = False

issues = github.search_issues(query=(filt))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the () around filt needed? If so, why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the pygithub syntax for that command...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

pr_list = [repo.get_pull(issue.number) for issue in issues]

# Should only be one matching PR but just in case, go through paginated list
for pr in pr_list:
if pr.merged:
userlog.info("%s - '%s': MERGED", example['name'], pr.title)
elif pr.state == 'open':
userlog.info("%s - '%s': PENDING", example['name'], pr.title)
elif pr.state == 'closed':
userlog.info("%s - '%s': CLOSED NOT MERGED", example['name'], pr.title)
else:
userlog.error("%s: Cannot find a pull request for %s", example['name'], tag)

if __name__ == '__main__':

parser = argparse.ArgumentParser(description=__doc__,
Expand All @@ -441,6 +484,7 @@ def create_work_directory(path):
exclusive = parser.add_mutually_exclusive_group(required=True)
exclusive.add_argument('-f', '--fork', help="Update a fork", action='store_true')
exclusive.add_argument('-b', '--branch', help="Update a branch", action='store_true')
exclusive.add_argument('-s', '--status', help="Show examples update status", action='store_true')

args = parser.parse_args()

Expand All @@ -451,8 +495,6 @@ def create_work_directory(path):
sys.exit(1)
json_data = json.load(config)

# Create working directory
create_work_directory('examples')

github = Github(args.github_token)
config = json_data['update-config']
Expand All @@ -462,6 +504,15 @@ def create_work_directory(path):
src = "master"
dst = None

if args.status:

# This option should only be called after an update has been performed
check_update_status(json_data['examples'], github, tag)
exit(0)

# Create working directory
create_work_directory('examples')

if args.fork:
user = config['via-fork']['github-user']
elif args.branch:
Expand Down