Additional fixes for running Python 3 in Windows #7092

cmonr · 2018-06-03T01:49:29Z

Description

An additional set of small fixes for running Mbed OS and Mbed CLI functions within Windows.
Once #7078 is merged, this will be rebased to remove duplicate commits.

Pull request type

[x] Fix
[ ] Refactor
[ ] New target
[ ] Feature
[ ] Breaking change

cmonr · 2018-06-04T14:19:15Z

/morph build

Had to remove part of test that was incompatible with Py3 on Windows.

… iteration over values. Py3 no longer supports dictionaries that self-modify their item lists during iteration.

Py3 open(...) returns a BufferedReader instead of a file.

cmonr · 2018-06-04T14:22:44Z

/morph build

bridadan · 2018-06-04T15:14:16Z

tools/build_api.py

@@ -459,7 +459,7 @@ def merge_region_list(region_list, destination, notify, padding=b'\xFF'):
    notify.info("Space used after regions merged: 0x%x" %
                (merged.maxaddr() - merged.minaddr() + 1))
    with open(destination, "wb+") as output:
-        merged.tofile(output, format=format.strip("."))
+        merged.tofile(destination, format=format.strip("."))


This block doesn't make much sense. Did you mean to remove the open() call above? Currently, output isn't being used.

Haha, you're right. I'm thinking that thewith open... could even be removed completely.

The tofile api happens to support passing the file path. The difficulty with this port was that In Py3, open returns a BufferedStream instead of a file, and I didn't have much luck finding something that worked in both versions.

API Ref: https://media.readthedocs.org/pdf/python-intelhex/latest/python-intelhex.pdf

Ok no worries, sounds like removing the with open... statement is what you're after then.

bridadan · 2018-06-04T15:14:55Z

tools/notifier/term.py

@@ -14,6 +14,7 @@
 # limitations under the License.

 from __future__ import print_function, division, absolute_import
+from past.builtins import basestring


Doesn't look like this import is used anywhere

It's not intuitive at all, but including basestring in this way affects how string concatenation occurs in line 77. Without it, Py3 defaults to treating strings as bytes, which breaks the concatenation.

Ah I see, thanks for clarifying. I just checked the cheatsheet and I see what you're talking about: http://python-future.org/compatible_idioms.html#basestring

If you end up pushing more commits to this, do you think you could add a comment pointing that reference so others know why its there?

Ah, yeah. Will do. There's been so much back and forth between testing across two separate OSs that I let commit quality suffer.

Oh no its no big deal, its honestly fine how it is. Just trying to make it easier for future maintenance.

mbed-ci · 2018-06-04T16:50:22Z

Build : SUCCESS

Build number : 2232
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7092/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build
/morph mbed2-build

cmonr · 2018-06-04T18:19:39Z

/morph build

mbed-ci · 2018-06-04T18:33:14Z

Test : FAILURE

Build number : 2024
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7092/2024

cmonr · 2018-06-04T18:38:25Z

^^^ Ignore that. It's from the first build which was stopped due to the update.

mbed-ci · 2018-06-04T19:19:29Z

Build : SUCCESS

Build number : 2236
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7092/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build
/morph mbed2-build

mbed-ci · 2018-06-04T20:43:13Z

Test : SUCCESS

Build number : 2025
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7092/2025

cmonr · 2018-06-04T22:38:07Z

Grumblegrumblegrumble......

Accidentally pushed a commit to this branch. Restarting CI after restoring the branch to it's approved commit history.

/morph build

cmonr · 2018-06-04T22:40:27Z

@bridadan @theotherjimmy Can I get one of y'all to approve?

mbed-ci · 2018-06-04T23:51:49Z

Build : SUCCESS

Build number : 2238
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7092/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build
/morph mbed2-build

mbed-ci · 2018-06-05T01:30:31Z

Test : SUCCESS

Build number : 2027
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7092/2027

mbed-ci · 2018-06-05T02:41:22Z

Exporter Build : SUCCESS

Build number : 1862
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/7092/

cmonr added needs: review release-version: 5.9.0-rc2 labels Jun 3, 2018

cmonr requested review from theotherjimmy and bridadan June 3, 2018 01:49

theotherjimmy previously approved these changes Jun 4, 2018

View reviewed changes

cmonr added needs: CI and removed needs: review labels Jun 4, 2018

cmonr and others added 6 commits June 4, 2018 09:21

Encapsulated Windows file separator for proper regex parsing

8b5a760

Modified memap path separator parsing to support Py3.

2abae9e

Had to remove part of test that was incompatible with Py3 on Windows.

Sets module no longer needed to use set()

27d0255

Added basestring import for proper string concatenation in Py3

6e9b7ea

Modified LazyDict to inherit from object instead of dict, and removed…

e655377

… iteration over values. Py3 no longer supports dictionaries that self-modify their item lists during iteration.

Modified IntelHex tofile parameter to use path.

cdbae99

Py3 open(...) returns a BufferedReader instead of a file.

cmonr dismissed theotherjimmy’s stale review via cdbae99 June 4, 2018 14:21

cmonr force-pushed the py3-in-windows branch from 0ee7167 to cdbae99 Compare June 4, 2018 14:21

theotherjimmy previously approved these changes Jun 4, 2018

View reviewed changes

cmonr mentioned this pull request Jun 4, 2018

Added warning message about Mbed OS Py3 compatability ARMmbed/mbed-cli#689

Merged

bridadan suggested changes Jun 4, 2018

View reviewed changes

Removed extraneous file open

f689ace

cmonr dismissed theotherjimmy’s stale review via f689ace June 4, 2018 17:02

bridadan previously approved these changes Jun 4, 2018

View reviewed changes

cmonr mentioned this pull request Jun 4, 2018

Introduced docs changes to indicate Python 3 compatability ARMmbed/mbed-os-5-docs#554

Merged

cmonr dismissed bridadan’s stale review via 0c6e687 June 4, 2018 21:45

cmonr force-pushed the py3-in-windows branch 2 times, most recently from 6b38f33 to f689ace Compare June 4, 2018 22:36

cmonr mentioned this pull request Jun 4, 2018

Python 3 makefile export issue #7111

Merged

bridadan approved these changes Jun 4, 2018

View reviewed changes

cmonr added the risk: G label Jun 4, 2018

cmonr merged commit 80dde0b into ARMmbed:master Jun 5, 2018

cmonr removed needs: CI risk: G labels Jun 5, 2018

maciejbocianski mentioned this pull request Jun 6, 2018

NRF52_DK: SLEEP enable #7073

Merged

theotherjimmy mentioned this pull request Jun 6, 2018

Export make_gcc_arm error with v1.7.2 + Python 3 ARMmbed/mbed-cli#692

Closed

cmonr mentioned this pull request Jun 8, 2018

Export make_gcc_arm results in TypeError with v1.7.3 + Python3 ARMmbed/mbed-cli#694

Closed

Additional fixes for running Python 3 in Windows #7092

Additional fixes for running Python 3 in Windows #7092

Uh oh!

Conversation

cmonr commented Jun 3, 2018

Description

Pull request type

Uh oh!

cmonr commented Jun 4, 2018

Uh oh!

cmonr commented Jun 4, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmonr Jun 4, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmonr Jun 4, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mbed-ci commented Jun 4, 2018

Triggering tests

Uh oh!

cmonr commented Jun 4, 2018

Uh oh!

mbed-ci commented Jun 4, 2018

Uh oh!

cmonr commented Jun 4, 2018

Uh oh!

mbed-ci commented Jun 4, 2018

Triggering tests

Uh oh!

mbed-ci commented Jun 4, 2018

Uh oh!

cmonr commented Jun 4, 2018

Uh oh!

cmonr commented Jun 4, 2018

Uh oh!

mbed-ci commented Jun 4, 2018

Triggering tests

Uh oh!

mbed-ci commented Jun 5, 2018

Uh oh!

mbed-ci commented Jun 5, 2018

Uh oh!

Uh oh!

cmonr Jun 4, 2018 •

edited

Loading

cmonr Jun 4, 2018 •

edited

Loading