Skip to content

Update CircleCI to run parallel Python jobs #668

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 23, 2018
Merged

Update CircleCI to run parallel Python jobs #668

merged 6 commits into from
May 23, 2018

Conversation

cmonr
Copy link
Contributor

@cmonr cmonr commented May 11, 2018

Replaces circle.yml with new CircleCI format.

py3 test is disabled until #667 comes in.

Updated config format to version 2.

Copy link
Contributor

@bridadan bridadan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hooray for Python 3 testing!

steps:
- run: sudo add-apt-repository -y ppa:terry.guo/gcc-arm-embedded
- run: sudo apt-get -y update
- run: sudo apt-get -y install gcc-arm-none-eabi=4.9.3.2015q3-1trusty1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we update this to GCC 6.x? Pretty sure Mbed OS doesn't recommend using 4.9.x anymore.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woah. You're right, but I'd prefer to change only what's needed for the topic at hand.

Definitely going to need to submit another PR to get that updated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating GCC 6.x would only work for newer Mbed OS, but older cases will fail (like Morpheus). And the Morpheus routine is used for comprehensive multi-level dependencies handling, including removal of libraries etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That make sense, but we should be testing with the recommended toolchain for each Mbed OS release. So if that means installing two different versions of GCC then we may need to do that.

mbed compile -S
mbed toolchain -S
mbed target -S
mbed export -S
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of copy and pasting these commands, can the common ones be extracted to either a script or circleci job?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. I also think there should be one test per feature release of mbed-os, but that was out of scope for this PR.

- run: mkdir .tests
- run: cd .tests && mbed new new-test
- run: |-
cd .tests/new-test/mbed-os
git fetch origin refs/pull/6886/head:pr6886
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fyi: There was one last issue in mbed-os that this testing revealed, and is fixed with ARMmbed/mbed-os#6886.

Once that PR is in, these two lines will need to be updated to origin/master until 5.9.0 is released to make sure the updates are pulled in correctly.

@cmonr
Copy link
Contributor Author

cmonr commented May 11, 2018

Forgot that this PR doesn't have the mbed-cli py3 fixes.

This is how far the CircleCI script gets with py3 before failing, assuming this PR and #667 are merged together: https://circleci.com/gh/cmonr/mbed-cli/157

The problem appears to be with https://os.mbed.com/teams/Morpheus/code/mbed-os/ when running the hg tests. Afaik, this won't be fixed until an mbed-os release is made.

I could use some help in seeing if this can be worked around, similar to the line 118 workaround for git.

@bridadan
Copy link
Contributor

@cmonr I think the problem is that https://os.mbed.com/teams/Morpheus/code/mbed-os/ is using an extremely out of date version of Mbed OS. In fact, that version was never actually released officially.

Sounds like we just need to switch up the example to something wayyy more more recent. I'd suggest just using something like blinky or another maintained example.

@screamerbg
Copy link
Contributor

@cmonr It's fine if we continue to check the Python 2 cases and that the Python 3 use only Mbed OS 5.9 and above.

cmonr added 4 commits May 22, 2018 10:24
- Uses version 2 of CircleCI yaml
- Capabile of running python jobs in parallel
- Removes version check from setup.py
Checkout will be removed once 5.9 is released.
Copy link
Contributor

@screamerbg screamerbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see inline

setup.py Outdated
@@ -32,5 +32,5 @@
'mbed-cli=mbed.mbed:main',
]
},
python_requires='>=2.7.10, <3',
python_requires='>=2.7.10',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this change should come with the other PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Will revert this.

@@ -0,0 +1,145 @@
version: 2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please update the circle_tests.py script which allows to run all commands from the circle.yml file and emulate circle CI test run?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now works across both versions!

@cmonr
Copy link
Contributor Author

cmonr commented May 22, 2018

@bridadan @screamerbg @theotherjimmy Mind rereviewing?

@screamerbg
Copy link
Contributor

Looks great!

@screamerbg screamerbg merged commit 56f59d6 into ARMmbed:master May 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants