Ci documentation added #762

OPpuolitaival · 2018-10-09T13:01:50Z

No description provided.

OPpuolitaival · 2018-10-09T13:02:20Z

@0xc0170 @adbridge please review

0xc0170 · 2018-10-09T14:51:50Z

@ARMmbed/mbed-os-maintainers

0xc0170 · 2018-10-09T14:52:04Z

@bulislaw pls review

0xc0170 · 2018-10-09T15:00:38Z

docs/reference/contributing/guidelines/ci.md

+
+### Travis CI
+
+In Mbed OS [Travis CI](https://travis-ci.org/) is used as primary automatic testing and checking run environment.


can we point here to our Travis page (mbed-os one) ?

0xc0170 · 2018-10-09T15:01:01Z

docs/reference/contributing/guidelines/ci.md

+
+In Mbed OS [Travis CI](https://travis-ci.org/) is used as primary automatic testing and checking run environment.
+
+Travis configuration is located in .travis.yml file in Mbed OS root directory. Mbed OS use public travis so test results are publicly available and there are public [documentation available](https://docs.travis-ci.com/).


reference here our .travis.ymls file (url)

@AnotherButler is it OK to add this url? It is so many times commented that do not link to the github

0xc0170 · 2018-10-09T15:29:59Z

docs/reference/contributing/guidelines/ci.md

+In Mbed OS [Travis CI](https://travis-ci.org/) is used as primary automatic testing and checking run environment.
+
+Travis configuration is located in .travis.yml file in Mbed OS root directory. Mbed OS use public travis so test results are publicly available and there are public [documentation available](https://docs.travis-ci.com/).
+


What kind of tests jenkins runs?

Should travis also explain each jobs : docs, astyle etc ? like below in the jenkins?

Good point!

bulislaw · 2018-10-10T03:25:50Z

@screamerbg first stab at documenting CI testing as we discussed some time ago

bulislaw

It focuses mainly on the tools we use rather than what sort of test we run, could we expand on the details?

OPpuolitaival · 2018-10-12T10:59:47Z

@bulislaw better now?

AnotherButler · 2018-10-17T15:07:17Z

@0xc0170 Do those changes resolve all of your comments?

0xc0170 · 2018-10-19T13:28:06Z

@0xc0170 Do those changes resolve all of your comments?

I'll review shortly, and send feedback

0xc0170 · 2018-10-19T15:09:49Z

@ashok-rao @MarceloSalazar Can you review this addition?

0xc0170 · 2018-10-22T09:40:09Z

Note, there's upcoming update to the mbed ci job, that will depend on these changes.

0xc0170 · 2018-10-22T10:23:49Z

Note, there's upcoming update to the mbed ci job, that will depend on these changes.

MarceloSalazar · 2018-10-22T11:43:00Z

Looks good, however, this is something for @screamerbg to have a look and comment

OPpuolitaival · 2018-10-22T11:46:29Z

Latest changes are not visible in web UI because of problems in github. You can review by fetching the branch by using git.

0xc0170 · 2018-10-24T09:42:53Z

@AnotherButler Can you review? I'll poke @ARMmbed/mbed-os-maintainers to get more reviews here.

Let's progress this important document (more updates are coming this way in the following days)

NirSonnenschein

looks pretty good, please see my comments (mostly small text fixups)

NirSonnenschein · 2018-10-24T10:53:19Z

docs/reference/contributing/guidelines/ci.md

+
+### Jenkins
+
+[Jenkins](https://jenkins.io/) is used to run tests behind firewall. There are multiple reasons to run tests in Jenkins but typically those are related to high amount of concurrent builds or shared secrets.


I'm not sure about what you are trying to say with this paragraph, is it a description of what Jenkins is? is it a description of what we use Jenkins for? is it a justification of the use of Jenkins.
this seems to be a mix of all three, and it is not really clear what you are trying to say. in the Travis section above it is more clearly about what we use Travis for (and to a lesser extent how).

NirSonnenschein · 2018-10-24T10:54:58Z

docs/reference/contributing/guidelines/ci.md

+[Jenkins](https://jenkins.io/) is used to run tests behind firewall. There are multiple reasons to run tests in Jenkins but typically those are related to high amount of concurrent builds or shared secrets.
+
+#### How it works:
+* Jenkins uses [scripted pipeline syntax](https://jenkins.io/doc/book/pipeline/)


Jenkins uses [scripted pipeline syntax] => Jenkins uses a [scripted pipeline syntax]

NirSonnenschein · 2018-10-24T10:55:41Z

docs/reference/contributing/guidelines/ci.md

+
+#### How it works:
+* Jenkins uses [scripted pipeline syntax](https://jenkins.io/doc/book/pipeline/)
+* There is Jenkinsfile in Mbed OS root folder but that is just trigger for tests. Jenkins scripts are not publicly available at the moment.


I would flip the order of these two sentences

NirSonnenschein · 2018-10-24T10:56:02Z

docs/reference/contributing/guidelines/ci.md

+#### How it works:
+* Jenkins uses [scripted pipeline syntax](https://jenkins.io/doc/book/pipeline/)
+* There is Jenkinsfile in Mbed OS root folder but that is just trigger for tests. Jenkins scripts are not publicly available at the moment.
+* Jenkins select required tests dynamically based on the code changes. For example. no tests are executed if only markdown (.md) file changes


Jenkins select => Jenkins selects

maybe also
For example. => For example:

if only markdown (.md) file changes => if only markdown (.md) files are changed

NirSonnenschein · 2018-10-24T11:04:31Z

docs/reference/contributing/guidelines/ci.md

+* Jenkins uses [scripted pipeline syntax](https://jenkins.io/doc/book/pipeline/)
+* There is Jenkinsfile in Mbed OS root folder but that is just trigger for tests. Jenkins scripts are not publicly available at the moment.
+* Jenkins select required tests dynamically based on the code changes. For example. no tests are executed if only markdown (.md) file changes
+* Jenkins run first small amount of tests to provide fast feedback and then more tests


Jenkins run first small amount => Jenkins first runs small amount
also maybe:
more tests => additional tests

NirSonnenschein · 2018-10-24T11:07:05Z

docs/reference/contributing/guidelines/ci.md

+* **jenkins-ci/greentea-test** - Run [greentea tests](/docs/tools/testing/testing_greentea.html)
+* **jenkins-ci/mbed2-build-ARM** - Build Mbed OS 2 with ARM
+ * `python tools/build_release.py -p <target> -t <toolchain>`
+* **jenkins-ci/mbed2-build-GCC_ARM** - Build Mbed OS 2 with GCC_ARM. Related commands:


no related commands after "Related commands:"

NirSonnenschein · 2018-10-24T11:09:18Z

docs/reference/contributing/guidelines/workflow.md

@@ -186,7 +186,7 @@ Time: 3 days for reviewers to leave feedback after the maintainers add the "need

 #### The CI (Continuous Integration) testing

-There are many CI systems available. Which CI tests we run against a particular pull request depends on the effect that pull request has on the code base. Irrespective of which CI tests run, Mbed OS has an all green policy, meaning that all the CI jobs that are triggered must pass before we merge the pull request.
+There are many [CI systems available](/docs/reference/contributing/guidelines/ci.html). Which CI tests we run against a particular pull request depends on the effect that pull request has on the code base. Irrespective of which CI tests run, Mbed OS has an all green policy, meaning that all the CI jobs that are triggered must pass before we merge the pull request.


I would add what the CI systems are available for (e.g. There are many [CI systems available] for testing Mbed OS pull requests)

NirSonnenschein · 2018-10-24T11:12:56Z

docs/reference/contributing/guidelines/ci.md

+
+In Mbed OS [Travis CI](https://travis-ci.org/ARMmbed/mbed-os) is used as primary automatic testing and checking run environment.
+
+Travis configuration is located in [.travis.yml](https://github.com/ARMmbed/mbed-os/blob/master/.travis.yml) file in Mbed OS root directory. Mbed OS use public travis so test results are publicly available and there are public [documentation available](https://docs.travis-ci.com/).


Travis configuration is located in .travis.yml=> The Travis configuration is located in the .travis.yml file

NirSonnenschein · 2018-10-24T11:13:37Z

docs/reference/contributing/guidelines/ci.md

+
+In Mbed OS [Travis CI](https://travis-ci.org/ARMmbed/mbed-os) is used as primary automatic testing and checking run environment.
+
+Travis configuration is located in [.travis.yml](https://github.com/ARMmbed/mbed-os/blob/master/.travis.yml) file in Mbed OS root directory. Mbed OS use public travis so test results are publicly available and there are public [documentation available](https://docs.travis-ci.com/).


Mbed OS use public travis => Mbed OS uses public travis

NirSonnenschein

Thanks, looks even better now.

AnotherButler · 2018-10-25T16:32:28Z

Query: How often do we add, remove or change tests?

cmonr · 2018-10-25T17:03:33Z

Query: How often do we add, remove or change tests?

Making a context check here that tests here refer to PR merge checks

Not often. It's under flux right now because we're @OPpuolitaival's team is getting the new CI up and running. Travis CI and Jenkins don't change often.

AnotherButler · 2018-10-25T20:00:14Z

So we don't expect to update these files every feature release or more?

cmonr · 2018-10-25T20:41:30Z

I would think not.
I should hope not?

OPpuolitaival · 2018-10-26T07:13:36Z

@AnotherButler There will be changes at least in next 6 months because there are a lot of tests which are missing and we start adding those when base tests are stable.

0xc0170 · 2018-10-31T15:12:34Z

docs/reference/contributing/guidelines/ci.md

+#### Tests
+
+* **continuous-integration/jenkins/pr-head** Jenkins main pipeline script execution status
+* **jenkins-ci/build-ARM** - Build Mbed OS and examples with ARM. Related commands:


@ARMmbed/mbed-os-maintainers What do you think about these checks inthe PR ?

My vote would be as it is now (build, exporters, mbed 2 build, test) rather than each toolchain own one (would end up with 20 checks?)

I don't see a particular problem with having up to 20 merge checks.
Plus, I think it would complicate the pipeline a bit to have the subjobs merge reporting after the fact.

@cmonr reporting is done already.. sub jobs report the status on checks and main pipeline send the summary post in the end of run.

Make initial edits to file.

AnotherButler

Please address my queries.

AnotherButler · 2018-10-31T15:57:00Z

docs/reference/contributing/guidelines/ci.md

+#### Tests
+
+- **continuous-integration/jenkins/pr-head** - Jenkins main pipeline script execution status.
+- **jenkins-ci/build-ARM** - Build Mbed OS and examples with ARM. Related commands:


Query: What does it mean that this builds with ARM?

AnotherButler · 2018-10-31T15:57:08Z

docs/reference/contributing/guidelines/ci.md

+- **jenkins-ci/exporter** - Export and build exported code. Related commands:
+   - `python -u mbed-os/tools/test/examples/examples.py export <exporter> --mcu <target>`.
+- **jenkins-ci/greentea-test** - Run [greentea tests](/docs/tools/testing/testing_greentea.html).
+- **jenkins-ci/mbed2-build-ARM** - Build Mbed OS 2 with ARM. Related commands:


Query: What does it mean that this builds with ARM?

Think it means arm-compiler

AnotherButler · 2018-10-31T15:57:25Z

docs/reference/contributing/guidelines/ci.md

+
+#### Tests
+
+- **continuous-integration/travis-ci/pr** - Main run main.


What does this mean?

I'm not sure how to clarify this. Could someone please explain what this means?

This job is the main job that runs all of the other travis-ci sub-jobs. Not sure what kind of wording would work for this.

"Run other testing jobs."?

Does this phrasing work?

@OPpuolitaival Do you think you could clarify this? Looks like it should be the last thing for this PR.

I went with the phrasing I suggested. If you have another suggestion, please put up a new PR.

Update phrasing for clarity.

Fix broken link.

0xc0170 requested changes Oct 9, 2018

View reviewed changes

bulislaw reviewed Oct 10, 2018

View reviewed changes

0xc0170 approved these changes Oct 19, 2018

View reviewed changes

Olli-Pekka Puolitaival added 2 commits October 22, 2018 12:50

Ci documentation added

3878590

Add travis test list and links

be2b507

OPpuolitaival force-pushed the ci branch from e3b0642 to be2b507 Compare October 22, 2018 09:51

Add checks which are coming soon

c2778a0

Sort checks and add tools tests

7637fc8

NirSonnenschein suggested changes Oct 24, 2018

View reviewed changes

Changes based on github comments

10ee044

NirSonnenschein approved these changes Oct 24, 2018

View reviewed changes

0xc0170 reviewed Oct 31, 2018

View reviewed changes

Edit ci.md

161f36e

Make initial edits to file.

AnotherButler suggested changes Oct 31, 2018

View reviewed changes

Olli-Pekka Puolitaival and others added 3 commits October 31, 2018 18:12

Fix arm compiler and add link to that

79994f3

Update phrasing in ci.md

ea90b07

Update phrasing for clarity.

Fix link in workflow.md

ce30733

Fix broken link.

AnotherButler approved these changes Nov 16, 2018

View reviewed changes

Merge branch 'development' into ci

f83299c

AnotherButler merged commit 2971803 into ARMmbed:development Nov 16, 2018


		### Travis CI

		In Mbed OS [Travis CI](https://travis-ci.org/) is used as primary automatic testing and checking run environment.


		### Jenkins

		[Jenkins](https://jenkins.io/) is used to run tests behind firewall. There are multiple reasons to run tests in Jenkins but typically those are related to high amount of concurrent builds or shared secrets.


		In Mbed OS [Travis CI](https://travis-ci.org/ARMmbed/mbed-os) is used as primary automatic testing and checking run environment.

		Travis configuration is located in [.travis.yml](https://github.com/ARMmbed/mbed-os/blob/master/.travis.yml) file in Mbed OS root directory. Mbed OS use public travis so test results are publicly available and there are public [documentation available](https://docs.travis-ci.com/).


		#### Tests

		- continuous-integration/travis-ci/pr - Main run main.

Ci documentation added #762

Ci documentation added #762

Uh oh!

Conversation

OPpuolitaival commented Oct 9, 2018

Uh oh!

OPpuolitaival commented Oct 9, 2018

Uh oh!

0xc0170 commented Oct 9, 2018

Uh oh!

0xc0170 commented Oct 9, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

What kind of tests jenkins runs?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bulislaw commented Oct 10, 2018

Uh oh!

bulislaw left a comment

Choose a reason for hiding this comment

Uh oh!

OPpuolitaival commented Oct 12, 2018

Uh oh!

AnotherButler commented Oct 17, 2018

Uh oh!

0xc0170 commented Oct 19, 2018

Uh oh!

0xc0170 commented Oct 19, 2018

Uh oh!

0xc0170 commented Oct 22, 2018

Uh oh!

0xc0170 commented Oct 22, 2018

Uh oh!

MarceloSalazar commented Oct 22, 2018

Uh oh!

OPpuolitaival commented Oct 22, 2018

Uh oh!

0xc0170 commented Oct 24, 2018

Uh oh!

NirSonnenschein left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NirSonnenschein left a comment

Choose a reason for hiding this comment

Uh oh!

AnotherButler commented Oct 25, 2018

Uh oh!

cmonr commented Oct 25, 2018

Uh oh!

AnotherButler commented Oct 25, 2018

Uh oh!

cmonr commented Oct 25, 2018

Uh oh!