Skip to content

Add coverage with coveralls and make use of travis stages #1835

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 6, 2019
Merged

Add coverage with coveralls and make use of travis stages #1835

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
af728d9
[coverage] Add coverage with `coveralls` :jeans:
opacam May 31, 2019
d514c37
[travis] Implement build stages
opacam May 31, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .coveragerc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[run]
omit =
*test*

[report]
exclude_lines =
pragma: no cover
def __repr__
raise NotImplementedError
if __name__ == .__main__.:
71 changes: 48 additions & 23 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,22 @@ dist: xenial # needed for more recent python 3 and python3-venv

language: generic

stages:
- lint
- test

services:
- docker

before_install:
- travis_retry sudo apt update -qq
- travis_retry sudo apt install -qq --no-install-recommends python2.7 python3 python3-venv python3-virtualenv
# to successfully send the coveralls reports we need pyOpenSSL
- travis_retry sudo apt install -qq --no-install-recommends
python2.7 python3 python3-venv python3-virtualenv python3-pip
python3-setuptools python3-openssl
# (venv/virtualenv are both used by tests/test_pythonpackage.py)
- sudo pip install tox>=2.0
- sudo pip3 install coveralls
# https://github.com/travis-ci/travis-ci/issues/6069#issuecomment-266546552
- git remote set-branches --add origin master
- git fetch
Expand All @@ -20,25 +28,42 @@ env:
global:
- ANDROID_SDK_HOME=/opt/android/android-sdk
- ANDROID_NDK_HOME=/opt/android/android-ndk
matrix:
- COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python3_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements libffi,sdl2,pyjnius,kivy,python3,openssl,requests,sqlite3,setuptools'
# overrides requirements to skip `peewee` pure python module, see:
# https://github.com/kivy/python-for-android/issues/1263#issuecomment-390421054
- COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python2_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements sdl2,pyjnius,kivy,python2,openssl,requests,sqlite3,setuptools'
- COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python2.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --bootstrap sdl2 --requirements python2,numpy'
# builds only the recipes that moved
- COMMAND='. venv/bin/activate && ./ci/rebuild_updated_recipes.py'

before_script:
# we want to fail fast on tox errors without having to `docker build` first
- tox -- tests/ --ignore tests/test_pythonpackage.py
# (we ignore test_pythonpackage.py since these run way too long!! test_pythonpackage_basic.py will still be run.)

script:
# Run a background process to make sure that travis will not kill our tests in
# case that the travis log doesn't produce any output for more than 10 minutes
- while sleep 540; do echo "==== Still running (travis, don't kill me) ===="; done &
- docker build --tag=p4a --file Dockerfile.py3 .
- docker run -e CI p4a /bin/sh -c "$COMMAND"
# kill the background process started before run docker
- kill %1

jobs:
include:
- stage: lint
name: "Tox tests and coverage"
script:
# we want to fail fast on tox errors without having to `docker build` first
- tox -- tests/ --ignore tests/test_pythonpackage.py
# (we ignore test_pythonpackage.py since these run way too long!!
# test_pythonpackage_basic.py will still be run.)
after_success:
- coveralls

- &testing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not aware of this syntax and also the <<: *testing below. Could you share me a doc to read or a keyword to look for?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes, no problem, I learned this syntax for this pr, they are called Yaml Anchors & Aliases and are great to create templates, I discovered this in this article:

Also mentioned in Travis docs examples
And here some more information: atlassian - yaml anchors

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The yaml Anchors & Aliases that I wrote works the following way:

First we define a template named testing using the keywords &testing

    - &testing
      stage: test
      before_script:
        # build docker image
        - docker build --tag=p4a --file Dockerfile.py3 .
        # Run a background process to make sure that travis will not kill our tests in
        # case that the travis log doesn't produce any output for more than 10 minutes
        - while sleep 540; do echo "==== Still running (travis, don't kill me) ===="; done &
      script:
        - docker run -e CI -e TRAVIS_JOB_ID="$TRAVIS_JOB_ID" -e TRAVIS_BRANCH="$TRAVIS_BRANCH" p4a /bin/sh -c "$COMMAND"
      after_script:
        # kill the background process started before run docker
        - kill %1

The following code will take the above defined template, actually will also be part of the template but we override later for the other jobs:

      name: Python 3 basic
      # overrides requirements to skip `peewee` pure python module, see:
      # https://github.com/kivy/python-for-android/issues/1263#issuecomment-390421054
      env: COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python3_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements libffi,sdl2,pyjnius,kivy,python3,openssl,requests,sqlite3,setuptools'

For the following jobs we want to use the same scripts defined in &testing so we use the keywords <<:*testing but we want to override the name and env defined in the template so we do the following:

    - <<: *testing
      name: Python 2 basic
      env: COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python2_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements sdl2,pyjnius,kivy,python2,openssl,requests,sqlite3,setuptools'

The same for the other jobs...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation makes sense 👍
Our Travis file start to get more complex, but that's the way it is I guess.
Dropping python2 at some point may help slightly

stage: test
before_script:
# build docker image
- docker build --tag=p4a --file Dockerfile.py3 .
# Run a background process to make sure that travis will not kill our tests in
# case that the travis log doesn't produce any output for more than 10 minutes
- while sleep 540; do echo "==== Still running (travis, don't kill me) ===="; done &
script:
- docker run -e CI -e TRAVIS_JOB_ID="$TRAVIS_JOB_ID" -e TRAVIS_BRANCH="$TRAVIS_BRANCH" p4a /bin/sh -c "$COMMAND"
after_script:
# kill the background process started before run docker
- kill %1
name: Python 3 basic
# overrides requirements to skip `peewee` pure python module, see:
# https://github.com/kivy/python-for-android/issues/1263#issuecomment-390421054
env: COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python3_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements libffi,sdl2,pyjnius,kivy,python3,openssl,requests,sqlite3,setuptools'
- <<: *testing
name: Python 2 basic
env: COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python2_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements sdl2,pyjnius,kivy,python2,openssl,requests,sqlite3,setuptools'
- <<: *testing
name: Python 2 numpy
env: COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python2.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --bootstrap sdl2 --requirements python2,numpy'
- <<: *testing
name: Rebuild updated recipes
env: COMMAND='. venv/bin/activate && ./ci/rebuild_updated_recipes.py'
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ python-for-android
==================

[![Build Status](https://travis-ci.org/kivy/python-for-android.svg?branch=master)](https://travis-ci.org/kivy/python-for-android)
[![Coverage Status](https://coveralls.io/repos/github/kivy/python-for-android/badge.svg?branch=master&kill_cache=1)](https://coveralls.io/github/kivy/python-for-android?branch=master)
[![Backers on Open Collective](https://opencollective.com/kivy/backers/badge.svg)](#backers)
[![Sponsors on Open Collective](https://opencollective.com/kivy/sponsors/badge.svg)](#sponsors)

Expand Down
8 changes: 8 additions & 0 deletions tox.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,20 @@ deps =
mock
pytest
virtualenv
py3: coveralls
# makes it possible to override pytest args, e.g.
# tox -- tests/test_graph.py
commands = pytest {posargs:tests/}
passenv = TRAVIS TRAVIS_*
setenv =
PYTHONPATH={toxinidir}

[testenv:py3]
# for py3 env we will get code coverage
commands =
coverage run --branch --source=pythonforandroid -m pytest {posargs:tests/}
coverage report -m

[testenv:pep8]
deps = flake8
commands = flake8 pythonforandroid/ tests/ ci/
Expand Down