Skip to content

Fix apk argument parsing oversight #1355

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 47 commits into from
Nov 10, 2018
Merged

Fix apk argument parsing oversight #1355

merged 47 commits into from
Nov 10, 2018

Conversation

wolfgangp
Copy link
Contributor

Due to iteration until unknown_args[:-1] the very last argument could be left without its path expanded (if it looked like "--add-jar=some.jar", i.e. contained an equals sign)
This new implementation will fail when something like "--icon=" is the last element of args.unknown_args.
If "--icon=some/path.png" is passed, it's fine.
If "--icon", "some/path.png" is passed, it's fine, too.

wolfgangp and others added 30 commits January 25, 2018 16:52
Adds python3crystax support to setuptools recipe.
Also removed unused imports.
Makes sure the arm cross linker is being used for LDSHARED.
Fixes configure script complaining about CFLAGS and LDFLAGS and
simplified `gevent.patch` (less is more), since flags are now
sanitized during `get_recipe_env()` call.

Also depends on #1249 (symlink system Python to hostpython3crystax).
This is a first step to python-for-android continuous integration.
It currently only verifies that p4a creates an APK by cross compiling
a limited set of recipes.

How it works; On git push Travis "executes" `.travis.yml` instructions:

1. the `before_install` section runs `docker build` to prepare the environment
2. the `script` section is the actual test, building the APK with `docker run`
3. based the exit status Travis build will be green or red

For example editing `pythonforandroid/recipes/hostpython2/__init__.py` and
introducing an error e.g. replace `-j5` with `--wrong-flag` would make it fail.

Things to improve:

 - improve `.travis.yml` readability
 - test more recipes
 - bring Python3 and Crystax support
 - speed up build/run by caching downloads
 - and much more
This reverts commit b88767d, reversing
changes made to dab32b7.
First tries to return local recipe path or defaults to core one.
Has the following requirements:
sdl2,pyjnius,kivy,python2
refs #1263
Has the following requirements:
sdl2,pyjnius,kivy,python2,openssl,requests,sqlite3

peewee had to be skipped because pure python modules fail with
nested virtualenv issue, see:
#1263 (comment)

refs #1263
Not only this speeds up build, but also workaround Travis 4M logs limits,
since it creates one log per build rather than all in the same file.

Also adds `-qq` flag to `apt` and `--quiet` to `pip`, to reduce logs even more.
Has the following requirements:
sdl2,pyjnius,kivy,python3crystax
Also updated Dockerfile to download and install CrystaX NDK.
Refs #1263
On Travis CrystaX download is redirected from
https://eu.crystax.net/download/crystax-ndk-10.3.2-linux-x86.tar.xz
to
https://us.crystax.net/download/crystax-ndk-10.3.2-linux-x86.tar.xz
Adding `--location` flag makes it possible to automatically follow
that redirect.
Also replaced tar by bsdtar, since tar was failing with the following:
```
tar: crystax-ndk-10.3.2/sources/cpufeatures: Directory renamed before its status could be extracted
```
Plus excluded some folders from the extract command to save time and space.
Reduces decompressed size from 7.6G to 6.3G.
Also added `gnutls_handshake` flag to `curl` to workaround random `gnutls_handshake()`
issues on CrystaX download only. The error was:
```
curl: (35) gnutls_handshake() failed: Error in the pull function.
```
Covers `setuptools` case in both `python2` and `hostpython3crystax`.
The `hostpython3crystax` build is supposed to fail until branch
`feature/ticket1154_hostpython3crystax_build_dir_and_synlink` is merged.
The merge will be done in next commit to demonstrate the fix.
Also adds android recipe case.
Plus removed the `uname -a` and `p4a apk --help` simple test cases since
the maximum of parallel run seems to be 5.
Creates an empty hostpython3crystax build directory and symlinks
system python3 to it, fixes #1154
Makes sure lint checks are ran before `docker build` so we fail fast.
Currently `flake8` errors are ignored except for `E101` and `W191`.
This will demonstrate the fail fast case.
Next commit will fix these two errors in the code base.
  - E101 indentation contains mixed spaces and tabs
  - W191 indentation contains tabs
This change may introduce regressions since some recipes were importing
from wrong module. e.g. importing from `pythonforandroid.toolchain`
rather than from `pythonforandroid.recipe` and `pythonforandroid.toolchain`
has a lot of apparently "unused" imports.
Fixes or ignores some undefined names.
Plus deletes `get_directory()` dead code.
Spacing related issues:

  - E203 whitespace before ','
  - E231 missing whitespace after ','
  - E261 at least two spaces before inline comment
  - E302 expected 2 blank lines
  - E303 too many blank lines
  - E305 expected 2 blank lines after class or function definition
  - W291 trailing whitespace
  - W293 blank line contains whitespace
  - W391 blank line at end of file
Fixes `find_library()` `OSError`/`FileNotFoundError` exception when app
deployed external SD storage.
Apps deployed on external SD storage have different root prefix.
External SD storage apps root dir prefix is like:
```
/mnt/expand/<sd-card-id>/user/0/<package.domain>.<package.name>/files/app
```
While internal storage apps root dir prefix is:
```
/data/data/<package.domain>.<package.name>/files/app
```
Hence the `[0:4]` trick doesn't work.
Adds CrystaX/Python3 support to 9 recipes:

  - cffi: fixes build in both Python2 and Python3
  - decorator: updates version and adds python3crystax support
  - idna: updates version
  - pycryptodome: fixes version string & adds python3crystax support
  - python3crystax: adds python3crystax support
  - pyyaml: updates version & drops host python via target python
  - requests: adds python3crystax support
  - scrypt: fixes version & adds python3crystax support
  - secp256k1: adds python3crystax support

These recipes were tested with CrystaX/Python3 in
https://github.com/AndreMiras/EtherollApp
See last `buildozer android debug` build success:
https://travis-ci.org/AndreMiras/EtherollApp/jobs/386381704
AndreMiras and others added 17 commits September 6, 2018 17:34
This changes affect almost all recipes inherited from PythonRecipe. Some recipes where affected by the changes and were modified in order to succesfully build. One of the changes forces to get python headers and linkages for all the CythonRecipe,so, all inherited recipes from CythonRecipe should be reviewed and cleaned up if necessary and all the recipes who inherited from PythonRecipe probably should be reviewed, taking into account the fact pointed by user @AndreMiras, some flags of some recipes are wrong:

  - CFLAGS may only be used to specify C compiler flags, for macro definitions use CPPFLAGS
  - LDFLAGS may only be used to specify linker flags, for libraries use LIBS

Resolves: #668
See also: #1307
The final docker image size is reduced about 2GB by cleaning up the apt files and removing unneeded compressed files.
ignores .DS_Store files while comparing build-tools versions
Newer releases of setuptools on pypi are only available as zip archives.

In order to support higher versions of setuptools, zip should be
preferred over tar.gz. Otherwise, installing e.g. setuptools==40.0.0
will fail.
This patch extends the existing numpy recipe to also support
python3crystax. Additionally, the existing recipe code is cleaned up a
bit and numpy is upgraded to version 1.15.1.
@Zen-CODE
Copy link
Member

This looks good, but would it be possible to rebase so we can merge this as one commit? Otherwise all the commits will add a lot of "noise" to the merge...

Thanks

@KeyWeeUsr
Copy link
Contributor

@Zen-CODE https://blog.github.com/2016-04-01-squash-your-commits/ :)

Copy link
Member

@Zen-CODE Zen-CODE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable enough to me...And does not seem to break anything :-)

@AndreMiras AndreMiras merged commit 09ee2db into kivy:master Nov 10, 2018
@AndreMiras
Copy link
Member

Sorry we forgot about merging that one, thanks for the fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.