recipe for libxml2 #999

yaki29 · 2017-01-27T20:28:52Z

This is a recipe for libxml2. If there are still some changes to be done I would be happy to do them.

inclement · 2017-01-28T00:37:32Z

How about adding the full suite of libxml2, libxslt and lxml to the one PR? Since they're probably mostly going to be used together, this would make testing easier.

inclement

I've made a few comments that you will want to address for this to work properly. Beyond that, have you tested it? Since this recipe has no dependencies, you should be able to build it once the recipe works.

inclement · 2017-01-28T00:38:36Z

pythonforandroid/recipes/libxml2/__init__.py

+
+
+class Libxml2Recipe(CythonRecipe):
+    version = '2.9.4'


Presumably this version is much newer than the one the old toolchain used? It's hopefully fine and definitely should be targeted, but just to note it could be a source of problems.

inclement · 2017-01-28T00:39:03Z

pythonforandroid/recipes/libxml2/__init__.py

+
+class Libxml2Recipe(CythonRecipe):
+    version = '2.9.4'
+    url = 'ftp://xmlsoft.org/libxml2/libxml2-{version}.tar.gz'


I'm not sure if p4a will work with ftp urls, have you tested it?

It used to work with the old_toolchain ... so it would work with the new one. Wouldn't it?

It definitely can work, I'm just not sure if ftp:// is handled properly right now. It shouldn't be a real problem.

inclement · 2017-01-28T00:41:00Z

pythonforandroid/recipes/libxml2/__init__.py

+
+        try:
+            sh.sed('runtest\$(EXEEXT) \\/ \\/', 'Makefile')
+        except:


What are these try/except blocks for? If you expect a specific error, catch it explicitly, catching everything is bad style since you'll find it hard to debug problems that you didn't expect as they're swallowed the same way as the one you did anticipate. Since the old toolchain didn't expect any errors, you probably just don't want the try/except at all - if it fails, it's not for an expected reason so you can let p4a crash.

@inclement You are right it would be difficult debug problems with all try and except statements.
I'll remove them.

inclement · 2017-01-28T00:42:44Z

pythonforandroid/recipes/libxml2/__init__.py

+        env = super(Libxml2Recipe, self).get_recipe_env(arch)
+
+        try:
+            sh.sed('runtest\$(EXEEXT) \\/ \\/', 'Makefile')


You might not need to escape the $ character since this is a python string rather than shell code.

inclement · 2017-01-28T00:44:35Z

pythonforandroid/recipes/libxml2/__init__.py

+        except:
+            pass
+        try:
+            sh.make('-j', '$MAKE_JOBS')


$MAKE_JOBS was an environment variable in the old toolchain, but this won't be picked up here, and in fact I don't think it exists any more. For now you could just set -j5 (possibly to be improved later).

inclement · 2017-01-28T00:45:31Z

pythonforandroid/recipes/libxml2/__init__.py

+from os.path import join
+
+
+class Libxml2Recipe(CythonRecipe):


This isn't a Cython module, so you'll want to just use Recipe as the base class.

yaki29 · 2017-01-28T11:25:40Z

@inclement I'm working on libxslt, I'll send a pr when all are ready and also add BeautifulSoup4 recipe with it which was my real target. I'll also make changes to this recipe.

inclement · 2017-01-28T23:18:57Z

pythonforandroid/recipes/libxslt/__init__.py

+    patches = ['fix-dlopen.patch']
+
+    def should_build(self, arch):
+        return not exists(join(self.get_build_container_dir(arch.arch),"libxslt/.libs/libxslt.a"))


I think you can use self.get_build_dir(arch.arch) and remove the libxslt folder from the path - this should work the same, but will also work if the extracted libxslt folder is something unexpected. This is a small thing though, it probably isn't important to having the recipe actually work.

@inclement Yes, it can cause problems. I have changed it.

inclement · 2017-01-31T22:47:54Z

@yaki29 This all looks good on a quick check, nice work. But, have you tried it, does it work?

dessant · 2017-02-01T23:50:35Z

"Robots and Screen Scraping: You may not use data mining, robots, screen scraping, or similar data gathering and extraction tools on this site, except with our express written consent as noted below. "

http://www.imdb.com/conditions

We could use a site which does not forbid scraping. There are some other cleanups to do in the example, i'm not sure what is happening with those globals, pep8 method names etc. Though it's also not clear if the example is even necessary, there is nothing kivy/p4a specific needed for bs4 to work in terms of code.

inclement · 2017-02-01T23:53:27Z

++ to @dessant's comments, although I'll note that I asked for this example as these are very useful for testing p4a - I don't know anything about beautifulsoup or its compiled dependencies, so it would be difficult to test without an example.

inclement

I've added a few comments on specific parts of the testapps. As @dessant noted, please also try to follow pep8.

inclement · 2017-02-01T23:54:43Z

testapps/BeautifulSoup4/main.py

+
+			self.Pop(movie,imdb,actor,director,writer,url)
+
+		except Exception:


When catching exceptions, work out what exception you expect and catch that one specifically. Catching Exception itself will make the code ignore just about anything, which can hide all manner of problems.

inclement · 2017-02-01T23:55:09Z

testapps/BeautifulSoup4/main.py

+		on_press: root.detail(name.text)
+
+''')
+global writer


You don't need to declare these as global.

inclement · 2017-02-01T23:55:35Z

testapps/BeautifulSoup4/main.py

+		popup = Popup(title="ERROR",content=Label(text="Bro, Check your movie name"),size_hint=(0.8,0.8))
+		popup.open()
+
+	def Pop(self,movie,imdb,actor,director,writer,url):


It would be preferable to use kv to set up these labels.

yaki29 · 2017-02-01T23:58:20Z

@dessant @inclement I didn't know that imdb doesn't let you scrap it's data. Sorry for that. I'll correct those mistakes pointed by you.

KeyWeeUsr · 2017-02-04T13:51:57Z

@yaki29 I think you could use OpenLibrary or e.g. a static website that has no big database in the background.

dessant · 2017-02-04T14:10:23Z

Scraping the content of the h1 tag from https://httpbin.org/html could be enough, and keeping the example simple.

yaki29 · 2017-02-04T16:32:54Z

Thanks @dessant @KeyWeeUsr . I was about to commit the changes. I was little busy for last two days because of some college work. I'll send a commit.

inclement · 2017-02-05T17:15:46Z

pythonforandroid/recipes/libxml2/__init__.py

-            pass
+        sh.sed('runtest$(EXEEXT) \/ \/', 'Makefile')
+        sh.sed('testrecurse$(EXEEXT)$//', 'Makefile')
+        sh.make('-j', '5')


These lines should be placed in the build_arch method, and you need to navigate to the right directory, something like

def build_arch(self, arch): with current_directory(self.get_build_dir(arch)): sh.sed(....)

inclement · 2017-02-05T17:16:39Z

pythonforandroid/recipes/beautifulsoup4/__init__.py

+import sh
+from os.path import join
+
+class BSoup4Recipe(PythonRecipe):


The fact that you need to set the compiler stuff seems to suggest that this is not a pure python recipe, so maybe you need to use CythonRecipe or CompiledComponentsPythonRecipe instead.

dessant · 2017-02-07T16:57:42Z

testapps/BeautifulSoup4/main.py

+class PopupScreen(BoxLayout):
+    def error(self):
+        popup = Popup(title="ERROR",
+            content=Label(text="Check Your Network Connection"),


If you must catch exceptions in this example, please show the exception message instead of assuming what the error is.

@dessant Will do the correction.

inclement · 2017-03-04T21:32:31Z

What's the current status of this, is it working for you?

yaki29 · 2017-03-05T03:56:17Z

pythonforandroid/recipes/libxml2/__init__.py


-        sh.sed('runtest$(EXEEXT) \/ \/', 'Makefile')
-        sh.sed('testrecurse$(EXEEXT)$//', 'Makefile')


@inclement it is having error here(with this recipe alone others are working fine ( I have not tested bs4 recipe as it depends on this one)).

yaki29 · 2017-03-05T03:57:23Z

@inclement I'll look into it and try to make it work.

FilipDem · 2017-03-18T07:27:05Z

Hi, is there any idea when this could be merged? I use lxml and want to migrate to the new toolchain to solve the problem with the old OpenSSL (on Google Play). Thanks for an update...

yaki29 · 2017-03-18T07:49:34Z

@FilipDem I'm right now working on kvCompiler for gsoc 2017. But I assure you I'll spare some time and complete it.

FilipDem · 2017-09-30T09:51:58Z

Hi Yaki29,
I cannot find yet the lxml recipe in the master branch. Is this normal?
Thanks for your help!

FilipDem · 2017-10-29T14:40:52Z

Is there any idea when the lxml will be available with the new toolchain? I am using this intensively and porting to cElementTree give really problems. Thanks for the info.

inclement · 2017-10-29T22:32:45Z

@FilipDem This isn't something I'm likely to attack directly any time soon, so it probably won't get into p4a unless someone else submits a PR.

FilipDem · 2017-10-30T07:45:49Z

Thanks for the information, Inclement. Is there any alternative? I tried using cElementTree, but it was not successful.

inclement · 2017-10-31T00:44:16Z

@FilipDem I'm afraid I'm not familiar with these libraries and their alternatives.

If you are able, it would help to provide a simple runnable example using lxml functionality, so that anyone who does try to make a recipe has something to test to check if it's actually working.

eacpereira · 2017-11-13T20:45:38Z

I just added #1166 which should implement an lxml recipe. Needs testing, tho.

goffi-contrib · 2018-11-05T07:12:16Z

Hello everybody, a PR based on #1166 has been merged, so this one can now be closed (ping @AndreMiras). Thanks to everybody who worked on that.

AndreMiras · 2018-11-05T07:15:17Z

Fixed in #1428 thanks for the heads-up

recipe for libxml2 added

99599aa

inclement reviewed Jan 28, 2017

View reviewed changes

lxml,libxslt,libxml2,beautifulsoup4 recipe added

c4b8969

inclement reviewed Jan 28, 2017

View reviewed changes

yaki29 force-pushed the master branch from 136839c to c4b8969 Compare January 31, 2017 22:07

get_build_dir changed

9a0579b

inclement mentioned this pull request Feb 1, 2017

BeautifulSoup4 testapp added #1004

Closed

BeautifulSoup4 testapp added

a658345

inclement reviewed Feb 1, 2017

View reviewed changes

bs4 testapp modified

dcf8721

inclement reviewed Feb 5, 2017

View reviewed changes

dessant reviewed Feb 7, 2017

View reviewed changes

testapp for libxml2, modified bs4 testapp, libxml2 recipe modified

3a1728b

inclement closed this Mar 4, 2017

inclement reopened this Mar 4, 2017

inclement mentioned this pull request Mar 4, 2017

Added beginnings of lxml recipes #997

Closed

yaki29 commented Mar 5, 2017

View reviewed changes

Merge branch 'master' into master

c2cb83d

AndreMiras closed this Nov 5, 2018


		self.Pop(movie,imdb,actor,director,writer,url)

		except Exception:


		sh.sed('runtest$(EXEEXT) \/ \/', 'Makefile')
		sh.sed('testrecurse$(EXEEXT)$//', 'Makefile')

recipe for libxml2 #999

recipe for libxml2 #999

Uh oh!

Conversation

yaki29 commented Jan 27, 2017

Uh oh!

inclement commented Jan 28, 2017

Uh oh!

inclement left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yaki29 commented Jan 28, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

inclement commented Jan 31, 2017

Uh oh!

dessant commented Feb 1, 2017

Uh oh!

inclement commented Feb 1, 2017

Uh oh!

inclement left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yaki29 commented Feb 1, 2017

Uh oh!

KeyWeeUsr commented Feb 4, 2017

Uh oh!

dessant commented Feb 4, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yaki29 commented Feb 4, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

inclement commented Mar 4, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yaki29 commented Mar 5, 2017

Uh oh!

FilipDem commented Mar 18, 2017

Uh oh!

yaki29 commented Mar 18, 2017

Uh oh!

FilipDem commented Sep 30, 2017

Uh oh!

FilipDem commented Oct 29, 2017

dessant commented Feb 4, 2017 •

edited

Loading