Adding e2e test for system-package sample

[dinagraves]

No description provided.

[dinagraves]

@tmatsuo My e2e test is in the same folder as the unit tests for this one - in the markup tutorial it was in a higher directory. The noxfile sets all of the tests to only run for python 3.7 - which I think is fine. Any concerns?

[tmatsuo]

@dinagraves
The only downside of the current hierarchy is that we skip main_test.py for other python versions.
I'm ok with it, but if you want to catch failures for other python versions, you'd better have the same hierarchy as markdown-preview project.

[dinagraves]

@tmatsuo The Dockerfile specifies python 3.8, so I think it should only matter that it passes in 3.8. I'll update the noxfile to use that instead of 3.7.

[tmatsuo]

I see. Currently we don't have 3.8 presubmit activated. Maybe we should. I'll add an internal Kokoro config.

[tmatsuo]

I updated the internal configs and now python3.8 presubmit and continuous builds should be activated. I'll try to run them by "Update branch"

[averikitsch]

Instead of using _SUFFIX here, can we use $COMMIT_SHA as the image tag?

[dinagraves]

Hrm - I have to get the URL in the test and to do that I need the name of the service. Not sure how I would get the COMMIT SHA into pytest to get the URL?

[tmatsuo]

Maybe you can just use the ${_SUFFIX} as the tag, instead of a part of the image name?

I mean

'--tag=gcr.io/$PROJECT_ID/sys-package:${_SUFFIX}'

@averikitsch WDYT?

[dinagraves]

If it's just part of the image name and not the service, wouldn't I create problems by deploying the image to the same service name if other tests are also running? Wouldn't I overwrite it?

[tmatsuo]

You need to keep the service name as sys-package-${_SUFFIX}.

[dinagraves]

Looks like COMMIT_SHA only works on builds that are triggered by a commit. Since I'm explicitly calling the build in the test, it doesn't have the substitution available and is null.

Results in error:

Step #0: invalid argument "gcr.io/python-docs-samples-tests/sys-package:" for "-t, --tag" flag: invalid reference format

[dinagraves]

@tmatsuo Would it make sense to set the noxfile_config at the run/ directory level? I'm doing the same e2e tests for all of the samples.

[tmatsuo]

@dinagraves Probably yes :) Especially if all the samples only support python 3.8

[tmatsuo]

@dinagraves Sorry I was probably wrong.

We copy noxfile.py from parent directory.
We don't copy noxfile_config.py from parent directory.

with the current implementation.

So now you need to have noxfile_config.py in each directory.

If you don't like the behavior, please file an issue so that we can discuss further there.

[dinagraves]

@tmatsuo Got it - putting noxfile_config back in the sample directories