Skip to content

Commit 8e2416f

Browse files
Jonathan DEKHTIARlgarithm
Jonathan DEKHTIAR
authored and
lgarithm
committed
Docker Integration (#648)
* no message * Configuration CircleCI fixed * Python added * Config Fix * CircleCI test fix * Readme Updated * CircleCI conditional build added * Error Fix * Error fix * Changelog Updated * CircleCI config updated with pinned version * Missing Package added "requests" * CircleCI conf updated * CircleCI updated * CircleCI test * Test CircleCI * Test CircleCI * Circle CI TEST * Circle CI Build Test * Circle BUILD TEST * problematic line removed * problematic line removed * DockerFiles Fixed * double quote fix * Test CircleCI * Docker Push Re-Organised * Test Workflow CircleCI * Code Checkout Added * Test CircleCI release Dev * Version Prefix fix * Last Version Build Restored * Nightly Builds added * Nightly build test * CircleCI - Runs Name Changed * Nightly builds errors fix * Test still OK CircleCI * Test CircleCI still OK * Nightly builds added * CircleCI config Updated * Renamed test builds * CircleCI Approval Push Added on Tag * Approval Test * Test Approval * CircleCI Config Cleaned for Merge
1 parent e965d86 commit 8e2416f

File tree

12 files changed

+759
-25
lines changed

12 files changed

+759
-25
lines changed

‎.circleci/config.yml

Lines changed: 548 additions & 0 deletions
Large diffs are not rendered by default.

‎CHANGELOG.md

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -71,26 +71,30 @@ To release a new version, please update the changelog as followed:
7171
### Added
7272
- API:
7373
- `tl.alphas` and `tl.alphas_like` added following the tf.ones/zeros and tf.zeros_like/ones_like (by @DEKHTIARJonathan in #580)
74-
- Tutorials:
75-
- `tutorial_tfslim` has been introduced to show how to use `SlimNetsLayer` (by @2wins in #560).
76-
- Test:
77-
- `test_utils_predict.py` added to reproduce and fix issue #288 (by @2wins in #566)
78-
- `Layer_DeformableConvolution_Test` added to reproduce issue #572 with deformable convolution (by @DEKHTIARJonathan in #573)
79-
- `Array_Op_Alphas_Test` and `Array_Op_Alphas_Like_Test` added to test `tensorlayer/array_ops.py` file (by @DEKHTIARJonathan in #580)
80-
- `test_optimizer_amsgrad.py` added to test `AMSGrad` optimizer (by @DEKHTIARJonathan in #636)
81-
- `test_logging.py` added to insure robustness of the logging API (by @DEKHTIARJonathan in #645)
8274
- CI Tool:
8375
- [Stale Probot](https://github.com/probot/stale) added to clean stale issues (by @DEKHTIARJonathan in #573)
8476
- [Changelog Probot](https://github.com/mikz/probot-changelog) Configuration added (by @DEKHTIARJonathan in #637)
8577
- Travis Builds now handling a matrix of TF Version from TF==1.6.0 to TF==1.8.0 (by @DEKHTIARJonathan in #644)
86-
- Layer:
87-
- ElementwiseLambdaLayer added to use custom function to connect multiple layer inputs (by @One-sixth in #579)
78+
- CircleCI added to build and upload Docker Containers for each PR merged and tag release (by @DEKHTIARJonathan in #648)
79+
- Docker:
80+
- Containers for each release and for each PR merged on master built (by @DEKHTIARJonathan in #648)
81+
- Containers built in the following configurations: py2+cpu, py2+gpu, py3+cpu, and py3+gpu (by @DEKHTIARJonathan in #648)
8882
- Documentation:
8983
- Release semantic version added on index page (by @DEKHTIARJonathan in #633)
9084
- Optimizers page added (by @DEKHTIARJonathan in #636)
9185
- `AMSGrad` added on Optimizers page added (by @DEKHTIARJonathan in #636)
86+
- Layer:
87+
- ElementwiseLambdaLayer added to use custom function to connect multiple layer inputs (by @One-sixth in #579)
9288
- Optimizer:
9389
- AMSGrad Optimizer added based on `On the Convergence of Adam and Beyond (ICLR 2018)` (by @DEKHTIARJonathan in #636)
90+
- Test:
91+
- `test_utils_predict.py` added to reproduce and fix issue #288 (by @2wins in #566)
92+
- `Layer_DeformableConvolution_Test` added to reproduce issue #572 with deformable convolution (by @DEKHTIARJonathan in #573)
93+
- `Array_Op_Alphas_Test` and `Array_Op_Alphas_Like_Test` added to test `tensorlayer/array_ops.py` file (by @DEKHTIARJonathan in #580)
94+
- `test_optimizer_amsgrad.py` added to test `AMSGrad` optimizer (by @DEKHTIARJonathan in #636)
95+
- `test_logging.py` added to insure robustness of the logging API (by @DEKHTIARJonathan in #645)
96+
- Tutorials:
97+
- `tutorial_tfslim` has been introduced to show how to use `SlimNetsLayer` (by @2wins in #560).
9498

9599
### Changed
96100
- Tensorflow CPU & GPU dependencies moved to separated requirement files in order to allow PyUP.io to parse them (by @DEKHTIARJonathan in #573)

‎Dockerfile

Lines changed: 0 additions & 6 deletions
This file was deleted.

‎Dockerfile.gpu

Lines changed: 0 additions & 6 deletions
This file was deleted.

‎README.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,15 +13,16 @@
1313
</div>
1414
</a>
1515

16-
[![Build Status](https://travis-ci.org/tensorlayer/tensorlayer.svg?branch=master)](https://travis-ci.org/tensorlayer/tensorlayer)
16+
[![Build Status](https://img.shields.io/travis/tensorlayer/tensorlayer.svg?label=Travis&branch=master)](https://travis-ci.org/tensorlayer/tensorlayer)
1717
[![PyPI version](https://badge.fury.io/py/tensorlayer.svg)](https://pypi.org/project/tensorlayer/)
1818
[![Github commits (since latest release)](https://img.shields.io/github/commits-since/tensorlayer/tensorlayer/latest.svg)](https://github.com/tensorlayer/tensorlayer/compare/1.8.5...master)
1919
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/tensorlayer.svg)](https://pypi.org/project/tensorlayer/)
2020
[![Supported TF Version](https://img.shields.io/badge/tensorflow-1.6.0+-blue.svg)](https://github.com/tensorflow/tensorflow/releases)
2121
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/ca2a29ddcf7445588beff50bee5406d9)](https://app.codacy.com/app/tensorlayer/tensorlayer)
22-
[![Docker Pulls](https://img.shields.io/docker/pulls/tensorlayer/tensorlayer.svg?maxAge=604800)](https://hub.docker.com/r/tensorlayer/tensorlayer/)
2322

24-
[![Documentation Status](https://readthedocs.org/projects/tensorlayer/badge/?version=stable)](https://tensorlayer.readthedocs.io/)
23+
[![CircleCI](https://img.shields.io/circleci/project/github/tensorlayer/tensorlayer.svg?label=Docker%20Build&branch=master)](https://circleci.com/gh/tensorlayer/tensorlayer/tree/master)
24+
[![Docker Pulls](https://img.shields.io/docker/pulls/tensorlayer/tensorlayer.svg?maxAge=604800)](https://hub.docker.com/r/tensorlayer/tensorlayer/)
25+
[![Documentation Status](https://img.shields.io/readthedocs/tensorlayer/stable.svg?label=ReadTheDocs)](https://tensorlayer.readthedocs.io/)
2526
[![PyUP Updates](https://pyup.io/repos/github/tensorlayer/tensorlayer/shield.svg)](https://pyup.io/repos/github/tensorlayer/tensorlayer/)
2627
[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/tensorlayer/Lobby)
2728

‎docker/DockerLint.bat

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
docker pull hadolint/hadolint:latest
2+
3+
docker run --rm -i hadolint/hadolint hadolint --ignore DL3007 - < python2\cpu\Dockerfile
4+
docker run --rm -i hadolint/hadolint hadolint --ignore DL3007 - < python2\gpu\Dockerfile
5+
docker run --rm -i hadolint/hadolint hadolint --ignore DL3007 - < python3\cpu\Dockerfile
6+
docker run --rm -i hadolint/hadolint hadolint --ignore DL3007 - < python3\gpu\Dockerfile
7+
8+
PAUSE;

‎docker/pypi_list.py

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
import argparse
2+
import requests
3+
import logging
4+
5+
import pip._internal
6+
7+
if __name__ == "__main__":
8+
9+
parser = argparse.ArgumentParser(description='Get the nth version of a given package')
10+
parser.add_argument('--package', type=str, required=True, help='The PyPI you want to inspect')
11+
parser.add_argument('--nth_last_version', type=int, default=1, help='The nth last package will be retrieved')
12+
parser.add_argument('--prerelease', help='Get PreRelease Package Version', action='store_true')
13+
parser.add_argument('--debug', help='Print debug information', action='store_true')
14+
15+
args = parser.parse_args()
16+
17+
# create logger
18+
logger = logging.getLogger("PyPI_CLI")
19+
20+
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
21+
22+
ch = logging.StreamHandler()
23+
ch.setLevel(logging.DEBUG)
24+
ch.setFormatter(formatter)
25+
logger.addHandler(ch)
26+
27+
if args.debug:
28+
logger.setLevel(logging.DEBUG)
29+
30+
logger.debug("Package: %s" % args.package)
31+
logger.debug("nth_last_version: %s" % args.nth_last_version)
32+
logger.debug("prerelease: %s" % args.prerelease)
33+
logger.debug("debug: %s" % args.debug)
34+
35+
finder = pip._internal.index.PackageFinder(
36+
[],
37+
['https://pypi.python.org/simple'],
38+
session=requests.Session()
39+
)
40+
results = finder.find_all_candidates(args.package)
41+
tmp_versions = [str(p.version) for p in results]
42+
43+
logger.debug("%s" % tmp_versions)
44+
45+
versions = list()
46+
for el in tmp_versions:
47+
if el not in versions:
48+
versions.append(el)
49+
50+
pos = -1
51+
nth_version = 1
52+
53+
while True:
54+
fetched_version = versions[pos]
55+
56+
logger.debug("Version: %s" % fetched_version)
57+
58+
if nth_version == args.nth_last_version:
59+
if args.prerelease or not ("rc" in fetched_version or "a" in fetched_version or "b" in fetched_version):
60+
break
61+
else:
62+
pos -= 1
63+
continue
64+
65+
pos -= 1
66+
nth_version += 1
67+
68+
print(fetched_version)

‎docker/python2/cpu/Dockerfile

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
FROM tensorflow/tensorflow:latest
2+
3+
LABEL version="1.0" maintainer="Jonathan DEKHTIAR <[email protected]>"
4+
5+
ARG TL_VERSION
6+
7+
RUN if [ -z "$TL_VERSION" ]; then \
8+
echo "Building a Nightly Release" \
9+
&& apt-get update \
10+
&& apt-get install -y git \
11+
&& mkdir /tensorlayer_dist/ \
12+
&& cd /tensorlayer_dist/ \
13+
&& git clone https://github.com/tensorlayer/tensorlayer.git \
14+
&& cd tensorlayer \
15+
&& pip install -e .[db,dev,test,doc] \
16+
&& rm -rf /var/lib/apt/lists/* ; \
17+
else \
18+
echo "Building Tag Release:" "$TL_VERSION" \
19+
&& pip install tensorlayer=="$TL_VERSION" ; \
20+
fi

‎docker/python2/gpu/Dockerfile

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
FROM tensorflow/tensorflow:latest-gpu
2+
3+
LABEL version="1.0" maintainer="Jonathan DEKHTIAR <[email protected]>"
4+
5+
ARG TL_VERSION
6+
7+
RUN if [ -z "$TL_VERSION" ]; then \
8+
echo "Building a Nightly Release" \
9+
&& apt-get update \
10+
&& apt-get install -y git \
11+
&& mkdir /tensorlayer_dist/ \
12+
&& cd /tensorlayer_dist/ \
13+
&& git clone https://github.com/tensorlayer/tensorlayer.git \
14+
&& cd tensorlayer \
15+
&& pip install -e .[db,dev,test,doc] \
16+
&& rm -rf /var/lib/apt/lists/* ; \
17+
else \
18+
echo "Building Tag Release:" "$TL_VERSION" \
19+
&& pip install tensorlayer=="$TL_VERSION" ; \
20+
fi

‎docker/python3/cpu/Dockerfile

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
FROM tensorflow/tensorflow:latest-py3
2+
3+
LABEL version="1.0" maintainer="Jonathan DEKHTIAR <[email protected]>"
4+
5+
ARG TL_VERSION
6+
7+
RUN if [ -z "$TL_VERSION" ]; then \
8+
echo "Building a Nightly Release" \
9+
&& apt-get update \
10+
&& apt-get install -y git \
11+
&& mkdir /tensorlayer_dist/ \
12+
&& cd /tensorlayer_dist/ \
13+
&& git clone https://github.com/tensorlayer/tensorlayer.git \
14+
&& cd tensorlayer \
15+
&& pip install -e .[db,dev,test,doc] \
16+
&& rm -rf /var/lib/apt/lists/* ; \
17+
else \
18+
echo "Building Tag Release:" "$TL_VERSION" \
19+
&& pip install tensorlayer=="$TL_VERSION" ; \
20+
fi

‎docker/python3/gpu/Dockerfile

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
FROM tensorflow/tensorflow:latest-gpu-py3
2+
3+
LABEL version="1.0" maintainer="Jonathan DEKHTIAR <[email protected]>"
4+
5+
ARG TL_VERSION
6+
7+
RUN if [ -z "$TL_VERSION" ]; then \
8+
echo "Building a Nightly Release" \
9+
&& apt-get update \
10+
&& apt-get install -y git \
11+
&& mkdir /tensorlayer_dist/ \
12+
&& cd /tensorlayer_dist/ \
13+
&& git clone https://github.com/tensorlayer/tensorlayer.git \
14+
&& cd tensorlayer \
15+
&& pip install -e .[db,dev,test,doc] \
16+
&& rm -rf /var/lib/apt/lists/* ; \
17+
else \
18+
echo "Building Tag Release:" "$TL_VERSION" \
19+
&& pip install tensorlayer=="$TL_VERSION" ; \
20+
fi

‎docker/version_prefix.py

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
import argparse
2+
import logging
3+
4+
if __name__ == "__main__":
5+
6+
parser = argparse.ArgumentParser(description='Determine the version prefix to apply depending on the version name')
7+
8+
parser.add_argument(
9+
'--version',
10+
type=str,
11+
required=True,
12+
help='The Package Version to be installed in the container'
13+
)
14+
15+
parser.add_argument('--debug', help='Print debug information', action='store_true')
16+
17+
args = parser.parse_args()
18+
19+
# create logger
20+
logger = logging.getLogger("VERSION_PREFIX_CLI")
21+
22+
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
23+
24+
ch = logging.StreamHandler()
25+
ch.setLevel(logging.DEBUG)
26+
ch.setFormatter(formatter)
27+
logger.addHandler(ch)
28+
29+
if args.debug:
30+
logger.setLevel(logging.DEBUG)
31+
32+
logger.debug("Package Version: %s" % args.version)
33+
34+
if "rc" in args.version or "a" in args.version or "b" in args.version:
35+
print("latest-dev")
36+
else:
37+
print("latest")

0 commit comments

Comments
 (0)