Skip to content

update: Update awscli version and remove related pins #267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 11, 2020
Merged

update: Update awscli version and remove related pins #267

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3d8e710
Update awscli version to latest
saimidu Jan 10, 2020
7cc67da
Update awscli, remove dependent pins
saimidu Jan 10, 2020
732a1e7
Update setup.py package versions to latest
saimidu Jan 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions docker/1.15.0/py2/Dockerfile.cpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,9 +100,6 @@ RUN pip install --no-cache-dir -U \
keras_preprocessing==1.1.0 \
requests==2.22.0 \
keras==2.3.1 \
# botocore requires python-dateutil<2.8.1
"python-dateutil<2.8.1" \
awscli==1.16.296 \
mpi4py==3.0.2 \
"cryptography>=2.3" \
"sagemaker-tensorflow>=1.15,<1.16" \
Expand All @@ -111,10 +108,9 @@ RUN pip install --no-cache-dir -U \
${TF_URL} \
&& pip install --no-cache-dir -U \
$FRAMEWORK_SUPPORT_INSTALLABLE \
awscli==1.16.314 \
&& rm -f $FRAMEWORK_SUPPORT_INSTALLABLE \
&& pip install --no-cache-dir -U \
# awscli requires PyYAML<5.2
"PyYAML<5.2" \
horovod==0.18.2

COPY dockerd-entrypoint.py /usr/local/bin/dockerd-entrypoint.py
Expand Down
6 changes: 1 addition & 5 deletions docker/1.15.0/py2/Dockerfile.gpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,9 +133,6 @@ RUN pip install --no-cache-dir -U \
keras_preprocessing==1.1.0 \
requests==2.22.0 \
keras==2.3.1 \
# botocore requires python-dateutil<2.8.1
"python-dateutil<2.8.1" \
awscli==1.16.296 \
mpi4py==3.0.2 \
"cryptography>=2.3" \
"sagemaker-tensorflow>=1.15,<1.16" \
Expand All @@ -144,13 +141,12 @@ RUN pip install --no-cache-dir -U \
${TF_URL} \
&& pip install --no-cache-dir -U \
$FRAMEWORK_SUPPORT_INSTALLABLE \
awscli==1.16.314 \
&& rm -f $FRAMEWORK_SUPPORT_INSTALLABLE

# Install Horovod, temporarily using CUDA stubs
RUN ldconfig /usr/local/cuda/targets/x86_64-linux/lib/stubs \
&& HOROVOD_GPU_ALLREDUCE=NCCL HOROVOD_WITH_TENSORFLOW=1 pip install --no-cache-dir \
# awscli requires PyYAML<5.2
"PyYAML<5.2" \
horovod==0.18.2 \
&& ldconfig

Expand Down
8 changes: 1 addition & 7 deletions docker/1.15.0/py3/Dockerfile.cpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,6 @@ RUN pip3 --no-cache-dir install --upgrade \
RUN ln -s $(which python3) /usr/local/bin/python \
&& ln -s $(which pip3) /usr/bin/pip

# install PyYAML==5.1.2 to avoid conflict with latest awscli
# python-dateutil==2.8.0 to satisfy botocore associated with latest awscli
RUN pip install --no-cache-dir -U \
numpy==1.17.4 \
scipy==1.2.2 \
Expand All @@ -100,12 +98,9 @@ RUN pip install --no-cache-dir -U \
keras_applications==1.0.8 \
keras_preprocessing==1.1.0 \
keras==2.3.1 \
# botocore requires python-dateutil<2.8.1
"python-dateutil<2.8.1" \
requests==2.22.0 \
smdebug==0.4.14 \
sagemaker-experiments==0.1.3 \
awscli==1.16.296 \
mpi4py==3.0.2 \
"cryptography>=2.3" \
"sagemaker-tensorflow>=1.15,<1.16" \
Expand All @@ -114,11 +109,10 @@ RUN pip install --no-cache-dir -U \
&& pip install --force-reinstall --no-cache-dir -U \
${TF_URL} \
&& pip install --force-reinstall --no-cache-dir -U \
# awscli requires PyYAML<5.2
"PyYAML<5.2" \
horovod==0.18.2 \
&& pip install --no-cache-dir -U \
$FRAMEWORK_SUPPORT_INSTALLABLE \
awscli==1.16.314 \
&& rm -f $FRAMEWORK_SUPPORT_INSTALLABLE

COPY dockerd-entrypoint.py /usr/local/bin/dockerd-entrypoint.py
Expand Down
8 changes: 1 addition & 7 deletions docker/1.15.0/py3/Dockerfile.gpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,8 +125,6 @@ RUN ln -s $(which python3) /usr/local/bin/python \

COPY $FRAMEWORK_SUPPORT_INSTALLABLE .

# install PyYAML==5.1.2 to avoid conflict with latest awscli
# python-dateutil==2.8.0 to satisfy botocore associated with latest awscli
RUN pip install --no-cache-dir -U \
numpy==1.17.4 \
scipy==1.2.2 \
Expand All @@ -138,11 +136,8 @@ RUN pip install --no-cache-dir -U \
keras_preprocessing==1.1.0 \
requests==2.22.0 \
keras==2.3.1 \
# botocore requires python-dateutil<2.8.1
"python-dateutil<2.8.1" \
smdebug==0.4.14 \
sagemaker-experiments==0.1.3 \
awscli==1.16.296 \
mpi4py==3.0.2 \
"cryptography>=2.3" \
"sagemaker-tensorflow>=1.15,<1.16" \
Expand All @@ -152,13 +147,12 @@ RUN pip install --no-cache-dir -U \
${TF_URL} \
&& pip install --no-cache-dir -U \
$FRAMEWORK_SUPPORT_INSTALLABLE \
awscli==1.16.314 \
&& rm -f $FRAMEWORK_SUPPORT_INSTALLABLE

# Install Horovod, temporarily using CUDA stubs
RUN ldconfig /usr/local/cuda-10.0/targets/x86_64-linux/lib/stubs \
&& HOROVOD_GPU_ALLREDUCE=NCCL HOROVOD_WITH_TENSORFLOW=1 pip install --no-cache-dir \
# awscli requires PyYAML<5.2
"PyYAML<5.2" \
horovod==0.18.2 \
&& ldconfig

Expand Down
8 changes: 4 additions & 4 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,13 +53,13 @@ def read_version():
'Programming Language :: Python :: 3.6',
],

install_requires=['sagemaker-containers>=2.4.6', 'numpy', 'scipy', 'sklearn',
install_requires=['sagemaker-containers>=2.6.2', 'numpy', 'scipy', 'sklearn',
'pandas', 'Pillow', 'h5py'],
extras_require={
'test': ['tox', 'flake8', 'pytest', 'pytest-cov', 'pytest-xdist', 'mock',
'sagemaker==1.19.1', 'tensorflow<2.0', 'docker-compose', 'boto3==1.10.32',
'six==1.13.0', 'python-dateutil>=2.1,<2.8.1', 'botocore==1.13.32',
'requests-mock', 'awscli==1.16.296'],
'sagemaker==1.50.1', 'tensorflow<2.0', 'docker-compose', 'boto3==1.10.50',
'six==1.13.0', 'python-dateutil>=2.1,<2.8.1', 'botocore==1.13.50',
'requests-mock', 'awscli==1.16.314'],
'benchmark': ['click']
},
)