Skip to content

Commit 96bfc27

Browse files
nadiayanadiaya
authored
feature: Add support for TF-2.0.0. (#1191)
* feature: Add support for TF-2.0.0. * doc: Update TF documentation.
1 parent 67380a7 commit 96bfc27

File tree

8 files changed

+69
-58
lines changed

8 files changed

+69
-58
lines changed

‎README.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -193,7 +193,7 @@ TensorFlow SageMaker Estimators
193193

194194
By using TensorFlow SageMaker Estimators, you can train and host TensorFlow models on Amazon SageMaker.
195195

196-
Supported versions of TensorFlow: ``1.4.1``, ``1.5.0``, ``1.6.0``, ``1.7.0``, ``1.8.0``, ``1.9.0``, ``1.10.0``, ``1.11.0``, ``1.12.0``, ``1.13.1``, ``1.14``.
196+
Supported versions of TensorFlow: ``1.4.1``, ``1.5.0``, ``1.6.0``, ``1.7.0``, ``1.8.0``, ``1.9.0``, ``1.10.0``, ``1.11.0``, ``1.12.0``, ``1.13.1``, ``1.14.``, ``1.15.0``, ``2.0.0``.
197197

198198
Supported versions of TensorFlow for Elastic Inference: ``1.11.0``, ``1.12.0``, ``1.13.1``, ``1.14``.
199199

‎doc/using_tf.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ For general information about using the SageMaker Python SDK, see :ref:`overview
2020

2121
.. contents::
2222

23-
Supported versions of TensorFlow for Elastic Inference: ``1.11.0``, ``1.12.0``.
23+
Supported versions of TensorFlow for Elastic Inference: ``1.11``, ``1.12``, ``1.13``, ``1.14``.
2424

2525

2626
*****************************

‎src/sagemaker/tensorflow/README.rst

Lines changed: 39 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -20,52 +20,52 @@ Documentation of the previous Legacy Mode versions: `1.4.1 <https://github.com/a
2020
| You can find the Legacy Mode README `here <https://github.com/aws/sagemaker-python-sdk/tree/v1.12.0/src/sagemaker/tensorflow#tensorflow-sagemaker-estimators-and-models>`_. |
2121
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2222

23-
Supported versions of TensorFlow for Elastic Inference: ``1.11``, ``1.12``, ``1.13``, ``1.15``.
23+
Supported versions of TensorFlow for Elastic Inference: ``1.11``, ``1.12``, ``1.13``, ``1.14``.
2424

2525
For information about using TensorFlow with the SageMaker Python SDK, see https://sagemaker.readthedocs.io/en/stable/using_tf.html.
2626

2727
SageMaker TensorFlow Docker containers
2828
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2929

30-
The containers include the following Python packages:
30+
The latest containers include the following Python packages:
3131

32-
+--------------------------------+---------------+
33-
| Dependencies | Script Mode |
34-
+--------------------------------+---------------+
35-
| awscli | 1.16.130 |
36-
+--------------------------------+---------------+
37-
| boto3 | Latest |
38-
+--------------------------------+---------------+
39-
| botocore | Latest |
40-
+--------------------------------+---------------+
41-
| h5py | 2.9.0 |
42-
+--------------------------------+---------------+
43-
| keras | 2.2.4 |
44-
+--------------------------------+---------------+
45-
| mpi4py | 3.0.1 |
46-
+--------------------------------+---------------+
47-
| numpy | 1.16.2 |
48-
+--------------------------------+---------------+
49-
| pandas | 0.24.2 |
50-
+--------------------------------+---------------+
51-
| pip | 19.1.1 |
52-
+--------------------------------+---------------+
53-
| Pillow | 5.4.1 |
54-
+--------------------------------+---------------+
55-
| Python | 2.7 or 3.6 |
56-
+--------------------------------+---------------+
57-
| requests | 2.21.0 |
58-
+--------------------------------+---------------+
59-
| sagemaker-containers | >=2.4.6 |
60-
+--------------------------------+---------------+
61-
| sagemaker-tensorflow-container | Latest |
62-
+--------------------------------+---------------+
63-
| scipy | 1.2.1 |
64-
+--------------------------------+---------------+
65-
| sklean | 0.20.3 |
66-
+--------------------------------+---------------+
67-
| tensorflow | 1.15.0 |
68-
+--------------------------------+---------------+
32+
+--------------------------------+---------------+---------------+
33+
| Dependencies | TF 1.15 | TF 2.0 |
34+
+--------------------------------+---------------+---------------+
35+
| awscli | 1.16.296 | 1.16.303 |
36+
+--------------------------------+---------------+---------------+
37+
| boto3 | Latest | Latest |
38+
+--------------------------------+---------------+---------------+
39+
| botocore | Latest | Latest |
40+
+--------------------------------+---------------+---------------+
41+
| h5py | 2.9.0 | 2.10.0 |
42+
+--------------------------------+---------------+---------------+
43+
| horovod | 0.18.2 | 0.18.2 |
44+
+--------------------------------+---------------+---------------+
45+
| keras | 2.3.1 | 2.3.1 |
46+
+--------------------------------+---------------+---------------+
47+
| mpi4py | 3.0.2 | 3.0.3 |
48+
+--------------------------------+---------------+---------------+
49+
| numpy | 1.17.4 | 1.17.4 |
50+
+--------------------------------+---------------+---------------+
51+
| pandas | 0.24.2 | 0.25.3 |
52+
+--------------------------------+---------------+---------------+
53+
| pip | 19.3.1 | 19.3.1 |
54+
+--------------------------------+---------------+---------------+
55+
| Pillow | 6.1.0 | 6.2.1 |
56+
+--------------------------------+---------------+---------------+
57+
| Python | 2.7 or 3.6 | 2.7 or 3.6 |
58+
+--------------------------------+---------------+---------------+
59+
| requests | 2.22.0 | 2.22.0 |
60+
+--------------------------------+---------------+---------------+
61+
| sagemaker-containers | >=2.6.1 | >=2.6.2 |
62+
+--------------------------------+---------------+---------------+
63+
| sagemaker-tensorflow-container | 1.15.0.1.0.0 | 2.0.0.1.0.0 |
64+
+--------------------------------+---------------+---------------+
65+
| scipy | 1.2.2 | 1.2.2 |
66+
+--------------------------------+---------------+---------------+
67+
| tensorflow | 1.15.0 | 2.0.0 |
68+
+--------------------------------+---------------+---------------+
6969

7070
Script Mode TensorFlow Docker images support both Python 2.7 and Python 3.6. The Docker images extend Ubuntu 16.04.
7171

‎src/sagemaker/tensorflow/estimator.py

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -197,13 +197,15 @@ class TensorFlow(Framework):
197197

198198
__framework_name__ = "tensorflow"
199199

200-
LATEST_VERSION = "1.15"
200+
LATEST_VERSION = "2.0.0"
201201
"""The latest version of TensorFlow included in the SageMaker pre-built Docker images."""
202202

203+
_LATEST_1X_VERSION = "1.15.0"
204+
203205
_LOWEST_SCRIPT_MODE_ONLY_VERSION = [1, 13]
204-
# 1.15.0 still supports py2
206+
# 2.0.0 still supports py2
205207
# we will need to update this version number if future versions still support py2
206-
_HIGHEST_PYTHON_2_VERSION = [1, 15, 0]
208+
_HIGHEST_PYTHON_2_VERSION = [2, 0, 0]
207209

208210
def __init__(
209211
self,

‎tests/conftest.py

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ def pytest_addoption(parser):
5656
"--rl-ray-full-version", action="store", default=RLEstimator.RAY_LATEST_VERSION
5757
)
5858
parser.addoption("--sklearn-full-version", action="store", default=SKLEARN_VERSION)
59-
parser.addoption("--tf-full-version", action="store", default=TensorFlow.LATEST_VERSION)
59+
parser.addoption("--tf-full-version", action="store")
6060
parser.addoption("--ei-tf-full-version", action="store", default=TensorFlow.LATEST_VERSION)
6161
parser.addoption("--xgboost-full-version", action="store", default=SKLEARN_VERSION)
6262

@@ -246,9 +246,13 @@ def sklearn_full_version(request):
246246
return request.config.getoption("--sklearn-full-version")
247247

248248

249-
@pytest.fixture(scope="module")
249+
@pytest.fixture(scope="module", params=[TensorFlow._LATEST_1X_VERSION, TensorFlow.LATEST_VERSION])
250250
def tf_full_version(request):
251-
return request.config.getoption("--tf-full-version")
251+
tf_version = request.config.getoption("--tf-full-version")
252+
if tf_version is None:
253+
return request.param
254+
else:
255+
return tf_version
252256

253257

254258
@pytest.fixture(scope="module")

‎tests/integ/test_tf_script_mode.py

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@
3939
TAGS = [{"Key": "some-key", "Value": "some-value"}]
4040

4141

42-
def test_mnist_with_checkpoint_config(sagemaker_session, instance_type):
42+
def test_mnist_with_checkpoint_config(sagemaker_session, instance_type, tf_full_version):
4343
checkpoint_s3_uri = "s3://{}/checkpoints/tf-{}".format(
4444
sagemaker_session.default_bucket(), sagemaker_timestamp()
4545
)
@@ -51,7 +51,7 @@ def test_mnist_with_checkpoint_config(sagemaker_session, instance_type):
5151
train_instance_type=instance_type,
5252
sagemaker_session=sagemaker_session,
5353
script_mode=True,
54-
framework_version=TensorFlow.LATEST_VERSION,
54+
framework_version=tf_full_version,
5555
py_version=tests.integ.PYTHON_VERSION,
5656
metric_definitions=[{"Name": "train:global_steps", "Regex": r"global_step\/sec:\s(.*)"}],
5757
checkpoint_s3_uri=checkpoint_s3_uri,
@@ -82,7 +82,7 @@ def test_mnist_with_checkpoint_config(sagemaker_session, instance_type):
8282
assert actual_training_checkpoint_config == expected_training_checkpoint_config
8383

8484

85-
def test_server_side_encryption(sagemaker_session):
85+
def test_server_side_encryption(sagemaker_session, tf_full_version):
8686
boto_session = sagemaker_session.boto_session
8787
with kms_utils.bucket_with_encryption(boto_session, ROLE) as (bucket_with_kms, kms_key):
8888
output_path = os.path.join(
@@ -97,7 +97,7 @@ def test_server_side_encryption(sagemaker_session):
9797
train_instance_type="ml.c5.xlarge",
9898
sagemaker_session=sagemaker_session,
9999
script_mode=True,
100-
framework_version=TensorFlow.LATEST_VERSION,
100+
framework_version=tf_full_version,
101101
py_version=tests.integ.PYTHON_VERSION,
102102
code_location=output_path,
103103
output_path=output_path,
@@ -125,7 +125,7 @@ def test_server_side_encryption(sagemaker_session):
125125

126126

127127
@pytest.mark.canary_quick
128-
def test_mnist_distributed(sagemaker_session, instance_type):
128+
def test_mnist_distributed(sagemaker_session, instance_type, tf_full_version):
129129
estimator = TensorFlow(
130130
entry_point=SCRIPT,
131131
role=ROLE,
@@ -134,7 +134,7 @@ def test_mnist_distributed(sagemaker_session, instance_type):
134134
sagemaker_session=sagemaker_session,
135135
py_version=tests.integ.PYTHON_VERSION,
136136
script_mode=True,
137-
framework_version=TensorFlow.LATEST_VERSION,
137+
framework_version=tf_full_version,
138138
distributions=PARAMETER_SERVER_DISTRIBUTION,
139139
)
140140
inputs = estimator.sagemaker_session.upload_data(
@@ -159,6 +159,7 @@ def test_mnist_async(sagemaker_session, cpu_instance_type):
159159
py_version=tests.integ.PYTHON_VERSION,
160160
sagemaker_session=sagemaker_session,
161161
script_mode=True,
162+
# testing py-sdk functionality, no need to run against all TF versions
162163
framework_version=TensorFlow.LATEST_VERSION,
163164
tags=TAGS,
164165
)
@@ -191,7 +192,7 @@ def test_mnist_async(sagemaker_session, cpu_instance_type):
191192
_assert_model_name_match(sagemaker_session.sagemaker_client, endpoint_name, model_name)
192193

193194

194-
def test_deploy_with_input_handlers(sagemaker_session, instance_type):
195+
def test_deploy_with_input_handlers(sagemaker_session, instance_type, tf_full_version):
195196
estimator = TensorFlow(
196197
entry_point="training.py",
197198
source_dir=TFS_RESOURCE_PATH,
@@ -201,7 +202,7 @@ def test_deploy_with_input_handlers(sagemaker_session, instance_type):
201202
py_version=tests.integ.PYTHON_VERSION,
202203
sagemaker_session=sagemaker_session,
203204
script_mode=True,
204-
framework_version=TensorFlow.LATEST_VERSION,
205+
framework_version=tf_full_version,
205206
tags=TAGS,
206207
)
207208

‎tests/integ/test_tuner.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -590,7 +590,7 @@ def test_tuning_mxnet(sagemaker_session, mxnet_full_version, cpu_instance_type):
590590

591591

592592
@pytest.mark.canary_quick
593-
def test_tuning_tf_script_mode(sagemaker_session, cpu_instance_type):
593+
def test_tuning_tf_script_mode(sagemaker_session, cpu_instance_type, tf_full_version):
594594
resource_path = os.path.join(DATA_DIR, "tensorflow_mnist")
595595
script_path = os.path.join(resource_path, "mnist.py")
596596

@@ -602,7 +602,7 @@ def test_tuning_tf_script_mode(sagemaker_session, cpu_instance_type):
602602
script_mode=True,
603603
sagemaker_session=sagemaker_session,
604604
py_version=PYTHON_VERSION,
605-
framework_version=TensorFlow.LATEST_VERSION,
605+
framework_version=tf_full_version,
606606
)
607607

608608
hyperparameter_ranges = {"epochs": IntegerParameter(1, 2)}

‎tests/unit/test_tf_estimator.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -982,7 +982,7 @@ def test_script_mode_deprecated_args(sagemaker_session):
982982

983983
def test_py2_version_deprecated(sagemaker_session):
984984
with pytest.raises(AttributeError) as e:
985-
_build_tf(sagemaker_session=sagemaker_session, framework_version="1.15.1", py_version="py2")
985+
_build_tf(sagemaker_session=sagemaker_session, framework_version="2.0.1", py_version="py2")
986986

987987
msg = "Python 2 containers are only available until January 1st, 2020. Please use a Python 3 container."
988988
assert msg in str(e.value)
@@ -993,6 +993,10 @@ def test_py2_version_is_not_deprecated(sagemaker_session):
993993
sagemaker_session=sagemaker_session, framework_version="1.15.0", py_version="py2"
994994
)
995995
assert estimator.py_version == "py2"
996+
estimator = _build_tf(
997+
sagemaker_session=sagemaker_session, framework_version="2.0.0", py_version="py2"
998+
)
999+
assert estimator.py_version == "py2"
9961000

9971001

9981002
def test_py3_is_default_version_before_tf1_14(sagemaker_session):

0 commit comments

Comments
 (0)