Merge remote-tracking branch 'aws/zwei' into require-framework-version

Author: metrizable

doc/frameworks/tensorflow/upgrade_from_legacy.rst

@@ -2,7 +2,7 @@
 Upgrade from Legacy TensorFlow Support
 ######################################
 
-With v2 of the SageMaker Python SDK, support for legacy SageMaker TensorFlow images has been deprecated.
+With version 2.0 and later of the SageMaker Python SDK, support for legacy SageMaker TensorFlow images has been deprecated.
 This guide explains how to upgrade your SageMaker Python SDK usage.
 
 For more information about using TensorFlow with the SageMaker Python SDK, see `Use TensorFlow with the SageMaker Python SDK <using_tf.html>`_.
@@ -67,13 +67,13 @@ For more information about implementing your own handlers, see `How to implement
 Continue with Legacy Versions
 *****************************
 
-While not recommended, you can still use a legacy TensorFlow version with v2 of the SageMaker Python SDK.
+While not recommended, you can still use a legacy TensorFlow version with version 2.0 and later of the SageMaker Python SDK.
 In order to do so, you need to change how a few parameters are defined.
 
 Training
 ========
 
-When creating an estimator, v2 requires the following changes:
+When creating an estimator, the Python SDK version 2.0 and later requires the following changes:
 
 #. Explicitly specify the ECR image URI via ``image_name``.
    To determine the URI, you can use :func:`sagemaker.fw_utils.create_image_uri`.
@@ -87,7 +87,7 @@ the difference in code would be as follows:
 
     from sagemaker.tensorflow import TensorFlow
 
-    # v1
+    # v1.x
     estimator = TensorFlow(
         ...
         source_dir="code",
@@ -99,7 +99,7 @@ the difference in code would be as follows:
         requirements_file="requirements.txt",
     )
 
-    # v2
+    # v2.0 and later
     estimator = TensorFlow(
         ...
         source_dir="code",
@@ -123,7 +123,7 @@ To provide a requirements file, define a hyperparameter named "sagemaker_require
 Inference
 =========
 
-Using a legacy TensorFlow version for endpoints and batch transform can be achieved with v2 of the SageMaker Python SDK with some minor changes to your code.
+Using a legacy TensorFlow version for endpoints and batch transform can be achieved with version 2.0 and later of the SageMaker Python SDK with some minor changes to your code.
 
 From an Estimator
 -----------------
@@ -134,16 +134,16 @@ To specify the number of model server workers, you need to set it through an env
 
 .. code:: python
 
-    # v1
+    # v1.x
     estimator.deploy(..., model_server_workers=4)
 
-    # v2
+    # v2.0 and later
     estimator.deploy(..., env={"MODEL_SERVER_WORKERS": 4})
 
 From a Trained Model
 --------------------
 
-If you are starting with a trained model, v2 requires the following changes:
+If you are starting with a trained model, the Python SDK version 2.0 and later requires the following changes:
 
 #. Use the the :class:`sagemaker.model.FrameworkModel` class.
 #. Explicitly specify the ECR image URI via ``image``.
@@ -155,7 +155,7 @@ the difference in code would be as follows:
 
 .. code:: python
 
-    # v1
+    # v1.x
     from sagemaker.tensorflow import TensorFlowModel
 
     model = TensorFlowModel(
@@ -165,7 +165,7 @@ the difference in code would be as follows:
         model_server_workers=4,
     )
 
-    # v2
+    # v2.0 and later
     from sagemaker.model import FrameworkModel
 
     model = FrameworkModel(

doc/index.rst

@@ -20,6 +20,7 @@ Overview
     :maxdepth: 2
 
     overview
+    v2
 
 The SageMaker Python SDK APIs:
 

doc/v2.rst

@@ -0,0 +1,86 @@
+##########################################################
+Use Version 2.0 of the SageMaker Python SDK (experimental)
+##########################################################
+
+Development on v2.0.0 of the SageMaker Python SDK is underway.
+For more info on our plans, see https://github.com/aws/sagemaker-python-sdk/issues/1459.
+
+.. contents::
+    :local:
+
+************
+Installation
+************
+
+.. warning::
+    Version 2.0.0 is currently experimental, so proceed with caution. If you do run into issues or have any other feedback,
+    please let us know by `opening an issue <https://github.com/aws/sagemaker-python-sdk/issues/new/choose>`_
+    or `commenting on our planning issue <https://github.com/aws/sagemaker-python-sdk/issues/1459>`_.
+
+To install the latest release candidate:
+
+.. code:: bash
+
+    pip install [email protected]:aws/[email protected]
+
+To install the latest version of v2:
+
+.. code:: bash
+
+    pip install [email protected]:aws/sagemaker-python-sdk.git@zwei
+
+If you are executing this pip install command in a notebook, make sure to restart your kernel.
+
+*******
+Changes
+*******
+
+To see what changes have been made, see the `CHANGELOG <https://github.com/aws/sagemaker-python-sdk/blob/zwei/CHANGELOG.md>`_.
+
+*******************************
+Automatically Upgrade Your Code
+*******************************
+
+To help make your transition as seamless as possible, v2 of the SageMaker Python SDK comes with a command-line tool to automate updating your code.
+It automates as much as possible, but there are still syntactical and stylistic changes that cannot be performed by the script.
+
+.. warning::
+    While the tool is intended to be easy to use, we recommend using it as part of a process that includes testing before and after you run the tool.
+
+Usage
+=====
+
+Currently, the tool supports only converting one file at a time:
+
+.. code::
+
+    $ sagemaker-upgrade-v2 --in-file input.py --out-file output.py
+    $ sagemaker-upgrade-v2 --in-file input.ipynb --out-file output.ipynb
+
+You can apply it to a set of files using a loop:
+
+.. code:: bash
+
+    $ for file in $(find input-dir); do sagemaker-upgrade-v2 --in-file $file --out-file output-dir/$file; done
+
+Limitations
+===========
+
+Aliased Imports
+---------------
+
+The tool checks for a limited number of patterns when looking for constructors.
+For example, if you are using a TensorFlow estimator, only the following invocation styles are handled:
+
+.. code:: python
+
+    TensorFlow()
+    sagemaker.tensorflow.TensorFlow()
+    sagemaker.tensorflow.estimator.TensorFlow()
+
+If you have aliased an import, e.g. ``from sagemaker.tensorflow import TensorFlow as TF``, the tool does not take care of updating its parameters.
+
+TensorFlow Serving
+------------------
+
+If you are using the ``sagemaker.tensorflow.serving.Model`` class, the tool does not take care of adding a framework version or changing it to ``sagemaker.tensorflow.TensorFlowModel``.

src/sagemaker/cli/compatibility/v2/__init__.py

@@ -10,5 +10,5 @@
 # distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
 # ANY KIND, either express or implied. See the License for the specific
 # language governing permissions and limitations under the License.
-"""Tools to assist with upgrading to v2 of the SageMaker Python SDK."""
+"""Tools to assist with upgrading to version 2.0 and later of the SageMaker Python SDK."""
 from __future__ import absolute_import

src/sagemaker/cli/compatibility/v2/files.py

@@ -30,7 +30,7 @@ class FileUpdater(object):
 
     def __init__(self, input_path, output_path):
         """Creates a ``FileUpdater`` for updating a file so that
-        it is compatible with v2 of the SageMaker Python SDK.
+        it is compatible with version 2.0 and later of the SageMaker Python SDK.
 
         Args:
             input_path (str): Location of the input file.
@@ -44,8 +44,8 @@ def __init__(self, input_path, output_path):
     @abstractmethod
     def update(self):
         """Reads the input file, updates the code so that it is
-        compatible with v2 of the SageMaker Python SDK, and writes the
-        updated code to an output file.
+        compatible with version 2.0 and later of the SageMaker Python SDK,
+        and writes the updated code to an output file.
         """
 
     def _make_output_dirs_if_needed(self):
@@ -66,21 +66,22 @@ class PyFileUpdater(FileUpdater):
 
     def update(self):
         """Reads the input Python file, updates the code so that it is
-        compatible with v2 of the SageMaker Python SDK, and writes the
-        updated code to an output file.
+        compatible with version 2.0 and later of the SageMaker Python SDK,
+        and writes the updated code to an output file.
         """
         output = self._update_ast(self._read_input_file())
         self._write_output_file(output)
 
     def _update_ast(self, input_ast):
         """Updates an abstract syntax tree (AST) so that it is compatible
-        with v2 of the SageMaker Python SDK.
+        with version 2.0 and later of the SageMaker Python SDK.
 
         Args:
-            input_ast (ast.Module): AST to be updated for use with Python SDK v2.
+            input_ast (ast.Module): AST to be updated for use with
+                the Python SDK version 2.0 and later.
 
         Returns:
-            ast.Module: Updated AST that is compatible with Python SDK v2.
+            ast.Module: Updated AST that is compatible with the Python SDK version 2.0 and later.
         """
         return ASTTransformer().visit(input_ast)
 
@@ -115,7 +116,7 @@ class JupyterNotebookFileUpdater(FileUpdater):
 
     def update(self):
         """Reads the input Jupyter notebook file, updates the code so that it is
-        compatible with v2 of the SageMaker Python SDK, and writes the
+        compatible with version 2.0 and later of the SageMaker Python SDK, and writes the
         updated code to an output file.
         """
         nb_json = self._read_input_file()
@@ -128,7 +129,7 @@ def update(self):
 
     def _update_code_from_cell(self, cell):
         """Updates the code from a code cell so that it is
-        compatible with v2 of the SageMaker Python SDK.
+        compatible with version 2.0 and later of the SageMaker Python SDK.
 
         Args:
             cell (dict): A dictionary representation of a code cell from

src/sagemaker/cli/compatibility/v2/modifiers/tf_legacy_mode.py

@@ -10,7 +10,9 @@
 # distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
 # ANY KIND, either express or implied. See the License for the specific
 # language governing permissions and limitations under the License.
-"""Classes to modify TensorFlow legacy mode code to be compatible with SageMaker Python SDK v2."""
+"""Classes to modify TensorFlow legacy mode code to be compatible
+with version 2.0 and later of the SageMaker Python SDK.
+"""
 from __future__ import absolute_import
 
 import ast

src/sagemaker/cli/compatibility/v2/modifiers/tfs.py

@@ -10,7 +10,9 @@
 # distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
 # ANY KIND, either express or implied. See the License for the specific
 # language governing permissions and limitations under the License.
-"""Classes to modify TensorFlow Serving code to be compatible with SageMaker Python SDK v2."""
+"""Classes to modify TensorFlow Serving code to be compatible
+with version 2.0 and later of the SageMaker Python SDK.
+"""
 from __future__ import absolute_import
 
 import ast
@@ -57,7 +59,8 @@ def node_should_be_modified(self, node):
         )
 
     def modify_node(self, node):
-        """Modifies the ``ast.Call`` node to use the v2 classes for TensorFlow Serving:
+        """Modifies the ``ast.Call`` node to use the classes for TensorFlow Serving available in
+        version 2.0 and later of the Python SDK:
 
         - ``sagemaker.tensorflow.TensorFlowModel``
         - ``sagemaker.tensorflow.TensorFlowPredictor``
@@ -72,7 +75,7 @@ def modify_node(self, node):
             node.func.value = node.func.value.value
 
     def _new_cls_name(self, cls_name):
-        """Returns the v2 class name."""
+        """Returns the updated class name."""
         return "TensorFlow{}".format(cls_name)
 
 

src/sagemaker/cli/compatibility/v2/sagemaker_upgrade_v2.py

@@ -10,7 +10,7 @@
 # distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
 # ANY KIND, either express or implied. See the License for the specific
 # language governing permissions and limitations under the License.
-"""A tool to upgrade SageMaker Python SDK code to be compatible with v2."""
+"""A tool to upgrade SageMaker Python SDK code to be compatible with version 2.0 and later."""
 from __future__ import absolute_import
 
 import argparse
@@ -22,7 +22,7 @@
 
 
 def _update_file(input_file, output_file):
-    """Updates a file to be compatible with v2 of the SageMaker Python SDK,
+    """Updates a file to be compatible with version 2.0 and later of the SageMaker Python SDK,
     and write the updated source to the output file.
 
     Args:
@@ -53,7 +53,8 @@ def _update_file(input_file, output_file):
 def _parse_args():
     """Parses CLI arguments."""
     parser = argparse.ArgumentParser(
-        description="A tool to convert files to be compatible with v2 of the SageMaker Python SDK. "
+        description="A tool to convert files to be compatible with "
+        "version 2.0 and later of the SageMaker Python SDK. "
         "Simple usage: sagemaker-upgrade-v2 --in-file foo.py --out-file bar.py"
     )
     parser.add_argument(