Update TFS EI example to use TensorFlow serving container.

Author: ChoiByungWook

sagemaker-python-sdk/tensorflow_using_elastic_inference_with_your_own_model/entry.py renamed to sagemaker-python-sdk/tensorflow_serving_using_elastic_inference_with_your_own_model/entry.py

@@ -4,17 +4,17 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Using Amazon Elastic Inference with a pre-trained TensorFlow model on SageMaker\n",
+    "# Using Amazon Elastic Inference with a pre-trained TensorFlow Serving model on SageMaker\n",
     "\n",
-    "This notebook demonstrates how to enable and use Amazon Elastic Inference with our predefined SageMaker TensorFlow containers.\n",
+    "This notebook demonstrates how to enable and use Amazon Elastic Inference with our predefined SageMaker TensorFlow Serving containers.\n",
     "\n",
     "Amazon Elastic Inference (EI) is a resource you can attach to your Amazon EC2 instances to accelerate your deep learning (DL) inference workloads. EI allows you to add inference acceleration to an Amazon SageMaker hosted endpoint or Jupyter notebook for a fraction of the cost of using a full GPU instance. For more information please visit: https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html\n",
     "\n",
-    "This notebook's main objective is to show how to create an endpoint, backed by an Elastic Inference, to serve our pre-trained TensorFlow model for predictions. With a more efficient cost per performance, Amazon Elastic Inference can prove to be useful for those looking to use GPUs for higher inference performance at a lower cost.\n",
+    "This notebook's main objective is to show how to create an endpoint, backed by an Elastic Inference, to serve our pre-trained TensorFlow Serving model for predictions. With a more efficient cost per performance, Amazon Elastic Inference can prove to be useful for those looking to use GPUs for higher inference performance at a lower cost.\n",
     "\n",
     "1. [The model](#The-model)\n",
     "1. [Setup role for SageMaker](#Setup-role-for-SageMaker)\n",
-    "1. [Load the TensorFlow Model on Amazon SageMaker using Python SDK](#Load-the-TensorFlow-Model-on-Amazon-SageMaker-using-Python-SDK)\n",
+    "1. [Load the TensorFlow Serving Model on Amazon SageMaker using Python SDK](#Load-the-TensorFlow-Serving-Model-on-Amazon-SageMaker-using-Python-SDK)\n",
     "1. [Deploy the trained Model to an Endpoint with EI](#Deploy-the-trained-Model-to-an-Endpoint-with-EI)\n",
     "    1. [Using EI with a SageMaker notebook instance](#Using-EI-with-a-SageMaker-notebook-instance)\n",
     "    1. [Invoke the Endpoint to get inferences](#Invoke-the-Endpoint-to-get-inferences)\n",
@@ -38,7 +38,7 @@
     "\n",
     "The pre-trained model we will be using for this example is a NCHW ResNet-50 model from the [official Tensorflow model Github repository](https://github.com/tensorflow/models/tree/master/official/resnet#pre-trained-model). For more information in regards to deep residual networks, please check [here](https://github.com/tensorflow/models/tree/master/official/resnet). It isn't a requirement to train our model on SageMaker to use SageMaker for serving our model.\n",
     "\n",
-    "SageMaker expects our models to be compressed in a tar.gz format in S3. Thankfully, our model already comes in that format. The predefined SageMaker TensorFlow containers utilize TensorFlow serving for loading and handling inferences, so it is expecting a SavedModel. The [predefined SageMaker Tensorflow](https://github.com/aws/sagemaker-tensorflow-container/blob/master/src/tf_container/serve.py#L108) container expects the base folder name for the saved model to be `export/Servo` followed by a verison and your SavedModel artifacts.\n",
+    "SageMaker expects our models to be compressed in a tar.gz format in S3. Thankfully, our model already comes in that format. The predefined TensorFlow Serving containers use REST API for handling inferences, for more informationm, please see [Deploying to TensorFlow Serving Endpoints](https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/tensorflow/deploying_tensorflow_serving.rst#making-predictions-against-a-sagemaker-endpoint).\n",
     "\n",
     "To host our model for inferences in SageMaker, we need to first upload the SavedModel to S3. This can be done through the AWS console or AWS command line.\n",
     "\n",
@@ -85,15 +85,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Load the TensorFlow Model on Amazon SageMaker using Python SDK\n",
+    "## Load the TensorFlow Serving Model on Amazon SageMaker using Python SDK\n",
     "\n",
-    "We can use the SageMaker Python SDK to load our pre-trained TensorFlow model for hosting in SageMaker for predictions.\n",
+    "We can use the SageMaker Python SDK to load our pre-trained TensorFlow Serving model for hosting in SageMaker for predictions.\n",
     "\n",
-    "There are a few parameters that our TensorFlowModel is expecting.\n",
+    "There are a few parameters that our TensorFlow Serving Model is expecting.\n",
     "1. `model_data` - The S3 location of a model tar.gz file to load in SageMaker\n",
-    "2. `entry_point` - Path (absolute or relative) to the Python source file which should be executed as the entry point to model hosting. If a blank file is provided, default configurations will be used for model hosting.\n",
-    "3. `role` - An IAM role name or ARN for SageMaker to access AWS resources on your behalf.\n",
-    "4. `framework_version` - TensorFlow serving version you want to use for handling your inference request .\n"
+    "2. `role` - An IAM role name or ARN for SageMaker to access AWS resources on your behalf.\n",
+    "3. `framework_version` - TensorFlow Serving version you want to use for handling your inference request .\n"
    ]
   },
   {
@@ -102,12 +101,11 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from sagemaker.tensorflow import TensorFlowModel\n",
+    "from sagemaker.tensorflow.serving import Model\n",
     "\n",
-    "tensorflow_model = TensorFlowModel(model_data=saved_model,\n",
-    "                                   entry_point='entry.py',\n",
-    "                                   role=role,\n",
-    "                                   framework_version='1.12')"
+    "tensorflow_model = Model(model_data=saved_model,\n",
+    "                         role=role,\n",
+    "                         framework_version='1.12')"
    ]
   },
   {
@@ -118,9 +116,7 @@
     "\n",
     "The `deploy()` method creates an endpoint which serves prediction requests in real-time.\n",
     "\n",
-    "The only change required for utilizing EI with our SageMaker TensorFlow containers only requires providing an `accelerator_type` parameter, which determines which type of EI accelerator to attach to your endpoint. The supported types of accelerators can be found here: https://aws.amazon.com/sagemaker/pricing/instance-types/\n",
-    "\n",
-    "No code changes are necessary for your model, as our predefined TensorFlow containers utilizes TensorFlow serving, which has been modified to utilize EI for inference, as long as an EI accelerator is attached to the endpoint."
+    "The only change required for utilizing EI with our SageMaker TensorFlow Serving containers only requires providing an `accelerator_type` parameter, which determines which type of EI accelerator to attach to your endpoint. The supported types of accelerators can be found here: https://aws.amazon.com/sagemaker/pricing/instance-types/\n"
    ]
   },
   {
@@ -167,7 +163,7 @@
     "import numpy as np\n",
     "random_input = np.random.rand(1, 1, 3, 3)\n",
     "\n",
-    "prediction = predictor.predict({'input': random_input.tolist()})\n",
+    "prediction = predictor.predict({'inputs': random_input.tolist()})\n",
     "\n",
     "print(prediction)"
    ]