add tensorflow serving docs #468
Conversation
Codecov Report
@@ Coverage Diff @@
## master #468 +/- ##
==========================================
- Coverage 93.98% 93.94% -0.05%
==========================================
Files 57 57
Lines 4259 4261 +2
==========================================
Hits 4003 4003
- Misses 256 258 +2
Continue to review full report at Codecov.
|
CHANGELOG.rst
Outdated
| @@ -5,6 +5,8 @@ CHANGELOG | |||
| 1.14.2-dev | |||
| ========== | |||
|
|
|||
| * enhancement: add content_type parameter to sagemaker.tensorflow.serving.Predictor | |||
There was a problem hiding this comment.
nitpick: I'd make content_type and sagemaker.tensorflow.serving.Predictor monospace
src/sagemaker/tensorflow/README.rst
Outdated
| @@ -229,7 +228,7 @@ More details on how to create input functions can be find in `Building Input Fun | |||
| Creating a ``serving_input_fn`` | |||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |||
|
|
|||
| ``serving_input_fn`` is used to define the shapes and types of the inputs the model accepts when the model is exported for Tensorflow Serving. It is optional, but required for deploying the trained model to a SageMaker endpoint. | |||
| ``serving_input_fn`` is used to define the shapes and types of the inputs the model accepts when the model is exported for Tensorflow Serving. It is optional, but is required to create the SavedModel bundle needed to deploying the trained model to a SageMaker endpoint. | |||
There was a problem hiding this comment.
I think it sounds a little weird to have "it is optional, but is required". maybe something like "it is required only for x; otherwise it is optional" instead?
src/sagemaker/tensorflow/README.rst
Outdated
| estimator. | ||
| will generate a `TensorFlow SavedModel <https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/saved_model/README.md>`_ | ||
| bundle ready for deployment. Your model will be available in S3 at the ``output_path`` location | ||
| that you specified when you created your `sagemaker.tensorflow.TensorFlow` estimator. |
There was a problem hiding this comment.
add another set of backticks around sagemaker.tensorflow.TensorFlow
src/sagemaker/tensorflow/README.rst
Outdated
| @@ -614,188 +612,25 @@ Note that TensorBoard is not supported when passing wait=False to ``fit``. | |||
| Deploying TensorFlow Serving models | |||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |||
|
|
|||
| After a ``TensorFlow`` Estimator has been fit, it saves a ``TensorFlow Serving`` model in | |||
| the S3 location defined by ``output_path``. You can call ``deploy`` on a ``TensorFlow`` | |||
| After a TensorFlow Estimator has been fit, it saves a TensorFlow ``SavedModel`` in | |||
There was a problem hiding this comment.
I don't think "Estimator" needs to be capitalized here
src/sagemaker/tensorflow/README.rst
Outdated
| Deploying directly from model artifacts | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| - The first option uses a Python-based server that allows you to specify your own custom | ||
| input and output handling functions in a python script. This is the default option. |
| The common functionality can be extended by the addiction of the following two functions to your training script: | ||
|
|
||
| Overriding input preprocessing with an ``input_fn`` | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
There was a problem hiding this comment.
extend the header lines on ll. 153 and 172
| Deploying from an Estimator | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| After a TensorFlow Estimator has been fit, it saves a TensorFlow ``SavedModel`` in |
There was a problem hiding this comment.
same capitalization comment about "Estimator"
| 'predictions': [3.5, 4.0, 5.5] | ||
| } | ||
|
|
||
| The format of the input and the output data correspond directly to the request and response formats |
There was a problem hiding this comment.
either s/format/formats or s/correspond/corresponds (I think it depends on if you count "data" as plural there)
| The format of the input and the output data correspond directly to the request and response formats | ||
| of the ``Predict`` method in the `TensorFlow Serving REST API <https://www.tensorflow.org/serving/api_rest>`_. | ||
|
|
||
| If your SavedModel includes the right ``signature_def``, you can also make Classify or Regress requests: |
There was a problem hiding this comment.
should "SavedModel" be monospace here?
There was a problem hiding this comment.
I think not. I Corrected the one or two places where it was to match.
| you are likely to have `./multi/model1/export/Servo/...` and `./multi/model2/export/Servo/...`. In both cases, | ||
| "Servo" is the base name for the SaveModel files. When serving multiple models, each model needs a unique | ||
| basename, so one or both of these will need to be changed. The `/export/` part of the path isn't needed | ||
| either, so you can simplify the layout at the same time: |
| - Starts ``initial_instance_count`` EC2 instances of the type ``instance_type``. | ||
| - On each instance, it will do the following steps: | ||
|
|
||
| - start a Docker container optimized for TensorFlow Serving, see `SageMaker TensorFlow Serving containers <https://github.com/aws/sagemaker-tensorflow-serving-container>`_. |
| To use this feature, you will need to: | ||
|
|
||
| #. create a multi-model archive file | ||
| #. create a SageMaker and deploy it to an Endpoint |
There was a problem hiding this comment.
create a SageMaker model**
| # result is prediction from 'model2' | ||
| result = model2_predictor.predict(...) | ||
|
|
||
| Making predictions with the AWS CLI |
There was a problem hiding this comment.
If I am not mistaken this doesn't work with localmode. I think we should make that clear here.
There was a problem hiding this comment.
good catch. there is a way to make it work with local mode, but it requires a change to local mode. I will add that.
| The ``predictor.predict`` method call takes one parameter, the input ``data`` for which you want the SageMaker Endpoint | ||
| to provide inference. ``predict`` will serialize the input data, and send it in as request to the SageMaker Endpoint by | ||
| an ``InvokeEndpoint`` SageMaker operation. ``InvokeEndpoint`` operation requests can be made by ``predictor.predict``, by | ||
| boto3 ``sageMaker.runtime`` client or by AWS CLI. |
There was a problem hiding this comment.
this should probably be "SageMaker Runtime client". if you want, could also add a link to https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker-runtime.html
src/sagemaker/tensorflow/README.rst
Outdated
| @@ -614,188 +617,25 @@ Note that TensorBoard is not supported when passing wait=False to ``fit``. | |||
| Deploying TensorFlow Serving models | |||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |||
|
|
|||
| After a ``TensorFlow`` Estimator has been fit, it saves a ``TensorFlow Serving`` model in | |||
| the S3 location defined by ``output_path``. You can call ``deploy`` on a ``TensorFlow`` | |||
| After a TensorFlow estimator has been fit, it saves a TensorFlow ``SavedModel`` in | |||
There was a problem hiding this comment.
another SavedModel in backticks
| Deploying from an Estimator | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| After a TensorFlow estimator has been fit, it saves a TensorFlow ``SavedModel`` in |
There was a problem hiding this comment.
and another monospaced SavedModel
| @@ -164,14 +164,19 @@ def __init__(self, config=None): | |||
| self.config = config | |||
| self.serving_port = get_config_value('local.serving_port', config) or 8080 | |||
|
|
|||
| def invoke_endpoint(self, Body, EndpointName, ContentType, Accept): # pylint: disable=unused-argument | |||
| def invoke_endpoint(self, Body, EndpointName, # pylint: disable=unused-argument | |||
There was a problem hiding this comment.
We have agreed on adding some unit tests in a follow up pr for this later.
There was a problem hiding this comment.
lgtm. not sure if @icywang86rui still has comments
edit: see she beat me to it
jesterhazy
dismissed stale reviews from laurenyu and icywang86rui
via
0ca9c32
Description of changes:
Merge Checklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.