Skip to content

Commit 4bd2f3f

Browse files
Danlaurenyu
Dan
authored and
laurenyu
committed
add readme (#13)
1 parent 94700f5 commit 4bd2f3f

File tree

2 files changed

+198
-7
lines changed

2 files changed

+198
-7
lines changed

‎README.md

Lines changed: 0 additions & 7 deletions
This file was deleted.

‎README.rst

Lines changed: 198 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,198 @@
1+
=================================
2+
SageMaker MXNet Serving Container
3+
=================================
4+
5+
SageMaker MXNet Serving Container is an open-source library for making Docker images for serving MXNet on Amazon SageMaker.
6+
7+
This library provides default pre-processing, predict and postprocessing for certain MXNet model types.
8+
9+
This library utilizes the `SageMaker Inference Toolkit <https://github.com/aws/sagemaker-inference-toolkit>`__ for starting up the model server, which is responsible for handling inference requests.
10+
11+
Only MXNet version 1.4 and higher are supported. For previous versions, see `SageMaker MXNet container <https://github.com/aws/sagemaker-mxnet-container>`__.
12+
13+
-----------------
14+
Table of Contents
15+
-----------------
16+
.. contents::
17+
:local:
18+
19+
Getting Started
20+
---------------
21+
22+
Prerequisites
23+
~~~~~~~~~~~~~
24+
25+
Make sure you have installed all of the following prerequisites on your development machine:
26+
27+
- `Docker <https://www.docker.com/>`__
28+
- For GPU testing: `nvidia-docker2 <https://github.com/NVIDIA/nvidia-docker>`__
29+
30+
Recommended
31+
^^^^^^^^^^^
32+
33+
- A Python environment management tool (e.g. `PyEnv <https://github.com/pyenv/pyenv>`__,
34+
`VirtualEnv <https://virtualenv.pypa.io/en/stable/>`__)
35+
36+
Building Images
37+
---------------
38+
39+
The Dockerfiles in this repository are intended to be used for building Docker images to run inference endpoints on `Amazon SageMaker <https://aws.amazon.com/documentation/sagemaker/>`__.
40+
41+
The current master branch of this repository contains Dockerfiles and support code for MXNet versions 1.4.0 and higher. For previous versions, see `SageMaker MXNet container <https://github.com/aws/sagemaker-mxnet-container>`__.
42+
43+
Before building these images, you need to have the pip-installable binary of this repository.
44+
45+
To create the SageMaker MXNet Container Python package:
46+
47+
::
48+
49+
# Create the binary
50+
git clone https://github.com/aws/sagemaker-mxnet-serving-container.git
51+
cd sagemaker-mxnet-serving-container
52+
python setup.py sdist
53+
54+
Once you have created this binary, you can then build the image.
55+
The Dockerfiles expect one build arguments:
56+
57+
- ``py_version``: the Python version (either 2.7 or 3.6).
58+
59+
To build an image:
60+
61+
::
62+
63+
# All build instructions assume you're building from the repository root directory.
64+
65+
# CPU
66+
docker build -t preprod-mxnet-serving:<tag> \
67+
--build-arg py_version=<python_version> \
68+
-f docker/<mxnet_version>/final/Dockerfile.cpu .
69+
70+
# GPU
71+
docker build -t preprod-mxnet-serving:<tag> \
72+
--build-arg py_version=<python_version> \
73+
-f docker/<mxnet_version>/final/Dockerfile.gpu .
74+
75+
Don't forget the period at the end of the command!
76+
77+
::
78+
79+
# Example
80+
81+
# CPU
82+
docker build -t preprod-mxnet-serving:1.4.0-cpu-py3 --build-arg py_version=3.6 -f docker/1.4.0/final/Dockerfile.cpu .
83+
84+
# GPU
85+
docker build -t preprod-mxnet-serving:1.4.0-gpu-py3 --build-arg py_version=3.6 -f docker/1.4.0/final/Dockerfile.gpu .
86+
87+
88+
Amazon Elastic Inference with MXNet in SageMaker
89+
------------------------------------------------
90+
`Amazon Elastic Inference <https://aws.amazon.com/machine-learning/elastic-inference/>`__ allows you to to attach
91+
low-cost GPU-powered acceleration to Amazon EC2 and Amazon SageMaker instances to reduce the cost running deep
92+
learning inference by up to 75%. Currently, Amazon Elastic Inference supports TensorFlow, Apache MXNet, and ONNX
93+
models, with more frameworks coming soon.
94+
95+
Support for using MXNet with Amazon Elastic Inference in SageMaker is supported in the public SageMaker MXNet containers.
96+
97+
* For information on how to use the Python SDK to create an endpoint with Amazon Elastic Inference and MXNet in SageMaker, see `Deploying MXNet Models <https://sagemaker.readthedocs.io/en/stable/using_mxnet.html#deploying-mxnet-models>`__.
98+
* For information on how Amazon Elastic Inference works, see `How EI Works <https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html#ei-how-it-works>`__.
99+
* For more information in regards to using Amazon Elastic Inference in SageMaker, see `Amazon SageMaker Elastic Inference <https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html>`__.
100+
* For notebook examples on how to use Amazon Elastic Inference with MXNet through the Python SDK in SageMaker, see `EI Sample Notebooks <https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html#ei-intro-sample-nb>`__.
101+
102+
Building the SageMaker Elastic Inference MXNet Serving container
103+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
104+
Amazon Elastic Inference is designed to be used with AWS enhanced versions of TensorFlow serving or Apache MXNet. These enhanced
105+
versions of the frameworks are automatically built into containers when you use the Amazon SageMaker Python SDK, or you can
106+
download them as binary files and import them into your own Docker containers. The enhanced MXNet binaries are available on Amazon S3 at https://s3.console.aws.amazon.com/s3/buckets/amazonei-apachemxnet.
107+
108+
The SageMaker MXNet containers with Amazon Elastic Inference support were built utilizing the
109+
same instructions listed `above <https://github.com/aws/sagemaker-mxnet-serving-container#building-images>`__ with the
110+
`EIA Dockerfile <https://github.com/aws/sagemaker-mxnet-serving-container/blob/master/docker/1.4.0/final/Dockerfile.eia>`__.
111+
112+
The only difference is the specified Dockerfile.
113+
114+
::
115+
116+
# Example
117+
118+
# EI
119+
docker build -t preprod-mxnet-serving-eia:1.4.0-cpu-py3 --build-arg py_version=3.6 -f docker/1.4.0/final/Dockerfile.eia .
120+
121+
122+
* For information about downloading and installing the enhanced binary for Apache MXNet, see `Install Amazon EI Enabled Apache MXNet <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ei-mxnet.html#ei-apache>`__.
123+
* For information on which versions of MXNet is supported for Elastic Inference within SageMaker, see `MXNet SageMaker Estimators <https://github.com/aws/sagemaker-python-sdk#mxnet-sagemaker-estimators>`__.
124+
125+
Running the tests
126+
-----------------
127+
128+
Running the tests requires tox.
129+
130+
::
131+
132+
git clone https://github.com/aws/sagemaker-mxnet-serving-container.git
133+
cd sagemaker-mxnet-serving-container
134+
tox
135+
136+
Tests are defined in `test/ <https://github.com/aws/sagemaker-mxnet-serving-container/tree/master/test>`__ and include unit and integration tests.
137+
The integration tests include both running the Docker containers locally and running them on SageMaker.
138+
The tests are compatible with only the Docker images built by Dockerfiles in the current branch.
139+
140+
All test instructions should be run from the top level directory
141+
142+
Unit Tests
143+
~~~~~~~~~~
144+
145+
To run unit tests:
146+
147+
::
148+
149+
tox test/unit
150+
151+
Local Integration Tests
152+
~~~~~~~~~~~~~~~~~~~~~~~
153+
154+
Running local integration tests require `Docker <https://www.docker.com/>`__ and `AWS credentials <https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html>`__,
155+
as the integration tests make calls to a couple AWS services.
156+
Local integration tests on GPU require `nvidia-docker2 <https://github.com/NVIDIA/nvidia-docker>`__.
157+
You Docker image must also be built in order to run the tests against it.
158+
159+
Local integration tests use the following pytest arguments:
160+
161+
- ``docker-base-name``: the Docker image's repository. Defaults to 'preprod-mxnet'.
162+
- ``framework-version``: the MXNet version. Defaults to the latest supported version.
163+
- ``py-version``: the Python version. Defaults to '3'.
164+
- ``processor``: CPU or GPU. Defaults to 'cpu'.
165+
- ``tag``: the Docker image's tag. Defaults to <mxnet_version>-<processor>-py<py-version>
166+
167+
To run local integration tests:
168+
169+
::
170+
171+
tox test/integration/local -- --docker-base-name <your_docker_image> \
172+
--tag <your_docker_image_tag> \
173+
--py-version <2_or_3> \
174+
--framework-version <mxnet_version> \
175+
--processor <cpu_or_gpu>
176+
177+
::
178+
179+
# Example
180+
tox test/integration/local -- --docker-base-name preprod-mxnet-serving \
181+
--tag 1.4.0-cpu-py3 \
182+
--py-version 3 \
183+
--framework-version 1.4.0 \
184+
--processor cpu
185+
186+
187+
Contributing
188+
------------
189+
190+
Please read `CONTRIBUTING.md <https://github.com/aws/sagemaker-mxnet-serving-container/blob/master/CONTRIBUTING.md>`__
191+
for details on our code of conduct, and the process for submitting pull requests to us.
192+
193+
License
194+
-------
195+
196+
SageMaker MXNet Containers is licensed under the Apache 2.0 License.
197+
It is copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
198+
The license is available at: http://aws.amazon.com/apache2.0/

0 commit comments

Comments
 (0)