Skip to content

Edited the tf script mode notebook #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 27, 2018
Merged

Edited the tf script mode notebook #90

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5c19ada
edited tf script mode notebook
eslesar-aws Oct 17, 2018
c6e365f
removed checkpoint
eslesar-aws Oct 17, 2018
fbb1f61
removed error output from notebook
eslesar-aws Oct 18, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 32 additions & 30 deletions examples/script_mode_train_any_tf_script_in_sage_maker.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,21 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Using the Script Mode to train any TensorFlow script from GitHub in SageMaker\n",
"# Use Script Mode to train any TensorFlow script from GitHub in SageMaker\n",
"\n",
"In this tutorial, we show how simple it is to train a TensorFlow script in SageMaker using the new Script Mode Tensorflow Container.\n",
"In this tutorial, you train a TensorFlow script in SageMaker using the new Script Mode Tensorflow Container.\n",
"\n",
"The example we chose is [Multi-layer Recurrent Neural Networks (LSTM, RNN) for character-level language models in Python using Tensorflow](https://github.com/sherjilozair/char-rnn-tensorflow) but this same technique can be used to other scripts or repositories including [TensorFlow Model Zoo](https://github.com/tensorflow/models) and [TensorFlow benchmark scripts](https://github.com/tensorflow/benchmarks/tree/master/scripts/tf_cnn_benchmarks)."
"For this example, you use [Multi-layer Recurrent Neural Networks (LSTM, RNN) for character-level language models in Python using Tensorflow](https://github.com/sherjilozair/char-rnn-tensorflow), but you can use the same technique for other scripts or repositories. For example, [TensorFlow Model Zoo](https://github.com/tensorflow/models) and [TensorFlow benchmark scripts](https://github.com/tensorflow/benchmarks/tree/master/scripts/tf_cnn_benchmarks)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setting up the environment\n",
"Let's start by creating a SageMaker session and specifying:\n",
"- The S3 bucket and prefix that you want to use for training and model data. It should be within the same region as the Notebook Instance, training, and hosting.\n",
"- The IAM role allows SageMaker services to access your data. See the documentation [for how to create these](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html).\n"
"## Set up the environment\n",
"Let's start by creating a SageMaker session and specifying the following:\n",
"- The S3 bucket and prefix to use for training and model data. The bucket should be in the same region as the Notebook Instance, training instance(s), and hosting instance(s). This example uses the default bucket that a SageMaker `Session` creates.\n",
"- The IAM role that allows SageMaker services to access your data. For more information about using IAM roles in SageMaker, see [Amazon SageMaker Roles](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html).\n"
]
},
{
Expand All @@ -40,7 +40,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Clone the repository"
"### Clone the repository\n",
"Run the following command to clone the repository that contains the example:"
]
},
{
Expand Down Expand Up @@ -73,7 +74,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Getting the data"
"### Get the data\n",
"For training data, use plain text versions of Sherlock Holmes stories."
]
},
{
Expand All @@ -90,7 +92,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Testing locally"
"## Test locally"
]
},
{
Expand Down Expand Up @@ -130,7 +132,7 @@
"metadata": {},
"source": [
"\n",
"We can use [Local Mode](https://github.com/aws/sagemaker-python-sdk#local-mode) to simulate SageMaker locally before submit training:"
"Use [Local Mode](https://github.com/aws/sagemaker-python-sdk#local-mode) to run the script locally in the notebook instance before you run a SageMaker training job:"
]
},
{
Expand All @@ -147,7 +149,7 @@
"\n",
"estimator = ScriptModeTensorFlow(entry_point='train.py',\n",
" source_dir='char-rnn-tensorflow',\n",
" train_instance_type='local', \n",
" train_instance_type='local', # Run in local mode\n",
" train_instance_count=1,\n",
" hyperparameters=hyperparameters,\n",
" role=role)\n",
Expand All @@ -161,14 +163,14 @@
"source": [
"## How Script Mode executes the script in the container\n",
"\n",
"The cell above downloads a Python 3 CPU container locally and simulates SageMaker training. When training starts, script mode installs the user script as a Python module. The module name matches the script name. In the case above, **train.py** is transformed into a Python module named **train**.\n",
"The above cell downloads a Python 3 CPU container locally and simulates a SageMaker training job. When training starts, script mode installs the user script as a Python module. The module name matches the script name. In this case, **train.py** is transformed into a Python module named **train**.\n",
"\n",
"After that, the Python interpreter executes the user module, passing **hyperparameters** as script arguments. The example above will be executed as follow:\n",
"After that, the Python interpreter executes the user module, passing **hyperparameters** as script arguments. The example above is executed as follows:\n",
"```bash\n",
"python -m train --num-epochs 1 --data-dir /opt/ml/input/data/training --save-dir /opt/ml/model\n",
"```\n",
"\n",
"A user provide script consumes the hyperparameters using any argument parsing library, [in the example above](https://github.com/sherjilozair/char-rnn-tensorflow/blob/master/train.py#L11):\n",
"The **train** module consumes the hyperparameters using any argument parsing library. [The example we're using](https://github.com/sherjilozair/char-rnn-tensorflow/blob/master/train.py#L11) uses the Python [argparse](https://docs.python.org/3/library/argparse.html) library:\n",
"\n",
"```python\n",
"parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)\n",
Expand All @@ -183,16 +185,16 @@
"\n",
"Let's explain the values of **--data_dir** and **--save-dir**:\n",
"\n",
"- **/opt/ml/input/data/training** is the directory inside the container where the training data is downloaded. The data was downloaded in this folder because **training** is the channel name defined in ```estimator.fit({'training': inputs})```. See [training data](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo.html#your-algorithms-training-algo-running-container-trainingdata) for more information. \n",
"- **/opt/ml/input/data/training** is the directory inside the container where the training data is downloaded. The data is downloaded to this folder because **training** is the channel name defined in ```estimator.fit({'training': inputs})```. See [training data](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo.html#your-algorithms-training-algo-running-container-trainingdata) for more information. \n",
"\n",
"- **/opt/ml/model** use this directory to save models, checkpoints or any other data. Any data saved in this folder is saved in the S3 bucket defined for training. See [model data](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo.html#your-algorithms-training-algo-envvariables) for more information.\n",
"- **/opt/ml/model** use this directory to save models, checkpoints, or any other data. Any data saved in this folder is saved in the S3 bucket defined for training. See [model data](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo.html#your-algorithms-training-algo-envvariables) for more information.\n",
"\n",
"### Reading additional information from the container\n",
"\n",
"Very often, a user script needs additional information from the container that is not available in ```hyperparameters```.\n",
"SageMaker Containers writes this information as **environment variables** that are available inside the script.\n",
"Often, a user script needs additional information from the container that is not available in ```hyperparameters```.\n",
"SageMaker containers write this information as **environment variables** that are available inside the script.\n",
"\n",
"For example, the example above can read information about the **training** channel provided in the training job request:\n",
"For example, the example above can read information about the **training** channel provided in the training job request by adding the environment variable `SM_CHANNEL_TRAINING` as the default value for the `--data_dir` argument:\n",
"\n",
"```python\n",
"if __name__ == '__main__':\n",
Expand All @@ -201,7 +203,7 @@
" parser.add_argument('--data_dir', type=str, default=os.environ['SM_CHANNEL_TRAINING'])\n",
"```\n",
"\n",
"Script Mode displays the list of the environment variables available in the training logs. You can find the [entire list here](https://github.com/aws/sagemaker-containers/blob/master/README.md#environment-variables-full-specification)."
"Script mode displays the list of available environment variables in the training logs. You can find the [entire list here](https://github.com/aws/sagemaker-containers/blob/master/README.md#environment-variables-full-specification)."
]
},
{
Expand All @@ -215,7 +217,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"We need to upload the dataset to an S3 bucket so SageMaker can access the data during training.\n"
"After you test the training job locally, upload the dataset to an S3 bucket so SageMaker can access the data during training.\n"
]
},
{
Expand All @@ -231,7 +233,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"You can change the estimator argument **train_instance_type** to any SageMaker ml instance available for training. For example:"
"To train in SageMaker, change the estimator argument **train_instance_type** to any SageMaker ml instance available for training. For example:"
]
},
{
Expand Down Expand Up @@ -268,7 +270,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Script Mode will install your source_dir in the container as a [Python package](https://github.com/aws/sagemaker-containers/blob/master/src/sagemaker_containers/_modules.py#L100). You can include a [requirements.txt file in the root folder of your source_dir to install any pip dependencies](https://github.com/aws/sagemaker-containers/blob/master/src/sagemaker_containers/_modules.py#L111). You can, for example, install the lastest version of tensorflow in the container:\n",
"Script Mode installs the contents of your `source_dir` folder in the container as a [Python package](https://github.com/aws/sagemaker-containers/blob/master/src/sagemaker_containers/_modules.py#L100). You can include a [requirements.txt file in the root folder of your source_dir to install any pip dependencies](https://github.com/aws/sagemaker-containers/blob/master/src/sagemaker_containers/_modules.py#L111). You can, for example, install the lastest version of TensorFlow in the container:\n",
"\n",
"content of requirements.txt\n",
"```\n",
Expand All @@ -281,7 +283,7 @@
"metadata": {},
"source": [
"# Installing apt-get packages and other dependencies\n",
"You can define a setup.py file in your source_dir to install other dependencies. The example below will install [TensorFlow for C](https://www.tensorflow.org/install/lang_c) in the container."
"You can define a `setup.py` file in your `source_dir` folder to install other dependencies. The example below installs [TensorFlow for C](https://www.tensorflow.org/install/lang_c) in the container."
]
},
{
Expand Down Expand Up @@ -385,21 +387,21 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"display_name": "Python 3",
"language": "python",
"name": "python2"
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.15"
"pygments_lexer": "ipython3",
"version": "3.6.5"
}
},
"nbformat": 4,
Expand Down