|
2 | 2 | "cells": [
|
3 | 3 | {
|
4 | 4 | "cell_type": "markdown",
|
5 |
| - "id": "ee5abd3d", |
| 5 | + "id": "e950fa8e", |
6 | 6 | "metadata": {},
|
7 | 7 | "source": [
|
8 |
| - "# SKLearn Script Mode + Bring Your Own Model\n", |
| 8 | + "# Train a SKLearn Model using Script Mode\n", |
9 | 9 | "\n",
|
10 |
| - "- [Documentation](https://sagemaker.readthedocs.io/en/stable/frameworks/sklearn/using_sklearn.html)\n", |
11 |
| - "- Dataset: [Iris](https://archive.ics.uci.edu/ml/datasets/iris)" |
| 10 | + "The aim of this notebook is to demonstrate how to train and deploy a scikit-learn model in Amazon SageMaker. The method used is called Script Mode, in which we write a script to train our model and submit it to the SageMaker Python SDK. For more information, feel free to read [Using Scikit-learn with the SageMaker Python SDK](https://sagemaker.readthedocs.io/en/stable/frameworks/sklearn/using_sklearn.html).\n", |
| 11 | + "\n", |
| 12 | + "## Runtime\n", |
| 13 | + "This notebook takes approximately 15 minutes to run.\n", |
| 14 | + "\n", |
| 15 | + "## Contents\n", |
| 16 | + "1. [Download data](#Download-data)\n", |
| 17 | + "1. [Prepare data](#Prepare-data)\n", |
| 18 | + "1. [Train model](#Train-model)\n", |
| 19 | + "1. [Deploy and test endpoint](#Deploy-and-test-endpoint)\n", |
| 20 | + "1. [Cleanup](#Cleanup)" |
12 | 21 | ]
|
13 | 22 | },
|
14 | 23 | {
|
15 | 24 | "cell_type": "markdown",
|
16 |
| - "id": "04fb9160", |
| 25 | + "id": "a16db1a6", |
17 | 26 | "metadata": {},
|
18 | 27 | "source": [
|
19 |
| - "# Read Data" |
| 28 | + "## Download data \n", |
| 29 | + "Download the [Iris Data Set](https://archive.ics.uci.edu/ml/datasets/iris), which is the data used to trained the model in this demo." |
20 | 30 | ]
|
21 | 31 | },
|
22 | 32 | {
|
23 | 33 | "cell_type": "code",
|
24 | 34 | "execution_count": null,
|
25 |
| - "id": "1de023a4", |
| 35 | + "id": "a670c242", |
26 | 36 | "metadata": {},
|
27 | 37 | "outputs": [],
|
28 | 38 | "source": [
|
|
39 | 49 | "df.head()"
|
40 | 50 | ]
|
41 | 51 | },
|
| 52 | + { |
| 53 | + "cell_type": "markdown", |
| 54 | + "id": "7c03b3d2", |
| 55 | + "metadata": {}, |
| 56 | + "source": [ |
| 57 | + "## Prepare data\n", |
| 58 | + "Next, we prepare the data for training by first converting the labels from string to integers. Then we split the data into a train dataset (80% of the data) and test dataset (the remaining 20% of the data) before saving them into CSV files. Then, these files are uploaded to S3 where the SageMaker SDK can access and use them to train the model." |
| 59 | + ] |
| 60 | + }, |
42 | 61 | {
|
43 | 62 | "cell_type": "code",
|
44 | 63 | "execution_count": null,
|
45 |
| - "id": "3eb3ab67", |
| 64 | + "id": "72748b04", |
46 | 65 | "metadata": {},
|
47 | 66 | "outputs": [],
|
48 | 67 | "source": [
|
|
56 | 75 | {
|
57 | 76 | "cell_type": "code",
|
58 | 77 | "execution_count": null,
|
59 |
| - "id": "e75f4bc4", |
| 78 | + "id": "fb5ea6cf", |
60 | 79 | "metadata": {},
|
61 | 80 | "outputs": [],
|
62 | 81 | "source": [
|
|
71 | 90 | {
|
72 | 91 | "cell_type": "code",
|
73 | 92 | "execution_count": null,
|
74 |
| - "id": "61c07854", |
| 93 | + "id": "48770a6b", |
75 | 94 | "metadata": {},
|
76 | 95 | "outputs": [],
|
77 | 96 | "source": [
|
|
80 | 99 | "test.to_csv(\"test.csv\", index=False)"
|
81 | 100 | ]
|
82 | 101 | },
|
83 |
| - { |
84 |
| - "cell_type": "markdown", |
85 |
| - "id": "90842a0d", |
86 |
| - "metadata": {}, |
87 |
| - "source": [ |
88 |
| - "# Upload Data to S3" |
89 |
| - ] |
90 |
| - }, |
91 | 102 | {
|
92 | 103 | "cell_type": "code",
|
93 | 104 | "execution_count": null,
|
94 |
| - "id": "f79c8e43", |
| 105 | + "id": "ba40dab3", |
95 | 106 | "metadata": {},
|
96 | 107 | "outputs": [],
|
97 | 108 | "source": [
|
|
107 | 118 | },
|
108 | 119 | {
|
109 | 120 | "cell_type": "markdown",
|
110 |
| - "id": "40d87112", |
| 121 | + "id": "9d52c534", |
111 | 122 | "metadata": {},
|
112 | 123 | "source": [
|
113 |
| - "# Train Estimator" |
| 124 | + "## Train model\n", |
| 125 | + "The model is trained using the SageMaker SDK's Estimator class. Firstly, get the execution role for training. This role allows us to access the S3 bucket in the last step, where the train and test data set is located." |
114 | 126 | ]
|
115 | 127 | },
|
116 | 128 | {
|
117 | 129 | "cell_type": "code",
|
118 | 130 | "execution_count": null,
|
119 |
| - "id": "98d6eb28", |
| 131 | + "id": "f7cbdad2", |
120 | 132 | "metadata": {},
|
121 | 133 | "outputs": [],
|
122 | 134 | "source": [
|
|
125 | 137 | "print(role)"
|
126 | 138 | ]
|
127 | 139 | },
|
| 140 | + { |
| 141 | + "cell_type": "markdown", |
| 142 | + "id": "10cdcfb6", |
| 143 | + "metadata": {}, |
| 144 | + "source": [ |
| 145 | + "Then, it is time to define the SageMaker SDK Estimator class. We use an Estimator class specifically desgined to train scikit-learn models called `SKLearn`. In this estimator, we define the following parameters:\n", |
| 146 | + "1. The script that we want to use to train the model (i.e. `entry_point`). This is the heart of the Script Mode method. Additionally, set the `script_mode` parameter to `True`.\n", |
| 147 | + "1. The role which allows us access to the S3 bucket containing the train and test data set (i.e. `role`)\n", |
| 148 | + "1. How many instances we want to use in training (i.e. `instance_count`) and what type of instance we want to use in training (i.e. `instance_type`)\n", |
| 149 | + "1. Which version of scikit-learn to use (i.e. `framework_version`)\n", |
| 150 | + "1. Training hyperparameters (i.e. `hyperparameters`)\n", |
| 151 | + "\n", |
| 152 | + "After setting these parameters, the `fit` function is invoked to train the model." |
| 153 | + ] |
| 154 | + }, |
128 | 155 | {
|
129 | 156 | "cell_type": "code",
|
130 | 157 | "execution_count": null,
|
131 |
| - "id": "df113b51", |
| 158 | + "id": "ac14dcb7", |
132 | 159 | "metadata": {},
|
133 | 160 | "outputs": [],
|
134 | 161 | "source": [
|
|
153 | 180 | },
|
154 | 181 | {
|
155 | 182 | "cell_type": "markdown",
|
156 |
| - "id": "5a8762c7", |
| 183 | + "id": "3813b62c", |
157 | 184 | "metadata": {},
|
158 | 185 | "source": [
|
159 |
| - "# Deploy Endpoint" |
| 186 | + "## Deploy and test endpoint\n", |
| 187 | + "After training the model, it is time to deploy it as an endpoint. To do so, we invoke the `deploy` function within the scikit-learn estimator. As shown in the code below, one can define the number of instances (i.e. `initial_instance_count`) and instance type (i.e. `instance_type`) used to deploy the model." |
160 | 188 | ]
|
161 | 189 | },
|
162 | 190 | {
|
163 | 191 | "cell_type": "code",
|
164 | 192 | "execution_count": null,
|
165 |
| - "id": "9d39a7af", |
| 193 | + "id": "06aace5c", |
166 | 194 | "metadata": {},
|
167 | 195 | "outputs": [],
|
168 | 196 | "source": [
|
|
176 | 204 | },
|
177 | 205 | {
|
178 | 206 | "cell_type": "markdown",
|
179 |
| - "id": "6e1c4ac0", |
| 207 | + "id": "bbc747e1", |
180 | 208 | "metadata": {},
|
181 | 209 | "source": [
|
182 |
| - "# Test Endpoint\n", |
183 |
| - "- Can use [invoke endpoint](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker-runtime.html) or [predictor](https://sagemaker.readthedocs.io/en/stable/frameworks/sklearn/sagemaker.sklearn.html#scikit-learn-predictor), using invoke endpoint for this example. \n", |
184 |
| - "- For predictor make sure to [serialize](https://sagemaker.readthedocs.io/en/stable/api/inference/serializers.html) properly." |
| 210 | + "After the endpoint has been completely deployed, it can be invoked using the [SageMaker Runtime Client](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker-runtime.html) (which is the method used in the code cell below) or [Scikit Learn Predictor](https://sagemaker.readthedocs.io/en/stable/frameworks/sklearn/sagemaker.sklearn.html#scikit-learn-predictor). If you plan to use the latter method, make sure to use a [Serializer](https://sagemaker.readthedocs.io/en/stable/api/inference/serializers.html) to serialize your data properly." |
185 | 211 | ]
|
186 | 212 | },
|
187 | 213 | {
|
188 | 214 | "cell_type": "code",
|
189 | 215 | "execution_count": null,
|
190 |
| - "id": "02bb9960", |
| 216 | + "id": "85491166", |
191 | 217 | "metadata": {},
|
192 | 218 | "outputs": [],
|
193 | 219 | "source": [
|
|
209 | 235 | },
|
210 | 236 | {
|
211 | 237 | "cell_type": "markdown",
|
212 |
| - "id": "8bae026f", |
| 238 | + "id": "90f26921", |
213 | 239 | "metadata": {},
|
214 | 240 | "source": [
|
215 |
| - "# Cleanup" |
| 241 | + "## Cleanup\n", |
| 242 | + "If the model and endpoint are no longer in use, they should be deleted to save costs and free up resources." |
216 | 243 | ]
|
217 | 244 | },
|
218 | 245 | {
|
219 | 246 | "cell_type": "code",
|
220 | 247 | "execution_count": null,
|
221 |
| - "id": "15f90752", |
| 248 | + "id": "ec5a3a83", |
222 | 249 | "metadata": {},
|
223 | 250 | "outputs": [],
|
224 | 251 | "source": [
|
|
0 commit comments