Merge pull request aws#311 from awslabs/bt-enhancements

Add new BlazingText notebooks

Author: djarpin

introduction_to_amazon_algorithms/blazingtext_hosting_pretrained_fasttext/blazingtext_hosting_pretrained_fasttext.ipynb

@@ -0,0 +1,231 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Introduction\n",
+    "\n",
+    "In this notebook, we demonstrate how BlazingText supports hosting of pre-trained Text Classification and Word2Vec models [FastText models](https://fasttext.cc/docs/en/english-vectors.html). BlazingText is a GPU accelerated version of FastText. FastText is a shallow Neural Network model used to perform both word embedding generation (unsupervised) and text classification (supervised). BlazingText uses custom CUDA kernels to accelerate the training process of FastText but the underlying algorithm is same for both the algorithms. Therefore, if you have a model trained with FastText or if one of the pre-trained models made available by FastText team is sufficient for your use case, then you can take advantage of Hosting support for BlazingText to setup SageMaker endpoints for realtime predictions using FastText models. It can help you avoid to train with BlazingText algorithm if your use-case is covered by the pre-trained models available from FastText."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To start the proceedings, we will specify few of the important parameter like IAM Role and S3 bucket location which is required for SageMaker to facilitate model hosting. SageMaker Python SDK helps us to retrieve the IAM role and also helps you to operate easily with S3 resources. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sagemaker\n",
+    "from sagemaker import get_execution_role\n",
+    "import boto3\n",
+    "import json\n",
+    "\n",
+    "sess = sagemaker.Session()\n",
+    "\n",
+    "role = get_execution_role()\n",
+    "print(role) # This is the role that SageMaker would use to leverage AWS resources (S3, CloudWatch) on your behalf\n",
+    "\n",
+    "bucket = sess.default_bucket() # Replace with your own bucket name if needed\n",
+    "print(bucket)\n",
+    "prefix = 'fasttext/pretrained' #Replace with the prefix under which you want to store the data if needed"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "region_name = boto3.Session().region_name"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "container = sagemaker.amazon.amazon_estimator.get_image_uri(region_name, \"blazingtext\", \"latest\")\n",
+    "print('Using SageMaker BlazingText container: {} ({})'.format(container, region_name))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Hosting the [Language Idenfication model](https://fasttext.cc/docs/en/language-identification.html) by FastText\n",
+    "\n",
+    "For the example, we will leverage the pre-trained model available by FastText for Language Identification. Language Identification is the first step of many NLP applications where after the language of the input text is identified, specific models for that language needs to be applied for various other downstream tasks. Language Identification underneath is a Text Classification model which uses the language IDs as the class labels and hence FastText can be directly used for the training. FastText pretrained language model supports identification of 176 different languages. "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Here we will download the Language Identification (Text Classification) model [1] from [FastText website](https://fasttext.cc/docs/en/language-identification.html).  \n",
+    "\n",
+    "[1] A. Joulin, E. Grave, P. Bojanowski, T. Mikolov, Bag of Tricks for Efficient Text Classification"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!wget -O model.bin https://s3-us-west-1.amazonaws.com/fasttext-vectors/supervised_models/lid.176.bin"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next we will `tar` the model and upload it to S3 with the help of utilities available from Python SDK. We'll delete the local copies of the data as it's not required anymore."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!tar -czvf langid.tar.gz model.bin\n",
+    "model_location = sess.upload_data(\"langid.tar.gz\", bucket=bucket, key_prefix=prefix)\n",
+    "!rm langid.tar.gz model.bin"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Creating SageMaker Inference Endpoint\n",
+    "\n",
+    "Next we'll create a SageMaker inference endpoint with the BlazingText container. This endpoint will be compatible with the pre-trained models available from FastText and can be used for inference directly without any modification. The inference endpoint works with content-type of `application/json`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "lang_id = sagemaker.Model(model_data=model_location, image=container, role=role, sagemaker_session=sess)\n",
+    "lang_id.deploy(initial_instance_count = 1,instance_type = 'ml.m4.xlarge')\n",
+    "predictor = sagemaker.RealTimePredictor(endpoint=lang_id.endpoint_name, \n",
+    "                                   sagemaker_session=sess,\n",
+    "                                   serializer=json.dumps,\n",
+    "                                   deserializer=sagemaker.predictor.json_deserializer)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "collapsed": true
+   },
+   "source": [
+    "Next we'll pass few sentences from various languages to the endpoint to verify that the language identification works as expected."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "sentences = [\"hi which language is this?\",\n",
+    "             \"mon nom est Pierre\",\n",
+    "             \"Dem Jungen gab ich einen Ball.\",\n",
+    "             \"আমি বাড়ি যাবো.\"]\n",
+    "payload = {\"instances\" : sentences}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "predictions = predictor.predict(payload)\n",
+    "print(predictions)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "FastText expects the class label to be prefixed by `__label__` and that's why when we are performing inference with pre-trained model provided by FastText, we can see that the output label is prefixed with `__label__`. With a little preprocessing, we can strip the `__label__` prefix from the response."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import copy\n",
+    "predictions_copy = copy.deepcopy(predictions) # Copying predictions object because we want to change the labels in-place\n",
+    "for output in predictions_copy:\n",
+    "    output['label'] = output['label'][0][9:].upper() #__label__ has length of 9\n",
+    "\n",
+    "print(predictions_copy)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Stop / Close the Endpoint (Optional)\n",
+    "Finally, we should delete the endpoint before we close the notebook if we don't need to keep the endpoint running for serving realtime predictions."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "sess.delete_endpoint(predictor.endpoint)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Similarly, we can host any pre-trained [FastText word2vec model](https://fasttext.cc/docs/en/pretrained-vectors.html) using SageMaker BlazingText hosting."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "conda_python3",
+   "language": "python",
+   "name": "conda_python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.2"
+  },
+  "notice": "Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.  Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}