Merge pull request #1784 from yehudaorel/development

Development

Author: jimmytwei

AI-and-Analytics/Features-and-Functionality/IntelTensorFlow_ModelZoo_Inference_with_FP32_Int8/ResNet50_Inference_gpu.ipynb

@@ -0,0 +1,141 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#  Getting Started with [Intel Model Zoo](https://github.com/IntelAI/models)\n",
+    "\n",
+    "This code sample will serve as a sample use case to perform TensorFlow ResNet50v1.5 inference on a synthetic data implementing a FP32/FP16 and Int8 pre-trained model. The pre-trained model published as part of Intel Model Zoo will be used in this sample. "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Select precision and download model\n",
+    "Select the precision that you would like to run resnet50 model with. `fp32` , `fp16` or `int8`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "precision = \"fp32\"  # or \"int8\" or \"fp16\"\n",
+    "batch_size = 1024\n",
+    "output_dir= './''"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "initial_cwd = os.getcwd()\n",
+    "model_bucket = 'https://storage.googleapis.com/intel-optimized-tensorflow/models/gpu/'\n",
+    "model_precision = '_int8' if 'int8' == precision else ''\n",
+    "model_file = 'resnet50_v1' + model_precision + '.pb'\n",
+    "model_download_path = os.path.join(model_bucket, model_file)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#download Intel's pretrained resnet50 model\n",
+    "if not os.path.exists(os.path.join(os.getcwd(), model_file)):\n",
+    "    ! wget $model_download_path\n",
+    "model_local_path = os.path.join(os.getcwd(), model_file)\n",
+    "if not os.path.exists(model_local_path):\n",
+    "    raise Exception(\"Failed to download pretrained Model file {}\", model_download_path)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We will be using a synthetic dataset of size 244x244.\n",
+    "It is important to set optimial batch_size, MKL run-time settings, TensorFlow's inter-intra number of threads to enable compute and data layer optimizations. We have identified optimial settings for popular topologies including ResNet50 to maximize GPU utlization. For more details on Run-time settings refer to blogs [maximize XPU performance](https://software.intel.com/en-us/articles/maximize-tensorflow-performance-on-cpu-considerations-and-recommendations-for-inference), [Intel Model Zoo tutorials](https://github.com/IntelAI/models/tree/master/docs). \n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%cd /opt/intel/oneapi/modelzoo/latest"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Batch and Online Inference"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Run inference using --batch-size=128 for throughput, or --batch-size=1 for latency\n",
+    "%run models/benchmarks/launch_benchmark.py \\\n",
+    "    --in-graph $model_local_path \\\n",
+    "    --model-name resnet50v1_5 \\\n",
+    "    --framework tensorflow \\\n",
+    "    --precision $precision \\\n",
+    "    --mode inference \\\n",
+    "    --batch-size=$batch_size \\\n",
+    "    --benchmark-only \\\n",
+    "    --output-dir=$output_dir \\\n",
+    "    --gpu "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The output(both stdout and stderr) is displayed on the command line console"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "os.chdir(initial_cwd)\n",
+    "print('[CODE_SAMPLE_COMPLETED_SUCCESFULLY]')"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}