Reinvent2023 keynote1 (#4488)

* adding notebooks for V7 LMI (#270)

Co-authored-by: “Vivek <“[email protected]”>

* feat: jumpstart llm load test benchmarking (#277)

* feat: initial jumpstart llm benchmarking commit

* fix: throughput robustness and pricing

* feat: jumpstart benchmarking, deploy endpoint and model specs

* feat: jumpstart benchmarking generalize concurrency probe

* fix: jumpstart latency benchmarking logging changes

* fix: adjust throughput computations

* fix: variety of cleanup to jumpstart llm benchmarking

* chore: clean up notebooks

* chore: black

* chore: cleanup jumpstart inference benchmarking

* fix: adjust error logging in concurrency probe

* chore: concurrency probe finalization

* chore: clean up notebooks

* chore: add tranformers requirement install

* chore: black

* chore: grammar

* chore: pip install change

* chore: adjust load metrics for missing token statistics

* [minor updates]

* pinned version for image generation notebook (#282)

* pinned version for image generation notebook

* re-triggered test

* pinned version

* dummy commit

* code formatting fix (#287)

* Badges vivek (#292)

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

* [adding badges for the lmi v7 notebooks]

---------

Co-authored-by: “Vivek <“[email protected]”>

* fix the badge issue (#289)

Co-authored-by: Harsha Reddy <[email protected]>

---------

Co-authored-by: Vivek Gangasani <[email protected]>
Co-authored-by: “Vivek <“[email protected]”>
Co-authored-by: ulrichkr <[email protected]>
Co-authored-by: vivekmadan2 <[email protected]>
Co-authored-by: xieyongliang <[email protected]>
Co-authored-by: Harsha Reddy <[email protected]>

Author: 7 people

inference/generativeai/llm-workshop/deploy-V7-lmi/llama2_70b-lmi-trtllm.ipynb

@@ -29,7 +29,7 @@
     "\n",
     "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.\n",
     "\n",
-    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
     "---"
    ]
@@ -782,35 +782,35 @@
     "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
     "\n",
     "\n",
-    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n",
+    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n",
     "\n",
-    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference.ipynb)\n"
+    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/inference|generativeai|llm-workshop|lab12-hosting-controlnet-models-on-sagemaker|stable-diffusion-webui-async-inference-sagemaker-notebook.ipynb)\n"
    ]
   }
  ],