Generative fill demo for re'Invent2023

inference/generativeai/llm-workshop/lab12-hosting-controlnet-models-on-sagemaker/Dockerfile.inference

@@ -1,23 +1,28 @@
 ARG REGION
 FROM 763104351884.dkr.ecr.$REGION.amazonaws.com/pytorch-inference:2.0.1-gpu-py310-cu118-ubuntu20.04-sagemaker
 
-RUN apt update && apt install wget git libgl1-mesa-glx libcairo2-dev -y
+RUN apt update && apt install wget git libgl1-mesa-glx libcairo2-dev libtcmalloc-minimal4 -y
 RUN rm -rf /opt/conda/lib/python3.10/site-packages/opencv_python-4.6.0.dist-info/
 RUN pip install opencv-python
-RUN pip install install -U -I --no-deps xformers==0.0.20
+RUN pip install xformers==0.0.22.post7 --extra-index-url https://download.pytorch.org/whl/cu118
 RUN pip install boto3 huggingface_hub triton deepspeed accelerate
 
 RUN mkdir -p /opt/ml/code
 WORKDIR /opt/ml/code
 
-ADD https://api.github.com/repos/xieyongliang/stable-diffusion-webui/git/refs/heads/api-v1.5.1 /opt/ml/webui.version.json
-RUN git clone https://github.com/xieyongliang/stable-diffusion-webui.git /opt/ml/code -b api-v1.5.1
+ADD https://api.github.com/repos/xieyongliang/stable-diffusion-webui/git/refs/heads/api /opt/ml/webui.version.json
+RUN git clone https://github.com/xieyongliang/stable-diffusion-webui.git /opt/ml/code -b api
 
 ADD https://api.github.com/repos/Mikubill/sd-webui-controlnet/git/refs/heads/main /opt/ml/controlnet.version.json
 RUN git clone https://github.com/Mikubill/sd-webui-controlnet.git /opt/ml/code/extensions/sd-webui-controlnet
 
+ADD https://api.github.com/repos/xieyongliang/inpaint-anything/git/refs/heads/main /opt/ml/inpaint-anything.version.json
+RUN git clone https://github.com/xieyongliang/inpaint-anything.git /opt/ml/code/extensions/inpaint-anything
+
 RUN python launch.py -f --exit --skip-torch-cuda-test
-RUN pip install -U pytorch_lightning==1.6.5 pydantic==1.10.11
+RUN pip install -U pytorch_lightning==1.6.5 pydantic==1.10.11 httpx==0.24.1
+RUN pip install -U torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu118
+
 RUN rm requirements.txt
 
 COPY serve /opt/ml/code

inference/generativeai/llm-workshop/lab12-hosting-controlnet-models-on-sagemaker/README.md

@@ -1,7 +1,89 @@
 # stable-diffusion-webui-api
 
-This repo shows how to use stable-diffusion-webui to build Amazon SageMaker Docker image for inference and training job and how to generate images based on the combination with Stable-diffusion models and ControlNet / Lora models.
+## Overview
 
-You will find 2 Jupyter Notebooks: 1 for sync ControlNet / Lora inference, and 1 for async ControlNet / Lora inference.
+In this notebook, we will explore how to build generative fill application and host Stable Diffusion/ ControlNet / segment anything models on SageMaker asynchronous endpoint using DLC container.
 
-Note: Amazon Web Services has no control or authority over the third-party generative AI service referenced in this Workshop, and does not make any representations or warranties that the third-party generative AI service is secure, virus-free, operational, or compatible with your production environment and standards. You are responsible for making your own independent assessment of the content provided in this Workshop, and take measures to ensure that you comply with your own specific quality control practices and standards, and the local rules, laws, regulations, licenses and terms of use that apply to you, your content, and the third-party generative AI service referenced in this Workshop. The content of this Workshop: (a) is for informational purposes only, (b) represents current Amazon Web Services product offerings and practices, which are subject to change without notice, and (c) does not create any commitments or assurances from Beijing Sinnet Technology Co., Ltd. (“Sinnet”), Ningxia Western Cloud Data Technology Co., Ltd. (“NWCD”), Amazon Connect Technology Services (Beijing) Co., Ltd. (“Amazon”), or their respective affiliates, suppliers or licensors.  Amazon Web Services’ content, products or services are provided “as is” without warranties, representations, or conditions of any kind, whether express or implied.  The responsibilities and liabilities of Sinnet, NWCD or Amazon to their respective customers are controlled by the applicable customer agreements. 
+You will find 2 Jupyter Notebooks: 1 for running with Amazon SageMaker Studio and 1 for running with Amazon SageMaker Notebook.
+
+## IAM role recommendations
+
+1) Running with Amazon SageMaker Studio
+
+* Permissions policies
+
+    ```
+    AmazonSageMakerFullAccess
+    AmazonEC2ContainerRegistryFullAccess
+    AWSCodeBuildAdminAccess
+    IAMFullAccess
+    ```
+
+* Trusted entities
+
+    ```
+    {
+        "Version": "2012-10-17",
+        "Statement": [
+            {
+                "Effect": "Allow",
+                "Principal": {
+                    "Service": [
+                        "sagemaker.amazonaws.com",
+                        "codebuild.amazonaws.com"
+                    ]
+                },
+                "Action": "sts:AssumeRole"
+            }
+        ]
+    }
+    ```
+
+* Tested image, kernel, and instance:
+    ```
+    image: Pytorch 2.0.1 Python 3.10 CPU Optimized
+    kernel: Python 3
+    instance: ml.m5.4xlarge
+
+    ```
+
+2) Running with Amazon SageMaker Notebook
+
+* Permission Policies
+
+    ```
+    AmazonSageMakerFullAccess
+    AmazonEC2ContainerRegistryFullAccess
+    ```
+
+* Trusted entities
+
+    ```
+    {
+        "Version": "2012-10-17",
+        "Statement": [
+            {
+                "Effect": "Allow",
+                "Principal": {
+                    "Service": [
+                        "ecs.amazonaws.com",
+                        "sagemaker.amazonaws.com"
+                    ]
+                },
+                "Action": "sts:AssumeRole"
+            }
+        ]
+    }
+    ```
+
+* Tested kernel:
+    ```
+    kernel: conda_pytorch_p310
+
+    ```
+
+## Note
+
+1. You may need to adjust IAM roles definition to achieve fine grained access control.
+
+2. Amazon Web Services has no control or authority over the third-party generative AI service referenced in this Workshop, and does not make any representations or warranties that the third-party generative AI service is secure, virus-free, operational, or compatible with your production environment and standards. You are responsible for making your own independent assessment of the content provided in this Workshop, and take measures to ensure that you comply with your own specific quality control practices and standards, and the local rules, laws, regulations, licenses and terms of use that apply to you, your content, and the third-party generative AI service referenced in this Workshop. The content of this Workshop: (a) is for informational purposes only, (b) represents current Amazon Web Services product offerings and practices, which are subject to change without notice, and (c) does not create any commitments or assurances from Beijing Sinnet Technology Co., Ltd. (“Sinnet”), Ningxia Western Cloud Data Technology Co., Ltd. (“NWCD”), Amazon Connect Technology Services (Beijing) Co., Ltd. (“Amazon”), or their respective affiliates, suppliers or licensors.  Amazon Web Services’ content, products or services are provided “as is” without warranties, representations, or conditions of any kind, whether express or implied.  The responsibilities and liabilities of Sinnet, NWCD or Amazon to their respective customers are controlled by the applicable customer agreements.