edit sagemaker_model_monitor/fairness_and_explainability/SageMaker-Model-Monitor-Fairness-and-Explainability.ipynb

Author: atqy

sagemaker_model_monitor/fairness_and_explainability/SageMaker-Model-Monitor-Fairness-and-Explainability.ipynb

@@ -179,10 +179,10 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# PART A: Capturing real-time inference data from Amazon SageMaker endpoints\n",
+    "## PART A: Capturing real-time inference data from Amazon SageMaker endpoints\n",
     "Create an endpoint to showcase the data capture capability in action. (In next parts, model monitors will be created to process the data.)\n",
     "\n",
-    "## Upload the pre-trained model to Amazon S3\n",
+    "### Upload the pre-trained model to Amazon S3\n",
     "As an example, this code uploads a pre-trained XGBoost model that is ready for to be deployed. This model was trained using the [XGBoost Churn Prediction Notebook](https://github.com/aws/amazon-sagemaker-examples/blob/master/introduction_to_applying_machine_learning/xgboost_customer_churn/xgboost_customer_churn.ipynb) in SageMaker."
    ]
   },
@@ -200,7 +200,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Deploy the model to Amazon SageMaker\n",
+    "### Deploy the model to Amazon SageMaker\n",
     "Start with deploying a pre-trained churn prediction model. Here, create the SageMaker `Model` object with the image and model data."
    ]
   },
@@ -257,7 +257,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Invoke the deployed model\n",
+    "### Invoke the deployed model\n",
     "\n",
     "Now send data to this endpoint to get inferences in real time. Because data capture is enabled in the previous steps, the request and response payload, along with some additional metadata, is saved in the Amazon S3 location specified in the DataCaptureConfig."
    ]
@@ -292,7 +292,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## View captured data\n",
+    "### View captured data\n",
     "\n",
     "Now list the data capture files stored in Amazon S3. There should be different files from different time periods organized based on the hour in which the invocation occurred. The format of the Amazon S3 path is:\n",
     "\n",
@@ -354,7 +354,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Start generating some artificial traffic\n",
+    "### Start generating some artificial traffic\n",
     "The cell below starts a thread to send some traffic to the endpoint. If there is no traffic, the monitoring jobs are marked as `Failed` since there is no data to process."
    ]
   },
@@ -421,7 +421,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Start generating some fake ground truth\n",
+    "### Start generating some fake ground truth\n",
     "\n",
     "Besides captures, model bias monitoring execution also requires ground truth data. In real use cases, ground truth data should be regularly collected and uploaded to designated S3 location. In this example notebook, below code snippet is used to generate fake ground truth data. The first-party merge container will combine captures and ground truth data, and the merged data will be passed to model bias monitoring job for analysis. Similar to captures, the model bias monitoring execution will fail if there's no data to merge."
    ]
@@ -493,7 +493,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# PART B: Model Bias Monitor"
+    "## PART B: Model Bias Monitor"
    ]
   },
   {
@@ -521,7 +521,7 @@
    "metadata": {},
    "source": [
     "\n",
-    "## Create a baselining job\n",
+    "### Create a baselining job\n",
     "\n",
     "A baselining job runs predictions on training dataset and suggests constraints. `suggest_baseline()` method starts a `SageMakerClarifyProcessor` processing job using SageMaker Clarify container to generate the constraints.\n",
     "\n",
@@ -532,7 +532,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Configurations\n",
+    "#### Configurations\n",
     "\n",
     "Information about the input data need to be provided to the processor."
    ]
@@ -624,7 +624,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Kick off baselining job"
+    "#### Kick off baselining job"
    ]
   },
   {
@@ -666,7 +666,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Schedule model bias monitor\n",
+    "### Schedule model bias monitor\n",
     "\n",
     "With above constraints collected, now call `create_monitoring_schedule()` method to schedule a hourly monitor, to analyze the data with monitoring schedule. If a baselining job has been submitted, then the monitor will automatically pick up analysis configuration from the baselining job. But if the baselining step is skipped, or the capture dataset has different nature than the training dataset, then analysis configuration has to be provided.\n",
     "\n",
@@ -709,7 +709,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Wait for the first execution\n",
+    "### Wait for the first execution\n",
     "\n",
     "The schedule starts jobs at the previously specified intervals. Code below wait util time crosses the hour boundary (in UTC) to see executions kick off.\n",
     "\n",
@@ -775,7 +775,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Wait for the execution to finish\n",
+    "### Wait for the execution to finish\n",
     "\n",
     "In the previous cell, the first execution has started. This section waits for the execution to finish so that its analysis results are available. Here are the possible terminal states and what each of them mean:\n",
     "\n",
@@ -826,7 +826,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Inspect execution results\n",
+    "### Inspect execution results\n",
     "\n",
     "List the generated reports,"
    ]
@@ -890,7 +890,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# PART C: Model Explainability Monitor\n",
+    "## PART C: Model Explainability Monitor\n",
     "\n",
     "Model explainability monitor can explain the predictions of a deployed model producing inferences and detect feature attribution drift on a regular basis."
    ]
@@ -912,7 +912,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Create a baselining job\n",
+    "### Create a baselining job\n",
     "\n",
     "Similary, a baselining job can be scheduled to suggest constraints for model explainability monitor."
    ]
@@ -921,7 +921,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Configuration"
+    "#### Configuration"
    ]
   },
   {
@@ -984,7 +984,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Kick off baselining job\n",
+    "#### Kick off baselining job\n",
     "\n",
     "The same model_config is required, because the explainability baselining job needs to create shadow endpoint to get predictions for generated synthetic dataset."
    ]
@@ -1031,7 +1031,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Schedule model explainability monitor"
+    "### Schedule model explainability monitor"
    ]
   },
   {
@@ -1070,7 +1070,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Wait for execution and inspect analysis results\n",
+    "### Wait for execution and inspect analysis results\n",
     "\n",
     "Once created the schedule is started by default, here wait for the its first execution to start, then stop the schedule to avoid incurring charges."
    ]
@@ -1170,7 +1170,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# PART D: Cleanup\n",
+    "## PART D: Cleanup\n",
     "\n",
     "The endpoint can keep running and capturing data, but if there is no plan to collect more data or use this endpoint further, it should be deleted to avoid incurring additional charges. Note that deleting endpoint does not delete the data that was captured during the model invocations."
    ]