Merge branch 'master' into fix-allow-docker-6

Author: jmahlik

CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## v2.110.0 (2022-09-27)
+
+### Features
+
+ * Support KeepAlivePeriodInSeconds for Training APIs
+ * added ANALYSIS_CONFIG_SCHEMA_V1_0 in clarify
+ * add model monitor image accounts for ap-southeast-3
+
+### Bug Fixes and Other Changes
+
+ * huggingface release test
+ * Fixing the logic to return instanceCount for heterogeneousClusters
+ * Disable type hints in doc signature and add PipelineVariable annotations in docstring
+ * estimator hyperparameters in script mode
+
+### Documentation Changes
+
+ * Added link to example notebook for Pipelines local mode
+
 ## v2.109.0 (2022-09-09)
 
 ### Features

VERSION

@@ -1 +1 @@
-2.109.1.dev0
+2.110.1.dev0

doc/frameworks/pytorch/using_pytorch.rst

@@ -415,20 +415,25 @@ Before a model can be served, it must be loaded. The SageMaker PyTorch model ser
 
 .. code:: python
 
-    def model_fn(model_dir)
+    def model_fn(model_dir, context)
+
+``context`` is an optional argument that contains additional serving information, such as the GPU ID and batch size.
+If specified in the function declaration, the context will be created and passed to the function by SageMaker.
+For more information about ``context``, see the `Serving Context class <https://github.com/pytorch/serve/blob/master/ts/context.py>`_.
 
 SageMaker will inject the directory where your model files and sub-directories, saved by ``save``, have been mounted.
 Your model function should return a model object that can be used for model serving.
 
 The following code-snippet shows an example ``model_fn`` implementation.
-It loads the model parameters from a ``model.pth`` file in the SageMaker model directory ``model_dir``.
+It loads the model parameters from a ``model.pth`` file in the SageMaker model directory ``model_dir``. As explained in the preceding example,
+``context`` is an optional argument that passes additional information.
 
 .. code:: python
 
     import torch
     import os
 
-    def model_fn(model_dir):
+    def model_fn(model_dir, context):
         model = Your_Model()
         with open(os.path.join(model_dir, 'model.pth'), 'rb') as f:
             model.load_state_dict(torch.load(f))
@@ -482,13 +487,13 @@ function in the chain. Inside the SageMaker PyTorch model server, the process lo
 .. code:: python
 
     # Deserialize the Invoke request body into an object we can perform prediction on
-    input_object = input_fn(request_body, request_content_type)
+    input_object = input_fn(request_body, request_content_type, context)
 
     # Perform prediction on the deserialized object, with the loaded model
-    prediction = predict_fn(input_object, model)
+    prediction = predict_fn(input_object, model, context)
 
     # Serialize the prediction result into the desired response content type
-    output = output_fn(prediction, response_content_type)
+    output = output_fn(prediction, response_content_type, context)
 
 The above code sample shows the three function definitions:
 
@@ -536,9 +541,13 @@ it should return an object that can be passed to ``predict_fn`` and have the fol
 
 .. code:: python
 
-    def input_fn(request_body, request_content_type)
+    def input_fn(request_body, request_content_type, context)
 
-Where ``request_body`` is a byte buffer and ``request_content_type`` is a Python string
+Where ``request_body`` is a byte buffer and ``request_content_type`` is a Python string.
+
+``context`` is an optional argument that contains additional serving information, such as the GPU ID and batch size.
+If specified in the function declaration, the context will be created and passed to the function by SageMaker.
+For more information about ``context``, see the `Serving Context class <https://github.com/pytorch/serve/blob/master/ts/context.py>`_.
 
 The SageMaker PyTorch model server provides a default implementation of ``input_fn``.
 This function deserializes JSON, CSV, or NPY encoded data into a torch.Tensor.
@@ -586,16 +595,19 @@ The ``predict_fn`` function has the following signature:
 
 .. code:: python
 
-    def predict_fn(input_object, model)
+    def predict_fn(input_object, model, context)
 
 Where ``input_object`` is the object returned from ``input_fn`` and
 ``model`` is the model loaded by ``model_fn``.
+If you are using multiple GPUs, then specify the ``context`` argument, which contains information such as the GPU ID for a dynamically-selected GPU and the batch size.
+One of the examples below demonstrates how to configure ``predict_fn`` with the ``context`` argument to handle multiple GPUs. For more information about ``context``, see the `Serving Context class <https://github.com/pytorch/serve/blob/master/ts/context.py>`_.
+If you are using CPUs or a single GPU, then you do not need to specify the ``context`` argument.
 
 The default implementation of ``predict_fn`` invokes the loaded model's ``__call__`` function on ``input_object``,
 and returns the resulting value. The return-type should be a torch.Tensor to be compatible with the default
 ``output_fn``.
 
-The example below shows an overridden ``predict_fn``:
+The following example shows an overridden ``predict_fn``:
 
 .. code:: python
 
@@ -609,6 +621,20 @@ The example below shows an overridden ``predict_fn``:
         with torch.no_grad():
             return model(input_data.to(device))
 
+The following example is for use cases with multiple GPUs and shows an overridden ``predict_fn`` that uses the ``context`` argument to dynamically select a GPU device for making predictions:
+
+.. code:: python
+
+    import torch
+    import numpy as np
+
+    def predict_fn(input_data, model):
+        device = torch.device("cuda:" + str(context.system_properties.get("gpu_id")) if torch.cuda.is_available() else "cpu")
+        model.to(device)
+        model.eval()
+        with torch.no_grad():
+            return model(input_data.to(device))
+
 If you implement your own prediction function, you should take care to ensure that:
 
 -  The first argument is expected to be the return value from input_fn.
@@ -664,11 +690,14 @@ The ``output_fn`` has the following signature:
 
 .. code:: python
 
-    def output_fn(prediction, content_type)
+    def output_fn(prediction, content_type, context)
 
 Where ``prediction`` is the result of invoking ``predict_fn`` and
-the content type for the response, as specified by the InvokeEndpoint request.
-The function should return a byte array of data serialized to content_type.
+the content type for the response, as specified by the InvokeEndpoint request. The function should return a byte array of data serialized to ``content_type``.
+
+``context`` is an optional argument that contains additional serving information, such as the GPU ID and batch size.
+If specified in the function declaration, the context will be created and passed to the function by SageMaker.
+For more information about ``context``, see the `Serving Context class <https://github.com/pytorch/serve/blob/master/ts/context.py>`_.
 
 The default implementation expects ``prediction`` to be a torch.Tensor and can serialize the result to JSON, CSV, or NPY.
 It accepts response content types of "application/json", "text/csv", and "application/x-npy".

requirements/extras/test_requirements.txt

@@ -13,7 +13,7 @@ black==22.3.0
 stopit==1.1.2
 apache-airflow==2.3.4
 apache-airflow-providers-amazon==4.0.0
-attrs==20.3.0
+attrs==22.1.0
 fabric==2.6.0
 requests==2.27.1
 sagemaker-experiments==0.1.35

setup.py

@@ -47,7 +47,7 @@ def read_requirements(filename):
 
 # Declare minimal set for installation
 required_packages = [
-    "attrs>=20.3.0,<22",
+    "attrs>=20.3.0,<23",
     "boto3>=1.20.21,<2.0",
     "google-pasta",
     "numpy>=1.9.0,<2.0",
@@ -58,6 +58,7 @@ def read_requirements(filename):
     "packaging>=20.0",
     "pandas",
     "pathos",
+    "schema",
 ]
 
 # Specific use case dependencies

src/sagemaker/amazon/amazon_estimator.py

@@ -16,7 +16,7 @@
 import json
 import logging
 import tempfile
-from typing import Union
+from typing import Union, Optional, Dict
 
 from six.moves.urllib.parse import urlparse
 
@@ -30,6 +30,7 @@
 from sagemaker.utils import sagemaker_timestamp
 from sagemaker.workflow.entities import PipelineVariable
 from sagemaker.workflow.pipeline_context import runnable_by_pipeline
+from sagemaker.workflow import is_pipeline_variable
 
 logger = logging.getLogger(__name__)
 
@@ -40,18 +41,20 @@ class AmazonAlgorithmEstimatorBase(EstimatorBase):
     This class isn't intended to be instantiated directly.
     """
 
-    feature_dim = hp("feature_dim", validation.gt(0), data_type=int)
-    mini_batch_size = hp("mini_batch_size", validation.gt(0), data_type=int)
-    repo_name = None
-    repo_version = None
+    feature_dim: hp = hp("feature_dim", validation.gt(0), data_type=int)
+    mini_batch_size: hp = hp("mini_batch_size", validation.gt(0), data_type=int)
+    repo_name: Optional[str] = None
+    repo_version: Optional[str] = None
+
+    DEFAULT_MINI_BATCH_SIZE: Optional[int] = None
 
     def __init__(
         self,
-        role,
-        instance_count=None,
-        instance_type=None,
-        data_location=None,
-        enable_network_isolation=False,
+        role: str,
+        instance_count: Optional[Union[int, PipelineVariable]] = None,
+        instance_type: Optional[Union[str, PipelineVariable]] = None,
+        data_location: Optional[str] = None,
+        enable_network_isolation: Union[bool, PipelineVariable] = False,
         **kwargs
     ):
         """Initialize an AmazonAlgorithmEstimatorBase.
@@ -62,16 +65,16 @@ def __init__(
                 endpoints use this role to access training data and model
                 artifacts. After the endpoint is created, the inference code
                 might use the IAM role, if it needs to access an AWS resource.
-            instance_count (int): Number of Amazon EC2 instances to use
+            instance_count (int or PipelineVariable): Number of Amazon EC2 instances to use
                 for training. Required.
-            instance_type (str): Type of EC2 instance to use for training,
+            instance_type (str or PipelineVariable): Type of EC2 instance to use for training,
                 for example, 'ml.c4.xlarge'. Required.
             data_location (str or None): The s3 prefix to upload RecordSet
                 objects to, expressed as an S3 url. For example
                 "s3://example-bucket/some-key-prefix/". Objects will be saved in
                 a unique sub-directory of the specified location. If None, a
                 default data location will be used.
-            enable_network_isolation (bool): Specifies whether container will
+            enable_network_isolation (bool or PipelineVariable): Specifies whether container will
                 run in network isolation mode. Network isolation mode restricts
                 the container access to outside networks (such as the internet).
                 Also known as internet-free mode (default: ``False``).
@@ -113,8 +116,14 @@ def data_location(self):
         return self._data_location
 
     @data_location.setter
-    def data_location(self, data_location):
+    def data_location(self, data_location: str):
         """Placeholder docstring"""
+        if is_pipeline_variable(data_location):
+            raise TypeError(
+                "Invalid input: data_location should be a plain string "
+                "rather than a pipeline variable - ({}).".format(type(data_location))
+            )
+
         if not data_location.startswith("s3://"):
             raise ValueError(
                 'Expecting an S3 URL beginning with "s3://". Got "{}"'.format(data_location)
@@ -198,12 +207,12 @@ def _prepare_for_training(self, records, mini_batch_size=None, job_name=None):
     @runnable_by_pipeline
     def fit(
         self,
-        records,
-        mini_batch_size=None,
-        wait=True,
-        logs=True,
-        job_name=None,
-        experiment_config=None,
+        records: "RecordSet",
+        mini_batch_size: Optional[int] = None,
+        wait: bool = True,
+        logs: bool = True,
+        job_name: Optional[str] = None,
+        experiment_config: Optional[Dict[str, str]] = None,
     ):
         """Fit this Estimator on serialized Record objects, stored in S3.
 
@@ -301,6 +310,20 @@ def record_set(self, train, labels=None, channel="train", encrypt=False):
             channel=channel,
         )
 
+    def _get_default_mini_batch_size(self, num_records: int):
+        """Generate the default mini_batch_size"""
+        if is_pipeline_variable(self.instance_count):
+            logger.warning(
+                "mini_batch_size is not given in .fit() and instance_count is a "
+                "pipeline variable (%s) which is only interpreted in pipeline execution time. "
+                "Thus setting mini_batch_size to 1, since it can't be greater than "
+                "number of records per instance_count, otherwise the training job fails.",
+                type(self.instance_count),
+            )
+            return 1
+
+        return min(self.DEFAULT_MINI_BATCH_SIZE, max(1, int(num_records / self.instance_count)))
+
 
 class RecordSet(object):
     """Placeholder docstring"""
@@ -461,7 +484,7 @@ def upload_numpy_to_s3_shards(
             raise ex
 
 
-def get_image_uri(region_name, repo_name, repo_version=1):
+def get_image_uri(region_name, repo_name, repo_version="1"):
     """Deprecated method. Please use sagemaker.image_uris.retrieve().
 
     Args: