Merge branch 'update-hf-neuronx-dlcs' of https://github.com/JingyaHuang/sagemaker-python-sdk into update-hf-neuronx-dlcs

Author: JingyaHuang

.pylintrc

@@ -42,7 +42,7 @@ unsafe-load-any-extension=no
 # A comma-separated list of package or module names from where C extensions may
 # be loaded. Extensions are loading into the active Python interpreter and may
 # run arbitrary code
-extension-pkg-whitelist=numpy
+extension-pkg-allow-list=numpy,math,_struct,_hashlib
 
 # Allow optimization of some AST trees. This will activate a peephole AST
 # optimizer, which will apply various small optimizations. For instance, it can

CHANGELOG.md

@@ -1,5 +1,106 @@
 # Changelog
 
+## v2.159.0 (2023-05-23)
+
+### Features
+
+ * Add TF Serving 2.12.1 images to the SM PySDK
+
+### Bug Fixes and Other Changes
+
+ * Update the list of extension packages pylint is allowed to load
+
+## v2.158.0 (2023-05-22)
+
+### Features
+
+ * Enable default role for Spark processors
+ * SDK Defaults - S3 Params for Session
+ * Bump up images for DJL transformers Neuronx DLCs
+
+### Bug Fixes and Other Changes
+
+ * Relax local-mode PyPI requirements on urllib3
+
+### Documentation Changes
+
+ * Fix Tensorflow and PyTorch supported version in HuggingFaceProcessor
+ * Update doc for model_server_workers param in PyTorchModel
+
+## v2.157.0 (2023-05-18)
+
+### Features
+
+ * Handle use case where endpoint is created outside of python …
+
+### Bug Fixes and Other Changes
+
+ * Make type annotation of UploadedCode consistent
+ * Add SELinux label to local docker volumes
+
+## v2.156.0 (2023-05-17)
+
+### Features
+
+ * Partition support for DJLModel using SM Training job
+ * Update run-notebook-test to consider skips failures
+
+### Bug Fixes and Other Changes
+
+ * Update apache airflow and update test requirements
+ * Perform integrity checks for remote function execution
+ * Add p2 instances to integ tests
+ * Fix typo in logging message within ir mixin
+ * double Run create on load_run
+ * Update dtype logic for huggingface backend for new containers
+
+### Documentation Changes
+
+ * Update container version for SKLearn
+ * Add description for parameters in TransformInput
+
+## v2.155.0 (2023-05-15)
+
+### Features
+
+ * Add support for SageMaker Serverless inference Provisioned Concurrency feature
+
+### Bug Fixes and Other Changes
+
+ * Revert "fix: make RemoteExecutor context manager non-blocking on pend…
+ * Add BOM to no No P2 Availability region list
+
+## v2.154.0 (2023-05-11)
+
+### Features
+
+ * Add integ tests for remote_function, auto_capture functionality
+ * jumpstart model estimator classes
+
+### Bug Fixes and Other Changes
+
+ * integs - pytorch transformer deps and add test retry
+ * adding .lower() so new Pandas dtypes will match the type lookup.
+ * Pass KMS value to create processing job
+
+## v2.153.0 (2023-05-09)
+
+### Features
+
+ * Support npz archives in NumpyDeserializer
+ * Add FasterTransformer DJL support
+ * support for Sample Weights for SageMaker Autopilot
+
+### Bug Fixes and Other Changes
+
+ * retry is_run assertion
+ * Avoid 'AttributeError' for endpoint_name, if deploy() is not yet called
+ * Fix LambdaStep Creation
+ * Fix error when instance_count>1 in remote_function
+ * Remove deprecated update_endpoint from deploy() args in TensorFlowModel
+ * Update DJL deepspeed and fastertransformer DLC image uris
+ * remote_function python version mismatch issue
+
 ## v2.152.0 (2023-05-04)
 
 ### Features

VERSION

@@ -1 +1 @@
-2.152.1.dev0
+2.159.1.dev0

doc/frameworks/djl/using_djl.rst

@@ -221,6 +221,31 @@ see the `DJL Serving Documentation on Python Mode. <https://docs.djl.ai/docs/ser
 
 For more information about DJL Serving, see the `DJL Serving documentation. <https://docs.djl.ai/docs/serving/index.html>`_
 
+**************************
+Ahead of time partitioning
+**************************
+
+To optimize the deployment of large models that do not fit in a single GPU, the model’s tensor weights are partitioned at
+runtime and each partition is loaded in individual GPU. But runtime partitioning takes significant amount of time and
+memory on model loading. So, DJLModel offers an ahead of time partitioning capability for DeepSpeed and FasterTransformer
+engines, which lets you partition your model weights and save them before deployment. HuggingFace does not support
+tensor parallelism, so ahead of time partitioning cannot be done for it. In our experiment with GPT-J model, loading
+this model with partitioned checkpoints increased the model loading time by 40%.
+
+`partition` method invokes an Amazon SageMaker Training job to partition the model and upload those partitioned
+checkpoints to S3 bucket. You can either provide your desired S3 bucket to upload the partitioned checkpoints or it will be
+uploaded to the default SageMaker S3 bucket. Please note that this S3 bucket will be remembered for deployment. When you
+call `deploy` method after partition, DJLServing downloads the partitioned model checkpoints directly from the uploaded
+s3 url, if available.
+
+.. code::
+
+    # partitions the model using Amazon Sagemaker Training Job.
+    djl_model.partition("ml.g5.12xlarge")
+
+    predictor = deepspeed_model.deploy("ml.g5.12xlarge",
+                                        initial_instance_count=1)
+
 ***********************
 SageMaker DJL Classes
 ***********************

doc/frameworks/sklearn/using_sklearn.rst

@@ -7,7 +7,7 @@ With Scikit-learn Estimators, you can train and host Scikit-learn models on Amaz
 For information about supported versions of Scikit-learn, see the `AWS documentation <https://docs.aws.amazon.com/sagemaker/latest/dg/sklearn.html>`__.
 We recommend that you use the latest supported version because that's where we focus most of our development efforts.
 
-For more information about the framework, see the `Sciket-Learn <https://github.com/scikit-learn/scikit-learn>`_ repository.
+For more information about the framework, see the `Scikit-Learn <https://github.com/scikit-learn/scikit-learn>`_ repository.
 For general information about using the SageMaker Python SDK, see :ref:`overview:Using the SageMaker Python SDK`.
 
 .. contents::
@@ -31,7 +31,7 @@ To train a Scikit-learn model by using the SageMaker Python SDK:
 Prepare a Scikit-learn Training Script
 ======================================
 
-Your Scikit-learn training script must be a Python 3.6 compatible source file.
+Your Scikit-learn training script must be a Python 3.7 compatible source file.
 
 The training script is similar to a training script you might run outside of SageMaker, but you
 can access useful properties about the training environment through various environment variables.
@@ -140,7 +140,7 @@ directories ('train' and 'test').
 
     sklearn_estimator = SKLearn('sklearn-train.py',
                                 instance_type='ml.m4.xlarge',
-                                framework_version='0.20.0',
+                                framework_version='1.0-1',
                                 hyperparameters = {'epochs': 20, 'batch-size': 64, 'learning-rate': 0.1})
     sklearn_estimator.fit({'train': 's3://my-data-bucket/path/to/my/training/data',
                             'test': 's3://my-data-bucket/path/to/my/test/data'})
@@ -204,7 +204,7 @@ operation.
     # Train my estimator
     sklearn_estimator = SKLearn(entry_point='train_and_deploy.py',
                                 instance_type='ml.m4.xlarge',
-                                framework_version='0.20.0')
+                                framework_version='1.0-1')
     sklearn_estimator.fit('s3://my_bucket/my_training_data/')
 
     # Deploy my estimator to a SageMaker Endpoint and get a Predictor
@@ -478,7 +478,7 @@ The following code sample shows how to do this, using the ``SKLearnModel`` class
     sklearn_model = SKLearnModel(model_data="s3://bucket/model.tar.gz",
                                  role="SageMakerRole",
                                  entry_point="transform_script.py",
-                                 framework_version="0.20.0")
+                                 framework_version="1.0-1")
 
     predictor = sklearn_model.deploy(instance_type="ml.c4.xlarge", initial_instance_count=1)
 

requirements/extras/local_requirements.txt

@@ -1,4 +1,4 @@
-urllib3==1.26.8
+urllib3>=1.26.8,<1.26.15
 docker-compose==1.29.2
 docker>=5.0.2,<7.0.0
 PyYAML==5.4.1

requirements/extras/test_requirements.txt

@@ -12,14 +12,18 @@ awslogs==0.14.0
 black==22.3.0
 stopit==1.1.2
 # Update tox.ini to have correct version of airflow constraints file
-apache-airflow==2.5.1
+apache-airflow==2.6.0
 apache-airflow-providers-amazon==7.2.1
-attrs==22.1.0
+attrs>=23.1.0,<24
 fabric==2.6.0
-requests==2.27.1
+requests==2.31.0
 sagemaker-experiments==0.1.35
 Jinja2==3.0.3
 pyvis==0.2.1
 pandas>=1.3.5,<1.5
 scikit-learn==1.0.2
 cloudpickle==2.2.1
+scipy==1.7.3
+urllib3>=1.26.8,<1.26.15
+docker>=5.0.2,<7.0.0
+PyYAML==6.0

setup.py

@@ -47,8 +47,8 @@ def read_requirements(filename):
 
 # Declare minimal set for installation
 required_packages = [
-    "attrs>=20.3.0,<23",
-    "boto3>=1.26.28,<2.0",
+    "attrs>=23.1.0,<24",
+    "boto3>=1.26.131,<2.0",
     "cloudpickle==2.2.1",
     "google-pasta",
     "numpy>=1.9.0,<2.0",
@@ -60,7 +60,7 @@ def read_requirements(filename):
     "pandas",
     "pathos",
     "schema",
-    "PyYAML==5.4.1",
+    "PyYAML==6.0",
     "jsonschema",
     "platformdirs",
     "tblib==1.7.0",
@@ -75,7 +75,7 @@ def read_requirements(filename):
 # Meta dependency groups
 extras["all"] = [item for group in extras.values() for item in group]
 # Tests specific dependencies (do not need to be included in 'all')
-extras["test"] = (extras["all"] + read_requirements("requirements/extras/test_requirements.txt"),)
+extras["test"] = (read_requirements("requirements/extras/test_requirements.txt"),)
 
 setup(
     name="sagemaker",

src/sagemaker/accept_types.py

@@ -0,0 +1,103 @@
+# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"). You
+# may not use this file except in compliance with the License. A copy of
+# the License is located at
+#
+#     http://aws.amazon.com/apache2.0/
+#
+# or in the "license" file accompanying this file. This file is
+# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
+# ANY KIND, either express or implied. See the License for the specific
+# language governing permissions and limitations under the License.
+"""This module is for SageMaker accept types."""
+from __future__ import absolute_import
+from typing import List, Optional
+
+from sagemaker.jumpstart import artifacts, utils as jumpstart_utils
+
+
+def retrieve_options(
+    region: Optional[str] = None,
+    model_id: Optional[str] = None,
+    model_version: Optional[str] = None,
+    tolerate_vulnerable_model: bool = False,
+    tolerate_deprecated_model: bool = False,
+) -> List[str]:
+    """Retrieves the supported accept types for the model matching the given arguments.
+
+    Args:
+        region (str): The AWS Region for which to retrieve the supported accept types.
+            Defaults to ``None``.
+        model_id (str): The model ID of the model for which to
+            retrieve the supported accept types. (Default: None).
+        model_version (str): The version of the model for which to retrieve the
+            supported accept types. (Default: None).
+        tolerate_vulnerable_model (bool): True if vulnerable versions of model
+            specifications should be tolerated (exception not raised). If False, raises an
+            exception if the script used by this version of the model has dependencies with known
+            security vulnerabilities. (Default: False).
+        tolerate_deprecated_model (bool): True if deprecated models should be tolerated
+            (exception not raised). False if these models should raise an exception.
+            (Default: False).
+    Returns:
+        list: The supported accept types to use for the model.
+
+    Raises:
+        ValueError: If the combination of arguments specified is not supported.
+    """
+    if not jumpstart_utils.is_jumpstart_model_input(model_id, model_version):
+        raise ValueError(
+            "Must specify JumpStart `model_id` and `model_version` when retrieving accept types."
+        )
+
+    return artifacts._retrieve_supported_accept_types(
+        model_id,
+        model_version,
+        region,
+        tolerate_vulnerable_model,
+        tolerate_deprecated_model,
+    )
+
+
+def retrieve_default(
+    region: Optional[str] = None,
+    model_id: Optional[str] = None,
+    model_version: Optional[str] = None,
+    tolerate_vulnerable_model: bool = False,
+    tolerate_deprecated_model: bool = False,
+) -> str:
+    """Retrieves the default accept type for the model matching the given arguments.
+
+    Args:
+        region (str): The AWS Region for which to retrieve the default accept type.
+            Defaults to ``None``.
+        model_id (str): The model ID of the model for which to
+            retrieve the default accept type. (Default: None).
+        model_version (str): The version of the model for which to retrieve the
+            default accept type. (Default: None).
+        tolerate_vulnerable_model (bool): True if vulnerable versions of model
+            specifications should be tolerated (exception not raised). If False, raises an
+            exception if the script used by this version of the model has dependencies with known
+            security vulnerabilities. (Default: False).
+        tolerate_deprecated_model (bool): True if deprecated models should be tolerated
+            (exception not raised). False if these models should raise an exception.
+            (Default: False).
+    Returns:
+        str: The default accept type to use for the model.
+
+    Raises:
+        ValueError: If the combination of arguments specified is not supported.
+    """
+    if not jumpstart_utils.is_jumpstart_model_input(model_id, model_version):
+        raise ValueError(
+            "Must specify JumpStart `model_id` and `model_version` when retrieving accept types."
+        )
+
+    return artifacts._retrieve_default_accept_type(
+        model_id,
+        model_version,
+        region,
+        tolerate_vulnerable_model,
+        tolerate_deprecated_model,
+    )

src/sagemaker/amazon/amazon_estimator.py

@@ -20,7 +20,7 @@
 
 from six.moves.urllib.parse import urlparse
 
-from sagemaker import image_uris
+from sagemaker import image_uris, s3_utils
 from sagemaker.amazon import validation
 from sagemaker.amazon.hyperparameter import Hyperparameter as hp  # noqa
 from sagemaker.amazon.common import write_numpy_to_dense_tensor
@@ -93,8 +93,15 @@ def __init__(
             enable_network_isolation=enable_network_isolation,
             **kwargs
         )
-        data_location = data_location or "s3://{}/sagemaker-record-sets/".format(
-            self.sagemaker_session.default_bucket()
+
+        data_location = data_location or (
+            s3_utils.s3_path_join(
+                "s3://",
+                self.sagemaker_session.default_bucket(),
+                self.sagemaker_session.default_bucket_prefix,
+                "sagemaker-record-sets",
+                with_end_slash=True,
+            )
         )
         self._data_location = data_location