Merge branch 'master' into feature/large_pipeline

Author: ahsan-z-khan

CHANGELOG.md

@@ -1,5 +1,74 @@
 # Changelog
 
+## v2.71.0 (2021-12-06)
+
+### Features
+
+ * Add support for TF 2.6
+ * Adding PT 17/18 Repo
+ * Add profile_name support for Feature Store ingestion
+
+### Bug Fixes and Other Changes
+
+ * Fix non-existent variable name
+ * Add TF 2.6.2 on training
+ * Recreate static lineage test data
+
+## v2.70.0 (2021-12-02)
+
+### Features
+
+ * update boto3 minor version >= 1.20.18
+ * Add support for SageMaker lineage queries
+ * add CV shap explainability for SageMaker Clarify
+ * add NLP support for SageMaker Clarify
+ * Add support for ModelMonitor/Clarify integration in model building pipelines
+ * adding support for transformers 4.11 for SM Training Compiler
+ * SM Training Compiler with an UI to enable/disable compilation for HuggingFace DLCs to speedup training
+
+### Bug Fixes and Other Changes
+
+ * pin coveragepy
+ * Add support for PyTorch 1.9.1
+ * Update s3 path of scheduling analysis config on ClarifyCheckStep
+ * documentation/logging to indicate correct place for DEBUG artifacts from SM trcomp
+ * validate requested transformers version and use the best available version
+ * Install custom pkgs
+
+## v2.69.0 (2021-11-12)
+
+### Features
+
+ * Hugging Face Transformers 4.12 for Pt1.9/TF2.5
+
+## v2.68.0 (2021-11-02)
+
+### Features
+
+ * CompilationStep support for Sagemaker Pipelines
+
+## v2.67.0 (2021-11-01)
+
+### Deprecations and Removals
+
+ * deprecate Serverless Lambda model-predictor
+
+### Features
+
+ * add joinsource to DataConfig
+ * Add support for Partial Dependence Plots(PDP) in SageMaker Clarify
+
+### Bug Fixes and Other Changes
+
+ * localmode subprocess parent process not sending SIGTERM to child
+ * remove buildspec from repo
+
+## v2.66.2.post0 (2021-10-28)
+
+### Documentation Changes
+
+ * Update estimator docstrings to add Fast File Mode
+
 ## v2.66.2 (2021-10-27)
 
 ### Bug Fixes and Other Changes

VERSION

@@ -1 +1 @@
-2.66.3.dev0
+2.71.1.dev0

buildspec-deploy.yml

@@ -16,7 +16,3 @@ Model
     :undoc-members:
     :show-inheritance:
 
-.. autoclass:: sagemaker.serverless.model.LambdaModel
-    :members:
-    :undoc-members:
-    :show-inheritance:

buildspec-localmodetests.yml

@@ -31,3 +31,13 @@ Model Monitor
     :members:
     :undoc-members:
     :show-inheritance:
+
+.. automodule:: sagemaker.model_metrics
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+.. automodule:: sagemaker.drift_check_baselines
+    :members:
+    :undoc-members:
+    :show-inheritance:

buildspec-notebooktests.yml

@@ -7,8 +7,3 @@ Make real-time predictions against SageMaker endpoints with Python objects
     :members:
     :undoc-members:
     :show-inheritance:
-
-.. autoclass:: sagemaker.serverless.predictor.LambdaPredictor
-    :members:
-    :undoc-members:
-    :show-inheritance:

buildspec-release.yml

@@ -4,7 +4,15 @@ Hugging Face
 Hugging Face Estimator
 ----------------------
 
-.. autoclass:: sagemaker.huggingface.estimator.HuggingFace
+.. autoclass:: sagemaker.huggingface.HuggingFace
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Hugging Face Training Compiler Configuration
+--------------------------------------------
+
+.. autoclass:: sagemaker.huggingface.TrainingCompilerConfig
     :members:
     :undoc-members:
     :show-inheritance:
@@ -17,8 +25,8 @@ Hugging Face Model
     :undoc-members:
     :show-inheritance:
 
-HuggingFace Predictor
----------------------
+Hugging Face Predictor
+----------------------
 
 .. autoclass:: sagemaker.huggingface.model.HuggingFacePredictor
     :members:

buildspec-slowtests.yml

@@ -1063,50 +1063,6 @@ You can also find these notebooks in the **Advanced Functionality** section of t
 For information about using sample notebooks in a SageMaker notebook instance, see `Use Example Notebooks <https://docs.aws.amazon.com/sagemaker/latest/dg/howitworks-nbexamples.html>`__
 in the AWS documentation.
 
-********************
-Serverless Inference
-********************
-
-You can use the SageMaker Python SDK to perform serverless inference on Lambda.
-
-To deploy models to Lambda, you must complete the following prerequisites:
-
-- `Package your model and inference code as a container image. <https://docs.aws.amazon.com/lambda/latest/dg/images-create.html>`_
-- `Create a role that lists Lambda as a trusted entity. <https://docs.aws.amazon.com/lambda/latest/dg/lambda-intro-execution-role.html#permissions-executionrole-console>`_
-
-After completing the prerequisites, you can deploy your model to Lambda using
-the `LambdaModel`_ class.
-
-.. code:: python
-
-   from sagemaker.serverless import LambdaModel
-
-   image_uri = "123456789012.dkr.ecr.us-west-2.amazonaws.com/my-lambda-repository:latest"
-   role = "arn:aws:iam::123456789012:role/MyLambdaExecutionRole"
-
-   model = LambdaModel(image_uri=image_uri, role=role)
-   predictor = model.deploy("my-lambda-function", timeout=20, memory_size=4092)
-
-The ``deploy`` method returns a `LambdaPredictor`_  instance. Use the
-`LambdaPredictor`_ ``predict`` method to perform inference on Lambda.
-
-.. code:: python
-
-   url = "https://example.com/cat.jpeg"
-   predictor.predict({"url": url})  # {'class': 'tabby'}
-
-Once you are done performing inference on Lambda, free the `LambdaModel`_ and
-`LambdaPredictor`_ resources using the ``delete_model`` and ``delete_predictor``
-methods.
-
-.. code:: python
-
-   model.delete_model()
-   predictor.delete_predictor()
-
-.. _LambdaModel : https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.serverless.model.LambdaModel
-.. _LambdaPredictor : https://sagemaker.readthedocs.io/en/stable/api/inference/predictors.html#sagemaker.serverless.predictor.LambdaPredictor
-
 ******************
 SageMaker Workflow
 ******************

buildspec-unittests.yml

@@ -32,6 +32,11 @@ Conditions
 
 .. autoclass:: sagemaker.workflow.conditions.ConditionOr
 
+CheckJobConfig
+--------------
+
+.. autoclass:: sagemaker.workflow.check_job_config.CheckJobConfig
+
 Entities
 --------
 
@@ -134,3 +139,13 @@ Steps
 .. autoclass:: sagemaker.workflow.steps.CacheConfig
 
 .. autoclass:: sagemaker.workflow.lambda_step.LambdaStep
+
+.. autoclass:: sagemaker.workflow.steps.CompilationStep
+
+.. autoclass:: sagemaker.workflow.quality_check_step.QualityCheckConfig
+
+.. autoclass:: sagemaker.workflow.quality_check_step.QualityCheckStep
+
+.. autoclass:: sagemaker.workflow.clarify_check_step.ClarifyCheckConfig
+
+.. autoclass:: sagemaker.workflow.clarify_check_step.ClarifyCheckStep

buildspec.yml

@@ -34,7 +34,7 @@ def read_version():
 # Declare minimal set for installation
 required_packages = [
     "attrs",
-    "boto3>=1.16.32",
+    "boto3>=1.20.18",
     "google-pasta",
     "numpy>=1.9.0",
     "protobuf>=3.1",
@@ -69,6 +69,7 @@ def read_version():
         "pytest-rerunfailures",
         "pytest-timeout",
         "pytest-xdist",
+        "coverage<6.2",
         "mock",
         "contextlib2",
         "awslogs",

doc/api/inference/model.rst

@@ -49,7 +49,7 @@
 from sagemaker.local.local_session import LocalSession  # noqa: F401
 
 from sagemaker.model import Model, ModelPackage  # noqa: F401
-from sagemaker.model_metrics import ModelMetrics, MetricsSource  # noqa: F401
+from sagemaker.model_metrics import ModelMetrics, MetricsSource, FileSource  # noqa: F401
 from sagemaker.pipeline import PipelineModel  # noqa: F401
 from sagemaker.predictor import Predictor  # noqa: F401
 from sagemaker.processing import Processor, ScriptProcessor  # noqa: F401