sync with master branch

CHANGELOG.md

@@ -1,5 +1,144 @@
 # Changelog
 
+## v2.196.0 (2023-10-27)
+
+### Features
+
+ * inference instance type conditioned on training instance type
+
+### Bug Fixes and Other Changes
+
+ * improved jumpstart tagging
+
+## v2.195.1 (2023-10-26)
+
+### Bug Fixes and Other Changes
+
+ * Allow either instance_type or instance_group to be defined in…
+ * enhance image_uris unit tests
+
+## v2.195.0 (2023-10-25)
+
+### Features
+
+ * jumpstart gated model artifacts
+ * jumpstart extract generated text from response
+ * jumpstart contruct payload utility
+
+### Bug Fixes and Other Changes
+
+ * relax upper bound on urllib in local mode requirements
+ * bump urllib3 version
+ * allow smdistributed to be enabled with torch_distributed.
+ * fix URL links
+
+### Documentation Changes
+
+ * remove python 2 reference
+ * update framework version links
+
+## v2.194.0 (2023-10-19)
+
+### Features
+
+ * Added register step in Jumpstart model
+ * jumpstart instance specific metric definitions
+
+### Bug Fixes and Other Changes
+
+ * Updates for DJL 0.24.0 Release
+ * use getter for resource-metadata dict
+ * add method to Model class to check if repack is needed
+
+## v2.193.0 (2023-10-18)
+
+### Features
+
+ * jumpstart model artifact instance type variants
+ * jumpstart instance specific hyperparameters
+ * Feature Processor event based triggers (#1132)
+ * Support job checkpoint in remote function
+ * jumpstart model package arn instance type variants
+
+### Bug Fixes and Other Changes
+
+ * Fix hyperlinks in feature_processor.scheduler parameter descriptions
+ * add image_uris_unit_test pytest mark
+ * bump apache-airflow to `v2.7.2`
+ * clone distribution in validate_distribution
+ * fix flaky Inference Recommender integration tests
+
+### Documentation Changes
+
+ * Update PipelineModel.register documentation
+ * specify that input_shape in no longer required for torch 2.0 mod…
+
+## v2.192.1 (2023-10-13)
+
+### Bug Fixes and Other Changes
+
+ * update local mode schema
+ * import error in unsupported js regions
+ * Update Ec2 instance type to g5.4xlarge in test_huggingface_torch_distributed.py
+
+## v2.192.0 (2023-10-11)
+
+### Features
+
+ * jumpstart estimator enable infra check flag
+ * jumpstart default payloads
+ * allow non-python files in job dependencies
+ * allow configuring docker container in local mode
+
+### Bug Fixes and Other Changes
+
+ * js tagging s3 prefix
+ * Batch transform: Add support for split_type == "None" in local mode
+ * use correct line endings and s3 uris on windows
+ * Fixed bug in _create_training_details
+ * DJL Neuronx 0.24.0
+
+### Documentation Changes
+
+ * Include FeatureGroup's load_feature_definitions API documentation
+
+## v2.191.0 (2023-10-05)
+
+### Features
+
+ * Selective Step Execution milestone 2 features
+ * feature-processor extra data sources support
+
+## v2.190.0 (2023-10-04)
+
+### Features
+
+ * Add support for in-memory feature groups and collection type features in Feature Store.
+
+### Bug Fixes and Other Changes
+
+ * chore: xfail resource in use failure for specific test
+ * Add missing API docs for processors
+
+### Documentation Changes
+
+ * Bring back (de)serializers documentation
+ * Add missing AirFlow operators + link to airflow documentation
+
+## v2.189.0 (2023-10-03)
+
+### Features
+
+ * add feature processor APIs to public doc
+ * s3 prefix model data for JumpStartModel
+ * Model Package support for updating approval
+
+### Bug Fixes and Other Changes
+
+ * Add bucket owner check
+ * transform step  unit test
+ * Release TGI 1.1.0 Image
+
 ## v2.188.0 (2023-09-26)
 
 ### Features

VERSION

@@ -1 +1 @@
-2.188.1.dev0
+2.196.1.dev0

doc/amazon_sagemaker_featurestore.rst

@@ -230,9 +230,11 @@ The following code from the fraud detection example shows a minimal
        enable_online_store=True
    )
 
-Creating a feature group takes time as the data is loaded. You will need
-to wait until it is created before you can use it. You can check status
-using the following method.
+Creating a feature group takes time as the data is loaded. You will
+need to wait until it is created before you can use it. You can
+check status using the following method. Note that it can take
+approximately 10-15 minutes to provision an online ``FeatureGroup``
+with the ``InMemory`` ``StorageType``.
 
 .. code:: python
 
@@ -480,7 +482,9 @@ Feature Store `DatasetBuilder API Reference
 .. rubric:: Delete a feature group
    :name: bCe9CA61b78
 
-You can delete a feature group with the ``delete`` function.
+You can delete a feature group with the ``delete`` function. Note that it
+can take approximately 10-15 minutes to delete an online ``FeatureGroup``
+with the ``InMemory`` ``StorageType``.
 
 .. code:: python
 

doc/api/inference/deserializers.rst

@@ -1,8 +1,12 @@
 Deserializers
 -------------
 
-.. automodule:: sagemaker.deserializers
+.. automodule:: sagemaker.base_deserializers
     :members:
     :undoc-members:
     :show-inheritance:
 
+.. automodule:: sagemaker.deserializers
+    :members:
+    :undoc-members:
+    :show-inheritance:

doc/api/inference/serializers.rst

@@ -1,8 +1,12 @@
 Serializers
 -----------
 
-.. automodule:: sagemaker.serializers
+.. automodule:: sagemaker.base_serializers
     :members:
     :undoc-members:
     :show-inheritance:
 
+.. automodule:: sagemaker.serializers
+    :members:
+    :undoc-members:
+    :show-inheritance:

doc/api/prep_data/feature_store.rst

@@ -6,7 +6,6 @@ Feature Group
 
 .. autoclass:: sagemaker.feature_store.feature_group.FeatureGroup
     :members:
-    :exclude-members: load_feature_definitions
     :show-inheritance:
 
 .. autoclass:: sagemaker.feature_store.feature_group.AthenaQuery
@@ -41,6 +40,25 @@ Feature Definition
     :members:
     :show-inheritance:
 
+.. autoclass:: sagemaker.feature_store.feature_definition.CollectionTypeEnum
+    :members:
+    :show-inheritance:
+
+.. autoclass:: sagemaker.feature_store.feature_definition.CollectionType
+    :members:
+    :show-inheritance:
+
+.. autoclass:: sagemaker.feature_store.feature_definition.ListCollectionType
+    :members:
+    :show-inheritance:
+
+.. autoclass:: sagemaker.feature_store.feature_definition.SetCollectionType
+    :members:
+    :show-inheritance:
+
+.. autoclass:: sagemaker.feature_store.feature_definition.VectorCollectionType
+    :members:
+    :show-inheritance:
 
 Inputs
 ******
@@ -77,6 +95,10 @@ Inputs
     :members:
     :show-inheritance:
 
+.. autoclass:: sagemaker.feature_store.inputs.OnlineStoreStorageTypeEnum
+    :members:
+    :show-inheritance:
+
 .. autoclass:: sagemaker.feature_store.inputs.ResourceEnum
     :members:
     :show-inheritance:
@@ -120,3 +142,41 @@ Feature Store
 .. autoclass:: sagemaker.feature_store.feature_store.FeatureStore
     :members:
     :show-inheritance:
+
+
+@feature_processor Decorator
+****************************
+
+.. autodecorator:: sagemaker.feature_store.feature_processor.feature_processor
+
+
+Feature Processor Data Source
+*****************************
+
+.. autoclass:: sagemaker.feature_store.feature_processor.FeatureGroupDataSource
+    :members:
+    :show-inheritance:
+
+.. autoclass:: sagemaker.feature_store.feature_processor.CSVDataSource
+    :members:
+    :show-inheritance:
+
+.. autoclass:: sagemaker.feature_store.feature_processor.ParquetDataSource
+    :members:
+    :show-inheritance:
+
+
+Feature Processor Scheduler
+***************************
+
+.. automethod:: sagemaker.feature_store.feature_processor.to_pipeline
+
+.. automethod:: sagemaker.feature_store.feature_processor.schedule
+
+.. automethod:: sagemaker.feature_store.feature_processor.execute
+
+.. automethod:: sagemaker.feature_store.feature_processor.delete_schedule
+
+.. automethod:: sagemaker.feature_store.feature_processor.describe
+
+.. automethod:: sagemaker.feature_store.feature_processor.list_pipelines

doc/conf.py

@@ -109,6 +109,8 @@
 # autosectionlabel
 autosectionlabel_prefix_document = True
 
+autodoc_mock_imports = ["pyspark", "feature_store_pyspark", "py4j"]
+
 
 def setup(app):
     sys.stdout.write("Generating JumpStart model table...")

doc/frameworks/huggingface/sagemaker.huggingface.rst

@@ -32,3 +32,11 @@ Hugging Face Predictor
     :members:
     :undoc-members:
     :show-inheritance:
+
+Hugging Face Processor
+----------------------
+
+.. autoclass:: sagemaker.huggingface.processing.HuggingFaceProcessor
+    :members:
+    :undoc-members:
+    :show-inheritance:

doc/frameworks/mxnet/sagemaker.mxnet.rst

@@ -3,25 +3,33 @@ MXNet Classes
 
 
 MXNet Estimator
----------------------------
+---------------
 
 .. autoclass:: sagemaker.mxnet.estimator.MXNet
     :members:
     :undoc-members:
     :show-inheritance:
 
 MXNet Model
----------------------------
+-----------
 
 .. autoclass:: sagemaker.mxnet.model.MXNetModel
     :members:
     :undoc-members:
     :show-inheritance:
 
 MXNet Predictor
----------------------------
+---------------
 
 .. autoclass:: sagemaker.mxnet.model.MXNetPredictor
     :members:
     :undoc-members:
     :show-inheritance:
+
+MXNet Processor
+---------------
+
+.. autoclass:: sagemaker.mxnet.processing.MXNetProcessor
+    :members:
+    :undoc-members:
+    :show-inheritance:

doc/frameworks/pytorch/sagemaker.pytorch.rst

@@ -24,3 +24,11 @@ PyTorch Predictor
     :members:
     :undoc-members:
     :show-inheritance:
+
+PyTorch Processor
+-----------------
+
+.. autoclass:: sagemaker.pytorch.processing.PyTorchProcessor
+    :members:
+    :undoc-members:
+    :show-inheritance:

doc/frameworks/tensorflow/sagemaker.tensorflow.rst

@@ -33,3 +33,12 @@ TensorFlow Serving Predictor
     :members:
     :undoc-members:
     :show-inheritance:
+
+
+TensorFlow Processor
+--------------------
+
+.. autoclass:: sagemaker.tensorflow.processing.TensorFlowProcessor
+    :members:
+    :undoc-members:
+    :show-inheritance: