Merge branch 'master' into upcoming_regions

Author: chuyang-deng

CHANGELOG.md

@@ -1,5 +1,109 @@
 # Changelog
 
+## v2.0.1 (2020-08-05)
+
+### Bug Fixes and Other Changes
+
+ * use pathlib.PurePosixPath for S3 URLs and Unix paths
+ * fix regions for updated RL images
+
+### Documentation Changes
+
+ * update CHANGELOG to reflect v2.0.0 changes
+
+### Testing and Release Infrastructure
+
+ * remove v2-incompatible notebooks from notebook build
+
+## v2.0.0 (2020-08-04)
+
+### Breaking Changes
+
+ * rename s3_input to TrainingInput
+ * Move _NumpyDeserializer to sagemaker.deserializers.NumpyDeserializer
+ * rename numpy_to_record_serializer to RecordSerializer
+ * Move _CsvDeserializer to sagemaker.deserializers and rename to CSVDeserializer
+ * Move _JsonSerializer to sagemaker.serializers.JSONSerializer
+ * Move _NPYSerializer to sagemaker.serializers and rename to NumpySerializer
+ * Move _JsonDeserializer to sagemaker.deserializers.JSONDeserializer
+ * Move _CsvSerializer to sagemaker.serializers.CSVSerializer
+ * preserve script path when S3 source_dir is provided
+ * use image_uris.retrieve() for XGBoost URIs
+ * deprecate sagemaker.amazon.amazon_estimator.get_image_uri()
+ * deprecate fw_registry module and use image_uris.retrieve() for SparkML
+ * deprecate Python SDK CLI
+ * Remove the content_types module
+ * deprecate unused parameters
+ * deprecate fw_utils.create_image_uri()
+ * use images_uris.retrieve() for Debugger
+ * deprecate fw_utils.parse_s3_url in favor of s3.parse_s3_url
+ * deprecate unused functions from utils and fw_utils
+ * Remove content_type and accept parameters from Predictor
+ * Add parameters to deploy and remove parameters from create_model
+ * Add LibSVM serializer for XGBoost predictor
+ * move ShuffleConfig from sagemaker.session to sagemaker.inputs
+ * deprecate get_ecr_image_uri_prefix
+ * rename estimator.train_image() to estimator.training_image_uri()
+ * deprecate is_version_equal_or_higher and is_version_equal_or_lower
+ * default wait=True for HyperparameterTuner.fit() and Transformer.transform()
+ * remove unused bin/sagemaker-submit file
+
+### Features
+
+ * start new module for retrieving prebuilt SageMaker image URIs
+ * handle separate training/inference images and EI in image_uris.retrieve
+ * add support for Amazon algorithms in image_uris.retrieve()
+ * Add pandas deserializer
+ * Remove LegacySerializer and LegacyDeserializer
+ * Add sparse matrix serializer
+ * Add v2 SerDe compatability
+ * Add JSON Lines serializer
+ * add framework upgrade tool
+ * add 1p algorithm image_uris migration tool
+ * Update migration tool to support breaking changes to create_model
+ * support PyTorch 1.6 training
+
+### Bug Fixes and Other Changes
+
+ * handle named variables in v2 migration tool
+ * add modifier for s3_input class
+ * add XGBoost support to image_uris.retrieve()
+ * add MXNet configuration to image_uris.retrieve()
+ * add remaining Amazon algorithms for image_uris.retrieve()
+ * add PyTorch configuration for image_uris.retrieve()
+ * make image_scope optional for some images in image_uris.retrieve()
+ * separate logs() from attach()
+ * use image_uris.retrieve instead of fw_utils.create_image_uri for DLC frameworks
+ * use images_uris.retrieve() for scikit-learn classes
+ * use image_uris.retrieve() for RL images
+ * Rename BaseDeserializer.deserialize data parameter
+ * Add allow_pickle parameter to NumpyDeserializer
+ * Fix scipy.sparse imports
+ * Improve code style of SerDe compatibility
+ * use image_uris.retrieve for Neo and Inferentia images
+ * use generated RL version fixtures and update Ray version
+ * use image_uris.retrieve() for ModelMonitor default image
+ * use _framework_name for 'protected' attribute
+ * Fix JSONLinesDeserializer
+ * upgrade TFS version and fix py_versions KeyError
+ * Fix PandasDeserializer tests to more accurately mock response
+ * don't require instance_type for image_uris.retrieve() if only one option
+ * ignore code cells with shell commands in v2 migration tool
+ * Support multiple Accept types
+
+### Documentation Changes
+
+ * fix pip install command
+ * document name changes for TFS classes
+ * document v2.0.0 changes
+ * update KFP full pipeline
+
+### Testing and Release Infrastructure
+
+ * generate Chainer latest version fixtures from config
+ * use generated TensorFlow version fixtures
+ * use generated MXNet version fixtures
+
 ## v1.72.0 (2020-07-29)
 
 ### Features

README.rst

@@ -1,8 +1,3 @@
-+-------------------------------------------------------------------------------------------------+
-| **NOTE**: We are working on v2.0.0. See https://github.com/aws/sagemaker-python-sdk/issues/1459 |
-| for more info on our plans and to leave feedback!                                               |
-+-------------------------------------------------------------------------------------------------+
-
 .. image:: https://github.com/aws/sagemaker-python-sdk/raw/master/branding/icon/sagemaker-banner.png
     :height: 100px
     :alt: SageMaker

VERSION

@@ -1 +1 @@
-2.0.0.rc1
+2.0.2.dev0

src/sagemaker/image_uri_config/chainer.json

@@ -9,6 +9,7 @@
     "versions": {
         "4.0.0": {
             "registries": {
+                "af-south-1": "313743910680",
                 "ap-east-1": "057415533634",
                 "ap-northeast-1": "520713654638",
                 "ap-northeast-2": "520713654638",
@@ -20,6 +21,7 @@
                 "cn-northwest-1": "423003514399",
                 "eu-central-1": "520713654638",
                 "eu-north-1": "520713654638",
+                "eu-south-1": "048378556238",
                 "eu-west-1": "520713654638",
                 "eu-west-2": "520713654638",
                 "eu-west-3": "520713654638",
@@ -37,6 +39,7 @@
         },
         "4.1.0": {
             "registries": {
+                "af-south-1": "313743910680",
                 "ap-east-1": "057415533634",
                 "ap-northeast-1": "520713654638",
                 "ap-northeast-2": "520713654638",
@@ -48,6 +51,7 @@
                 "cn-northwest-1": "423003514399",
                 "eu-central-1": "520713654638",
                 "eu-north-1": "520713654638",
+                "eu-south-1": "048378556238",
                 "eu-west-1": "520713654638",
                 "eu-west-2": "520713654638",
                 "eu-west-3": "520713654638",
@@ -65,6 +69,7 @@
         },
         "5.0.0": {
             "registries": {
+                "af-south-1": "313743910680",
                 "ap-east-1": "057415533634",
                 "ap-northeast-1": "520713654638",
                 "ap-northeast-2": "520713654638",
@@ -76,6 +81,7 @@
                 "cn-northwest-1": "423003514399",
                 "eu-central-1": "520713654638",
                 "eu-north-1": "520713654638",
+                "eu-south-1": "048378556238",
                 "eu-west-1": "520713654638",
                 "eu-west-2": "520713654638",
                 "eu-west-3": "520713654638",

src/sagemaker/image_uri_config/coach-mxnet.json

@@ -5,6 +5,7 @@
         "0.11": {
             "py_versions": ["py3"],
             "registries": {
+                "af-south-1": "313743910680",
                 "ap-east-1": "057415533634",
                 "ap-northeast-1": "520713654638",
                 "ap-northeast-2": "520713654638",
@@ -16,6 +17,7 @@
                 "cn-northwest-1": "423003514399",
                 "eu-central-1": "520713654638",
                 "eu-north-1": "520713654638",
+                "eu-south-1": "048378556238",
                 "eu-west-1": "520713654638",
                 "eu-west-2": "520713654638",
                 "eu-west-3": "520713654638",
@@ -34,6 +36,7 @@
         "0.11.0": {
             "py_versions": ["py3"],
             "registries": {
+                "af-south-1": "313743910680",
                 "ap-east-1": "057415533634",
                 "ap-northeast-1": "520713654638",
                 "ap-northeast-2": "520713654638",
@@ -45,6 +48,7 @@
                 "cn-northwest-1": "423003514399",
                 "eu-central-1": "520713654638",
                 "eu-north-1": "520713654638",
+                "eu-south-1": "048378556238",
                 "eu-west-1": "520713654638",
                 "eu-west-2": "520713654638",
                 "eu-west-3": "520713654638",

src/sagemaker/image_uri_config/coach-tensorflow.json

@@ -5,6 +5,7 @@
         "0.10": {
             "py_versions": ["py3"],
             "registries": {
+                "af-south-1": "313743910680",
                 "ap-east-1": "057415533634",
                 "ap-northeast-1": "520713654638",
                 "ap-northeast-2": "520713654638",
@@ -16,6 +17,7 @@
                 "cn-northwest-1": "423003514399",
                 "eu-central-1": "520713654638",
                 "eu-north-1": "520713654638",
+                "eu-south-1": "048378556238",
                 "eu-west-1": "520713654638",
                 "eu-west-2": "520713654638",
                 "eu-west-3": "520713654638",
@@ -34,6 +36,7 @@
         "0.10.1": {
             "py_versions": ["py3"],
             "registries": {
+                "af-south-1": "313743910680",
                 "ap-east-1": "057415533634",
                 "ap-northeast-1": "520713654638",
                 "ap-northeast-2": "520713654638",
@@ -45,6 +48,7 @@
                 "cn-northwest-1": "423003514399",
                 "eu-central-1": "520713654638",
                 "eu-north-1": "520713654638",
+                "eu-south-1": "048378556238",
                 "eu-west-1": "520713654638",
                 "eu-west-2": "520713654638",
                 "eu-west-3": "520713654638",
@@ -63,6 +67,7 @@
         "0.11": {
             "py_versions": ["py3"],
             "registries": {
+                "af-south-1": "313743910680",
                 "ap-east-1": "057415533634",
                 "ap-northeast-1": "520713654638",
                 "ap-northeast-2": "520713654638",
@@ -74,6 +79,7 @@
                 "cn-northwest-1": "423003514399",
                 "eu-central-1": "520713654638",
                 "eu-north-1": "520713654638",
+                "eu-south-1": "048378556238",
                 "eu-west-1": "520713654638",
                 "eu-west-2": "520713654638",
                 "eu-west-3": "520713654638",
@@ -92,6 +98,7 @@
         "0.11.0": {
             "py_versions": ["py3"],
             "registries": {
+                "af-south-1": "313743910680",
                 "ap-east-1": "057415533634",
                 "ap-northeast-1": "520713654638",
                 "ap-northeast-2": "520713654638",
@@ -103,6 +110,7 @@
                 "cn-northwest-1": "423003514399",
                 "eu-central-1": "520713654638",
                 "eu-north-1": "520713654638",
+                "eu-south-1": "048378556238",
                 "eu-west-1": "520713654638",
                 "eu-west-2": "520713654638",
                 "eu-west-3": "520713654638",
@@ -121,6 +129,7 @@
         "0.11.1": {
             "py_versions": ["py3"],
             "registries": {
+                "af-south-1": "313743910680",
                 "ap-east-1": "057415533634",
                 "ap-northeast-1": "520713654638",
                 "ap-northeast-2": "520713654638",
@@ -132,6 +141,7 @@
                 "cn-northwest-1": "423003514399",
                 "eu-central-1": "520713654638",
                 "eu-north-1": "520713654638",
+                "eu-south-1": "048378556238",
                 "eu-west-1": "520713654638",
                 "eu-west-2": "520713654638",
                 "eu-west-3": "520713654638",
@@ -157,11 +167,8 @@
                 "ap-southeast-2": "462105765813",
                 "ca-central-1": "462105765813",
                 "eu-central-1": "462105765813",
-                "eu-north-1": "462105765813",
                 "eu-west-1": "462105765813",
                 "eu-west-2": "462105765813",
-                "eu-west-3": "462105765813",
-                "sa-east-1": "462105765813",
                 "us-east-1": "462105765813",
                 "us-east-2": "462105765813",
                 "us-west-1": "462105765813",