Merge branch 'master' into shortUri

Author: ahsan-z-khan

CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## v2.23.6 (2021-01-20)
+
+### Bug Fixes and Other Changes
+
+ * add artifact, action, context to virsualizer
+
+## v2.23.5 (2021-01-18)
+
+### Bug Fixes and Other Changes
+
+ * increase time allowed for trial components to index
+
 ## v2.23.4.post0 (2021-01-14)
 
 ### Documentation Changes

VERSION

@@ -1 +1 @@
-2.23.5.dev0
+2.23.7.dev0

doc/api/training/smd_model_parallel_release_notes/smd_model_parallel_change_log.md

@@ -1,3 +1,44 @@
+# Sagemaker Distributed Model Parallel 1.2.0 Release Notes
+
+- New Features
+- Bug Fixes
+- Known Issues
+
+## New Features
+
+### PyTorch
+
+#### Add support for PyTorch 1.7
+
+- Adds support for `gradient_as_bucket_view` (PyTorch 1.7 only), `find_unused_parameters` (PyTorch 1.7 only) and `broadcast_buffers` options to `smp.DistributedModel`. These options behave the same as the corresponding options (with the same names) in
+`torch.DistributedDataParallel` API. Please refer to the [SageMaker distributed model parallel API documentation](https://sagemaker.readthedocs.io/en/stable/api/training/smd_model_parallel_pytorch.html#smp.DistributedModel) for more information.
+
+- Adds support for `join` (PyTorch 1.7 only) context manager, which is to be used in conjunction with an instance of `smp.DistributedModel` to be able to train with uneven inputs across participating processes.
+
+- Adds support for `_register_comm_hook` (PyTorch 1.7 only) which will register the callable as a communication hook for DDP. NOTE: Like in DDP, this is an experimental API and subject to change.
+
+### Tensorflow
+
+- Adds support for Tensorflow 2.4
+
+## Bug Fixes
+
+### PyTorch
+
+- `Serialization`: Fix a bug with serialization/flattening where instances of subclasses of dict/OrderedDicts were serialized/deserialized or internally flattened/unflattened as
+regular dicts.
+
+### Tensorflow
+
+- Fix a bug that may cause a hang during evaluation when there is no model input for one partition.
+
+## Known Issues
+
+### PyTorch
+
+- A performance regression was observed when training on SMP with PyTorch 1.7.1 compared to 1.6. The rootcause was found to be the slowdown in performance of `.grad` method calls in PyTorch 1.7.1 compared to 1.6. Please see the related discussion: https://github.com/pytorch/pytorch/issues/50636.
+
+
 # Sagemaker Distributed Model Parallel 1.1.0 Release Notes
 
 - New Features

src/sagemaker/lineage/visualizer.py

@@ -37,6 +37,9 @@ def show(
         pipeline_execution_step=None,
         model_package_arn=None,
         endpoint_arn=None,
+        artifact_arn=None,
+        context_arn=None,
+        actions_arn=None,
     ):
         """Generate a dataframe containing all incoming and outgoing lineage entities.
 
@@ -55,6 +58,9 @@ def show(
             pipeline_execution_step (obj, optional): Pipeline execution step. Defaults to None.
             model_package_arn (str, optional): Model package arn. Defaults to None.
             endpoint_arn (str, optional): Endpoint arn. Defaults to None.
+            artifact_arn (str, optional): Artifact arn. Defaults to None.
+            context_arn (str, optional): Context arn. Defaults to None.
+            actions_arn (str, optional): Action arn. Defaults to None.
 
         Returns:
             DataFrame: Pandas dataframe containing lineage associations.
@@ -75,6 +81,12 @@ def show(
             start_arn = self._get_start_arn_from_model_package_arn(model_package_arn)
         elif endpoint_arn:
             start_arn = self._get_start_arn_from_endpoint_arn(endpoint_arn)
+        elif artifact_arn:
+            start_arn = artifact_arn
+        elif context_arn:
+            start_arn = context_arn
+        elif actions_arn:
+            start_arn = actions_arn
 
         return self._get_associations_dataframe(start_arn)
 

tests/integ/sagemaker/lineage/test_artifact.py

@@ -78,10 +78,12 @@ def test_list(artifact_objs, sagemaker_session):
 
 
 def test_downstream_trials(trial_associated_artifact, trial_obj, sagemaker_session):
-    # wait for TC to index
-    time.sleep(3)
-
-    trials = trial_associated_artifact.downstream_trials(sagemaker_session=sagemaker_session)
+    # allow trial components to index, 30 seconds max
+    for i in range(3):
+        time.sleep(10)
+        trials = trial_associated_artifact.downstream_trials(sagemaker_session=sagemaker_session)
+        if len(trials) > 0:
+            break
 
     assert len(trials) == 1
     assert trial_obj.trial_name in trials