feature: add support for Std:Join for pipelines #2103

metrizable · 2021-01-22T17:48:17Z

Issue #, if available:

Description of changes:

Changes include:

Support for SageMaker Pipelines DSL function Std:Join

Testing done:

unit:

❯ tox --parallel 3 -- -rfE --disable-warnings tests/unit
✔ OK black-format in 14.852 seconds
✔ OK flake8 in 33.196 seconds
✔ OK pylint in 45.187 seconds
✔ OK docstyle in 31.303 seconds
✔ OK twine in 9.889 seconds
✔ OK doc8 in 13.097 seconds
✔ OK sphinx in 56.37 seconds
✔ OK py36 in 6 minutes, 7.25 seconds
✔ OK py38 in 5 minutes, 38.155 seconds
✔ OK py37 in 6 minutes, 14.6 seconds

integ:

❯ IGNORE_COVERAGE="#" tox -e py38 -- -rfE --no-header --disable-warnings --cov-report= -s tests/integ/test_workflow.py::test_three_step_definition
============================================================== test session starts ==============================================================
collected 1 item

tests/integ/test_workflow.py .



============================================================== 1 passed in 10.56s ===============================================================
____________________________________________________________________ summary ____________________________________________________________________
  py38: commands succeeded
  congratulations :)
  12.40s user 0.76s system 77% cpu 17.027 total

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2021-01-22T17:57:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 9e626f2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/workflow/functions.py

tests/unit/sagemaker/workflow/test_functions.py

sagemaker-bot · 2021-01-22T18:11:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: fbdac3f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T18:11:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 641f940
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

owen-t

function
integ test
unit test

LGTM

sagemaker-bot · 2021-01-22T18:28:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 9e626f2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T18:31:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 9e626f2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T18:40:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: fbdac3f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T18:44:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 641f940
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T18:45:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 641f940
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T18:46:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: fbdac3f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T18:49:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: fbdac3f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

tests/integ/test_workflow.py

* Map image name to image uri * fix bug in test

sagemaker-bot · 2021-01-22T21:12:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: fc83d96
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T21:12:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5bc5d16
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T21:43:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: fc83d96
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T21:46:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 5bc5d16
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T21:47:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5bc5d16
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T21:48:10Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: fc83d96
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-01-22T22:00:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: fc83d96
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

metrizable added 2 commits January 22, 2021 09:33

feature: add support for Std:Join for pipelines

449684e

Merge branch 'master' into std-join

9e626f2

metrizable commented Jan 22, 2021

View reviewed changes

src/sagemaker/workflow/functions.py Outdated Show resolved Hide resolved

tests/unit/sagemaker/workflow/test_functions.py Outdated Show resolved Hide resolved

metrizable added 2 commits January 22, 2021 10:02

Update src/sagemaker/workflow/functions.py

641f940

Update tests/unit/sagemaker/workflow/test_functions.py

fbdac3f

owen-t previously approved these changes Jan 22, 2021

View reviewed changes

navaj0 reviewed Jan 22, 2021

View reviewed changes

tests/integ/test_workflow.py Show resolved Hide resolved

navaj0 previously approved these changes Jan 22, 2021

View reviewed changes

metrizable and others added 2 commits January 22, 2021 13:02

fix: ensure region is specified for workflow client

ccca21f

feature: Map image name to image uri (aws#2100)

5bc5d16

* Map image name to image uri * fix bug in test

metrizable dismissed stale reviews from navaj0 and owen-t via 5bc5d16 January 22, 2021 21:03

Merge branch 'master' into std-join

fc83d96

ahsan-z-khan approved these changes Jan 22, 2021

View reviewed changes

ahsan-z-khan merged commit 9211562 into aws:master Jan 22, 2021

feature: add support for Std:Join for pipelines #2103

feature: add support for Std:Join for pipelines #2103

Uh oh!

Conversation

metrizable commented Jan 22, 2021

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

Uh oh!

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

owen-t left a comment

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 22, 2021

AWS CodeBuild CI Report

Uh oh!

Uh oh!