feature: Add ModelStep for SageMaker Model Building Pipeline #3076

qidewenwhen · 2022-04-22T06:25:31Z

Description of changes: feature: Add ModelStep for SageMaker Model Building Pipeline

Testing done: unit tests and integ tests

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2022-04-22T06:30:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: f33dff7
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T07:00:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: f33dff7
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T07:15:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: f33dff7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T07:18:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: f33dff7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T07:30:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: f33dff7
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T18:52:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: b8abfef
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T19:19:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: b8abfef
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T19:34:22Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: b8abfef
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T19:40:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 8f99dd6
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T19:46:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: b8abfef
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T20:18:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 8f99dd6
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T20:20:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 8f99dd6
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T20:21:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 8f99dd6
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T20:33:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 8f99dd6
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T21:36:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: b8abfef
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T22:43:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: ca06cbe
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2022-04-22T22:56:43Z

Codecov Report

Merging #3076 (bbd589c) into master (92b1d47) will increase coverage by 0.05%.
The diff coverage is 92.27%.

@@            Coverage Diff             @@
##           master    #3076      +/-   ##
==========================================
+ Coverage   89.51%   89.57%   +0.05%     
==========================================
  Files         199      200       +1     
  Lines       17177    17284     +107     
==========================================
+ Hits        15376    15482     +106     
- Misses       1801     1802       +1

Impacted Files	Coverage Δ
src/sagemaker/session.py	`71.03% <78.26%> (+0.16%)`	⬆️
src/sagemaker/tensorflow/model.py	`95.00% <83.33%> (+4.57%)`	⬆️
src/sagemaker/pipeline.py	`84.93% <90.00%> (+1.59%)`	⬆️
src/sagemaker/workflow/model_step.py	`91.56% <91.56%> (ø)`
src/sagemaker/workflow/_utils.py	`96.03% <92.59%> (+0.19%)`	⬆️
src/sagemaker/model.py	`92.58% <100.00%> (+0.68%)`	⬆️
src/sagemaker/workflow/pipeline_context.py	`100.00% <100.00%> (ø)`
src/sagemaker/workflow/step_collections.py	`97.61% <100.00%> (-1.32%)`	⬇️
src/sagemaker/workflow/steps.py	`96.84% <100.00%> (+0.06%)`	⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 92b1d47...bbd589c. Read the comment docs.

sagemaker-bot · 2022-04-22T23:00:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: db65419
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T23:12:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: ca06cbe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T23:22:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ca06cbe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T23:30:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 603f2df
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T23:30:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: ca06cbe
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-04-22T23:39:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 603f2df
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T18:39:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: b68eff6
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T18:50:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: c27aa04
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T18:54:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: c27aa04
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T18:58:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: b68eff6
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/workflow/model_step.py

src/sagemaker/workflow/step_collections.py

tests/integ/sagemaker/workflow/test_model_steps.py

sagemaker-bot · 2022-05-12T19:13:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: c27aa04
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T19:23:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c27aa04
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T20:46:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: b68eff6
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T20:59:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 742abd3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T20:59:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 742abd3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T20:59:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 742abd3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T21:00:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 742abd3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T21:16:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: bbd589c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T21:28:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: bbd589c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

staubhp

Approved pending TODOs on documentation

sagemaker-bot · 2022-05-12T21:43:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: bbd589c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T21:56:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: bbd589c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-12T23:47:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: bbd589c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-05-13T01:20:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: bbd589c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

qidewenwhen force-pushed the model branch from f33dff7 to b8abfef Compare April 22, 2022 18:44

qidewenwhen force-pushed the model branch from b8abfef to 8f99dd6 Compare April 22, 2022 19:31

qidewenwhen force-pushed the model branch from 8f99dd6 to ca06cbe Compare April 22, 2022 22:23

qidewenwhen requested review from jerrypeng7773 and navaj0 April 22, 2022 22:23

qidewenwhen force-pushed the model branch from ca06cbe to db65419 Compare April 22, 2022 22:56

qidewenwhen force-pushed the model branch from db65419 to 603f2df Compare April 22, 2022 23:06

staubhp reviewed May 12, 2022

View reviewed changes

src/sagemaker/workflow/model_step.py Outdated Show resolved Hide resolved

src/sagemaker/workflow/model_step.py Show resolved Hide resolved

src/sagemaker/workflow/step_collections.py Show resolved Hide resolved

tests/integ/sagemaker/workflow/test_model_steps.py Show resolved Hide resolved

qidewenwhen force-pushed the model branch from c27aa04 to 742abd3 Compare May 12, 2022 20:54

Dewen Qi added 3 commits May 12, 2022 13:54

feature: Add ModelStep for SageMaker Model Building Pipeline

df82b0f

change: Update tests

dd91e29

change: Clean code and add retry to integ tests

bbd589c

qidewenwhen force-pushed the model branch from 742abd3 to bbd589c Compare May 12, 2022 20:54

staubhp approved these changes May 12, 2022

View reviewed changes

mufaddal-rohawala approved these changes May 13, 2022

View reviewed changes

jerrypeng7773 mentioned this pull request May 13, 2022

Sagemaker Pipelines CreateModelStep doesn't support tags #2627

Closed

jerrypeng7773 mentioned this pull request May 13, 2022

RegisterModel Step in Sagemaker Pipelines fails with multiple SKLearn models in a PipelineModel #2701

Closed

navinsoni merged commit 8c52f1b into aws:master May 13, 2022

jerrypeng7773 pushed a commit to jerrypeng7773/sagemaker-python-sdk that referenced this pull request May 13, 2022

feature: Add ModelStep for SageMaker Model Building Pipeline (aws#3076)

9014064

jerrypeng7773 pushed a commit to jerrypeng7773/sagemaker-python-sdk that referenced this pull request May 15, 2022

feature: Add ModelStep for SageMaker Model Building Pipeline (aws#3076)

0b61fa4

feature: Add ModelStep for SageMaker Model Building Pipeline #3076

feature: Add ModelStep for SageMaker Model Building Pipeline #3076

Uh oh!

Conversation

qidewenwhen commented Apr 22, 2022

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

codecov-commenter commented Apr 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 22, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 12, 2022

codecov-commenter commented Apr 22, 2022 •

edited

Loading