support tuning step parameter range parameterization + support retry strategy in tuner #2551

jerrypeng7773 · 2021-08-03T18:43:11Z

Issue #, if available:

Description of changes:

support tuning step tuning configuration parameter range parameterization
support retry strategy for estimator used by tuner

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
[] I used the commit message format described in CONTRIBUTING
[] I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…rs to sagemaker.wrangler.ingestion

…step

…strategy in tuner

sagemaker-bot · 2021-08-03T18:54:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: ae55619
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-03T19:20:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ae55619
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-03T19:27:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: ae55619
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-03T19:27:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: ae55619
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

danabens · 2021-08-03T20:14:49Z

src/sagemaker/tuner.py

@@ -1580,6 +1583,9 @@ def _prepare_training_config(
        if parameter_ranges is not None:
            training_config["parameter_ranges"] = parameter_ranges

+        if estimator.max_retry_attempts is not None:
+            training_config["max_retry_attempts"] = estimator.max_retry_attempts


do we already have a standard mechanism to propagate API changes to pipeline steps? i.e. simultaneous release of new API features and pipeline support

from DSL level yes, as long as the model has it, we can support it simultaneously. However, in python SDK, when translating the estimator or tuner to request arguments, one will need to manually add the new fields. For instance, in this case, the estimator supports max_retry_attempts, however when passed to tuner, and during python object to request arguments translations, this is not included.

sagemaker-bot · 2021-08-03T23:23:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: ae55619
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-09T17:46:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5a6148a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-09T18:09:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5a6148a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-09T18:19:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5a6148a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-09T18:24:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 5a6148a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-09T18:28:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 5a6148a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-11T16:28:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 9b1d905
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-11T16:51:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 9b1d905
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-11T16:53:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 9b1d905
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-11T16:54:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 9b1d905
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-11T17:04:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 9b1d905
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

ahsan-z-khan · 2021-08-11T17:37:39Z

tests/integ/test_workflow.py

@@ -1098,14 +1098,17 @@ def test_tuning(
        role=role,
        framework_version="1.5.0",
        py_version="py3",
-        instance_count=1,
-        instance_type="ml.m5.xlarge",
+        instance_count=instance_count,


Where are these coming from? Do we have fixtures for instance_count and instance_type ?

line 1093 and 1094, they are the pipeline parameters

ahsan-z-khan

LGTM!

sagemaker-bot · 2021-08-11T17:51:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 282c9fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-11T18:15:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 282c9fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-11T18:17:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 282c9fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-11T18:23:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 282c9fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-08-11T18:27:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 282c9fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jerrypeng7773 and others added 30 commits May 10, 2021 20:18

add helper function to generate no-op (data ingestion only) recipe

868db55

Merge branch 'aws:master' into master

21bedbb

separate flow generation by source input type + move generation helpe…

854dd10

…rs to sagemaker.wrangler.ingestion

Merge branch 'aws:master' into master

8798b65

create an internal helper function to generate output node

69ae4bd

Merge branch 'master' of github.com:jerrypeng7773/sagemaker-python-sdk

a6a8449

Merge branch 'aws:master' into master

2aa256e

add ingestion test using dw processor via pipeline execution

06557a8

Merge branch 'aws:master' into master

dcbfd13

verify the fg query df

fc6522e

Merge branch 'master' into master

b6f9371

fix tests

86fa47d

Merge branch 'master' into master

05ccfa6

Merge branch 'aws:master' into master

0716e9f

add tuning step support

7ca5af4

fix docstyle check

8cf18b8

add helper function to get tuning step top performing model s3 uri

1f95b82

Merge branch 'aws:master' into master

1b9d66b

allow step depends on pass in step instance

5bc47bd

Merge branch 'aws:master' into master

603b934

Merge branch 'master' of github.com:jerrypeng7773/sagemaker-python-sdk

664f2a8

Merge branch 'master' into master

a8755ec

Merge branch 'aws:master' into master

e25d36c

Merge branch 'master' into accept-step-object-in-dependson-list

a9cfab4

resolve merge conflict

c0066ea

support passing step object to tuning step depends on list

e9ac9fa

fix test_workflow_with_clarify

eb6a523

add tuning step to docs

c19c426

allow step instance in depends on list for repack and reigster model …

450e4a5

…step

Merge branch 'master' into master

cb7be4a

support tuning step parameter range parameterization + support retry …

ae55619

…strategy in tuner

danabens reviewed Aug 3, 2021

View reviewed changes

danabens approved these changes Aug 3, 2021

View reviewed changes

Merge branch 'master' into master

5a6148a

ahsan-z-khan added the component: pipelines Relates to the SageMaker Pipeline Platform label Aug 10, 2021

Merge branch 'master' into master

9b1d905

ahsan-z-khan reviewed Aug 11, 2021

View reviewed changes

ahsan-z-khan approved these changes Aug 11, 2021

View reviewed changes

Merge branch 'master' into master

282c9fe

ahsan-z-khan merged commit 1d84830 into aws:master Aug 11, 2021

support tuning step parameter range parameterization + support retry strategy in tuner #2551

support tuning step parameter range parameterization + support retry strategy in tuner #2551

Uh oh!

Conversation

jerrypeng7773 commented Aug 3, 2021

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Aug 3, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 3, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 3, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 3, 2021

AWS CodeBuild CI Report

Uh oh!

danabens Aug 3, 2021

Choose a reason for hiding this comment

Uh oh!

jerrypeng7773 Aug 3, 2021

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Aug 3, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 9, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 9, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 9, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 9, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 9, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 11, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 11, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 11, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 11, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 11, 2021

AWS CodeBuild CI Report

Uh oh!

ahsan-z-khan Aug 11, 2021

Choose a reason for hiding this comment

Uh oh!

jerrypeng7773 Aug 11, 2021

Choose a reason for hiding this comment

Uh oh!

ahsan-z-khan left a comment

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Aug 11, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 11, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 11, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Aug 11, 2021

AWS CodeBuild CI Report