feature: SDK changes for TRCOMP support #3714

BruceChang9688 · 2023-03-13T18:24:49Z

Issue #, if available:

Description of changes:

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2023-03-13T18:29:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 1c68699
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T18:29:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 1c68699
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T18:31:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 1c68699
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T18:31:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 1c68699
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T18:36:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 668064b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T18:47:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 2083bac
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T19:43:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: c1955db
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T19:46:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: c1955db
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-03-13T19:46:28Z

Codecov Report

Merging #3714 (5b2daa4) into master (1b4892a) will decrease coverage by 0.77%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3714      +/-   ##
==========================================
- Coverage   89.84%   89.07%   -0.77%     
==========================================
  Files         984      231     -753     
  Lines       92292    22834   -69458     
==========================================
- Hits        82917    20339   -62578     
+ Misses       9375     2495    -6880

Impacted Files	Coverage Δ
.../sagemaker/huggingface/training_compiler/config.py	`91.30% <100.00%> (ø)`
...c/sagemaker/tensorflow/training_compiler/config.py	`96.00% <100.00%> (ø)`

... and 1213 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sagemaker-bot · 2023-03-13T19:48:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: c1955db
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T19:50:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: c1955db
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T21:24:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 50f4e14
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T21:30:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 50f4e14
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T21:36:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 50f4e14
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T21:50:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 50f4e14
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-13T23:42:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 50f4e14
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

navinsoni

@BruceChang9688 Can you please add unit tests for this change.

src/sagemaker/tensorflow/training_compiler/config.py

src/sagemaker/huggingface/training_compiler/config.py

src/sagemaker/tensorflow/training_compiler/config.py

…ker-python-sdk into trcomp-sdk

sagemaker-bot · 2023-03-14T17:26:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: df4bbdc
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-14T18:19:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 32b9c79
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-14T18:20:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 32b9c79
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-14T18:23:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 32b9c79
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-14T18:26:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 32b9c79
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-14T18:28:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 32b9c79
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

BruceChang9688 · 2023-03-14T21:00:10Z

@BruceChang9688 Can you please add unit tests for this change.

Hi @navinsoni I have added unit test for both min and max support version check

sagemaker-bot · 2023-03-14T21:49:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5b2daa4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-14T21:51:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 5b2daa4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-14T21:54:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5b2daa4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-14T21:57:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 5b2daa4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-14T23:52:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5b2daa4
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

navinsoni · 2023-03-16T18:19:04Z

src/sagemaker/huggingface/training_compiler/config.py

+                error_helper_string = (
+                    "SageMaker Training Compiler is only supported "
+                    "with HuggingFace PyTorch 1.9-1.11. "
+                    "Received pytorch_version={} which is unsupported."
+                )
+                raise ValueError(error_helper_string.format(estimator.pytorch_version))


nit: can you please variables same as in tensorflow for max and min version.
Otherwise LGTM

Thanks for the comment. Since we won't change this version support in the future for HuggingFace training compiler estimator support which has been restrict to 1.9-1.11, it is not a big deal for us to variables this now.

navinsoni

/bot run slow-tests

sagemaker-bot · 2023-03-16T19:22:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5b2daa4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

SDK changes for TRCOMP support

1c68699

BruceChang9688 requested a review from a team as a code owner March 13, 2023 18:24

BruceChang9688 requested review from navinsoni and removed request for a team March 13, 2023 18:24

fix syntax error

668064b

fix format issue

2083bac

fix pylint issue

c1955db

Merge branch 'master' into trcomp-sdk

50f4e14

knikure assigned navinsoni Mar 13, 2023

navinsoni reviewed Mar 14, 2023

View reviewed changes

src/sagemaker/tensorflow/training_compiler/config.py Outdated Show resolved Hide resolved

Lokiiiiii suggested changes Mar 14, 2023

View reviewed changes

src/sagemaker/huggingface/training_compiler/config.py Outdated Show resolved Hide resolved

src/sagemaker/huggingface/training_compiler/config.py Outdated Show resolved Hide resolved

src/sagemaker/tensorflow/training_compiler/config.py Outdated Show resolved Hide resolved

BruceChang9688 added 3 commits March 14, 2023 17:20

edit frameversion comment and specifier set function

071303c

Merge branch 'trcomp-sdk' of https://github.com/BruceChang9688/sagema…

3c0e511

…ker-python-sdk into trcomp-sdk

Merge branch 'master' into trcomp-sdk

df4bbdc

fix format issue

32b9c79

edit unit test for trcomp support version coverage

5b2daa4

Lokiiiiii approved these changes Mar 15, 2023

View reviewed changes

navinsoni reviewed Mar 16, 2023

View reviewed changes

navinsoni approved these changes Mar 16, 2023

View reviewed changes

navinsoni reviewed Mar 16, 2023

View reviewed changes

navinsoni merged commit 0d7a826 into aws:master Mar 16, 2023

nmadan pushed a commit to nmadan/sagemaker-python-sdk that referenced this pull request Apr 18, 2023

feature: SDK changes for TRCOMP support (aws#3714)

537c1a4

feature: SDK changes for TRCOMP support #3714

feature: SDK changes for TRCOMP support #3714

Uh oh!

Conversation

BruceChang9688 commented Mar 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

codecov-commenter commented Mar 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 13, 2023

AWS CodeBuild CI Report

Uh oh!

navinsoni left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sagemaker-bot commented Mar 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 14, 2023

AWS CodeBuild CI Report

BruceChang9688 commented Mar 13, 2023 •

edited

Loading

codecov-commenter commented Mar 13, 2023 •

edited

Loading