fix: Handle instance support for Hugging Face tests #3729

tejaschumbalkar · 2023-03-17T22:30:42Z

Issue #, if available:

Description of changes:

handle instance support for huggingface test

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2023-03-17T22:34:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 58cd44b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T22:36:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 58cd44b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

knikure · 2023-03-17T22:57:31Z

tests/integ/test_huggingface.py

 @pytest.mark.skip(
    reason="need to re enable it later t.corp:V609860141",
 )
 def test_huggingface_training_tf(
    sagemaker_session,
-    gpu_instance_type,
+    gpu_pytorch_instance_type,


This will fail since gpu_pytorch_instance_type fixture is doing validation for PyTorch fw_version check which can't be used for Tensorflow test. Can you write another fixture to get right GPU instance_type ?

Keeping to gpu_instance_type for now, we will revisit the defination once we are ready to support newer TF huggingface images.

tests/integ/test_huggingface.py

sagemaker-bot · 2023-03-17T23:06:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 58cd44b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T23:15:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 206cb13
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

tejaschumbalkar · 2023-03-17T23:18:10Z

tests/integ/test_huggingface.py

-    and integ.test_region() in integ.TRAINING_NO_P3_REGIONS,
-    reason="no ml.p2 or ml.p3 instances in this region",
-)
-@retry_with_instance_list(gpu_list(integ.test_region()))


removed this retry so that it does not pickup p2 instance during retry

sagemaker-bot · 2023-03-17T23:18:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 206cb13
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T23:22:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 06cf7eb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T23:24:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 06cf7eb
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T23:33:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 81976fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-17T23:38:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 81976fe
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-18T00:20:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 81976fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-18T00:23:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 81976fe
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-18T00:26:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 81976fe
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-18T00:45:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 7ada426
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-18T01:20:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 7ada426
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-18T01:32:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 7ada426
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-03-18T01:32:36Z

Codecov Report

Merging #3729 (b82b208) into master (0d7a826) will decrease coverage by 0.78%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3729      +/-   ##
==========================================
- Coverage   89.84%   89.06%   -0.78%     
==========================================
  Files         984      231     -753     
  Lines       92312    22834   -69478     
==========================================
- Hits        82936    20338   -62598     
+ Misses       9376     2496    -6880

see 1215 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sagemaker-bot · 2023-03-18T01:38:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 7ada426
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-18T03:44:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 7ada426
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-20T19:25:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 37fb0bb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…emaker-python-sdk into handle-p2-hf

sagemaker-bot · 2023-03-20T19:37:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 34fae70
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-20T20:10:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 34fae70
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-20T20:24:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 34fae70
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-20T20:30:13Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 34fae70
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-20T22:10:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: b82b208
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-20T22:56:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: b82b208
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-20T22:59:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: b82b208
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-20T23:09:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: b82b208
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-20T23:22:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: b82b208
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-21T00:21:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: b82b208
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-21T00:56:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: b82b208
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-21T01:03:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: b82b208
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-21T01:06:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: b82b208
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-03-21T01:10:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: b82b208
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Co-authored-by: Kevin <[email protected]> Co-authored-by: André Perez <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]>

NivekNey and others added 4 commits March 17, 2023 14:47

fix: type hint of PySparkProcessor __init__ (aws#3297)

427215e

fix: fix PySparkProcessor __init__ params type (aws#3354)

b760181

fix: Revert "fix: type hint of PySparkProcessor __init__" (aws#3524)

bebce21

handle instance support for hf images

58cd44b

tejaschumbalkar requested a review from a team as a code owner March 17, 2023 22:30

tejaschumbalkar requested review from vikas0203 and removed request for a team March 17, 2023 22:30

knikure reviewed Mar 17, 2023

View reviewed changes

remove retry

206cb13

tejaschumbalkar commented Mar 17, 2023

View reviewed changes

run black

06cf7eb

remove unused imports

81976fe

fix pytorch version fixture

7ada426

tejaschumbalkar added 5 commits March 20, 2023 12:23

handle instance support for hf images

bd5d152

remove retry

2d54223

run black

525585a

remove unused imports

68c75d1

fix pytorch version fixture

37fb0bb

knikure force-pushed the handle-p2-hf branch from 7ada426 to 37fb0bb Compare March 20, 2023 19:23

tejaschumbalkar added 2 commits March 20, 2023 12:28

Merge branch 'handle-p2-hf' of https://github.com/tejaschumbalkar/sag…

fdd1690

…emaker-python-sdk into handle-p2-hf

initialize the parameter

34fae70

knikure approved these changes Mar 20, 2023

View reviewed changes

Merge branch 'master' into handle-p2-hf

b82b208

knikure changed the title ~~handle instance support for huggingface test~~ fix: Handle instance support for Hugging Face tests Mar 21, 2023

knikure merged commit 8d282c1 into aws:master Mar 21, 2023

fix: Handle instance support for Hugging Face tests #3729

fix: Handle instance support for Hugging Face tests #3729

Uh oh!

Conversation

tejaschumbalkar commented Mar 17, 2023

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

Uh oh!

knikure Mar 17, 2023

Choose a reason for hiding this comment

Uh oh!

tejaschumbalkar Mar 17, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

Uh oh!

tejaschumbalkar Mar 17, 2023

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 17, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 18, 2023

AWS CodeBuild CI Report

Uh oh!

codecov-commenter commented Mar 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sagemaker-bot commented Mar 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 18, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 20, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 20, 2023

AWS CodeBuild CI Report

Uh oh!

codecov-commenter commented Mar 18, 2023 •

edited

Loading