feature: Add Neuronx Image uri - Transformers 4.28 - PyTorch 1.13 #3844

JingyaHuang · 2023-05-09T13:19:43Z

Issue #3845:

Description of changes:

Add Image URI for:

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

mufaddal-rohawala

/bot run all

sagemaker-bot · 2023-05-12T00:31:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: a5409a8
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-12T01:17:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: a5409a8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-12T01:21:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: a5409a8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-12T01:49:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: a5409a8
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-12T03:27:22Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: a5409a8
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-05-12T16:34:47Z

@JingyaHuang can you fix unit tests, by locally running - tox -e black-format

knikure

/bot run slow-tests

sagemaker-bot · 2023-05-17T02:34:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: a5409a8
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

knikure · 2023-05-18T00:12:19Z

@JingyaHuang Can you fix the black-check errors by running tox -e black-format command?
Submit your changes after successfully running ./.githooks/pre-push command.

trajanikant

/bot run all

sagemaker-bot · 2023-05-23T19:27:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 93bbc66
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-23T19:44:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 93bbc66
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-23T20:24:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 93bbc66
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-23T20:29:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 93bbc66
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-23T20:51:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 93bbc66
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-23T22:41:22Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 93bbc66
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

JingyaHuang · 2023-05-24T19:47:08Z

Hi folks @trajanikant @knikurem, thanks for reviewing. I did some tentative modifications to the tests, usually we only update image uri. Shall I remove those changes?

mufaddal-rohawala

/bot run all

sagemaker-bot · 2023-05-24T21:42:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 71c20dc
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-05-24T21:48:29Z

@JingyaHuang thanks for adding those tests, we need those. Can you please fix the unit tests to quickly unblock this PR. Just run tox -e black-format and commit those changes.

sagemaker-bot · 2023-05-25T17:30:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 4292f61
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-25T17:36:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 4292f61
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-25T17:46:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 4292f61
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-05-25T17:51:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 4292f61
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-05-25T19:39:00Z

Codecov Report

Merging #3844 (dae9950) into master (1c514bf) will decrease coverage by 0.71%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3844      +/-   ##
==========================================
- Coverage   90.06%   89.35%   -0.71%     
==========================================
  Files        1152      270     -882     
  Lines      106452    26311   -80141     
==========================================
- Hits        95877    23511   -72366     
+ Misses      10575     2800    -7775

Impacted Files	Coverage Δ
src/sagemaker/huggingface/model.py	`89.87% <ø> (ø)`
src/sagemaker/image_uris.py	`98.50% <100.00%> (ø)`

... and 1420 files with indirect coverage changes

sagemaker-bot · 2023-05-25T19:39:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 4292f61
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-05-26T00:28:02Z

@JingyaHuang This time some unit tests are failing, please have a look at the failed logs link in the sagemaker-bot comments above. Slow tests seems to be unrelated to the change, Il retry those.

mufaddal-rohawala

/bot run slow-tests

sagemaker-bot · 2023-05-26T01:24:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 4292f61
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala

/bot run slow-tests

sagemaker-bot · 2023-05-26T17:19:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 4292f61
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

claytonparnell · 2023-05-30T21:17:09Z

@JingyaHuang Hello, the unit test is failing because sagemaker/image_uris::_get_inference_tool is currently returning just neuron, rather than neuronx, so the config for huggingface-neuron is incorrectly getting fetched.

claytonparnell

/bot run slow-tests

claytonparnell · 2023-06-01T19:41:55Z

@JingyaHuang I have addressed the issue in recent commit, please take a look. Main difference - requires users to specify inference_tool="neuronx", as otherwise I didn't see a way to determine which inference tool we want to use in generating image uri. Let me know if there are any concerns with this approach.

sagemaker-bot · 2023-06-01T19:46:22Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 8711836
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/huggingface/model.py

sagemaker-bot · 2023-06-01T20:36:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: dae9950
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-01T20:42:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: dae9950
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-01T20:57:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: dae9950
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-01T21:05:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: dae9950
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-01T22:50:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: dae9950
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-02T04:22:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: dae9950
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

JingyaHuang requested a review from a team as a code owner May 9, 2023 13:19

JingyaHuang requested review from mufaddal-rohawala and removed request for a team May 9, 2023 13:19

knikure linked an issue May 9, 2023 that may be closed by this pull request

[Feature - Hugging Face] Add Neuronx DLCs to Image config - Transformers 4.28 - PyTorch 1.13 - Neuron SDK 2.9 #3845

Closed

2 tasks

knikure assigned mufaddal-rohawala May 9, 2023

mufaddal-rohawala approved these changes May 12, 2023

View reviewed changes

mufaddal-rohawala reviewed May 12, 2023

View reviewed changes

knikure reviewed May 16, 2023

View reviewed changes

trajanikant force-pushed the update-hf-neuronx-dlcs branch from a5409a8 to 93bbc66 Compare May 23, 2023 19:21

trajanikant reviewed May 23, 2023

View reviewed changes

trajanikant force-pushed the update-hf-neuronx-dlcs branch from 93bbc66 to 71c20dc Compare May 24, 2023 16:46

mufaddal-rohawala approved these changes May 24, 2023

View reviewed changes

mufaddal-rohawala reviewed May 24, 2023

View reviewed changes

mufaddal-rohawala reviewed May 26, 2023

View reviewed changes

knikure changed the title ~~[Feature - Hugging Face] Add Neuronx Image uri - Transformers 4.28 - PyTorch 1.13~~ feature: Add Neuronx Image uri - Transformers 4.28 - PyTorch 1.13 Jun 1, 2023

JingyaHuang and others added 3 commits June 1, 2023 12:36

add neuronx uri

93d10b8

fix style

c753997

Pass inference_tool for generating neuron/x image_uris

8711836

claytonparnell force-pushed the update-hf-neuronx-dlcs branch from 7015227 to 8711836 Compare June 1, 2023 19:36

claytonparnell reviewed Jun 1, 2023

View reviewed changes

Formatting...

dae9950

mufaddal-rohawala reviewed Jun 1, 2023

View reviewed changes

src/sagemaker/huggingface/model.py Show resolved Hide resolved

knikure merged commit 57f3bb9 into aws:master Jun 2, 2023

JingyaHuang deleted the update-hf-neuronx-dlcs branch June 2, 2023 08:24

feature: Add Neuronx Image uri - Transformers 4.28 - PyTorch 1.13 #3844

feature: Add Neuronx Image uri - Transformers 4.28 - PyTorch 1.13 #3844

Uh oh!

Conversation

JingyaHuang commented May 9, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

General

Tests

Uh oh!

mufaddal-rohawala left a comment

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented May 12, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 12, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 12, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 12, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 12, 2023

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented May 12, 2023

Uh oh!

knikure left a comment

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented May 17, 2023

AWS CodeBuild CI Report

Uh oh!

knikure commented May 18, 2023

Uh oh!

trajanikant left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented May 23, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 23, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 23, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 23, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 23, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 23, 2023

AWS CodeBuild CI Report

Uh oh!

JingyaHuang commented May 24, 2023

Uh oh!

mufaddal-rohawala left a comment • edited by knikure Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented May 24, 2023

AWS CodeBuild CI Report

Uh oh!

mufaddal-rohawala commented May 24, 2023

Uh oh!

sagemaker-bot commented May 25, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 25, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented May 25, 2023

AWS CodeBuild CI Report

Uh oh!

JingyaHuang commented May 9, 2023 •

edited

Loading

trajanikant left a comment •

edited

Loading

mufaddal-rohawala left a comment •

edited by knikure

Loading

codecov-commenter commented May 25, 2023 •

edited

Loading

claytonparnell left a comment •

edited

Loading