change: Upgrade DJL deepspeed versions #3921

lanking520 · 2023-06-09T15:18:19Z

Description of changes:

Update the container image of DLC.

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

knikure

/bot run all

sagemaker-bot · 2023-06-09T18:21:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: faec86a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-09T18:34:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: faec86a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-09T18:38:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: faec86a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-09T19:06:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: faec86a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-06-09T19:10:23Z

Codecov Report

Merging #3921 (3224417) into master (c1b2465) will decrease coverage by 0.93%.
The diff coverage is n/a.

❗ Current head 3224417 differs from pull request most recent head 40a9ef7. Consider uploading reports for the commit 40a9ef7 to get more accurate results

@@            Coverage Diff             @@
##           master    #3921      +/-   ##
==========================================
- Coverage   90.30%   89.38%   -0.93%     
==========================================
  Files        1292      271    -1021     
  Lines      112984    26424   -86560     
==========================================
- Hits       102031    23618   -78413     
+ Misses      10953     2806    -8147

see 1563 files with indirect coverage changes

sagemaker-bot · 2023-06-09T22:47:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: faec86a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

knikure

/bot run all

sagemaker-bot · 2023-06-13T16:38:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: f794a48
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-13T16:39:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: f794a48
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-13T17:12:10Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: f794a48
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-13T17:43:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: f794a48
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-13T19:32:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: f794a48
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

davidthomas426 · 2023-06-18T00:22:06Z

@lanking520 Looking at the logs, looks like a change is needed here too: https://github.com/aws/sagemaker-python-sdk/blob/master/tests/unit/sagemaker/image_uris/test_huggingface_llm.py#L52

davidthomas426 · 2023-06-18T00:22:56Z

The notebook test check failures are unrelated to this change:

/tensorflow_serving_pretrained_model_elastic_inference.ipynb: TIMED_OUT (PT7M13.603S) (no stdout was captured)
...
./amazon-sagemaker-examples/sagemaker-python-sdk/mxnet_onnx_export/mxnet_onnx_export.ipynb: TIMED_OUT (PT11M38.586S)

lanking520 · 2023-06-19T17:28:20Z

Just rebase the master. @knikure could we merge this PR since it only introduce small changes?

sagemaker-bot · 2023-06-19T18:24:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 3224417
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-19T18:37:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 3224417
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-19T18:40:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 3224417
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-19T19:12:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 3224417
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

knikure · 2023-06-19T20:20:39Z

@lanking520 Can you fix the failing unit-test?
tests/unit/sagemaker/image_uris/test_huggingface_llm.py:78: AssertionError

davidthomas426 · 2023-06-19T20:47:07Z

@lanking520 Looking at the logs, looks like a change is needed here too: https://github.com/aws/sagemaker-python-sdk/blob/master/tests/unit/sagemaker/image_uris/test_huggingface_llm.py#L52

Not sure if you saw this before, but this unit test is failing at this point.

sagemaker-bot · 2023-06-19T21:20:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 3224417
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

lanking520 · 2023-06-28T15:30:52Z

@knikure just rebased the changes, let's see

sagemaker-bot · 2023-06-28T17:03:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 40a9ef7
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-28T17:18:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 40a9ef7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-28T17:25:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 40a9ef7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-28T17:30:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 40a9ef7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-06-28T18:00:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 40a9ef7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

davidthomas426 · 2023-06-29T01:25:57Z

The unit tests succeeded this time.

The notebook test failures are unrelated to this change:

TIMED_OUT	./amazon-sagemaker-examples/sagemaker-python-sdk/mxnet_onnx_export/mxnet_onnx_export.ipynb	PT4M32.953S
...
TIMED_OUT	./amazon-sagemaker-examples/sagemaker-python-sdk/1P_kmeans_highlevel/kmeans_mnist.ipynb	PT9M28.25S
...
TIMED_OUT	./amazon-sagemaker-examples/sagemaker-python-sdk/scikit_learn_randomforest/Sklearn_on_SageMaker_end2end.ipynb	PT12M18.844S

lanking520 requested a review from a team as a code owner June 9, 2023 15:18

lanking520 requested review from jgoyani1 and removed request for a team June 9, 2023 15:18

frankfliu approved these changes Jun 9, 2023

View reviewed changes

knikure changed the title ~~upgrade deepspeed versions~~ change: Upgrade deepspeed versions Jun 9, 2023

knikure force-pushed the upgrade branch from 71b9265 to faec86a Compare June 9, 2023 17:44

knikure reviewed Jun 9, 2023

View reviewed changes

siddvenk approved these changes Jun 12, 2023

View reviewed changes

lanking520 force-pushed the upgrade branch from faec86a to 594716e Compare June 12, 2023 20:48

frankfliu approved these changes Jun 12, 2023

View reviewed changes

knikure self-assigned this Jun 13, 2023

knikure force-pushed the upgrade branch from 594716e to f794a48 Compare June 13, 2023 15:50

knikure changed the title ~~change: Upgrade deepspeed versions~~ change: Upgrade DJL deepspeed versions Jun 13, 2023

knikure reviewed Jun 13, 2023

View reviewed changes

knikure approved these changes Jun 19, 2023

View reviewed changes

upgrade deepspeed versions

40a9ef7

lanking520 force-pushed the upgrade branch from 3224417 to 40a9ef7 Compare June 28, 2023 15:30

frankfliu approved these changes Jun 28, 2023

View reviewed changes

knikure merged commit a8b6b1d into aws:master Jul 6, 2023

change: Upgrade DJL deepspeed versions #3921

change: Upgrade DJL deepspeed versions #3921

Uh oh!

Conversation

lanking520 commented Jun 9, 2023

Merge Checklist

General

Tests

Uh oh!

knikure left a comment

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Jun 9, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 9, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 9, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 9, 2023

AWS CodeBuild CI Report

Uh oh!

codecov-commenter commented Jun 9, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sagemaker-bot commented Jun 9, 2023

AWS CodeBuild CI Report

Uh oh!

knikure left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Jun 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 13, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 13, 2023

AWS CodeBuild CI Report

Uh oh!

davidthomas426 commented Jun 18, 2023

Uh oh!

davidthomas426 commented Jun 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lanking520 commented Jun 19, 2023

Uh oh!

sagemaker-bot commented Jun 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jun 19, 2023

AWS CodeBuild CI Report

Uh oh!

knikure commented Jun 19, 2023

Uh oh!

davidthomas426 commented Jun 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sagemaker-bot commented Jun 19, 2023

AWS CodeBuild CI Report

Uh oh!

lanking520 commented Jun 28, 2023

Uh oh!

sagemaker-bot commented Jun 28, 2023

codecov-commenter commented Jun 9, 2023 •

edited

Loading

knikure left a comment •

edited

Loading

davidthomas426 commented Jun 18, 2023 •

edited

Loading

davidthomas426 commented Jun 19, 2023 •

edited

Loading