Tune (local mode) support for Jumpstart Models #4532

makungaj1 · 2024-03-25T15:13:45Z

Issue #, if available:

Description of changes:
Currently, ModelBuilder provides tune capability only for HF models. This PR extends tune capability to JumpStart Models as well.

To leverage this feature, users bring in JumpStart model id, and create model with ModelBuilder.

model_builder = ModelBuilder(
    model=jumpstart_model_id,
    schema_builder=SchemaBuilder(sample_input, sample_output),
    model_path=m_path,
    mode=Mode.LOCAL_CONTAINER
)

model = model_builder.build()

model.tune()

This feature is only for Local_Mode.

Testing done:

Unit tests
Notebook (InProgress)
Integration tests (InProgress)

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

[x ] I have read the CONTRIBUTING doc
[x ] I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
[x ] I used the commit message format described in CONTRIBUTING
[x ] I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
[ x] I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

[x ] I have added tests that prove my fix is effective or that my feature works (if appropriate)
[x ] I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
[x ] I have checked that my tests are not configured for a specific region or account (if appropriate)
[x ] I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

This reverts commit 4be1679.

This reverts commit 1ec29e3.

Tune Support for JumpStart Models.

codecov · 2024-03-25T15:37:33Z

Codecov Report

Attention: Patch coverage is 62.72727% with 41 lines in your changes are missing coverage. Please review.

Project coverage is 87.44%. Comparing base (cc5d0dc) to head (d7c85e7).

Files	Patch %	Lines
src/sagemaker/serve/utils/tuning.py	36.36%	21 Missing ⚠️
src/sagemaker/serve/builder/jumpstart_builder.py	71.01%	20 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4532      +/-   ##
==========================================
+ Coverage   87.38%   87.44%   +0.06%     
==========================================
  Files         389      389              
  Lines       36776    36881     +105     
==========================================
+ Hits        32135    32249     +114     
+ Misses       4641     4632       -9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mufaddal-rohawala · 2024-03-25T16:25:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: d0ea680
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/serve/builder/jumpstart_builder.py

src/sagemaker/serve/utils/tuning.py

src/sagemaker/serve/builder/jumpstart_builder.py

mufaddal-rohawala · 2024-03-25T18:19:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: d0ea680
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/serve/builder/jumpstart_builder.py

src/sagemaker/serve/utils/tuning.py

tests/integ/sagemaker/serve/constants.py

Refactoring Refactoring Refactoring

mufaddal-rohawala · 2024-03-27T11:56:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: aa3a351
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/serve/utils/tuning.py

mufaddal-rohawala · 2024-03-27T18:48:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 266fe2b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

src/sagemaker/serve/utils/tuning.py

sagemaker-bot · 2024-03-27T22:15:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 9b18684
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2024-03-27T23:14:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 3fe54c0
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2024-03-28T17:25:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: e63de84
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2024-03-28T18:45:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: da2c0a2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2024-03-28T20:01:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 921f7d4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2024-03-28T21:24:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: d7c85e7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

zhaoqizqwang · 2024-03-28T21:42:18Z

Overriding codecov/patch since coverage is affected by logger.info & logger.warning lines

* JumpStart TGI tune support * TGI does not enable multiple model serving internal to the container * Tune support for JS model with DJL DLC * Pretty Print JS TGI Benchmark results * Pretty Print JS DJL Benchmark results * Debugging * Refactoring * Revert "Refactoring" This reverts commit 4be1679. * refactoring * Revert "Debugging" This reverts commit 1ec29e3. * Add Tests * Refactoring * Refactoring * Debugging * Refactoring and Debugging * Refactoring * Refactoring Refactoring Refactoring Refactoring * Refactoring * Add unit tests * Address PR Review Comments * Sharded is not supported for all model types * Refactoring * Refactoring * Refactoring * Refactoring * Refactoring * Addressed PR Review comments * Sharding support validation for TGI * Fix unit tests * Increase code coverage --------- Co-authored-by: Jonathan Makunga <[email protected]>

Jonathan Makunga and others added 19 commits March 18, 2024 14:33

JumpStart TGI tune support

fc17719

TGI does not enable multiple model serving internal to the container

e4a042a

Merge branch 'master' into js_tune

01eb282

Tune support for JS model with DJL DLC

a2be460

Pretty Print JS TGI Benchmark results

332b2ec

Pretty Print JS DJL Benchmark results

4d06e66

Debugging

1ec29e3

Refactoring

4be1679

Revert "Refactoring"

62d00b3

This reverts commit 4be1679.

refactoring

2952dd3

Revert "Debugging"

b0c0d16

This reverts commit 1ec29e3.

Add Tests

94a2bbf

Refactoring

6b7f5c1

Refactoring

223b73a

Debugging

b96e78f

Refactoring and Debugging

e08e1c0

Refactoring

15b2890

Merge branch 'master' into js_tune

c8289ff

Merge pull request #2 from makungaj1/js_tune

d0ea680

Tune Support for JumpStart Models.

makungaj1 requested a review from a team as a code owner March 25, 2024 15:13

makungaj1 requested review from akrishna1995 and removed request for a team March 25, 2024 15:13

samruds suggested changes Mar 25, 2024

View reviewed changes

gwang111 reviewed Mar 25, 2024

View reviewed changes

src/sagemaker/serve/builder/jumpstart_builder.py Outdated Show resolved Hide resolved

gwang111 reviewed Mar 25, 2024

View reviewed changes

src/sagemaker/serve/builder/jumpstart_builder.py Show resolved Hide resolved

jiapinw suggested changes Mar 25, 2024

View reviewed changes

Refactoring

1b4c864

Refactoring Refactoring Refactoring

Refactoring

aa3a351

Refactoring

266fe2b

gwang111 reviewed Mar 27, 2024

View reviewed changes

src/sagemaker/serve/utils/tuning.py Outdated Show resolved Hide resolved

gwang111 reviewed Mar 27, 2024

View reviewed changes

src/sagemaker/serve/utils/tuning.py Outdated Show resolved Hide resolved

gwang111 reviewed Mar 27, 2024

View reviewed changes

src/sagemaker/serve/utils/tuning.py Show resolved Hide resolved

samruds suggested changes Mar 27, 2024

View reviewed changes

src/sagemaker/serve/utils/tuning.py Outdated Show resolved Hide resolved

src/sagemaker/serve/utils/tuning.py Show resolved Hide resolved

src/sagemaker/serve/utils/tuning.py Outdated Show resolved Hide resolved

src/sagemaker/serve/utils/tuning.py Show resolved Hide resolved

Refactoring

9b18684

Merge branch 'master' into master

3fe54c0

samruds approved these changes Mar 28, 2024

View reviewed changes

Jonathan Makunga added 2 commits March 28, 2024 09:16

Refactoring

c615842

Addressed PR Review comments

e63de84

Sharding support validation for TGI

da2c0a2

samruds approved these changes Mar 28, 2024

View reviewed changes

gwang111 approved these changes Mar 28, 2024

View reviewed changes

Fix unit tests

921f7d4

Increase code coverage

d7c85e7

zhaoqizqwang self-requested a review March 28, 2024 21:43

zhaoqizqwang approved these changes Mar 28, 2024

View reviewed changes

zhaoqizqwang merged commit 951d2c4 into aws:master Mar 28, 2024

Tune (local mode) support for Jumpstart Models #4532

Tune (local mode) support for Jumpstart Models #4532

Uh oh!

Conversation

makungaj1 commented Mar 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

General

Tests

Uh oh!

codecov bot commented Mar 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mufaddal-rohawala commented Mar 25, 2024

AWS CodeBuild CI Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mufaddal-rohawala commented Mar 25, 2024

AWS CodeBuild CI Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mufaddal-rohawala commented Mar 27, 2024

AWS CodeBuild CI Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mufaddal-rohawala commented Mar 27, 2024

AWS CodeBuild CI Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sagemaker-bot commented Mar 27, 2024

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 27, 2024

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 28, 2024

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 28, 2024

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 28, 2024

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 28, 2024

AWS CodeBuild CI Report

Uh oh!

zhaoqizqwang commented Mar 28, 2024

Uh oh!

Uh oh!

makungaj1 commented Mar 25, 2024 •

edited

Loading

codecov bot commented Mar 25, 2024 •

edited

Loading