feat: jumpstart EULA models #3999

evakravi · 2023-07-13T23:58:12Z

Issue #, if available:

Description of changes:
JumpStart is releasing models that require the customer to accept an end-user license agreement (EULA). These models have protected model weights, so the artifacts cannot be stored in public S3 buckets.

Models that require EULA will rely use a ModelPackage to store the model weights. This paradigm requires modifying the metadata for JumpStart models, as well as modifying the workflow so that if a model package arn is detected, a Model is created from the model package versus the normal method of injecting public s3 artifacts into the container.

custom_attributes field was also added to Predictor class to support EULA models.
Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

akozd · 2023-07-14T00:23:19Z

src/sagemaker/jumpstart/artifacts/model_uris.py

+
+        model_artifact_key = None
+
+        for model_artifact_field_name in _MODEL_ARTIFACT_PRIORITY_LIST:


just from a code legibility standpoint this for-loop block would benefit from being extracted into a helper function

akozd · 2023-07-14T00:28:23Z

src/sagemaker/jumpstart/artifacts/script_uris.py

    if script_scope == JumpStartScriptScope.INFERENCE:
+        script_uri = getattr(model_specs, "hosting_script_uri", None)
+        if script_uri is not None:


Same here, I'm not a fan of the nested ifs. Not a blocker through

sagemaker-bot · 2023-07-14T00:35:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 427c22e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-14T00:47:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 427c22e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-14T00:56:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 427c22e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-14T01:26:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 427c22e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-14T01:32:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 427c22e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-07-14T01:32:51Z

Codecov Report

Merging #3999 (19668b1) into master (53fe9b7) will decrease coverage by 0.77%.
The diff coverage is 61.36%.

❗ Current head 19668b1 differs from pull request most recent head 422e0ba. Consider uploading reports for the commit 422e0ba to get more accurate results

@@            Coverage Diff             @@
##           master    #3999      +/-   ##
==========================================
- Coverage   90.18%   89.42%   -0.77%     
==========================================
  Files        1296      307     -989     
  Lines      114144    28274   -85870     
==========================================
- Hits       102940    25283   -77657     
+ Misses      11204     2991    -8213

Impacted Files	Coverage Δ
src/sagemaker/jumpstart/model.py	`77.41% <23.52%> (ø)`
...rc/sagemaker/jumpstart/artifacts/model_packages.py	`75.00% <75.00%> (ø)`
src/sagemaker/base_predictor.py	`88.54% <100.00%> (ø)`
src/sagemaker/jumpstart/artifacts/__init__.py	`100.00% <100.00%> (ø)`
src/sagemaker/jumpstart/factory/model.py	`97.12% <100.00%> (ø)`
src/sagemaker/jumpstart/types.py	`97.60% <100.00%> (ø)`

... and 1597 files with indirect coverage changes

sagemaker-bot · 2023-07-14T15:33:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 2367d16
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-14T16:04:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 2367d16
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-14T16:10:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5af2b17
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

whittech1 · 2023-07-14T16:19:13Z

src/sagemaker/jumpstart/types.py

@@ -351,6 +351,8 @@ class JumpStartModelSpecs(JumpStartDataHolderType):
        "inference_enable_network_isolation",
        "training_enable_network_isolation",
        "resource_name_base",
+        "eula_model",


Should we pass the eula filepath here? We may eventually need to make this retrievable for customers (otherwise they are just blindly accepting something)

whittech1 · 2023-07-14T16:20:21Z

src/sagemaker/jumpstart/utils.py

@@ -402,6 +402,12 @@ def verify_model_region_and_return_specs(
            f"JumpStart model ID '{model_id}' and version '{version}' " "does not support training."
        )

+    if model_specs.eula_model:
+        LOGGER.info(


See prior comment, we can perhaps give them the ability to retrieve the EULA from our public bucket

sagemaker-bot · 2023-07-14T16:41:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 5af2b17
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-14T16:46:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 5af2b17
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-14T16:58:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5af2b17
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

whittech1 · 2023-07-14T17:21:18Z

src/sagemaker/jumpstart/utils.py

+    if model_specs.eula_model:
+        LOGGER.info(
+            "Using model with end-user license agreement (EULA). "
+            "Deploying this model requires accepting EULA terms."


May want to provide a more exact call to action

sagemaker-bot · 2023-07-14T17:32:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5af2b17
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-14T18:16:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 01ff3e5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-14T18:22:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 01ff3e5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-14T18:33:57Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 01ff3e5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

akrishna1995 · 2023-07-14T23:12:17Z

@evakravi Please remove from draft and ping me when you need this to be reviewed from sagemaker team - Thanks

sagemaker-bot · 2023-07-15T00:24:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 09c05f4
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-15T16:29:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 19668b1
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-15T16:32:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 19668b1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-15T16:42:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 19668b1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-15T17:26:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 19668b1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-15T19:23:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 19668b1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

knikure · 2023-07-15T21:04:31Z

tests/unit/sagemaker/jumpstart/constants.py

@@ -2299,6 +2299,8 @@
    "training_script_key": "source-directory-tarballs/pytorch/transfer_learning/ic/v1.0.0/sourcedir.tar.gz",
    "training_prepacked_script_key": None,
    "hosting_prepacked_artifact_key": None,
+    "hosting_model_package_arns": None,
+    "hosting_eula_key": False,


We will have to change it to None

knikure

Changes looks good to me. Once this change is public we are planning to have a follow-up PR to add integ-test as well as unit tests under jumpstart module.

sagemaker-bot · 2023-07-15T21:40:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 422e0ba
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-15T21:54:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 422e0ba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-15T21:56:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 422e0ba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-15T22:22:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 422e0ba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-15T22:40:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 422e0ba
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-17T13:59:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 422e0ba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-17T13:59:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 422e0ba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-17T14:02:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 422e0ba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-07-17T14:04:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 422e0ba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

JGuinegagne

Could you also add unit tests for model package flow as well?

JGuinegagne · 2023-07-17T15:03:11Z

src/sagemaker/jumpstart/model.py


        super(JumpStartModel, self).__init__(**model_init_kwargs.to_kwargs_dict())

+    def _create_sagemaker_model(self, *args, **kwargs):  # pylint: disable=unused-argument
+        """Create a SageMaker Model Entity


nit^2: missing period . at the end of the docstring.

JGuinegagne · 2023-07-17T15:04:33Z

src/sagemaker/jumpstart/utils.py

+            model_id,
+            get_jumpstart_content_bucket(region=region),
+            region,
+            ".cn" if region.startswith("cn-") else "",


callout: this may not work for gov-cloud.

sagemaker-bot · 2023-07-17T15:35:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 422e0ba
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

evakravi requested a review from a team as a code owner July 13, 2023 23:58

evakravi requested review from vpbhargav and removed request for a team July 13, 2023 23:58

evakravi marked this pull request as draft July 13, 2023 23:58

evakravi changed the title ~~feat: jumpstart EULA models (initial commit)~~ feat: jumpstart EULA models Jul 13, 2023

akozd reviewed Jul 14, 2023

View reviewed changes

feat: jumpstart EULA models (initial commit)

2367d16

evakravi force-pushed the feat/jumpstart-eula-models branch from 427c22e to 2367d16 Compare July 14, 2023 15:27

chore: add support for custom attributes to predictor class

5af2b17

akozd approved these changes Jul 14, 2023

View reviewed changes

whittech1 approved these changes Jul 14, 2023

View reviewed changes

whittech1 reviewed Jul 14, 2023

View reviewed changes

chore: address PR comments, fix failing tests

01ff3e5

whittech1 approved these changes Jul 14, 2023

View reviewed changes

evakravi added 2 commits July 15, 2023 15:52

chore: improve EULA logging message

4913109

Merge remote-tracking branch 'origin' into feat/jumpstart-eula-models

19668b1

evakravi marked this pull request as ready for review July 15, 2023 15:54

knikure reviewed Jul 15, 2023

View reviewed changes

fix: unit test constant

422e0ba

knikure reviewed Jul 15, 2023

View reviewed changes

knikure approved these changes Jul 15, 2023

View reviewed changes

knikure self-assigned this Jul 15, 2023

Captainia approved these changes Jul 15, 2023

View reviewed changes

akozd approved these changes Jul 15, 2023

View reviewed changes

knikure merged commit 0a5b450 into aws:master Jul 15, 2023

JGuinegagne reviewed Jul 17, 2023

View reviewed changes


		model_artifact_key = None

		for model_artifact_field_name in _MODEL_ARTIFACT_PRIORITY_LIST:

feat: jumpstart EULA models #3999

feat: jumpstart EULA models #3999

Uh oh!

Conversation

evakravi commented Jul 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

General

Tests

Uh oh!

akozd Jul 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akozd Jul 14, 2023

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

codecov-commenter commented Jul 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

whittech1 Jul 14, 2023

Choose a reason for hiding this comment

Uh oh!

whittech1 Jul 14, 2023

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

whittech1 Jul 14, 2023

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jul 14, 2023

AWS CodeBuild CI Report

Uh oh!

akrishna1995 commented Jul 14, 2023

Uh oh!

sagemaker-bot commented Jul 15, 2023

AWS CodeBuild CI Report

Uh oh!

evakravi commented Jul 13, 2023 •

edited

Loading

akozd Jul 14, 2023 •

edited

Loading

codecov-commenter commented Jul 14, 2023 •

edited

Loading