feat: allow configuring docker container in local mode #4153

trungleduc · 2023-10-03T16:55:16Z

Issue #, if available:

closes Add possibility of opening additional ports for Local Mode #4025
closes Increase shared memory for local mode #3331
closes Can't supply config to LocalSagemakerRuntimeClient via LocalSession #2482
closes Ability to expose additional ports from the container in Local mode #2729
Description of changes:
Add schema to validate the local mode config file
Add a new keyword: container_config to the local mode config file, this allows users to define configs for the container created by local mode.
Add validation to the LocalSession.config attribute.
Update documentation for the local mode

Testing done:

Unit tests added

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2023-10-03T17:57:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: ace8240
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-03T18:05:43Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: ace8240
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-03T18:36:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: ace8240
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-03T18:50:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: ace8240
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-03T18:51:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: ace8240
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-03T20:58:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: e6a8d85
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-03T21:08:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e6a8d85
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-03T21:27:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: e6a8d85
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-03T21:54:57Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e6a8d85
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-04T04:07:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: e6a8d85
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-04T11:22:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 40dba74
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-04T12:24:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 1f86451
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-04T12:25:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 1f86451
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-04T12:28:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 1f86451
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-04T12:51:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 1f86451
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-04T13:17:39Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 1f86451
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-04T20:34:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 891e8eb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-04T20:40:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 891e8eb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-04T21:30:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 891e8eb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-04T21:42:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 891e8eb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-04T22:22:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 891e8eb
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-05T20:28:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 0f8df75
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-05T21:05:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 0f8df75
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-06T13:21:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: c71eebb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-06T13:26:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: c71eebb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-06T13:35:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: c71eebb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-06T14:07:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: c71eebb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-09T14:53:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 328a9ad
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-09T15:05:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 328a9ad
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-09T15:39:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 328a9ad
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-09T15:51:54Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 328a9ad
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-09T16:02:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 328a9ad
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-09T19:58:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 33a1803
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-09T21:04:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 7dcc1f6
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2023-10-09T21:27:56Z

src/sagemaker/config/config_schema.py

+            },
+        },
+    },
+    "required": [LOCAL],


What does required mean here? the usage of defaults config is still optional correct?

I added it to prevent creating an empty config.yaml file or passing an empty dict to the local session config. This config is optional as always.

awaiting response on comments

sagemaker-bot · 2023-10-09T22:06:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 9233947
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-09T22:17:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 9233947
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-09T22:22:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 9233947
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-09T22:26:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 9233947
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-10-09T23:03:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 9233947
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

trungleduc force-pushed the docker-host-config branch 2 times, most recently from 43f24e8 to ace8240 Compare October 3, 2023 17:02

trungleduc changed the title ~~Allow configuring docker container in local mode~~ feat: allow configuring docker container in local mode Oct 3, 2023

trungleduc force-pushed the docker-host-config branch from ace8240 to e6a8d85 Compare October 3, 2023 20:05

trungleduc force-pushed the docker-host-config branch from e6a8d85 to 40dba74 Compare October 4, 2023 11:17

trungleduc force-pushed the docker-host-config branch from 40dba74 to 1f86451 Compare October 4, 2023 11:27

trungleduc force-pushed the docker-host-config branch from 891e8eb to 0f8df75 Compare October 5, 2023 20:12

trungleduc force-pushed the docker-host-config branch from c71eebb to 328a9ad Compare October 9, 2023 14:02

trungleduc added 4 commits October 9, 2023 21:02

Add container_default_config argument

f7b2272

Add container config to local mode config

64713f6

Add tests

328c4ac

Update documentation

f9a35d9

trungleduc force-pushed the docker-host-config branch from 328a9ad to f9a35d9 Compare October 9, 2023 19:02

trungleduc force-pushed the docker-host-config branch from 33a1803 to 7dcc1f6 Compare October 9, 2023 20:09

mufaddal-rohawala self-assigned this Oct 9, 2023

Update LocalSagemakerRuntimeClient

9233947

trungleduc force-pushed the docker-host-config branch from 7dcc1f6 to 9233947 Compare October 9, 2023 21:14

mufaddal-rohawala previously approved these changes Oct 9, 2023

View reviewed changes

goelakash approved these changes Oct 9, 2023

View reviewed changes

goelakash merged commit 412e8ba into aws:master Oct 9, 2023

feat: allow configuring docker container in local mode #4153

feat: allow configuring docker container in local mode #4153

Uh oh!

Conversation

trungleduc commented Oct 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Oct 3, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 3, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 3, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 3, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 3, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 3, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 3, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 3, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 3, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 4, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 4, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 4, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 4, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 4, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 4, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 4, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 4, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 4, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 4, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 4, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 4, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 5, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 5, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 6, 2023

trungleduc commented Oct 3, 2023 •

edited

Loading