Skip to content

Commit 1bb9eec

Browse files
makungaj1Captainianikhil-skNikhil Kulkarnijessicazhu3
authored
Sync branch (#4718)
* fix: mainline alt config parsing (#4602) * fix: parsing * fix: commit tests * fix: types * updated * fix * Add Triton v24.03 URI (#4605) Co-authored-by: Nikhil Kulkarni <[email protected]> * feature: support session tag chaining for training job (#4596) * feature: support session tag chaining for training job * fix: resolve typo * fix: resolve typo and build failure * fix: resolve typo and unit test failure --------- Co-authored-by: Jessica Zhu <jessicazhu3@[email protected]> * prepare release v2.217.0 * update development version to v2.217.1.dev0 * fix: properly close files in lineage queries and tests (#4587) Closes #4458 * feature: set default allow_pickle param to False (#4557) * breaking: set default allow_pickle param to False * breaking: fix unit tests and linting NumpyDeserializer will not allow deserialization unless allow_pickle flag is set to True explicitly * fix: black-check --------- Co-authored-by: Ashwin Krishna <[email protected]> * Fix:invalid component error with new metadata (#4634) * fix: invalid component name * tests * format * fix vulnerable model integ tests llama 2 * updated * fix: training dataset location * prepare release v2.218.0 * update development version to v2.218.1.dev0 * chore: update skipped flaky tests (#4644) * Update skipped flaky tests * flake8 * format * format * chore: release tgi 2.0.1 (#4642) * chore: release tgi 2.0.1 * minor fix --------- Co-authored-by: Zhaoqi <[email protected]> * fix: Fix UserAgent logging in Python SDK (#4647) * prepare release v2.218.1 * update development version to v2.218.2.dev0 * feature: allow choosing js payload by alias in private method * Updates for SMP v2.3.1 (#4660) Co-authored-by: Suhit Kodgule <[email protected]> * chore(deps): bump jinja2 from 3.1.3 to 3.1.4 in /doc (#4655) Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](pallets/jinja@3.1.3...3.1.4) --- updated-dependencies: - dependency-name: jinja2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump tqdm from 4.66.2 to 4.66.3 in /tests/data/serve_resources/mlflow/pytorch (#4650) Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.66.2 to 4.66.3. - [Release notes](https://github.com/tqdm/tqdm/releases) - [Commits](tqdm/tqdm@v4.66.2...v4.66.3) --- updated-dependencies: - dependency-name: tqdm dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump jinja2 from 3.1.3 to 3.1.4 in /requirements/extras (#4654) Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](pallets/jinja@3.1.3...3.1.4) --- updated-dependencies: - dependency-name: jinja2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * prepare release v2.219.0 * update development version to v2.219.1.dev0 * fix: skip flakey tests pending investigation (#4667) * change: update image_uri_configs 05-09-2024 07:17:41 PST * Add tensorflow_serving support for mlflow models and enable lineage tracking for mlflow models (#4662) * Initial commit for tensorflow_serving support of MLflow * Add integ tests for mlflow tf_serving * fix style issues * remove unused attributes from tf builder * Add deep ping for tf_serving local mode * Initial commit for lineage impl * Initial commit for tensorflow_serving support of MLflow * Add integ tests for mlflow tf_serving * fix style issues * remove unused attributes from tf builder * Add deep ping for tf_serving local mode * Add integ tests and uts * fix local mode for tf_serving * Allow lineage tracking only in sagemaker endpoint mode * fix regex pattern * fix style issues * fix regex pattern and hard coded py version in ut * fix missing session * Resolve pr comments and fix regex for mlflow registry and ids * fix: model builder race condition on sagemaker session (#4673) Co-authored-by: Jonathan Makunga <[email protected]> * feat: Add telemetry support for mlflow models (#4674) * Initial commit for telemetry support * Fix style issues and add more logger messages * fix value error messages in ut * feat: add new images for HF TGI release (#4677) * chore: add new images for HF TGI release * test * feature: AutoGluon 1.1.0 image_uris update (#4679) Co-authored-by: Ubuntu <[email protected]> * change: add debug logs to workflow container dist creation (#4682) * prepare release v2.220.0 * update development version to v2.220.1.dev0 * fix: Image URI should take precedence for HF models (#4684) * Fix: Image URI should take precedence for HF models * Fix formatting * Fix formatting * Fix formatting * Increase coverage - UT pass * feat: onboard tei image config to pysdk (#4681) * feat: onboard tei image config to pysdk * fix formatting issue * minor fix func name * fix unit tests --------- Co-authored-by: Mufaddal Rohawala <[email protected]> * fix: model builder limited container support for endpoint mode. (#4683) * Allow ModelBuilder's endpoint mode for Jumpstart models packaged with containers other than TGI and DJL * increase coverage * Add JS Support for MMS Serving * Add JS Support for MMS Serving * Unit tests * Refactoring * Refactoring * Refactoring --------- Co-authored-by: Jonathan Makunga <[email protected]> * change: Add more debuging (#4687) * change: cover tei with image_uris.retrieve API (#4689) * fix: JS Model with non-TGI/non-DJL deployment failure (#4688) * Debug * Debug * Debug * Debug * Debug * Debug * fix docstyle * Refactoring * Add Integ tests --------- Co-authored-by: Jonathan Makunga <[email protected]> * Feat: Pull latest tei container for sentence similiarity models on HuggingFace hub (#4686) * Update: Pull latest tei container for sentence similiarity models * Fix formatting * Address PR comments * Fix formatting * Fix check * Switch sentence similarity to be deployed on tgi * Fix formatting * Fix formatting * Fix formatting * Fix formatting * Introduce TEI builder with TGI server * Fix formmatting * Add integ test * Fix formatting * Add integ test * Add integ test * Add integ test * Add integ test * Add integ test * Fix formatting * Move to G5 for integ test * Fix formatting * Integ test updates * Integ test updates * Integ test updates * Fix formatting * Integ test updates * Move back to generate for ping * Integ test updates * Integ test updates * Fix: Add Image URI overrides for transformers models (#4693) * Fix: Add Image URI overrides for transformers models * Increase coverage * Fix formatting * prepare release v2.221.0 * update development version to v2.221.1.dev0 * Add tei cpu image (#4695) * Add tei cpu image * fix format issue * fix unit tests * fix typo * fix typo * Feat: Add TEI support for ModelBuilder (#4694) * Add TEI Serving * Add TEI Serving * Add TEI Serving * Add TEI Serving * Add TEI Serving * Add TEI Serving * Notebook testing * Notebook testing * Notebook testing * Refactoring * Refactoring * UT * UT * Refactoring * Test coverage * Refactoring * Refactoring --------- Co-authored-by: Jonathan Makunga <[email protected]> * Convert pytorchddp distribution to smdistributed distribution (#4698) * rewrite pytorchddp to smdistributed * remove instance type check * Update estimator.py * remove validate_pytorch_distribution * fix * fix unit tests * fix formatting * check instance type not None * prepare release v2.221.1 * update development version to v2.221.2.dev0 * Update: SM Endpoint Routing Strategy Support. (#4702) * RoutingConfig * Refactoring * Docstring * UT * Refactoring * Refactoring --------- Co-authored-by: Jonathan Makunga <[email protected]> * change: update image_uri_configs 05-29-2024 07:17:35 PST * Making project name in workflow files dynamic (#4708) * fix: Fix ci unit-tests (#4713) * chore(deps): bump requests from 2.31.0 to 2.32.2 in /tests/data/serve_resources/mlflow/pytorch (#4709) Bumps [requests](https://github.com/psf/requests) from 2.31.0 to 2.32.2. - [Release notes](https://github.com/psf/requests/releases) - [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md) - [Commits](psf/requests@v2.31.0...v2.32.2) --- updated-dependencies: - dependency-name: requests dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump apache-airflow from 2.9.0 to 2.9.1 in /requirements/extras (#4703) * chore(deps): bump apache-airflow in /requirements/extras Bumps [apache-airflow](https://github.com/apache/airflow) from 2.9.0 to 2.9.1. - [Release notes](https://github.com/apache/airflow/releases) - [Changelog](https://github.com/apache/airflow/blob/main/RELEASE_NOTES.rst) - [Commits](apache/airflow@2.9.0...2.9.1) --- updated-dependencies: - dependency-name: apache-airflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update tox.ini to bump apache-airflow --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kalyani Nikure <[email protected]> * chore(deps): bump mlflow from 2.10.2 to 2.12.1 in /tests/data/serve_resources/mlflow/pytorch (#4690) Bumps [mlflow](https://github.com/mlflow/mlflow) from 2.10.2 to 2.12.1. - [Release notes](https://github.com/mlflow/mlflow/releases) - [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.md) - [Commits](mlflow/mlflow@v2.10.2...v2.12.1) --- updated-dependencies: - dependency-name: mlflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump mlflow from 2.11.1 to 2.12.1 in /tests/data/serve_resources/mlflow/xgboost (#4692) Bumps [mlflow](https://github.com/mlflow/mlflow) from 2.11.1 to 2.12.1. - [Release notes](https://github.com/mlflow/mlflow/releases) - [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.md) - [Commits](mlflow/mlflow@v2.11.1...v2.12.1) --- updated-dependencies: - dependency-name: mlflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump mlflow from 2.11.1 to 2.12.1 in /tests/data/serve_resources/mlflow/tensorflow (#4691) Bumps [mlflow](https://github.com/mlflow/mlflow) from 2.11.1 to 2.12.1. - [Release notes](https://github.com/mlflow/mlflow/releases) - [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.md) - [Commits](mlflow/mlflow@v2.11.1...v2.12.1) --- updated-dependencies: - dependency-name: mlflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * change: Updates for DJL 0.28.0 release (#4701) * Sync Branch --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Haotian An <[email protected]> Co-authored-by: Nikhil Kulkarni <[email protected]> Co-authored-by: Nikhil Kulkarni <[email protected]> Co-authored-by: jessicazhu3 <[email protected]> Co-authored-by: Jessica Zhu <jessicazhu3@[email protected]> Co-authored-by: ci <ci> Co-authored-by: Justin <[email protected]> Co-authored-by: ASHWIN KRISHNA <[email protected]> Co-authored-by: Ashwin Krishna <[email protected]> Co-authored-by: Haixin Wang <[email protected]> Co-authored-by: Zhaoqi <[email protected]> Co-authored-by: Kalyani Nikure <[email protected]> Co-authored-by: Keerthan Vasist <[email protected]> Co-authored-by: SuhitK <[email protected]> Co-authored-by: Suhit Kodgule <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: sagemaker-bot <[email protected]> Co-authored-by: jiapinw <[email protected]> Co-authored-by: Jonathan Makunga <[email protected]> Co-authored-by: Prateek M Desai <[email protected]> Co-authored-by: Ubuntu <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Samrudhi Sharma <[email protected]> Co-authored-by: Tom Bousso <[email protected]> Co-authored-by: Zhaoqi <[email protected]> Co-authored-by: Tyler Osterberg <[email protected]>
1 parent 6fb935d commit 1bb9eec

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

54 files changed

+1642
-219
lines changed

‎.github/workflows/codebuild-ci.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ jobs:
5555
- name: Run Codestyle & Doc Tests
5656
uses: aws-actions/aws-codebuild-run-build@v1
5757
with:
58-
project-name: sagemaker-python-sdk-ci-codestyle-doc-tests
58+
project-name: ${{ github.event.repository.name }}-ci-codestyle-doc-tests
5959
source-version-override: 'refs/pull/${{ github.event.pull_request.number }}/head^{${{ github.event.pull_request.head.sha }}}'
6060
unit-tests:
6161
runs-on: ubuntu-latest
@@ -74,7 +74,7 @@ jobs:
7474
- name: Run Unit Tests
7575
uses: aws-actions/aws-codebuild-run-build@v1
7676
with:
77-
project-name: sagemaker-python-sdk-ci-unit-tests
77+
project-name: ${{ github.event.repository.name }}-ci-unit-tests
7878
source-version-override: 'refs/pull/${{ github.event.pull_request.number }}/head^{${{ github.event.pull_request.head.sha }}}'
7979
env-vars-for-codebuild: |
8080
PY_VERSION
@@ -93,5 +93,5 @@ jobs:
9393
- name: Run Integ Tests
9494
uses: aws-actions/aws-codebuild-run-build@v1
9595
with:
96-
project-name: sagemaker-python-sdk-ci-integ-tests
96+
project-name: ${{ github.event.repository.name }}-ci-integ-tests
9797
source-version-override: 'refs/pull/${{ github.event.pull_request.number }}/head^{${{ github.event.pull_request.head.sha }}}'

‎CHANGELOG.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,26 @@
11
# Changelog
22

3+
## v2.221.1 (2024-05-22)
4+
5+
### Bug Fixes and Other Changes
6+
7+
* Convert pytorchddp distribution to smdistributed distribution
8+
* Add tei cpu image
9+
10+
## v2.221.0 (2024-05-20)
11+
12+
### Features
13+
14+
* onboard tei image config to pysdk
15+
16+
### Bug Fixes and Other Changes
17+
18+
* JS Model with non-TGI/non-DJL deployment failure
19+
* cover tei with image_uris.retrieve API
20+
* Add more debuging
21+
* model builder limited container support for endpoint mode.
22+
* Image URI should take precedence for HF models
23+
324
## v2.220.0 (2024-05-15)
425

526
### Features

‎VERSION

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
2.220.1.dev0
1+
2.221.2.dev0

‎requirements/extras/test_requirements.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ awslogs==0.14.0
1212
black==24.3.0
1313
stopit==1.1.2
1414
# Update tox.ini to have correct version of airflow constraints file
15-
apache-airflow==2.9.0
15+
apache-airflow==2.9.1
1616
apache-airflow-providers-amazon==7.2.1
1717
attrs>=23.1.0,<24
1818
fabric==2.6.0

‎src/sagemaker/enums.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,3 +28,15 @@ class EndpointType(Enum):
2828
INFERENCE_COMPONENT_BASED = (
2929
"InferenceComponentBased" # Amazon SageMaker Inference Component Based Endpoint
3030
)
31+
32+
33+
class RoutingStrategy(Enum):
34+
"""Strategy for routing https traffics."""
35+
36+
RANDOM = "RANDOM"
37+
"""The endpoint routes each request to a randomly chosen instance.
38+
"""
39+
LEAST_OUTSTANDING_REQUESTS = "LEAST_OUTSTANDING_REQUESTS"
40+
"""The endpoint routes requests to the specific instances that have
41+
more capacity to process them.
42+
"""

‎src/sagemaker/fw_utils.py

Lines changed: 2 additions & 97 deletions
Original file line numberDiff line numberDiff line change
@@ -145,22 +145,6 @@
145145
],
146146
}
147147

148-
PYTORCHDDP_SUPPORTED_FRAMEWORK_VERSIONS = [
149-
"1.10",
150-
"1.10.0",
151-
"1.10.2",
152-
"1.11",
153-
"1.11.0",
154-
"1.12",
155-
"1.12.0",
156-
"1.12.1",
157-
"1.13.1",
158-
"2.0.0",
159-
"2.0.1",
160-
"2.1.0",
161-
"2.2.0",
162-
]
163-
164148
TORCH_DISTRIBUTED_GPU_SUPPORTED_FRAMEWORK_VERSIONS = [
165149
"1.13.1",
166150
"2.0.0",
@@ -795,7 +779,6 @@ def _validate_smdataparallel_args(
795779
796780
Raises:
797781
ValueError: if
798-
(`instance_type` is not in SM_DATAPARALLEL_SUPPORTED_INSTANCE_TYPES or
799782
`py_version` is not python3 or
800783
`framework_version` is not in SM_DATAPARALLEL_SUPPORTED_FRAMEWORK_VERSION
801784
"""
@@ -806,17 +789,10 @@ def _validate_smdataparallel_args(
806789
if not smdataparallel_enabled:
807790
return
808791

809-
is_instance_type_supported = instance_type in SM_DATAPARALLEL_SUPPORTED_INSTANCE_TYPES
810-
811792
err_msg = ""
812793

813-
if not is_instance_type_supported:
814-
# instance_type is required
815-
err_msg += (
816-
f"Provided instance_type {instance_type} is not supported by smdataparallel.\n"
817-
"Please specify one of the supported instance types:"
818-
f"{SM_DATAPARALLEL_SUPPORTED_INSTANCE_TYPES}\n"
819-
)
794+
if not instance_type:
795+
err_msg += "Please specify an instance_type for smdataparallel.\n"
820796

821797
if not image_uri:
822798
# ignore framework_version & py_version if image_uri is set
@@ -928,13 +904,6 @@ def validate_distribution(
928904
)
929905
if framework_name and framework_name == "pytorch":
930906
# We need to validate only for PyTorch framework
931-
validate_pytorch_distribution(
932-
distribution=validated_distribution,
933-
framework_name=framework_name,
934-
framework_version=framework_version,
935-
py_version=py_version,
936-
image_uri=image_uri,
937-
)
938907
validate_torch_distributed_distribution(
939908
instance_type=instance_type,
940909
distribution=validated_distribution,
@@ -968,13 +937,6 @@ def validate_distribution(
968937
)
969938
if framework_name and framework_name == "pytorch":
970939
# We need to validate only for PyTorch framework
971-
validate_pytorch_distribution(
972-
distribution=validated_distribution,
973-
framework_name=framework_name,
974-
framework_version=framework_version,
975-
py_version=py_version,
976-
image_uri=image_uri,
977-
)
978940
validate_torch_distributed_distribution(
979941
instance_type=instance_type,
980942
distribution=validated_distribution,
@@ -1023,63 +985,6 @@ def validate_distribution_for_instance_type(instance_type, distribution):
1023985
raise ValueError(err_msg)
1024986

1025987

1026-
def validate_pytorch_distribution(
1027-
distribution, framework_name, framework_version, py_version, image_uri
1028-
):
1029-
"""Check if pytorch distribution strategy is correctly invoked by the user.
1030-
1031-
Args:
1032-
distribution (dict): A dictionary with information to enable distributed training.
1033-
(Defaults to None if distributed training is not enabled.) For example:
1034-
1035-
.. code:: python
1036-
1037-
{
1038-
"pytorchddp": {
1039-
"enabled": True
1040-
}
1041-
}
1042-
framework_name (str): A string representing the name of framework selected.
1043-
framework_version (str): A string representing the framework version selected.
1044-
py_version (str): A string representing the python version selected.
1045-
image_uri (str): A string representing a Docker image URI.
1046-
1047-
Raises:
1048-
ValueError: if
1049-
`py_version` is not python3 or
1050-
`framework_version` is not in PYTORCHDDP_SUPPORTED_FRAMEWORK_VERSIONS
1051-
"""
1052-
if framework_name and framework_name != "pytorch":
1053-
# We need to validate only for PyTorch framework
1054-
return
1055-
1056-
pytorch_ddp_enabled = False
1057-
if "pytorchddp" in distribution:
1058-
pytorch_ddp_enabled = distribution.get("pytorchddp").get("enabled", False)
1059-
if not pytorch_ddp_enabled:
1060-
# Distribution strategy other than pytorchddp is selected
1061-
return
1062-
1063-
err_msg = ""
1064-
if not image_uri:
1065-
# ignore framework_version and py_version if image_uri is set
1066-
# in case image_uri is not set, then both are mandatory
1067-
if framework_version not in PYTORCHDDP_SUPPORTED_FRAMEWORK_VERSIONS:
1068-
err_msg += (
1069-
f"Provided framework_version {framework_version} is not supported by"
1070-
" pytorchddp.\n"
1071-
"Please specify one of the supported framework versions:"
1072-
f" {PYTORCHDDP_SUPPORTED_FRAMEWORK_VERSIONS} \n"
1073-
)
1074-
if "py3" not in py_version:
1075-
err_msg += (
1076-
f"Provided py_version {py_version} is not supported by pytorchddp.\n"
1077-
"Please specify py_version>=py3"
1078-
)
1079-
if err_msg:
1080-
raise ValueError(err_msg)
1081-
1082-
1083988
def validate_torch_distributed_distribution(
1084989
instance_type,
1085990
distribution,

‎src/sagemaker/huggingface/llm_utils.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,20 @@ def get_huggingface_llm_image_uri(
6565
image_scope="inference",
6666
inference_tool="neuronx",
6767
)
68+
if backend == "huggingface-tei":
69+
return image_uris.retrieve(
70+
"huggingface-tei",
71+
region=region,
72+
version=version,
73+
image_scope="inference",
74+
)
75+
if backend == "huggingface-tei-cpu":
76+
return image_uris.retrieve(
77+
"huggingface-tei-cpu",
78+
region=region,
79+
version=version,
80+
image_scope="inference",
81+
)
6882
if backend == "lmi":
6983
version = version or "0.24.0"
7084
return image_uris.retrieve(framework="djl-deepspeed", region=region, version=version)

‎src/sagemaker/huggingface/model.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -334,6 +334,7 @@ def deploy(
334334
endpoint_type=kwargs.get("endpoint_type", None),
335335
resources=kwargs.get("resources", None),
336336
managed_instance_scaling=kwargs.get("managed_instance_scaling", None),
337+
routing_config=kwargs.get("routing_config", None),
337338
)
338339

339340
def register(

‎src/sagemaker/image_uri_config/djl-lmi.json

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
{
2+
"scope": [
3+
"inference"
4+
],
5+
"versions": {
6+
"0.28.0": {
7+
"registries": {
8+
"af-south-1": "626614931356",
9+
"il-central-1": "780543022126",
10+
"ap-east-1": "871362719292",
11+
"ap-northeast-1": "763104351884",
12+
"ap-northeast-2": "763104351884",
13+
"ap-northeast-3": "364406365360",
14+
"ap-south-1": "763104351884",
15+
"ap-southeast-1": "763104351884",
16+
"ap-southeast-2": "763104351884",
17+
"ap-southeast-3": "907027046896",
18+
"ca-central-1": "763104351884",
19+
"cn-north-1": "727897471807",
20+
"cn-northwest-1": "727897471807",
21+
"eu-central-1": "763104351884",
22+
"eu-north-1": "763104351884",
23+
"eu-west-1": "763104351884",
24+
"eu-west-2": "763104351884",
25+
"eu-west-3": "763104351884",
26+
"eu-south-1": "692866216735",
27+
"me-south-1": "217643126080",
28+
"sa-east-1": "763104351884",
29+
"us-east-1": "763104351884",
30+
"us-east-2": "763104351884",
31+
"us-west-1": "763104351884",
32+
"us-west-2": "763104351884",
33+
"ca-west-1": "204538143572"
34+
},
35+
"repository": "djl-inference",
36+
"tag_prefix": "0.28.0-lmi10.0.0-cu124"
37+
}
38+
}
39+
}

‎src/sagemaker/image_uri_config/djl-neuronx.json

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,24 @@
33
"inference"
44
],
55
"versions": {
6+
"0.28.0": {
7+
"registries": {
8+
"ap-northeast-1": "763104351884",
9+
"ap-south-1": "763104351884",
10+
"ap-southeast-1": "763104351884",
11+
"ap-southeast-2": "763104351884",
12+
"eu-central-1": "763104351884",
13+
"eu-west-1": "763104351884",
14+
"eu-west-3": "763104351884",
15+
"sa-east-1": "763104351884",
16+
"us-east-1": "763104351884",
17+
"us-east-2": "763104351884",
18+
"us-west-2": "763104351884",
19+
"ca-west-1": "204538143572"
20+
},
21+
"repository": "djl-inference",
22+
"tag_prefix": "0.28.0-neuronx-sdk2.18.2"
23+
},
624
"0.27.0": {
725
"registries": {
826
"ap-northeast-1": "763104351884",

‎src/sagemaker/image_uri_config/djl-tensorrtllm.json

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,38 @@
33
"inference"
44
],
55
"versions": {
6+
"0.28.0": {
7+
"registries": {
8+
"af-south-1": "626614931356",
9+
"il-central-1": "780543022126",
10+
"ap-east-1": "871362719292",
11+
"ap-northeast-1": "763104351884",
12+
"ap-northeast-2": "763104351884",
13+
"ap-northeast-3": "364406365360",
14+
"ap-south-1": "763104351884",
15+
"ap-southeast-1": "763104351884",
16+
"ap-southeast-2": "763104351884",
17+
"ap-southeast-3": "907027046896",
18+
"ca-central-1": "763104351884",
19+
"cn-north-1": "727897471807",
20+
"cn-northwest-1": "727897471807",
21+
"eu-central-1": "763104351884",
22+
"eu-north-1": "763104351884",
23+
"eu-west-1": "763104351884",
24+
"eu-west-2": "763104351884",
25+
"eu-west-3": "763104351884",
26+
"eu-south-1": "692866216735",
27+
"me-south-1": "217643126080",
28+
"sa-east-1": "763104351884",
29+
"us-east-1": "763104351884",
30+
"us-east-2": "763104351884",
31+
"us-west-1": "763104351884",
32+
"us-west-2": "763104351884",
33+
"ca-west-1": "204538143572"
34+
},
35+
"repository": "djl-inference",
36+
"tag_prefix": "0.28.0-tensorrtllm0.9.0-cu122"
37+
},
638
"0.27.0": {
739
"registries": {
840
"af-south-1": "626614931356",

0 commit comments

Comments
 (0)