Skip to content

Commit bba18b6

Browse files
SSRraymondSSRraymond
authored
Merge branch 'master' into server
2 parents 560af86 + 57fd632 commit bba18b6

File tree

14 files changed

+67
-211
lines changed

14 files changed

+67
-211
lines changed

‎doc/frameworks/pytorch/using_pytorch.rst

Lines changed: 36 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -132,14 +132,49 @@ Using third-party libraries
132132
---------------------------
133133

134134
When running your training script on SageMaker, it will have access to some pre-installed third-party libraries including ``torch``, ``torchvision``, and ``numpy``.
135-
For more information on the runtime environment, including specific package versions, see `SageMaker PyTorch Docker containers <https://github.com/aws/deep-learning-containers/tree/master/pytorch>`_.
136135

137136
If there are other packages you want to use with your script, you can include a ``requirements.txt`` file in the same directory as your training script to install other dependencies at runtime. Both ``requirements.txt`` and your training script should be put in the same folder. You must specify this folder in ``source_dir`` argument when creating PyTorch estimator.
138137

139138
The function of installing packages using ``requirements.txt`` is supported for all PyTorch versions during training. When serving a PyTorch model, support for this function varies with PyTorch versions. For PyTorch 1.3.1 or newer, ``requirements.txt`` must be under folder ``code``. The SageMaker PyTorch Estimator will automatically save ``code`` in ``model.tar.gz`` after training (assuming you set up your script and ``requirements.txt`` correctly as stipulated in the previous paragraph). In the case of bringing your own trained model for deployment, you must save ``requirements.txt`` under folder ``code`` in ``model.tar.gz`` yourself or specify it through ``dependencies``. For PyTorch 1.2.0, ``requirements.txt`` is not supported for inference. For PyTorch 0.4.0 to 1.1.0, ``requirements.txt`` must be in ``source_dir``.
140139

141140
A ``requirements.txt`` file is a text file that contains a list of items that are installed by using ``pip install``. You can also specify the version of an item to install. For information about the format of a ``requirements.txt`` file, see `Requirements Files <https://pip.pypa.io/en/stable/user_guide/#requirements-files>`__ in the pip documentation.
142141

142+
If you were to use your own custom Docker Image, the `SageMaker Python SDK` and the `SageMaker Training Toolkit <https://github.com/aws/sagemaker-training-toolkit/>`__ need to be installed.
143+
144+
To do so, you can add the following lines to your ``requirements.txt`` file:
145+
146+
.. code:: text
147+
148+
sagemaker
149+
sagemaker-training
150+
151+
Deep Learning Framework-Specific SageMaker Toolkits and Containers
152+
------------------------------------------------------------------
153+
154+
Framework-specific Toolkits exist. You might want to use them in your applications for framework-specific features.
155+
156+
For Training Toolkits, see:
157+
158+
- `SageMaker PyTorch Training Toolkit <https://github.com/aws/sagemaker-pytorch-training-toolkit>`__
159+
- `SageMaker MXNet Training Toolkit <https://github.com/aws/sagemaker-mxnet-training-toolkit>`__
160+
- `SageMaker TensorFlow Training Toolkit <https://github.com/aws/sagemaker-tensorflow-training-toolkit>`__
161+
162+
For Inference Toolkits, see:
163+
164+
- `SageMaker PyTorch Inference Toolkit <https://github.com/aws/sagemaker-pytorch-inference-toolkit>`__
165+
- `SageMaker MXNet Inference Toolkit <https://github.com/aws/sagemaker-mxnet-training-toolkit>`__
166+
- `SageMaker TensorFlow Inference Toolkit <https://github.com/aws/sagemaker-tensorflow-inference-toolkit>`__
167+
- `SageMaker HuggingFace Inference Toolkit <https://github.com/aws/sagemaker-huggingface-inference-toolkit>`__
168+
169+
Moreover, for more information on the container runtime environment, including specific framework versions and configurations,
170+
see `AWS Deep Learning Containers <https://github.com/aws/deep-learning-containers/>`_. More specifically, see:
171+
172+
- `Images for PyTorch <https://github.com/aws/deep-learning-containers/tree/master/pytorch>`__
173+
- `Images for MXNet <https://github.com/aws/deep-learning-containers/tree/master/mxnet>`__
174+
- `Images for TensorFlow <https://github.com/aws/deep-learning-containers/tree/master/tensorflow>`__
175+
- `Images for HuggingFace <https://github.com/aws/deep-learning-containers/tree/master/huggingface>`__
176+
177+
143178
Create an Estimator
144179
===================
145180

‎inference-experience-dev-tester.sh

Lines changed: 0 additions & 53 deletions
This file was deleted.

‎requirements/extras/test_requirements.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ awslogs==0.14.0
1212
black==22.3.0
1313
stopit==1.1.2
1414
# Update tox.ini to have correct version of airflow constraints file
15-
apache-airflow==2.7.2
15+
apache-airflow==2.7.3
1616
apache-airflow-providers-amazon==7.2.1
1717
attrs>=23.1.0,<24
1818
fabric==2.6.0
@@ -31,7 +31,7 @@ pyspark==3.3.1
3131
sagemaker-feature-store-pyspark-3.3
3232
# TODO find workaround
3333
xgboost>=1.6.2,<=1.7.6
34-
pillow>=9.5.0,<=10.0.0
34+
pillow>=10.0.1,<=11
3535
transformers==4.32.0
3636
sentencepiece==0.1.99
3737
# https://github.com/triton-inference-server/server/issues/6246

‎src/sagemaker/image_uri_config/djl-neuronx.json

Lines changed: 0 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -3,123 +3,67 @@
33
"versions": {
44
"0.25.0": {
55
"registries": {
6-
"af-south-1": "626614931356",
7-
"il-central-1": "780543022126",
8-
"ap-east-1": "871362719292",
96
"ap-northeast-1": "763104351884",
10-
"ap-northeast-2": "763104351884",
11-
"ap-northeast-3": "364406365360",
127
"ap-south-1": "763104351884",
138
"ap-southeast-1": "763104351884",
149
"ap-southeast-2": "763104351884",
15-
"ap-southeast-3": "907027046896",
16-
"ca-central-1": "763104351884",
17-
"cn-north-1": "727897471807",
18-
"cn-northwest-1": "727897471807",
1910
"eu-central-1": "763104351884",
20-
"eu-north-1": "763104351884",
2111
"eu-west-1": "763104351884",
22-
"eu-west-2": "763104351884",
2312
"eu-west-3": "763104351884",
24-
"eu-south-1": "692866216735",
25-
"me-south-1": "217643126080",
2613
"sa-east-1": "763104351884",
2714
"us-east-1": "763104351884",
2815
"us-east-2": "763104351884",
29-
"us-west-1": "763104351884",
3016
"us-west-2": "763104351884"
3117
},
3218
"repository": "djl-inference",
3319
"tag_prefix": "0.25.0-neuronx-sdk2.15.0"
3420
},
3521
"0.24.0": {
3622
"registries": {
37-
"af-south-1": "626614931356",
38-
"il-central-1": "780543022126",
39-
"ap-east-1": "871362719292",
4023
"ap-northeast-1": "763104351884",
41-
"ap-northeast-2": "763104351884",
42-
"ap-northeast-3": "364406365360",
4324
"ap-south-1": "763104351884",
4425
"ap-southeast-1": "763104351884",
4526
"ap-southeast-2": "763104351884",
46-
"ap-southeast-3": "907027046896",
47-
"ca-central-1": "763104351884",
48-
"cn-north-1": "727897471807",
49-
"cn-northwest-1": "727897471807",
5027
"eu-central-1": "763104351884",
51-
"eu-north-1": "763104351884",
5228
"eu-west-1": "763104351884",
53-
"eu-west-2": "763104351884",
5429
"eu-west-3": "763104351884",
55-
"eu-south-1": "692866216735",
56-
"me-south-1": "217643126080",
5730
"sa-east-1": "763104351884",
5831
"us-east-1": "763104351884",
5932
"us-east-2": "763104351884",
60-
"us-west-1": "763104351884",
6133
"us-west-2": "763104351884"
6234
},
6335
"repository": "djl-inference",
6436
"tag_prefix": "0.24.0-neuronx-sdk2.14.1"
6537
},
6638
"0.23.0": {
6739
"registries": {
68-
"af-south-1": "626614931356",
69-
"il-central-1": "780543022126",
70-
"ap-east-1": "871362719292",
7140
"ap-northeast-1": "763104351884",
72-
"ap-northeast-2": "763104351884",
73-
"ap-northeast-3": "364406365360",
7441
"ap-south-1": "763104351884",
7542
"ap-southeast-1": "763104351884",
7643
"ap-southeast-2": "763104351884",
77-
"ap-southeast-3": "907027046896",
78-
"ca-central-1": "763104351884",
79-
"cn-north-1": "727897471807",
80-
"cn-northwest-1": "727897471807",
8144
"eu-central-1": "763104351884",
82-
"eu-north-1": "763104351884",
8345
"eu-west-1": "763104351884",
84-
"eu-west-2": "763104351884",
8546
"eu-west-3": "763104351884",
86-
"eu-south-1": "692866216735",
87-
"me-south-1": "217643126080",
8847
"sa-east-1": "763104351884",
8948
"us-east-1": "763104351884",
9049
"us-east-2": "763104351884",
91-
"us-west-1": "763104351884",
9250
"us-west-2": "763104351884"
9351
},
9452
"repository": "djl-inference",
9553
"tag_prefix": "0.23.0-neuronx-sdk2.12.0"
9654
},
9755
"0.22.1": {
9856
"registries": {
99-
"af-south-1": "626614931356",
100-
"il-central-1": "780543022126",
101-
"ap-east-1": "871362719292",
10257
"ap-northeast-1": "763104351884",
103-
"ap-northeast-2": "763104351884",
104-
"ap-northeast-3": "364406365360",
10558
"ap-south-1": "763104351884",
10659
"ap-southeast-1": "763104351884",
10760
"ap-southeast-2": "763104351884",
108-
"ap-southeast-3": "907027046896",
109-
"ca-central-1": "763104351884",
110-
"cn-north-1": "727897471807",
111-
"cn-northwest-1": "727897471807",
11261
"eu-central-1": "763104351884",
113-
"eu-north-1": "763104351884",
11462
"eu-west-1": "763104351884",
115-
"eu-west-2": "763104351884",
11663
"eu-west-3": "763104351884",
117-
"eu-south-1": "692866216735",
118-
"me-south-1": "217643126080",
11964
"sa-east-1": "763104351884",
12065
"us-east-1": "763104351884",
12166
"us-east-2": "763104351884",
122-
"us-west-1": "763104351884",
12367
"us-west-2": "763104351884"
12468
},
12569
"repository": "djl-inference",

‎src/sagemaker/image_uri_config/huggingface-neuron.json

Lines changed: 2 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -9,37 +9,21 @@
99
"py_versions": ["py37"],
1010
"repository": "huggingface-pytorch-inference-neuron",
1111
"registries": {
12-
"af-south-1": "626614931356",
13-
"il-central-1": "780543022126",
14-
"ap-east-1": "871362719292",
1512
"ap-northeast-1": "763104351884",
16-
"ap-northeast-2": "763104351884",
17-
"ap-northeast-3": "364406365360",
1813
"ap-south-1": "763104351884",
1914
"ap-south-2": "772153158452",
2015
"ap-southeast-1": "763104351884",
2116
"ap-southeast-2": "763104351884",
2217
"ap-southeast-4": "457447274322",
23-
"ca-central-1": "763104351884",
24-
"cn-north-1": "727897471807",
25-
"cn-northwest-1": "727897471807",
2618
"eu-central-1": "763104351884",
2719
"eu-central-2": "380420809688",
28-
"eu-north-1": "763104351884",
20+
"eu-south-2": "503227376785",
2921
"eu-west-1": "763104351884",
30-
"eu-west-2": "763104351884",
3122
"eu-west-3": "763104351884",
32-
"eu-south-1": "692866216735",
33-
"eu-south-2": "503227376785",
34-
"me-south-1": "217643126080",
23+
"il-central-1": "780543022126",
3524
"sa-east-1": "763104351884",
3625
"us-east-1": "763104351884",
3726
"us-east-2": "763104351884",
38-
"us-gov-east-1": "446045086412",
39-
"us-gov-west-1": "442386744353",
40-
"us-iso-east-1": "886529160074",
41-
"us-isob-east-1": "094389454867",
42-
"us-west-1": "763104351884",
4327
"us-west-2": "763104351884"
4428
},
4529
"container_version": {"inf": "ubuntu18.04"},

‎src/sagemaker/image_uri_config/huggingface-neuronx.json

Lines changed: 4 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -9,37 +9,21 @@
99
"py_versions": ["py38"],
1010
"repository": "huggingface-pytorch-training-neuronx",
1111
"registries": {
12-
"af-south-1": "626614931356",
13-
"il-central-1": "780543022126",
14-
"ap-east-1": "871362719292",
1512
"ap-northeast-1": "763104351884",
16-
"ap-northeast-2": "763104351884",
17-
"ap-northeast-3": "364406365360",
1813
"ap-south-1": "763104351884",
1914
"ap-south-2": "772153158452",
2015
"ap-southeast-1": "763104351884",
2116
"ap-southeast-2": "763104351884",
2217
"ap-southeast-4": "457447274322",
23-
"ca-central-1": "763104351884",
24-
"cn-north-1": "727897471807",
25-
"cn-northwest-1": "727897471807",
2618
"eu-central-1": "763104351884",
2719
"eu-central-2": "380420809688",
28-
"eu-north-1": "763104351884",
20+
"eu-south-2": "503227376785",
2921
"eu-west-1": "763104351884",
30-
"eu-west-2": "763104351884",
3122
"eu-west-3": "763104351884",
32-
"eu-south-1": "692866216735",
33-
"eu-south-2": "503227376785",
34-
"me-south-1": "217643126080",
23+
"il-central-1": "780543022126",
3524
"sa-east-1": "763104351884",
3625
"us-east-1": "763104351884",
3726
"us-east-2": "763104351884",
38-
"us-gov-east-1": "446045086412",
39-
"us-gov-west-1": "442386744353",
40-
"us-iso-east-1": "886529160074",
41-
"us-isob-east-1": "094389454867",
42-
"us-west-1": "763104351884",
4327
"us-west-2": "763104351884"
4428
},
4529
"container_version": {"trn": "ubuntu20.04"},
@@ -52,37 +36,21 @@
5236
"py_versions": ["py310"],
5337
"repository": "huggingface-pytorch-inference-neuronx",
5438
"registries": {
55-
"af-south-1": "626614931356",
56-
"il-central-1": "780543022126",
57-
"ap-east-1": "871362719292",
5839
"ap-northeast-1": "763104351884",
59-
"ap-northeast-2": "763104351884",
60-
"ap-northeast-3": "364406365360",
6140
"ap-south-1": "763104351884",
6241
"ap-south-2": "772153158452",
6342
"ap-southeast-1": "763104351884",
6443
"ap-southeast-2": "763104351884",
6544
"ap-southeast-4": "457447274322",
66-
"ca-central-1": "763104351884",
67-
"cn-north-1": "727897471807",
68-
"cn-northwest-1": "727897471807",
6945
"eu-central-1": "763104351884",
7046
"eu-central-2": "380420809688",
71-
"eu-north-1": "763104351884",
47+
"eu-south-2": "503227376785",
7248
"eu-west-1": "763104351884",
73-
"eu-west-2": "763104351884",
7449
"eu-west-3": "763104351884",
75-
"eu-south-1": "692866216735",
76-
"eu-south-2": "503227376785",
77-
"me-south-1": "217643126080",
50+
"il-central-1": "780543022126",
7851
"sa-east-1": "763104351884",
7952
"us-east-1": "763104351884",
8053
"us-east-2": "763104351884",
81-
"us-gov-east-1": "446045086412",
82-
"us-gov-west-1": "442386744353",
83-
"us-iso-east-1": "886529160074",
84-
"us-isob-east-1": "094389454867",
85-
"us-west-1": "763104351884",
8654
"us-west-2": "763104351884"
8755
},
8856
"container_version": {"inf": "ubuntu20.04"},

0 commit comments

Comments
 (0)