Skip to content

Commit 2a05313

Browse files
can-suncan-sun
authored
Merge branch 'master' into master
2 parents 43cdbf9 + e100e0a commit 2a05313

File tree

118 files changed

+5085
-437
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

118 files changed

+5085
-437
lines changed

‎.githooks/pre-push

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#!/bin/sh
2-
# this pre-push hook runs style checks and unit tests in python 3.6, 3.7, and 3.8 using tox.
2+
# this pre-push hook runs style checks and unit tests in python 3.8, 3.9, and 3.10 using tox.
33

44
set -e
55

@@ -12,5 +12,5 @@ start_time=`date +%s`
1212
tox -e sphinx,doc8 --parallel all
1313
./ci-scripts/displaytime.sh 'sphinx,doc8' $start_time
1414
start_time=`date +%s`
15-
tox -e py37,py38,py39 --parallel all -- tests/unit
16-
./ci-scripts/displaytime.sh 'py37,py38,py39 unit' $start_time
15+
tox -e py38,py39,py310 --parallel all -- tests/unit
16+
./ci-scripts/displaytime.sh 'py38,py39,py310 unit' $start_time

‎.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
build
33
src/*.egg-info
44
.cache
5-
.coverage
5+
.coverage*
66
sagemaker_venv*
77
*.egg-info
88
.tox

‎CHANGELOG.md

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,101 @@
11
# Changelog
22

3+
## v2.187.0 (2023-09-19)
4+
5+
### Features
6+
7+
* add HealthCheckConfig support
8+
* SkipModelValidation in modelRegistry
9+
10+
### Bug Fixes and Other Changes
11+
12+
* Update fw_utils.py - support 2.0.1 container for DDP and Torch distri…
13+
* bump apache-airflow to v2.7.1
14+
15+
## v2.186.0 (2023-09-14)
16+
17+
### Features
18+
19+
* TGI 1.0.3 Image URI Config
20+
21+
## v2.185.0 (2023-09-12)
22+
23+
### Features
24+
25+
* Local Mode - Add Support for Docker Compose V2
26+
27+
### Bug Fixes and Other Changes
28+
29+
* handle bad jumpstart default session
30+
* Add Data Wrangler TLV and version 3.x images
31+
32+
## v2.184.0.post0 (2023-09-11)
33+
34+
### Documentation Changes
35+
36+
* add interactive apps rst file
37+
38+
## v2.184.0 (2023-09-07)
39+
40+
### Features
41+
42+
* Enable notebook instances to get presigned url
43+
44+
### Bug Fixes and Other Changes
45+
46+
* update scikit-learn, scipy, and apache-airflow deps for dependabot
47+
* log message when sdk defaults not applied
48+
49+
## v2.183.0 (2023-09-05)
50+
51+
### Deprecations and Removals
52+
53+
* remove support for py37
54+
55+
### Features
56+
57+
* Neo service GA in TLV
58+
59+
### Bug Fixes and Other Changes
60+
61+
* Update pytorch.json with 2.0.1 for inference and training
62+
* get python version dynamically for remote function tests
63+
* HuggingFaceProcessor parameterized instance_type when image_uri is absent
64+
65+
## v2.182.0 (2023-08-29)
66+
67+
### Features
68+
69+
* image url for modelmonitor in TLV region
70+
* Enable spot training on remote decorator and executor
71+
72+
## v2.181.0 (2023-08-28)
73+
74+
### Features
75+
76+
* StabilityAI DLC Image URIs
77+
78+
### Bug Fixes and Other Changes
79+
80+
* temporarily skip kmeans notebook
81+
82+
## v2.180.0 (2023-08-24)
83+
84+
### Features
85+
86+
* Add presigned URLs for interactive apps
87+
* Add detail profiler V2 options and tests
88+
89+
## v2.179.0 (2023-08-21)
90+
91+
### Features
92+
93+
* attach method for jumpstart estimator
94+
95+
### Bug Fixes and Other Changes
96+
97+
* pipeline upsert failed to pass parallelism_config to update
98+
399
## v2.178.0 (2023-08-17)
4100

5101
### Features

‎README.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,6 @@ Supported Python Versions
8787

8888
SageMaker Python SDK is tested on:
8989

90-
- Python 3.7
9190
- Python 3.8
9291
- Python 3.9
9392
- Python 3.10

‎VERSION

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
2.178.1.dev0
1+
2.187.1.dev0

‎doc/api/training/debugger.rst

Lines changed: 1 addition & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ Debugger Rule APIs
2020
.. autoclass:: get_rule_container_image_uri
2121
:show-inheritance:
2222

23-
.. autoclass:: get_default_profiler_rule
23+
.. autoclass:: get_default_profiler_processing_job
2424
:show-inheritance:
2525

2626
.. class:: sagemaker.debugger.rule_configs
@@ -45,10 +45,6 @@ Debugger Rule APIs
4545
:show-inheritance:
4646
:inherited-members:
4747

48-
.. autoclass:: ProfilerRule
49-
:show-inheritance:
50-
:inherited-members:
51-
5248
Debugger Configuration APIs
5349
~~~~~~~~~~~~~~~~~~~~~~~~~~~
5450

@@ -60,43 +56,3 @@ Debugger Configuration APIs
6056

6157
.. autoclass:: TensorBoardOutputConfig
6258
:show-inheritance:
63-
64-
.. autoclass:: ProfilerConfig
65-
:show-inheritance:
66-
67-
Debugger Configuration APIs for Framework Profiling (Deprecated)
68-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
69-
70-
.. warning::
71-
72-
SageMaker Debugger deprecates the framework profiling feature starting from TensorFlow 2.11 and PyTorch 2.0. You can still use the feature in the previous versions of the frameworks and SDKs as follows.
73-
74-
* SageMaker Python SDK <= v2.130.0
75-
* PyTorch >= v1.6.0, < v2.0
76-
* TensorFlow >= v2.3.1, < v2.11
77-
78-
With the deprecation, SageMaker Debugger discontinues support for the APIs below this note.
79-
80-
See also `Amazon SageMaker Debugger Release Notes: March 16, 2023 <https://docs.aws.amazon.com/sagemaker/latest/dg/debugger-release-notes.html#debugger-release-notes-20230315>`_.
81-
82-
.. autoclass:: FrameworkProfile
83-
:show-inheritance:
84-
85-
.. autoclass:: DetailedProfilingConfig
86-
:show-inheritance:
87-
88-
.. autoclass:: DataloaderProfilingConfig
89-
:show-inheritance:
90-
91-
.. autoclass:: PythonProfilingConfig
92-
:show-inheritance:
93-
94-
.. autoclass:: PythonProfiler
95-
:show-inheritance:
96-
97-
.. autoclass:: cProfileTimer
98-
:show-inheritance:
99-
100-
.. automodule:: sagemaker.debugger.metrics_config
101-
:members: StepRange, TimeRange
102-
:undoc-members:

‎doc/api/training/index.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,12 @@ Training APIs
55
.. toctree::
66
:maxdepth: 4
77

8+
algorithm
89
analytics
910
automl
1011
debugger
1112
estimators
12-
algorithm
1313
tuner
1414
parameter
1515
processing
16+
profiler

‎doc/api/training/profiler.rst

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
Profiler
2+
--------
3+
4+
Amazon SageMaker Profiler provides full visibility
5+
into provisioned compute resources for training
6+
state-of-the-art deep learning models.
7+
The following SageMaker Profiler classes are
8+
for activating SageMaker Profiler while creating
9+
an estimator object of `:class:sagemaker.pytorch.estimator.PyTorch`
10+
or `:class:sagemaker.tensorflow.estimator.TensorFlow`.
11+
12+
.. contents::
13+
14+
.. currentmodule:: sagemaker.debugger
15+
16+
Profiler configuration modules
17+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
18+
19+
.. class:: sagemaker.Profiler(cpu_profiling_duration=3600)
20+
21+
A configuration class to activate
22+
`Amazon SageMaker Profiler <https://docs.aws.amazon.com/sagemaker/latest/dg/train-profile-computational-performance.html>`_.
23+
24+
To adjust the Profiler configuration instead of using the default configuration, use the following parameters.
25+
26+
**Parameters:**
27+
28+
- **cpu_profiling_duration** (*str*): Specify the time duration in seconds for
29+
profiling CPU activities. The default value is 3600 seconds.
30+
31+
**Example usage:**
32+
33+
.. code:: python
34+
35+
import sagemaker
36+
from sagemaker.pytorch import PyTorch
37+
from sagemaker import ProfilerConfig, Profiler
38+
39+
profiler_config = ProfilerConfig(
40+
profiler_params = Profiler(cpu_profiling_duration=3600)
41+
)
42+
43+
estimator = PyTorch(
44+
framework_version="2.0.0",
45+
... # Set up other essential parameters for the estimator class
46+
profiler_config=profiler_config
47+
)
48+
49+
For a complete instruction on activating and using SageMaker Profiler, see
50+
`Use Amazon SageMaker Profiler to profile activities on AWS compute resources
51+
<https://docs.aws.amazon.com/sagemaker/latest/dg/train-profile-computational-performance.html>`_.
52+
53+
.. autoclass:: sagemaker.ProfilerConfig
54+
55+
56+
Profiler Rule APIs
57+
~~~~~~~~~~~~~~~~~~
58+
59+
The following API is for setting up SageMaker Debugger's profiler rules
60+
to detect computational performance issues from training jobs.
61+
62+
.. autoclass:: ProfilerRule
63+
:inherited-members:
64+
65+
66+
Debugger Configuration APIs for Framework Profiling (Deprecated)
67+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
68+
69+
.. warning::
70+
71+
In favor of `Amazon SageMaker Profiler <https://docs.aws.amazon.com/sagemaker/latest/dg/train-profile-computational-performance.html>`_,
72+
SageMaker Debugger deprecates the framework profiling feature starting from TensorFlow 2.11 and PyTorch 2.0. You can still use the feature in the previous versions of the frameworks and SDKs as follows.
73+
74+
* SageMaker Python SDK <= v2.130.0
75+
* PyTorch >= v1.6.0, < v2.0
76+
* TensorFlow >= v2.3.1, < v2.11
77+
78+
With the deprecation, SageMaker Debugger discontinues support for the APIs below this note.
79+
80+
See also `Amazon SageMaker Debugger Release Notes: March 16, 2023 <https://docs.aws.amazon.com/sagemaker/latest/dg/debugger-release-notes.html#debugger-release-notes-20230315>`_.
81+
82+
.. autoclass:: FrameworkProfile
83+
:show-inheritance:
84+
85+
.. autoclass:: DetailedProfilingConfig
86+
:show-inheritance:
87+
88+
.. autoclass:: DataloaderProfilingConfig
89+
:show-inheritance:
90+
91+
.. autoclass:: PythonProfilingConfig
92+
:show-inheritance:
93+
94+
.. autoclass:: PythonProfiler
95+
:show-inheritance:
96+
97+
.. autoclass:: cProfileTimer
98+
:show-inheritance:
99+
100+
.. automodule:: sagemaker.debugger.metrics_config
101+
:members: StepRange, TimeRange
102+
:undoc-members:

‎doc/api/utility/featuregroup_utils.rst

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
FeatureGroup Utilities
2+
----------------------
3+
4+
.. automodule:: sagemaker.feature_store.feature_utils
5+
:members:
6+
:undoc-members:
7+
:show-inheritance:

‎doc/api/utility/interactive_apps.rst

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
Interactive Apps
2+
----------------
3+
4+
TensorBoard on SageMaker
5+
~~~~~~~~~~~~~~~~~~~~~~~~
6+
7+
.. automodule:: sagemaker.interactive_apps.tensorboard
8+
:members:
9+
:undoc-members:
10+
:show-inheritance:
11+
12+
SageMaker Profiler UI application
13+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
14+
15+
.. automodule:: sagemaker.interactive_apps.detail_profiler_app
16+
:members:
17+
:undoc-members:
18+
:show-inheritance:
19+

‎doc/overview.rst

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1550,14 +1550,15 @@ them to your local environment. This is a great way to test your deep learning s
15501550
managed training or hosting environments. Local Mode is supported for frameworks images (TensorFlow, MXNet, Chainer, PyTorch,
15511551
and Scikit-Learn) and images you supply yourself.
15521552

1553-
You can install necessary dependencies for this feature using pip; local mode also requires docker-compose which you can
1554-
install using the following steps (More info - https://github.com/docker/compose#where-to-get-docker-compose ):
1553+
You can install necessary dependencies for this feature using pip.
15551554

15561555
::
15571556

15581557
pip install 'sagemaker[local]' --upgrade
1559-
curl -L "https://github.com/docker/compose/releases/download/v2.7.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
1560-
chmod +x /usr/local/bin/docker-compose
1558+
1559+
1560+
Additionally, Local Mode also requires Docker Compose V2. Follow the guidelines in https://docs.docker.com/compose/install/ to install.
1561+
Make sure to have a Compose Version compatible with your Docker Engine installation. Check Docker Engine release notes https://docs.docker.com/engine/release-notes to find a compatible version.
15611562

15621563
If you want to keep everything local, and not use Amazon S3 either, you can enable "local code" in one of two ways:
15631564

‎requirements/extras/scipy_requirements.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
scipy==1.7.3
1+
scipy==1.10.1

‎requirements/extras/test_requirements.txt

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ awslogs==0.14.0
1212
black==22.3.0
1313
stopit==1.1.2
1414
# Update tox.ini to have correct version of airflow constraints file
15-
apache-airflow==2.6.3
15+
apache-airflow==2.7.1
1616
apache-airflow-providers-amazon==7.2.1
1717
attrs>=23.1.0,<24
1818
fabric==2.6.0
@@ -21,9 +21,9 @@ sagemaker-experiments==0.1.35
2121
Jinja2==3.0.3
2222
pyvis==0.2.1
2323
pandas>=1.3.5,<1.5
24-
scikit-learn==1.0.2
24+
scikit-learn==1.3.0
2525
cloudpickle==2.2.1
26-
scipy==1.7.3
26+
scipy==1.10.1
2727
urllib3>=1.26.8,<1.26.15
2828
docker>=5.0.2,<7.0.0
2929
PyYAML==6.0

0 commit comments

Comments
 (0)