Skip to content

Commit 3cdefeb

Browse files
beniericgverkesakrishna1995Ashwin Krishnaknikure
authored
Merge master into master-assets (#4515)
* fix: make sure gpus are found in local_gpu run (#4384) * fix: make sure gpus are found in local_gpu run * fix: black formatting * fix: adjust unit test * feat: pin dll version to support python3.11 to the sdk (#4472) Co-authored-by: Ashwin Krishna <[email protected]> * fix: Skip No Canvas regions for test_deploy_best_candidate (#4477) * prepare release v2.211.0 * update development version to v2.211.1.dev0 * change: Enhance model builder selection logic to include model size (#4429) * change: Enhance model builder selection logic to include model size * Fix conflicts * Address PR comments * fix formatting * fix formatting of test * Fix token in tasks.json * Increase coverage for tests * fix formatting * Fix requirements * Import code instead of importing accelerate * Fix formatting * Setup dependencies * change: Upgrade smp to version 2.2 (#4479) * upgrading smp to version 2.2 * fixing linting issue * fixing syntax error with multiline if statement * upgrading smp to version 2.2 * fixing linting issue * fixing syntax error with multiline if statement * fixing formatting --------- Co-authored-by: Andrew Tian <[email protected]> * feat: Update SM Python SDK for PT 2.2.0 SM DLC (#4481) * update pt2.2 sm training dlc pysdk * update pt2.2 sm inference dlc pysdk and region list * fix: Create custom tarfile extractall util to fix backward compatibility issue (#4476) * fix: Create custom tarfile extractall util to fix backward compatibility issue * Address review comments * fix logger.error statements * prepare release v2.212.0 * update development version to v2.212.1.dev0 * change: Update tblib constraint (#4452) * fix: make unit tests compatible with pytest-xdist (#4486) * fix: make unit tests compatible with pytest-xdist * fix failing test * feature: Add overriding logic in ModelBuilder when task is provided (#4460) * feat: Add Optional task to Model * Revert "feat: Add Optional task to Model" This reverts commit fd3e86b. * Add override logic in ModelBuilder with task provided * Adjusted formatting * Add extra unit tests for invalid inputs * Address PR comments * Add more test inputs to integration test * Add model_metadata field to ModelBuilder * Update doc * Update doc * Adjust formatting --------- Co-authored-by: Samrudhi Sharma <[email protected]> Co-authored-by: Xiong Zeng <[email protected]> * feature: Accept user-defined env variables for the entry-point (#4175) * fix: Move sagemaker pysdk version check after bootstrap in remote job (#4487) * change: enable github actions for PRs (#4489) * change: enable github actions for PRs * Update codebuild-ci.yml * trigger on pull_request_target * add source-version-override * fix permission * feature: Add ModelDataSource and SourceUri support for model package and while registering (#4492) Co-authored-by: Erick Benitez-Ramos <[email protected]> * feat: support JumpStart proprietary models (#4467) * feat: add proprietary manifest/specs parsing add unittests for test_cache small refactoring address comments and more unittests fix linting and fix more tests fix: pylint feat: JumpStartModel class for prop models * remove unused imports and fix docstyle * fix: remove unused args * fix: remove unused args * fix: more unused vars * fix: slow tests * fix: unittests * added more tests to cover some lines * remove estimator warn check * chore: address comments re performance * fix: address comments * complete list experience and other fixes * fix: pylint * add doc utils and fix pylint * fix: docstyle * fix: doc * fix: default payloads * fix: doc and tags and enums * fix: jumpstart doc * rename to open_weights and fix filtering * update filter name * doc update * fix: black * rename to proprietary model and fix unittests * address comments * fix: docstyle and flake8 * address more comments and fix doc * put back doc utils for future refactoring * add prop model title in doc * doc update --------- Co-authored-by: liujiaor <[email protected]> * chore: emit warning when no instance specific gated training env var is available, and raise exception when accept_eula flag is not supplied (#4485) * fix: raise exception when no instance specific gated training env var available * chore: raise client exception if accept_eula flag is not set for gated models * chore: address flake8 errors * chore: emit warning when instance type is chosen with no gated training artifacts * fix: sagemaker session region not being used (#4469) * fix: sagemaker session region not being used * chore: add unit tests * fix: remove all JUMPSTART_DEFAULT_REGION_NAME default arguments * chore: use get_region_fallback throughout * chore: remove unnecessary if statement * chore: remove unnecessary if statement (2) --------- Co-authored-by: Erick Benitez-Ramos <[email protected]> * fix: add PT 2.2 support for smdistributed, pytorchddp, and torch_distributed distributions (#4480) * Add support for smdistributed, pytorchddp, torch_distributed for PT 2.2 * formatting * formatting --------- Co-authored-by: liujiaor <[email protected]> * change: split coverage out from testenv in tox.ini (#4495) Co-authored-by: Ashwin Krishna <[email protected]> * change: add ci-health checks (#4493) * feat: tgi optimum 0.0.19, 0.0.20 releases (#4496) * feature: Add support for Streaming Inference (#4497) * feature: Add support for Streaming Inference * fix: codestyle-docs-test * fix: codestyle-docs-test * Add AutoML -> AutoMLV2 mapper (#4500) Co-authored-by: liujiaor <[email protected]> * Skip of tests which are long running and causing the ResourceLimitInUse exception (#4504) * Improvement of the tuner documentation (#4506) * prepare release v2.213.0 * update development version to v2.213.1.dev0 * fix:urge customers to install latest version (#4507) * fix: list jumpstart models with invalid version strings (#4511) * fix: list jumpstart models with invalid versions * docstyle * docstyle * pylint * add more test * fix * fix: skip failing pt test (#4512) * fix: skip failing pt test * black-format --------- Co-authored-by: gv <[email protected]> Co-authored-by: akrishna1995 <[email protected]> Co-authored-by: Ashwin Krishna <[email protected]> Co-authored-by: Kalyani Nikure <[email protected]> Co-authored-by: ci <ci> Co-authored-by: Samrudhi Sharma <[email protected]> Co-authored-by: adtian2 <[email protected]> Co-authored-by: Andrew Tian <[email protected]> Co-authored-by: Sirut Buasai <[email protected]> Co-authored-by: Danny Bushkanets <[email protected]> Co-authored-by: xiongz945 <[email protected]> Co-authored-by: Samrudhi Sharma <[email protected]> Co-authored-by: Xiong Zeng <[email protected]> Co-authored-by: martinRenou <[email protected]> Co-authored-by: qidewenwhen <[email protected]> Co-authored-by: mrudulmn <[email protected]> Co-authored-by: Haotian An <[email protected]> Co-authored-by: liujiaor <[email protected]> Co-authored-by: evakravi <[email protected]> Co-authored-by: ruhanprasad <[email protected]> Co-authored-by: Mufaddal Rohawala <[email protected]> Co-authored-by: Anton Repushko <[email protected]>
1 parent 12bcf05 commit 3cdefeb

File tree

141 files changed

+7160
-890
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

141 files changed

+7160
-890
lines changed

‎.github/workflows/codebuild-ci-health.yml

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
name: CI Health
2+
on:
3+
schedule:
4+
- cron: "0 */3 * * *"
5+
workflow_dispatch:
6+
7+
permissions:
8+
id-token: write # This is required for requesting the JWT
9+
10+
jobs:
11+
codestyle-doc-tests:
12+
runs-on: ubuntu-latest
13+
steps:
14+
- name: Configure AWS Credentials
15+
uses: aws-actions/configure-aws-credentials@v4
16+
with:
17+
role-to-assume: ${{ secrets.CI_AWS_ROLE_ARN }}
18+
aws-region: us-west-2
19+
role-duration-seconds: 10800
20+
- name: Run Codestyle & Doc Tests
21+
uses: aws-actions/aws-codebuild-run-build@v1
22+
with:
23+
project-name: sagemaker-python-sdk-ci-health-codestyle-doc-tests
24+
unit-tests:
25+
runs-on: ubuntu-latest
26+
strategy:
27+
fail-fast: false
28+
matrix:
29+
python-version: ["py38", "py39", "py310"]
30+
steps:
31+
- name: Configure AWS Credentials
32+
uses: aws-actions/configure-aws-credentials@v4
33+
with:
34+
role-to-assume: ${{ secrets.CI_AWS_ROLE_ARN }}
35+
aws-region: us-west-2
36+
role-duration-seconds: 10800
37+
- name: Run Unit Tests
38+
uses: aws-actions/aws-codebuild-run-build@v1
39+
with:
40+
project-name: sagemaker-python-sdk-ci-health-unit-tests
41+
env-vars-for-codebuild: |
42+
PY_VERSION
43+
env:
44+
PY_VERSION: ${{ matrix.python-version }}
45+
integ-tests:
46+
runs-on: ubuntu-latest
47+
steps:
48+
- name: Configure AWS Credentials
49+
uses: aws-actions/configure-aws-credentials@v4
50+
with:
51+
role-to-assume: ${{ secrets.CI_AWS_ROLE_ARN }}
52+
aws-region: us-west-2
53+
role-duration-seconds: 10800
54+
- name: Run Integ Tests
55+
uses: aws-actions/aws-codebuild-run-build@v1
56+
id: codebuild
57+
with:
58+
project-name: sagemaker-python-sdk-ci-health-integ-tests
59+
slow-tests:
60+
runs-on: ubuntu-latest
61+
steps:
62+
- name: Configure AWS Credentials
63+
uses: aws-actions/configure-aws-credentials@v4
64+
with:
65+
role-to-assume: ${{ secrets.CI_AWS_ROLE_ARN }}
66+
aws-region: us-west-2
67+
role-duration-seconds: 10800
68+
- name: Run Slow Tests
69+
uses: aws-actions/aws-codebuild-run-build@v1
70+
with:
71+
project-name: sagemaker-python-sdk-ci-health-slow-tests
72+
localmode-tests:
73+
runs-on: ubuntu-latest
74+
steps:
75+
- name: Configure AWS Credentials
76+
uses: aws-actions/configure-aws-credentials@v4
77+
with:
78+
role-to-assume: ${{ secrets.CI_AWS_ROLE_ARN }}
79+
aws-region: us-west-2
80+
role-duration-seconds: 10800
81+
- name: Run Local Mode Tests
82+
uses: aws-actions/aws-codebuild-run-build@v1
83+
with:
84+
project-name: sagemaker-python-sdk-ci-health-localmode-tests
85+
notebook-tests:
86+
runs-on: ubuntu-latest
87+
steps:
88+
- name: Configure AWS Credentials
89+
uses: aws-actions/configure-aws-credentials@v4
90+
with:
91+
role-to-assume: ${{ secrets.CI_AWS_ROLE_ARN }}
92+
aws-region: us-west-2
93+
role-duration-seconds: 10800
94+
- name: Run Notebook Tests
95+
uses: aws-actions/aws-codebuild-run-build@v1
96+
with:
97+
project-name: sagemaker-python-sdk-ci-health-notebook-tests

‎.github/workflows/codebuild-ci.yml

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
name: PR Checks
2+
on:
3+
pull_request_target:
4+
5+
concurrency:
6+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.head_ref }}
7+
cancel-in-progress: true
8+
9+
permissions:
10+
id-token: write # This is required for requesting the JWT
11+
12+
jobs:
13+
codestyle-doc-tests:
14+
runs-on: ubuntu-latest
15+
steps:
16+
- name: Configure AWS Credentials
17+
uses: aws-actions/configure-aws-credentials@v4
18+
with:
19+
role-to-assume: ${{ secrets.CI_AWS_ROLE_ARN }}
20+
aws-region: us-west-2
21+
role-duration-seconds: 10800
22+
- name: Run Codestyle & Doc Tests
23+
uses: aws-actions/aws-codebuild-run-build@v1
24+
with:
25+
project-name: sagemaker-python-sdk-ci-codestyle-doc-tests
26+
source-version-override: 'pr/${{ github.event.pull_request.number }}'
27+
unit-tests:
28+
runs-on: ubuntu-latest
29+
strategy:
30+
fail-fast: false
31+
matrix:
32+
python-version: ["py38", "py39", "py310"]
33+
steps:
34+
- name: Configure AWS Credentials
35+
uses: aws-actions/configure-aws-credentials@v4
36+
with:
37+
role-to-assume: ${{ secrets.CI_AWS_ROLE_ARN }}
38+
aws-region: us-west-2
39+
role-duration-seconds: 10800
40+
- name: Run Unit Tests
41+
uses: aws-actions/aws-codebuild-run-build@v1
42+
with:
43+
project-name: sagemaker-python-sdk-ci-unit-tests
44+
source-version-override: 'pr/${{ github.event.pull_request.number }}'
45+
env-vars-for-codebuild: |
46+
PY_VERSION
47+
env:
48+
PY_VERSION: ${{ matrix.python-version }}

‎CHANGELOG.md

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,66 @@
11
# Changelog
22

3+
## v2.213.0 (2024-03-15)
4+
5+
### Features
6+
7+
* Add support for Streaming Inference
8+
* tgi optimum 0.0.19, 0.0.20 releases
9+
* support JumpStart proprietary models
10+
* Add ModelDataSource and SourceUri support for model package and while registering
11+
* Accept user-defined env variables for the entry-point
12+
* Add overriding logic in ModelBuilder when task is provided
13+
14+
### Bug Fixes and Other Changes
15+
16+
* Improvement of the tuner documentation
17+
* Skip of tests which are long running and causing the ResourceLimitInUse exception
18+
* Add AutoML -> AutoMLV2 mapper
19+
* add ci-health checks
20+
* split coverage out from testenv in tox.ini
21+
* add PT 2.2 support for smdistributed, pytorchddp, and torch_distributed distributions
22+
* sagemaker session region not being used
23+
* chore: emit warning when no instance specific gated training env var is available, and raise exception when accept_eula flag is not supplied
24+
* enable github actions for PRs
25+
* Move sagemaker pysdk version check after bootstrap in remote job
26+
* make unit tests compatible with pytest-xdist
27+
* Update tblib constraint
28+
29+
## v2.212.0 (2024-03-06)
30+
31+
### Features
32+
33+
* Update SM Python SDK for PT 2.2.0 SM DLC
34+
35+
### Bug Fixes and Other Changes
36+
37+
* Create custom tarfile extractall util to fix backward compatibility issue
38+
* Upgrade smp to version 2.2
39+
* Enhance model builder selection logic to include model size
40+
41+
## v2.211.0 (2024-03-05)
42+
43+
### Features
44+
45+
* pin dll version to support python3.11 to the sdk
46+
* instance specific jumpstart host requirements
47+
* Add TensorFlow 2.14 image configs
48+
* Add AutoMLV2 support
49+
* Support selective pipeline execution between function step and regular step
50+
* Add new Triton DLC URIs
51+
52+
### Bug Fixes and Other Changes
53+
54+
* Skip No Canvas regions for test_deploy_best_candidate
55+
* make sure gpus are found in local_gpu run
56+
* Bump Apache Airflow version to 2.8.2
57+
* properly close sagemaker config file after loading config
58+
* remove enable_network_isolation from the python doc
59+
60+
### Documentation Changes
61+
62+
* Add doc for new feature processor APIs and classes
63+
364
## v2.210.0 (2024-02-28)
465

566
### Features

‎CONTRIBUTING.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -77,8 +77,8 @@ Before sending us a pull request, please ensure that:
7777
1. Install coverage using `pip install .[test]`
7878
1. cd into the sagemaker-python-sdk folder: `cd sagemaker-python-sdk` or `cd /environment/sagemaker-python-sdk`
7979
1. Run the following tox command and verify that all code checks and unit tests pass: `tox tests/unit`
80-
81-
You can also run a single test with the following command: `tox -e py310 -- -s -vv <path_to_file><file_name>::<test_function_name>`
80+
1. You can also run a single test with the following command: `tox -e py310 -- -s -vv <path_to_file><file_name>::<test_function_name>`
81+
1. You can run coverage via runcvoerage env : `tox -e runcoverage -- tests/unit` or `tox -e py310 -- tests/unit --cov=sagemaker --cov-append --cov-report xml`
8282
* Note that the coverage test will fail if you only run a single test, so make sure to surround the command with `export IGNORE_COVERAGE=-` and `unset IGNORE_COVERAGE`
8383
* Example: `export IGNORE_COVERAGE=- ; tox -e py310 -- -s -vv tests/unit/test_estimator.py::test_sagemaker_model_s3_uri_invalid ; unset IGNORE_COVERAGE`
8484

‎README.rst

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,10 @@ SageMaker Python SDK
2222
:target: https://sagemaker.readthedocs.io/en/stable/
2323
:alt: Documentation Status
2424

25+
.. image:: https://github.com/benieric/sagemaker-python-sdk/actions/workflows/codebuild-ci-health.yml/badge.svg
26+
:target: https://github.com/benieric/sagemaker-python-sdk/actions/workflows/codebuild-ci-health.yml
27+
:alt: CI Health
28+
2529
SageMaker Python SDK is an open source library for training and deploying machine learning models on Amazon SageMaker.
2630

2731
With the SDK, you can train and deploy models using popular deep learning frameworks **Apache MXNet** and **TensorFlow**.
@@ -63,11 +67,10 @@ Table of Contents
6367
Installing the SageMaker Python SDK
6468
-----------------------------------
6569

66-
The SageMaker Python SDK is built to PyPI and can be installed with pip as follows:
67-
70+
The SageMaker Python SDK is built to PyPI and the latest version of the SageMaker Python SDK can be installed with pip as follows
6871
::
6972

70-
pip install sagemaker
73+
pip install sagemaker==<Latest version from pyPI from https://pypi.org/project/sagemaker/>
7174

7275
You can install from source by cloning this repository and running a pip install command in the root directory of the repository:
7376

‎VERSION

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
2.210.1.dev0
1+
2.213.1.dev0

‎doc/doc_utils/jumpstart_doc_utils.py

Lines changed: 68 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -74,9 +74,12 @@ class Frameworks(str, Enum):
7474

7575
JUMPSTART_REGION = "eu-west-2"
7676
SDK_MANIFEST_FILE = "models_manifest.json"
77+
PROPRIETARY_SDK_MANIFEST_FILE = "proprietary-sdk-manifest.json"
7778
JUMPSTART_BUCKET_BASE_URL = "https://jumpstart-cache-prod-{}.s3.{}.amazonaws.com".format(
7879
JUMPSTART_REGION, JUMPSTART_REGION
7980
)
81+
PROPRIETARY_DOC_BUCKET = "https://jumpstart-cache-prod-us-west-2.s3.us-west-2.amazonaws.com"
82+
8083
TASK_MAP = {
8184
Tasks.IC: ProblemTypes.IMAGE_CLASSIFICATION,
8285
Tasks.IC_EMBEDDING: ProblemTypes.IMAGE_EMBEDDING,
@@ -152,18 +155,26 @@ class Frameworks(str, Enum):
152155
}
153156

154157

155-
def get_jumpstart_sdk_manifest():
156-
url = "{}/{}".format(JUMPSTART_BUCKET_BASE_URL, SDK_MANIFEST_FILE)
158+
def get_public_s3_json_object(url):
157159
with request.urlopen(url) as f:
158160
models_manifest = f.read().decode("utf-8")
159161
return json.loads(models_manifest)
160162

161163

162-
def get_jumpstart_sdk_spec(key):
163-
url = "{}/{}".format(JUMPSTART_BUCKET_BASE_URL, key)
164-
with request.urlopen(url) as f:
165-
model_spec = f.read().decode("utf-8")
166-
return json.loads(model_spec)
164+
def get_jumpstart_sdk_manifest():
165+
return get_public_s3_json_object(f"{JUMPSTART_BUCKET_BASE_URL}/{SDK_MANIFEST_FILE}")
166+
167+
168+
def get_proprietary_sdk_manifest():
169+
return get_public_s3_json_object(f"{PROPRIETARY_DOC_BUCKET}/{PROPRIETARY_SDK_MANIFEST_FILE}")
170+
171+
172+
def get_jumpstart_sdk_spec(s3_key: str):
173+
return get_public_s3_json_object(f"{JUMPSTART_BUCKET_BASE_URL}/{s3_key}")
174+
175+
176+
def get_proprietary_sdk_spec(s3_key: str):
177+
return get_public_s3_json_object(f"{PROPRIETARY_DOC_BUCKET}/{s3_key}")
167178

168179

169180
def get_model_task(id):
@@ -196,6 +207,45 @@ def get_model_source(url):
196207
return "Source"
197208

198209

210+
def create_proprietary_model_table():
211+
proprietary_content_intro = []
212+
proprietary_content_intro.append("\n")
213+
proprietary_content_intro.append(".. list-table:: Available Proprietary Models\n")
214+
proprietary_content_intro.append(" :widths: 50 20 20 20 20\n")
215+
proprietary_content_intro.append(" :header-rows: 1\n")
216+
proprietary_content_intro.append(" :class: datatable\n")
217+
proprietary_content_intro.append("\n")
218+
proprietary_content_intro.append(" * - Model ID\n")
219+
proprietary_content_intro.append(" - Fine Tunable?\n")
220+
proprietary_content_intro.append(" - Supported Version\n")
221+
proprietary_content_intro.append(" - Min SDK Version\n")
222+
proprietary_content_intro.append(" - Source\n")
223+
224+
sdk_manifest = get_proprietary_sdk_manifest()
225+
sdk_manifest_top_versions_for_models = {}
226+
227+
for model in sdk_manifest:
228+
if model["model_id"] not in sdk_manifest_top_versions_for_models:
229+
sdk_manifest_top_versions_for_models[model["model_id"]] = model
230+
else:
231+
if str(sdk_manifest_top_versions_for_models[model["model_id"]]["version"]) < str(
232+
model["version"]
233+
):
234+
sdk_manifest_top_versions_for_models[model["model_id"]] = model
235+
236+
proprietary_content_entries = []
237+
for model in sdk_manifest_top_versions_for_models.values():
238+
model_spec = get_proprietary_sdk_spec(model["spec_key"])
239+
proprietary_content_entries.append(" * - {}\n".format(model_spec["model_id"]))
240+
proprietary_content_entries.append(" - {}\n".format(False)) # TODO: support training
241+
proprietary_content_entries.append(" - {}\n".format(model["version"]))
242+
proprietary_content_entries.append(" - {}\n".format(model["min_version"]))
243+
proprietary_content_entries.append(
244+
" - `{} <{}>`__ |external-link|\n".format("Source", model_spec.get("url"))
245+
)
246+
return proprietary_content_intro + proprietary_content_entries + ["\n"]
247+
248+
199249
def create_jumpstart_model_table():
200250
sdk_manifest = get_jumpstart_sdk_manifest()
201251
sdk_manifest_top_versions_for_models = {}
@@ -249,19 +299,19 @@ def create_jumpstart_model_table():
249299
file_content_intro.append(" - Source\n")
250300

251301
dynamic_table_files = []
252-
file_content_entries = []
302+
open_weight_content_entries = []
253303

254304
for model in sdk_manifest_top_versions_for_models.values():
255305
model_spec = get_jumpstart_sdk_spec(model["spec_key"])
256306
model_task = get_model_task(model_spec["model_id"])
257307
string_model_task = get_string_model_task(model_spec["model_id"])
258308
model_source = get_model_source(model_spec["url"])
259-
file_content_entries.append(" * - {}\n".format(model_spec["model_id"]))
260-
file_content_entries.append(" - {}\n".format(model_spec["training_supported"]))
261-
file_content_entries.append(" - {}\n".format(model["version"]))
262-
file_content_entries.append(" - {}\n".format(model["min_version"]))
263-
file_content_entries.append(" - {}\n".format(model_task))
264-
file_content_entries.append(
309+
open_weight_content_entries.append(" * - {}\n".format(model_spec["model_id"]))
310+
open_weight_content_entries.append(" - {}\n".format(model_spec["training_supported"]))
311+
open_weight_content_entries.append(" - {}\n".format(model["version"]))
312+
open_weight_content_entries.append(" - {}\n".format(model["min_version"]))
313+
open_weight_content_entries.append(" - {}\n".format(model_task))
314+
open_weight_content_entries.append(
265315
" - `{} <{}>`__ |external-link|\n".format(model_source, model_spec["url"])
266316
)
267317

@@ -299,7 +349,10 @@ def create_jumpstart_model_table():
299349
f.writelines(file_content_single_entry)
300350
f.close()
301351

352+
proprietary_content_entries = create_proprietary_model_table()
353+
302354
f = open("doc_utils/pretrainedmodels.rst", "a")
303355
f.writelines(file_content_intro)
304-
f.writelines(file_content_entries)
356+
f.writelines(open_weight_content_entries)
357+
f.writelines(proprietary_content_entries)
305358
f.close()

‎doc/requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,4 @@ docutils==0.15.2
44
packaging==20.9
55
jinja2==3.1.3
66
schema==0.7.5
7+
accelerate>=0.24.1,<=0.27.0

‎requirements/extras/huggingface_requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
accelerate>=0.24.1,<=0.27.0

‎requirements/extras/test_requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,3 +39,4 @@ tritonclient[http]<2.37.0
3939
onnx==1.14.1
4040
# tf2onnx==1.15.1
4141
nbformat>=5.9,<6
42+
accelerate>=0.24.1,<=0.27.0

0 commit comments

Comments
 (0)