Skip to content

feature: separating sagemaker dependencies into more use case specific installable components. #1130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Nov 21, 2019
Merged

feature: separating sagemaker dependencies into more use case specific installable components. #1130

Changes from 5 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
c1f7ff0
feature: separating sagemaker dependencies into more use case specifi…
nadiaya Nov 20, 2019
35c01aa
Fix formatting.
nadiaya Nov 20, 2019
617d50f
Remove "core" from extra dependencies.
nadiaya Nov 20, 2019
ca5f0d6
Fix formatting.
nadiaya Nov 20, 2019
e98d6c8
Include all extra dependencies in the tests list.
nadiaya Nov 20, 2019
349c9d4
Add warning for urllib3 imports.
nadiaya Nov 20, 2019
c064a9f
Move fabric to test dependencies.
nadiaya Nov 20, 2019
978582d
Move tensorflow dependencies into a special extra subset.
nadiaya Nov 20, 2019
90d56bf
Keep local mode dependencies in the required list to be removed in th…
nadiaya Nov 20, 2019
afc10d1
Update documentation.
nadiaya Nov 20, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 25 additions & 20 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,35 @@ def read_version():
"numpy>=1.9.0",
"protobuf>=3.1",
"scipy>=0.19.0",
"urllib3>=1.21, <1.25",
"protobuf3-to-dict>=0.1.5",
"requests>=2.20.0, <2.21",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize this isn't strictly in scope, but the request bounds were somewhat defined by docker-compose - we could probably change it to requests>=2.20.0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, don't have bandwidth to do a proper check for it as part of this PR knowing how many problems it caused before.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair enough 😂

"fabric>=2.0",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize this was part of an old change, but fabric should actually be part of test requires

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

]

# Specific use case dependencies
extras = {"analytics": ["pandas"], "local": ["urllib3>=1.21, <1.25", "docker-compose>=1.23.0"]}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add one for tensorflow? (if people want to use the TF serializers/deserializers that require the library itself)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i didn't want to touch it, but actually why not :)

# Meta dependency groups
extras["all"] = [item for group in extras.values() for item in group]
# Tests specific dependencies (do not need to be included in 'all')
extras["test"] = (
[
extras["all"],
"tox==3.13.1",
"flake8",
"pytest==4.4.1",
"pytest-cov",
"pytest-rerunfailures",
"pytest-xdist",
"mock",
"tensorflow>=1.3.0",
"contextlib2",
"awslogs",
"black==19.3b0 ; python_version >= '3.6'",
"stopit==1.1.2",
"apache-airflow==1.10.5",
],
)

# enum is introduced in Python 3.4. Installing enum back port
if sys.version_info < (3, 4):
required_packages.append("enum34>=1.1.6")
Expand All @@ -70,24 +93,6 @@ def read_version():
"Programming Language :: Python :: 3.6",
],
install_requires=required_packages,
extras_require={
"test": [
"tox==3.13.1",
"flake8",
"pytest==4.4.1",
"pytest-cov",
"pytest-rerunfailures",
"pytest-xdist",
"mock",
"tensorflow>=1.3.0",
"contextlib2",
"awslogs",
"pandas",
"black==19.3b0 ; python_version >= '3.6'",
"stopit==1.1.2",
"apache-airflow==1.10.5",
"docker-compose>=1.23.0",
]
},
extras_require=extras,
entry_points={"console_scripts": ["sagemaker=sagemaker.cli.main:main"]},
)