Skip to content

fix: blazingtext tag and upgrade sparkml version #1904

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

fix: blazingtext tag and upgrade sparkml version #1904

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e8ffe70
fix: blazingtext tag and upgrade sparkml version
Sep 17, 2020
f031c53
update sparkml default version
Sep 17, 2020
d6d1012
fix pylint error
Sep 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/sagemaker/image_uri_config/blazingtext.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"scope": ["inference", "training"],
"versions": {
"1": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should remain as 1 and we should add latest as an alias to 1. That's the policy for all 1P algorithms.

"latest": {
"registries": {
"af-south-1": "455444449433",
"ap-east-1": "286214385809",
Expand Down
29 changes: 29 additions & 0 deletions src/sagemaker/image_uri_config/sparkml-serving.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,35 @@
"us-west-2": "246618743249"
},
"repository": "sagemaker-sparkml-serving"
},
"2.4": {
"registries": {
"af-south-1": "510948584623",
"ap-east-1": "651117190479",
"ap-northeast-1": "354813040037",
"ap-northeast-2": "366743142698",
"ap-south-1": "720646828776",
"ap-southeast-1": "121021644041",
"ap-southeast-2": "783357654285",
"ca-central-1": "341280168497",
"cn-north-1": "450853457545",
"cn-northwest-1": "451049120500",
"eu-central-1": "492215442770",
"eu-north-1": "662702820516",
"eu-west-1": "141502667606",
"eu-west-2": "764974769150",
"eu-west-3": "659782779980",
"eu-south-1": "978288397137",
"me-south-1": "801668240914",
"sa-east-1": "737474898029",
"us-east-1": "683313688378",
"us-east-2": "257758044811",
"us-gov-west-1": "414596584902",
"us-iso-east-1": "833128469047",
"us-west-1": "746614075791",
"us-west-2": "246618743249"
},
"repository": "sagemaker-sparkml-serving"
}
}
}
6 changes: 4 additions & 2 deletions src/sagemaker/sparkml/model.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,9 @@ class SparkMLModel(Model):
model .
"""

def __init__(self, model_data, role=None, spark_version=2.2, sagemaker_session=None, **kwargs):
def __init__(
self, model_data, role=None, spark_version="2.4", sagemaker_session=None, **kwargs
):
"""Initialize a SparkMLModel.

Args:
Expand All @@ -73,7 +75,7 @@ def __init__(self, model_data, role=None, spark_version=2.2, sagemaker_session=N
artifacts. After the endpoint is created, the inference code
might use the IAM role, if it needs to access an AWS resource.
spark_version (str): Spark version you want to use for executing the
inference (default: '2.2').
inference (default: '2.4').
sagemaker_session (sagemaker.session.Session): Session object which
manages interactions with Amazon SageMaker APIs and any other
AWS services needed. If not specified, the estimator creates one
Expand Down
3 changes: 2 additions & 1 deletion tests/unit/sagemaker/image_uris/test_algos.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,8 @@ def test_algo_uris(algo):

for region in regions.regions():
uri = image_uris.retrieve(algo, region)
assert expected_uris.algo_uri(algo, accounts[region], region) == uri
version = "latest" if algo == "blazingtext" else 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be a str?

Suggested change
version = "latest" if algo == "blazingtext" else 1
version = "latest" if algo == "blazingtext" else "1"

Copy link
Contributor

@iyerr3 iyerr3 Sep 18, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To reiterate point from my other comment: nothing special in BlazingText wrt to versioning. It should be same as other 1P algos

assert expected_uris.algo_uri(algo, accounts[region], region, version) == uri


def test_lda():
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/test_pipeline_model.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ def test_prepare_container_def(tfo, time, sagemaker_session):
{
"Environment": {"SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "text/csv"},
"Image": "246618743249.dkr.ecr.us-west-2.amazonaws.com"
+ "/sagemaker-sparkml-serving:2.2",
+ "/sagemaker-sparkml-serving:2.4",
"ModelDataUrl": "s3://bucket/model_2.tar.gz",
},
]
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/test_sparkml_serving.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ def sagemaker_session():

def test_sparkml_model(sagemaker_session):
sparkml = SparkMLModel(sagemaker_session=sagemaker_session, model_data=MODEL_DATA, role=ROLE)
assert sparkml.image_uri == image_uris.retrieve("sparkml-serving", REGION, version="2.2")
assert sparkml.image_uri == image_uris.retrieve("sparkml-serving", REGION, version="2.4")


def test_predictor_type(sagemaker_session):
Expand Down