Skip to content

fix: fix instance_type assignment logic #4719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

fix: fix instance_type assignment logic #4719

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/sagemaker/jumpstart/factory/estimator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,7 @@ def get_deploy_kwargs(
model_id=model_id,
model_from_estimator=True,
model_version=model_version,
instance_type=model_deploy_kwargs.instance_type if training_instance_type is None else None,
instance_type=model_deploy_kwargs.instance_type or training_instance_type,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change doesn't look right, the attribute is referring to the hosting instance type, it should not be passed a training instance type.

I think it should just be something like:

# if passed an instance type, use it
deploy_instance_type = instance_type

# otherwise, if passed a training instance, derive from training instance
if not instance_type and training_instance_type
  deploy_instance_type = instance_types.retrieve_default(
    model_id,
    model_version,
    training_instance_type=training_instance_type
  )
else:
  deploy_instance_type = model_deploy_kwargs.instance_type

Can we wait for SME @evakravi to look at this code change before merging?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I based the above change on the commit where it was introduced, which also added:

training_instance_type (str): In the case of a model fine-tuned on SageMaker, the training
            instance type used for the training job that produced the fine-tuned weights.
            Optionally supply this to get a inference instance type conditioned
            on the training instance, to ensure compatability of training artifact to inference
            instance. (Default: None).

Which I now notice should also:

- compatability
+ compatibility

Happy to wait for @evakravi to be back in office to discuss what is the best approach here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JGuinegagne The logic you have looks right. But let's definitely add unit tests so there's no doubt that this fixes the problem at hand.

region=region,
image_uri=image_uri,
source_dir=source_dir,
Expand Down
Loading