-
Notifications
You must be signed in to change notification settings - Fork 102
Remove node info from ML team serverless responses #2205
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Serverless will present the impression of having no underlying servers, so API responses in serverless should not contain any node information. This PR removes the node fields from the responses owned by the ML team.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
/** The deployent stats for each node that currently has the model allocated. */ | ||
/** | ||
* The deployment stats for each node that currently has the model allocated. | ||
* @availability stack |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are useful stats in this object so of which could be aggregated and lifted up to the top level as we have done with inference_count below.
We will want to review what model stats are presented in serverless, I'll open an issue for that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks.
The ES team is doing something similar for enrich stats - see elastic/elasticsearch#97472.
Changing the approach for node ID to still return the field but always set to "serverless" in serverless. This avoids causing problems for users who want to use the stateful client with serverless.
Co-authored-by: James Gowdy <[email protected]>
These will be aggregated and reported as a virtual node called "serverless" to keep the output format similar to current product.
We can avoid making a breaking change to current clients by returning a dummy serverless node to them if used against serverless. The serverless clients will still omit methods to get node information.
Following you can find the validation results for the APIs you have changed.
You can validate these APIs yourself by using the |
Serverless will present the impression of having no underlying servers,
so API responses in serverless should not contain any node information.
To avoid changing the shape of responses, we still return node fields
but always set to "serverless" in serverless. This avoids causing
problems for users who want to use the stateful client with serverless.
Trained model stats will be aggregated and reported as being for a
virtual node called "serverless".