Skip to content

Make Inference API task_settings optional #2994

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 10, 2024
Merged

Conversation

davidkyle
Copy link
Member

@davidkyle davidkyle commented Oct 8, 2024

task_settings are optional when creating an endpoint in the Inference API and may not be returned in the GET response. The spec incorrectly has task_settings as required for both creation (PUT) and retrieval (GET), this PR changes that

The Elasticsearch documentation contains an example where an endpoint is created without task_settings ,see creating the ELSER service.

Copy link
Contributor

github-actions bot commented Oct 8, 2024

Following you can find the validation results for the APIs you have changed.

API Status Request Response
inference.delete Missing test Missing test
inference.get 🟢 4/4 4/4
inference.inference Missing test Missing test
inference.put Missing test Missing test
inference.stream_inference 🟠 Missing type Missing type

You can validate these APIs yourself by using the make validate target.

@pquentin
Copy link
Member

pquentin commented Oct 9, 2024

Thanks! This pull request makes a response field optional, which is why I asked a review from static client maintainers here.

Copy link
Contributor

@l-trotta l-trotta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no problems for java

Copy link
Member

@pquentin pquentin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussion with the team, the conclusion is that making a response field optional is OK. User code handling the case where task_settings is set won't change. (Except in Rust, where it won't compile because the value will be wrapped in std::Option. But the Rust client isn't ready yet.)

Making a field optional in requests is actually more of an issue, because that means constructors need to change to include a variant without that field. Before that change, users will be forced to set this field.

But in any case, since this a bug fix to a tech preview API, this pull request is fine. I would guess that today multiple clients would be failing if given a payload without task settings anyway.

@pquentin pquentin merged commit de300c8 into main Oct 10, 2024
7 checks passed
@pquentin pquentin deleted the optional-task-settings branch October 10, 2024 14:03
github-actions bot pushed a commit that referenced this pull request Oct 10, 2024
l-trotta pushed a commit that referenced this pull request Oct 14, 2024
(cherry picked from commit de300c8)

Co-authored-by: David Kyle <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants