-
Notifications
You must be signed in to change notification settings - Fork 430
[Inference] request()
returns a request context to avoid redundant makeRequestOptions calls
#1314
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR adds different return types, not sure the complication is worth it (unless it saves network calls)
FWIW in the Python client we've deprecated the "custom requests" method to make it internal-only and gain some flexibility -which would prove useful here-. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice!
I would be in favor of removing the request
method from the public API as suggested by @Wauplin
Or at least, don't make the return type conditioned by a parameter (always return ResponseWrapper<T>
)
just to reiterate a bit, it's totally fine to call a "light" sync function twice to simplify things |
…ace.js into update-request-func
I added a deprecation for custom requests through |
AFAIU in this case |
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
ah yes, forgot that we call |
This comment was marked as outdated.
This comment was marked as outdated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Super nice - thanks for keeping the diff very low !!
I have some suggestion regarding syntax / code style, otherwise LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's goooo
Related to @SBrandeis's comment here #1292 (comment). This PR addresses the original concern about redundant
makeRequestOptions
calls introduced in #1292.The solution implemented here updates therequest
function to return both the response and a request context when needed, allowing provider-specific polling code to reuse this context without redundant calls tomakeRequestOptions
.This differs from the initial suggestion in the comment as each provider implements polling differently with different parameters / response formats. Making a generic
.poll
property would require mixing provider-specific logic into the core request function (we don't want that, right? 😄 ).In the end, we want to keep provider-specific logic isolated in their respective provider files (PR coming today to push that further!).