Add OVHcloud as an inference provider

[fabienric]

What

Adds OVHcloud as an inference provider.

Test Plan

Added new tests for OVHcloud both with and without streaming.

What Should Reviewers Focus On?

I used the Cerebras PR as an example.

[julien-c]

Hi @fabienric we are currently finishing a refactoring of Inference Providers integration code in #1315, this should be merged soon, but we will need to rewrite part of your implementation (should be even simpler to integrate), will ping again after it's been merged.

[hanouticelina]

Hi @fabienric,
We've merged a refactoring for Inference Provider integration into main, which should make adding new providers much easier.
Could you merge main into your branch and update your PR accordingly? it should be relatively straightforward with the new structure:
1 - You have to update the PROVIDERS mapping here: inference/src/lib/getProviderHelper.ts#L49 and add ovhcloud (let's ensure we respect the alphabetical order) :

import * as OvhCloud from "../providers/ovhcloud";
...
export const PROVIDERS: Record<InferenceProvider, Partial<Record<InferenceTask, TaskProviderHelper>>> = {
	...
	"ovhcloud": {
		"conversational": new OvhCloud.OvhCloudConversationalTask(),
	},
        ...

2 - Update packages/inference/src/providers/ovhcloud.ts to implement OvhCloudConversationalTask that inherits BaseConversationalTask:

import { BaseConversationalTask, BaseTextGenerationTask } from "./providerHelper";

export class OvhCloudConversationalTask extends BaseConversationalTask {
	constructor() {
		super("ovhcloud", "https://oai.endpoints.kepler.ai.cloud.ovh.net");
	}
}

and that's it :) let us know if you need any help! you can find more details in the documentation : https://huggingface.co/docs/inference-providers/register-as-a-provider#2-js-client-integration.

[julien-c]

(sorry for the moving parts @fabienric – we can help move this PR over the finish line if needed)

[fabienric]

Hi @hanouticelina and @julien-c,

Thank you for the feedback, refactoring and updated documentation.

I've implemented our provider ; it required more work than I expected to get the payload right for an OpenAI compatible endpoint (make sure that the seed, max_tokens and other generation parameters are correctly mapped: the base implementation put them in a parameters dictionary, which is ignored on our end). Maybe other providers are impacted by this issue?

I've also implemented the text generation task, but I've found that the streaming case is not covered by the base task (the getResponse returns a Promise<TextGenerationOutput> where we would need a way to return a TextGenerationStreamOutput). My test case passes using ChatCompletionStreamOutput but this doesn't seem right. Maybe I missed something here.

Available to discuss the matter further if required.

[Wauplin]

Maybe other providers are impacted by this issue?

Actually most (if not all) providers are implemented only for the conversational task which is by far the most used task. It covers both text-generation and image-text-to-text pipelines as long as the model is tagged as conversational (i.e. it has a chat template, etc.). That is why the text-generation implementation has been left aside a bit for now. Let us know if the intention is indeed to cover text-generation AND conversational, in which case we can think about better improvements for text-generation (streaming option, OAI-compatible API, etc.). I do think it'll better to ship conversational-only for now to get the integration running and then go back to "non-conversational text-generation"

[fabienric]

Hi @Wauplin and thanks for your feedback.

Actually my remark on the OpenAI compatible parameters also applies to the conversational case ; let me know if it's not the case.

I agree with you on the fact that the priority on our side is to get the conversational use case running. I can remove the code related to text-generation if needed but I think we can leave it as is.

Let me know when you're ready to merge.

[hanouticelina]

thank you @fabienric for the PR, I left some comments

[SBrandeis]

How should we proceed to test the Inference Endpoints?
Do we need to setup a OVH cloud account?

[fabienric]

Thank you @hanouticelina and @SBrandeis for your feedback and suggestions.
You're right, no need to change the payload for the ChatCompletion task. I thought the ChatCompletion task had the same kind of parameters argument than the TextGeneration task, hence the confusion.
It is fixed now.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[hanouticelina]

looks good @fabienric, thank you for the contribution!
as you probably noticed, we dropped VCR tapes which means all new providers PRs need a bit of manual testing (until we implement a better testing strategy), Did you already run some tests for the PR locally? if we want to test on our side, do we need an OVH Cloud account?

[fabienric]

Thank you, yes I already tested locally. I can send you an API key so you can test on your side as well. Le lun. 28 avr. 2025, 12:34, célina ***@***.***> a écrit :

…

***@***.**** approved this pull request. looks good @fabienric <https://github.com/fabienric>, thank you for the contribution! as you probably noticed, we dropped VCR tapes which means all new providers PRs need a bit of manual testing (until we implement a better testing strategy), Did you already run some tests for the PR locally? if we want to test on our side, do we need an OVH Cloud account? — Reply to this email directly, view it on GitHub <#1303 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPDMQEH4WEGDQU642OP72D23X72FAVCNFSM6AAAAABZPVMBVGVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDOOJYHA2DKMRSG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[julien-c]

(we do have a team ovh account already)

[fabienric]

@hanouticelina @julien-c ok, let me know if you need any help with the API token generation.

[hanouticelina]

I tested a couple of models for both chat completion and text generation tasks and it works as expected ✅ thank you @fabienric for the contribution!

[fabienric]

Thank you @hanouticelina!