feat: add batch support for `LLM.generate`

[micpst]

Feature description

Add support for list of prompts as an input for LLM.generate method, also add adjust the interface for the generate_streaming and generate with_metadata. I'm talking about something like this:

responses = await LLiteLLM.generate([ImageDescriberPrompt(...), ...])

It should also work for strings and other types:

responses = await LLiteLLM.generate(["Tell me the meaning of life", ...])

Motivation

Currently there is no way to send multiple prompts in a single request to the LLM. There are many use cases that can take advantage of this feature, like LLM as judge evaluation pipeline or LLM reranker that need to process the same prompts with different input data.

Additional context

No response

[ds-sebastianchwilczynski]

So far:

• I used singledispatch to separate logic of list vs single item. Inside, I used asyncio.gather over original generate calls -> Basically client sends tens of requests to the back end and it's on the server side to batch it.
• I've heard that we also need served side parallelism, where client sends one request with all messages, but this is problematic as leads to chain of changes as all functions so far assumes the get a single message.
• We decided that generate_streaming with batches doesn't make much sense so this will stay as it is.
• There is also a small problem with traces, rich supports only one session at a time - So if we run many sessions in parallel it simply doesn't work. OpenTelemetry on the other hand seems to treat many asynchronous calls as one.