Llama4 as Evaluator

[PrashantDixit0]

Describe the Feature
Llama4 is recently released and it is benchmarking as one of the best evaluators.

Why is the feature important for you?
With 10M of context window, I believe it will be game-changer evaluator.

Additional context
As its newly released, it will give a boost to Ragas also if we implement it faster and bring it show 😄

[sahusiddharth]

Hi @PrashantDixit0, we will soon be releasing a tutorial showing evaluations with Llama 4

If you are planning to do evaluations with Llama and using TogetherAI Api you can do it like this,

from langchain_together import ChatTogether
from ragas.llms import LangchainLLMWrapper

llm = ChatTogether(
    model="meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8",
)
evaluator_llm = LangchainLLMWrapper(llm)

if you are using something else if it is supported by Langchain, just wrap it with LangchainLLMWrapper and you are good to go.