Open
Description
Similar to #221
There are a number of places where AugmentedLLMs are first producing text responses and then translating to structured responses.
e.x. in the Orchestrator
async def generate_structured(
self,
message: str | MessageParamT | List[MessageParamT],
response_model: Type[ModelT],
request_params: RequestParams | None = None,
) -> ModelT:
"""Request a structured LLM generation and return the result as a Pydantic model."""
params = self.get_request_params(request_params)
result_str = await self.generate_str(message=message, request_params=params)
llm = self.llm_factory(
agent=Agent(
name="Structured Output",
instruction="Produce a structured output given a message",
)
)
structured_result = await llm.generate_structured(
message=result_str,
response_model=response_model,
request_params=params,
)
return structured_result
We should refactor AugmentedLLMs to use structured requests internally and eliminate anywhere we are gathering a string and pushing it through a completion to structure it
Metadata
Metadata
Assignees
Labels
No labels