Use tool calls on Gemini for structured responses

[strawgate]

When using Gemini we're currently using a second chat completion to produce structured responses. Gemini has a mode where you can require tool calls:

The Gemini API lets you control how the model uses the provided tools (function declarations). Specifically, you can set the mode within the function_calling_config.

ANY: The model is constrained to always predict a function call and guarantee function schema adherence. If allowed_function_names is not specified, the model can choose from any of the provided function declarations. If allowed_function_names is provided as a list, the model can only choose from the functions in that list. Use this mode when you require a function call in response to every prompt (if applicable).

Ideally,if you're trying to produce a plan of tasks you would set it to function calls only and give it a tool call for reporting back the results.