fix: update tool result content formatting in AnthropicAugmentedLLM #95
Conversation
StreetLamb
changed the title
StreetLamb
changed the title
There was a problem hiding this comment.
Great fix, @StreetLamb! Wondering about the proper place for this, but otherwise lgtm
| # Convert mcp CallToolResult content to anthropic ToolResultBlockParam content | ||
| tool_result_content: list[ | ||
| Union[TextBlockParam, ImageBlockParam] | ||
| ] = [] | ||
| for content_block in result.content: | ||
| if isinstance(content_block, TextContent): | ||
| tool_result_content.append( | ||
| TextBlockParam(type="text", text=content_block.text) | ||
| ) | ||
| elif isinstance(content_block, ImageContent): | ||
| tool_result_content.append( | ||
| ImageBlockParam( | ||
| type="image", | ||
| source=Base64ImageSourceParam( | ||
| type="base64", | ||
| data=content_block.data, | ||
| media_type=content_block.mimeType, | ||
| ), | ||
| ) | ||
| ) | ||
| elif isinstance(content_block, EmbeddedResource): | ||
| if isinstance( | ||
| content_block.resource, TextResourceContents | ||
| ): | ||
| return TextBlockParam( | ||
| type="text", text=content_block.resource.text | ||
| ) | ||
| else: | ||
| # Best effort to convert since this is not supported by anthropic ToolResultBlockParam. | ||
| return TextBlockParam( | ||
| type="text", | ||
| text=f"{content_block.resource.mimeType}:{content_block.resource.blob}", | ||
| ) | ||
|
|
There was a problem hiding this comment.
Should this logic be in or utilize the AnthropicMCPTypeConverter class below?
There was a problem hiding this comment.
I was planning to leverage on from_mcp_message_param method, but it expects a MessageParam containing a single content block, whereas the call_tool returns a result with a list of content blocks so they are not really compatible.
There was a problem hiding this comment.
What if we loop through the list of content blocks returned by the call_tool and use the from_mcp_message_param method?
Alternatively, we could add a from_mcp_tool_call method to the type converter. I feel like this would be a common enough issue across all model providers. But lmk what you think -- happy to merge in as is as well
There was a problem hiding this comment.
Added a new from_mcp_tool_result method to handle tool results from MCP servers. Let me know what you think!
) * fix: update tool result content formatting in AnthropicAugmentedLLM * fix: include type attribute in tool result content blocks * Enhance tool result content conversion * feat: add conversion method for MCP tool results to LLM input type * feat: refactor tool result handling in AnthropicAugmentedLLM
Fixes #94. Previously MCP server response from tool calling was not converted to Anthropic's ToolResultBlockParam content type.