Skip to content

Orchestrator likes to tell agents that they should call agents #223

Open
@strawgate

Description

@strawgate

Currently when planning steps the orchestrator is told to return:


FULL_PLAN_PROMPT_TEMPLATE = """You are tasked with orchestrating a plan to complete an objective.
You can analyze results from the previous steps already executed to decide if the objective is complete.
Your plan must be structured in sequential steps, with each step containing independent parallel subtasks.

Objective: {objective}

{plan_result}

If the previous results achieve the objective, return is_complete=True.
Otherwise, generate remaining steps needed.

You have access to the following MCP Servers (which are collections of tools/functions),
and Agents (which are collections of servers):

Agents:
{agents}

Generate a plan with all remaining steps needed.
Steps are sequential, but each Step can have parallel subtasks.
For each Step, specify a description of the step and independent subtasks that can run in parallel.
For each subtask specify:
    1. Clear description of the task that an LLM can execute  
    2. Name of 1 Agent OR List of MCP server names to use for the task
    
Return your response in the following JSON structure:
    {{
        "steps": [
            {{
                "description": "Description of step 1",
                "tasks": [
                    {{
                        "description": "Description of task 1",
                        "agent": "agent_name"  # For AgentTask
                    }},
                    {{
                        "description": "Description of task 2", 
                        "agent": "agent_name2"
                    }}
                ]
            }}
        ],
        "is_complete": false
    }}

You must respond with valid JSON only, with no triple backticks. No markdown formatting.
No extra text. Do not wrap in ```json code fences."""

The description in the task is not actually the description of the task, it's the information passed to the Agent. This causes the planner to produce a plan which says things like:

          {
            "description": "Ask the geography agent: What is the capital of France?",
            "agent": "geography"
          }

This causes the geography agent to return:

          {
            "description": "Ask the geography agent: What is the capital of France?",
            "result": "What is the capital of France?",
            "agent": "geography"
          }

This causes the planner to continuously plan additional steps where it thinks the geography agent is now asking it a question.

We should either fix the prompts to tell the planner that the description is the instruction provided to the agent:

Note: You are instructing these agents. The description you give is given directly to the agent as instructions.

appears to be sufficient.

The planner is using a structured response so it should be getting the Field metadata. So we could just override the Field definition on the description field to indicate that it holds the action to be performed by the agent:

    description: str = Field(description="Describes what the agent needs to do in order to complete the task")

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions