Skip to content

[Bug] History constructs assistant messages without [[ ## completed ## ]] tags. #8261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
marwood-p opened this issue May 22, 2025 · 0 comments · May be fixed by #8340
Open

[Bug] History constructs assistant messages without [[ ## completed ## ]] tags. #8261

marwood-p opened this issue May 22, 2025 · 0 comments · May be fixed by #8340
Labels
bug Something isn't working

Comments

@marwood-p
Copy link

marwood-p commented May 22, 2025

What happened?

The DSPy History example produces an incorrectly formatted assistant message.
https://dspy.ai/api/primitives/History/

The System message says the [[ ## answer ## ]] should be followed by [[ ## completed ## ]]. This is not a response from the LM! It is a answer constructed inside Adapter.format_assistant_message_content().

Steps to reproduce

import dspy

dspy.settings.configure(lm=dspy.LM("openai/gpt-4o-mini"))

class MySignature(dspy.Signature):
    question: str = dspy.InputField()
    history: dspy.History = dspy.InputField()
    answer: str = dspy.OutputField()

history = dspy.History(
    messages=[
        {"question": "What is the capital of France?", "answer": "Paris"},
        {"question": "What is the capital of Germany?", "answer": "Berlin"},
    ]
)

predict = dspy.Predict(MySignature)
outputs = predict(question="What is the capital of France?", history=history)

print('System message:')
print(dspy.settings.lm.history[-1]['messages'][0]['content'])
print('Constructed assistant message:')
print(dspy.settings.lm.history[-1]['messages'][2]['content'])
System message:
Your input fields are:
1. `question` (str)
2. `history` (History)
Your output fields are:
1. `answer` (str)
All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## question ## ]]
{question}

[[ ## history ## ]]
{history}

[[ ## answer ## ]]
{answer}

[[ ## completed ## ]]
In adhering to this structure, your objective is: 
        Given the fields `question`, `history`, produce the fields `answer`.
Constructed assistant message:
[[ ## answer ## ]]
Paris

The [[ ## answer ## ]] tag constructed by Adapter.format_assistant_message_content() should have a [[ ## completed ## ]] tag after it.

Thanks for your excellent work on DSPy!

DSPy version

2.6.23

@marwood-p marwood-p added the bug Something isn't working label May 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant