Skip to content

Commit d632202

Browse files
matteoservamatteo
authored andcommitted
enable_thinking and assistant prefill cannot be enabled at the same time
1 parent 2613869 commit d632202

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

tools/server/utils.hpp

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -737,6 +737,10 @@ static json oaicompat_completion_params_parse(
737737
throw std::runtime_error("Cannot have 2 or more assistant messages at the end of the list.");
738738
}
739739

740+
if(inputs.chat_template_kwargs.find("enable_thinking") != inputs.chat_template_kwargs.end()) {
741+
throw std::runtime_error("Assistant response prefill is incompatible with enable_thinking.");
742+
}
743+
740744
inputs.extract_reasoning = false;
741745
inputs.add_generation_prompt = true;
742746
}

0 commit comments

Comments
 (0)