Skip to content

Commit 46064b4

Browse files
committed
enable_thinking and assistant prefill cannot be enabled at the same time
1 parent 2950f62 commit 46064b4

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

tools/server/utils.hpp

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -768,6 +768,11 @@ static json oaicompat_chat_params_parse(
768768

769769
/* TODO: test this properly */
770770
inputs.reasoning_format = COMMON_REASONING_FORMAT_NONE;
771+
772+
if(inputs.chat_template_kwargs.find("enable_thinking") != inputs.chat_template_kwargs.end()) {
773+
throw std::runtime_error("Assistant response prefill is incompatible with enable_thinking.");
774+
}
775+
771776
inputs.add_generation_prompt = true;
772777
}
773778

0 commit comments

Comments
 (0)