Skip to content

Commit a59f8fd

Browse files
authored
Server: Enable setting default sampling parameters via command-line (#8402)
* Load server sampling parameters from the server context by default. * Wordsmithing comment
1 parent fd560fe commit a59f8fd

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

examples/server/server.cpp

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -884,7 +884,8 @@ struct server_context {
884884

885885
bool launch_slot_with_task(server_slot & slot, const server_task & task) {
886886
slot_params default_params;
887-
llama_sampling_params default_sparams;
887+
// Sampling parameter defaults are loaded from the global server context (but individual requests can still override them)
888+
llama_sampling_params default_sparams = params.sparams;
888889
auto & data = task.data;
889890

890891
if (data.count("__oaicompat") != 0) {

0 commit comments

Comments
 (0)