Skip to content

Commit 00aff35

Browse files
ngxsonggerganov
authored andcommitted
server : simplify state machine for slot (ggml-org#9283)
* server : simplify state machine for slot * add SLOT_STATE_DONE_PROMPT * pop_deferred_task * add missing notify_one * fix passkey test * metrics : add n_busy_slots_per_decode * fix test step * add test * maybe fix AddressSanitizer? * fix deque ? * missing lock * pop_deferred_task: also notify * Update examples/server/server.cpp Co-authored-by: Georgi Gerganov <[email protected]> --------- Co-authored-by: Georgi Gerganov <[email protected]>
1 parent ef27803 commit 00aff35

File tree

4 files changed

+147
-93
lines changed

4 files changed

+147
-93
lines changed

0 commit comments

Comments
 (0)