Skip to content

server : tuning tests #7388

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 20, 2024
Merged

server : tuning tests #7388

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f159c9d
server : don't pass temperature as string
ggerganov May 19, 2024
1899632
server : increase timeout
ggerganov May 19, 2024
8ed8fa9
tests : fix the fix 0.8f -> 0.8
ggerganov May 20, 2024
cc98fdd
tests : set explicit temperature
ggerganov May 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions examples/server/tests/features/results.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Feature: Results

Scenario Outline: consistent results with same seed
Given <n_slots> slots
And 0.0 temperature
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be set to temperature 1, not temperature 0. With temperature 0 you get greedy decoding in which case the seed is irrelevant. So this test can no longer detect issues with e.g. the RNG state being shared between slots.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, will push a fix now

Then the server is starting
Then the server is healthy

Expand All @@ -30,6 +31,7 @@ Feature: Results

Scenario Outline: different results with different seed
Given <n_slots> slots
And 1.0 temperature
Then the server is starting
Then the server is healthy

Expand Down
4 changes: 2 additions & 2 deletions examples/server/tests/features/steps/steps.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ async def step_wait_for_the_server_to_be_started(context, expecting_status):

case 'ready' | 'idle':
await wait_for_health_status(context, context.base_url, 200, 'ok',
timeout=10,
timeout=30,
params={'fail_on_no_slot': 0, 'include_slots': 0},
slots_idle=context.n_slots,
slots_processing=0,
Expand Down Expand Up @@ -883,7 +883,7 @@ async def request_completion(prompt,
"cache_prompt": cache_prompt,
"id_slot": id_slot,
"seed": seed if seed is not None else 42,
"temperature": temperature if temperature is not None else "0.8f",
"temperature": temperature if temperature is not None else 0.8,
"n_probs": 2,
},
headers=headers,
Expand Down
Loading