Skip to content

Commit c966dcf

Browse files
committed
test: tweak smoke test tool bodies to standardize response text
Tweak the tool bodies for smoke test GPTScripts to reduce ambiguity in the response. This prevents models -- like gpt-4o -- from doing things like failing to interpolate strings consistently between runs. Signed-off-by: Nick Hale <[email protected]>
1 parent 2bd60f8 commit c966dcf

File tree

2 files changed

+3
-4
lines changed

2 files changed

+3
-4
lines changed

pkg/tests/smoke/testdata/Bob/test.gpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
tools: bob
22

3-
Ask Bob "how are you doing" and repeat his reply exactly.
3+
Ask Bob "how are you doing" and repeat the response text exactly as given without saying anything else.
44

55
---
66
name: bob
77
description: I'm Bob, a friendly guy.
88
args: question: The question to ask Bob.
99

10-
When asked how I am doing, respond with exactly "Thanks for asking "${QUESTION}", I'm doing great fellow friendly AI tool!"
10+
When asked how I am doing, respond with the following exactly: "Thanks for asking ${QUESTION}! I'm doing great fellow friendly AI tool!" with ${QUESTION} replaced with the question text as given.

pkg/tests/smoke/testdata/BobAsShell/test.gpt

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
1-
21
tools: bob
32

4-
Ask Bob "how are you doing" and repeat his reply exactly.
3+
Ask Bob "how are you doing" and repeat the response text exactly as given without saying anything else.
54

65
---
76
name: bob

0 commit comments

Comments
 (0)