-
Notifications
You must be signed in to change notification settings - Fork 21
chore(cts): report benchmark results #3406
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
✔️ Code generated!
📊 Benchmark resultsBenchmarks performed on the method using a mock server, the results might not reflect the real-world performance.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wonderful
{ | ||
echo 'BENCHMARK_SECTION<<EOF' | ||
echo "<details>" | ||
echo "<summary>📊 Benchmark results</summary>" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we add a small disclaimer like
take those results with a grain of salt, best effort benchmarking solution, in order to highlight general performances of our Search API
or something like that? just so people don't expect those to be a source of truth or things like that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also we should mention that it's only tested on search?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added a small disclaimer, I would like to add more benchmark in the future so I'm not very specific.
echo 'BENCHMARK_SECTION<<EOF' | ||
echo "<details>" | ||
echo "<summary>📊 Benchmark results</summary>" | ||
echo "" # empty line is required to make the table work |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we still need it now that we have a comment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
need what ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this specific line 492 which add an empty line
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ahhh ok make sense
564e1c8
to
64f7b3d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧭 What and Why
Report benchmark results in the generation comment directly