Skip to content

chore(cts): report benchmark results #3406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 22, 2024
Merged

chore(cts): report benchmark results #3406

merged 4 commits into from
Jul 22, 2024

Conversation

millotp
Copy link
Collaborator

@millotp millotp commented Jul 21, 2024

🧭 What and Why

Report benchmark results in the generation comment directly

@millotp millotp self-assigned this Jul 21, 2024
@algolia-bot
Copy link
Collaborator

algolia-bot commented Jul 21, 2024

✔️ Code generated!

Name Link
🪓 Triggered by ae20c83379d98fcb6c961eb6f685432b8b19abcb
🍃 Generated commit ead2fc56a4659e0c67e74a534e026643a1c0d987
🌲 Generated branch generated/chore/report-benchmark
📊 Benchmark results

Benchmarks performed on the method using a mock server, the results might not reflect the real-world performance.

Language Rate
javascript 1373
php 1362
csharp 1076
java 909
ruby 897
swift 744
python 729
kotlin 466
go 454

@millotp millotp marked this pull request as ready for review July 22, 2024 09:03
@millotp millotp requested a review from a team as a code owner July 22, 2024 09:03
@millotp millotp requested review from morganleroi and shortcuts and removed request for a team July 22, 2024 09:03
Copy link
Member

@shortcuts shortcuts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wonderful

{
echo 'BENCHMARK_SECTION<<EOF'
echo "<details>"
echo "<summary>📊 Benchmark results</summary>"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add a small disclaimer like

take those results with a grain of salt, best effort benchmarking solution, in order to highlight general performances of our Search API

or something like that? just so people don't expect those to be a source of truth or things like that

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also we should mention that it's only tested on search?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a small disclaimer, I would like to add more benchmark in the future so I'm not very specific.

echo 'BENCHMARK_SECTION<<EOF'
echo "<details>"
echo "<summary>📊 Benchmark results</summary>"
echo "" # empty line is required to make the table work
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we still need it now that we have a comment?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need what ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this specific line 492 which add an empty line

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2024-07-22 at 14 43 11

yes otherwise it looks like this

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahhh ok make sense

Base automatically changed from chore/benchmark to main July 22, 2024 12:32
@millotp millotp force-pushed the chore/report-benchmark branch from 564e1c8 to 64f7b3d Compare July 22, 2024 12:45
Copy link
Member

@shortcuts shortcuts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Copy link

@millotp millotp enabled auto-merge (squash) July 22, 2024 13:00
@millotp millotp merged commit 0e13b69 into main Jul 22, 2024
21 checks passed
@millotp millotp deleted the chore/report-benchmark branch July 22, 2024 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants