Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As has been noted, the status page shows old benchmarks in the section on build success. This is because the query for errors is incorrect.
The code previously made the assumption that the set of benchmarks that have ever had an error and the set of benchmarks run for the artifact in question are the same, but this has long not been true (and is very, very not true now).
First, we need to change the sql query: When doing a left join, all rows from the left table (in this case error_series) will be in the resulting table with the rows nulled out where the inner join condition is not met. This is not what we want. Instead, we want an inner join which only selects row where the join condition is met.
However, the frontend still makes the assumption that the status page shows all the benchmarks that were run for the previous commit. But we can no longer rely on the
get_error
data giving us that data.There are two ways to go about fixing this:
Because option 2 is simpler, I went with that for now, but we can discuss whether we want to actually go with option 1.