Skip to content

Use subquery to instead In #10874

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 30, 2020

Conversation

lunny
Copy link
Member

@lunny lunny commented Mar 29, 2020

Sometimes id will be more than 500 that will result in 500.

@lunny lunny added the type/refactoring Existing code has been cleaned up. There should be no new functionality. label Mar 29, 2020
@lunny lunny added this to the 1.12.0 milestone Mar 29, 2020
@GiteaBot GiteaBot added the lgtm/need 1 This PR needs approval from one additional maintainer to be merged. label Mar 29, 2020
@lafriks lafriks added the skip-changelog This PR is irrelevant for the (next) changelog, for example bug fixes for unreleased features. label Mar 29, 2020
@@ -1153,6 +1155,14 @@ func (opts *IssuesOptions) setupSession(sess *xorm.Session) {
}
}
}

if len(opts.IncludedLabelNames) > 0 {
sess.In("issue.id", BuildLabelNamesIssueIDsCondition(opts.IncludedLabelNames))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guillep2k can we do better here? I'm suspicious that this In could be quite expensive.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WHERE EXISTS (SELECT ...) is the equivalent idiom, but depending on the engine it could be better optimized.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It all depends on how sophisticated the engine is. Some engines will rearrange the whole query and be very clever on optimizing the access... others will just take your statement more or less literally, and therefore your syntax might influence the results drastically. And versions are important too. I read from time to time "XX SQL now takes this construct and treats it like xxx for better performance".

Formally, WHERE EXISTS (SELECT ...) is equivalent to IN (SELECT ....); but if the engine is not too smart, EXISTS is a kind of join that is more optimizable than a plain LOOP JOIN (e.g. using a HASH JOIN).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But performing the IN() in a subquery is definitely more performant that SELECT ids, and then `SELECT ... WHERE ... IN(id list).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like @guillep2k said, it depends on the database engine. The sub query maybe cached or not. It may be faster than before or not. But it will give the right result however many there are labels. But for In with id, when there are over 500 variables, database will return error.

@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Mar 29, 2020
@mrsdizzie mrsdizzie mentioned this pull request Mar 29, 2020
6 tasks
@codecov-io
Copy link

Codecov Report

Merging #10874 into master will increase coverage by 0.00%.
The diff coverage is 34.61%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master   #10874   +/-   ##
=======================================
  Coverage   43.41%   43.42%           
=======================================
  Files         593      593           
  Lines       83332    83344   +12     
=======================================
+ Hits        36178    36190   +12     
+ Misses      42660    42659    -1     
- Partials     4494     4495    +1     
Impacted Files Coverage Δ
models/issue.go 51.17% <0.00%> (-0.25%) ⬇️
models/issue_label.go 61.60% <0.00%> (-1.45%) ⬇️
routers/api/v1/repo/issue.go 54.89% <75.00%> (+0.38%) ⬆️
modules/git/command.go 86.95% <0.00%> (-2.61%) ⬇️
services/pull/check.go 52.43% <0.00%> (-0.61%) ⬇️
services/pull/pull.go 34.70% <0.00%> (-0.30%) ⬇️
modules/queue/workerpool.go 58.00% <0.00%> (+1.06%) ⬆️
modules/git/repo.go 48.53% <0.00%> (+1.25%) ⬆️
services/pull/temp_repo.go 31.62% <0.00%> (+2.56%) ⬆️
services/pull/patch.go 64.51% <0.00%> (+2.58%) ⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5c3be56...769aa8e. Read the comment docs.

@guillep2k guillep2k merged commit f490291 into go-gitea:master Mar 30, 2020
@lunny lunny deleted the lunny/use_sub_query_instead_in branch March 30, 2020 07:19
@go-gitea go-gitea locked and limited conversation to collaborators Nov 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. skip-changelog This PR is irrelevant for the (next) changelog, for example bug fixes for unreleased features. type/refactoring Existing code has been cleaned up. There should be no new functionality.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants