Create a smaller index to be used on the summary page #1328
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
While rust-lang#1312 improved that query a lot, we can shave off even more time by giving it an index containing only the data it's going to need. This index would need to be refreshed daily by a heroku scheduler task at 00:01 UTC. I don't have a large enough dataset locally to get a great perf comparison, so it'd be great to have a test comparison on prod before we merge this. The funkiness in how we go about creating/dropping the index is due to the fact that we can't use bind parameters to create indices, nor can we use `CURRENT_DATE` directly. If we want to create the index concurrently (I don't think we need to), then we will have to do string concatenation in Rust.
jtgeibel
reviewed
Apr 6, 2018
| DROP INDEX IF EXISTS todays_recent_crate_downloads; | ||
| EXECUTE 'CREATE INDEX todays_recent_crate_downloads | ||
| ON crate_downloads (date) | ||
| WHERE date > date(''' || CURRENT_DATE::text || '''::date - INTERVAL ''90 days'')'; |
There was a problem hiding this comment.
Does the WHERE clause in the query need to match to ensure the index is used? Especially regarding CURRENT_TIMESTAMP vs CURRENT_DATE. The query is currently:
WHERE crate_downloads.date > date(CURRENT_TIMESTAMP - INTERVAL '90 days')
There was a problem hiding this comment.
I confirmed that the query as it is written today will use this index before opening the PR
|
#1363 is a better approach |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While #1312 improved that query a lot, we can shave off even more time
by giving it an index containing only the data it's going to need. This
index would need to be refreshed daily by a heroku scheduler task at
00:01 UTC.
I don't have a large enough dataset locally to get a great perf
comparison, so it'd be great to have a test comparison on prod before we
merge this.
The funkiness in how we go about creating/dropping the index is due to
the fact that we can't use bind parameters to create indices, nor can we
use
CURRENT_DATEdirectly. If we want to create the index concurrently(I don't think we need to), then we will have to do string concatenation
in Rust.