Skip to content

Commit 978a872

Browse files
committed
Create a smaller index to be used on the summary page
While rust-lang#1312 improved that query a lot, we can shave off even more time by giving it an index containing only the data it's going to need. This index would need to be refreshed daily by a heroku scheduler task at 00:01 UTC. I don't have a large enough dataset locally to get a great perf comparison, so it'd be great to have a test comparison on prod before we merge this. The funkiness in how we go about creating/dropping the index is due to the fact that we can't use bind parameters to create indices, nor can we use `CURRENT_DATE` directly. If we want to create the index concurrently (I don't think we need to), then we will have to do string concatenation in Rust.
1 parent 8d266d0 commit 978a872

File tree

3 files changed

+33
-0
lines changed

3 files changed

+33
-0
lines changed
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
DROP FUNCTION refresh_todays_recent_crate_downloads();
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
CREATE FUNCTION refresh_todays_recent_crate_downloads() RETURNS void AS $$
2+
BEGIN
3+
DROP INDEX IF EXISTS todays_recent_crate_downloads;
4+
EXECUTE 'CREATE INDEX todays_recent_crate_downloads
5+
ON crate_downloads (date)
6+
WHERE date > date(''' || CURRENT_DATE::text || '''::date - INTERVAL ''90 days'')';
7+
ANALYZE crate_downloads;
8+
END;
9+
$$ LANGUAGE plpgsql;

src/bin/update-daily-indices.rs

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
// Creates any partial indexes that need to be updated daily
2+
//
3+
// Should be run at just after midnight UTC on a daily basis
4+
5+
#![deny(warnings)]
6+
7+
extern crate cargo_registry;
8+
#[macro_use]
9+
extern crate diesel;
10+
11+
use diesel::prelude::*;
12+
use diesel::select;
13+
14+
fn main() {
15+
let conn = cargo_registry::db::connect_now().unwrap();
16+
make_indices(&conn).unwrap();
17+
}
18+
19+
no_arg_sql_function!(refresh_todays_recent_crate_downloads, ());
20+
fn make_indices(conn: &PgConnection) -> QueryResult<()> {
21+
select(refresh_todays_recent_crate_downloads).execute(conn)?;
22+
Ok(())
23+
}

0 commit comments

Comments
 (0)