Skip to content

Commit 9cf06d1

Browse files
committed
Better index version downloads for the update script
The query used to grab the next batch of unprocessed version downloads is now taking on average 9 seconds to run. It can take as much as 30 seconds. I suspect this is because the size of the `processed` index, or some other measurement PG is using changed so it decided that index wasn't the right one to use. That index wasn't very helpful anyway (it was just the `processed` column). We can do better by indexing the id with the where clause. `EXPLAIN ANALYZE` results: Before: ``` QUERY PLAN --------------------------------------------------------------------------------------------------------------------------------------------------------------------- Limit (cost=0.09..16204.73 rows=1000 width=21) (actual time=39279.902..39298.408 rows=1000 loops=1) -> Index Scan using version_downloads_pkey on version_downloads (cost=0.09..199171.32 rows=12291 width=21) (actual time=39279.900..39298.335 rows=1000 loops=1) Index Cond: (id > 0) Filter: (NOT processed) Rows Removed by Filter: 8544073 Planning time: 0.369 ms Execution time: 39298.459 ms (7 rows) ``` After: ``` QUERY PLAN -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Limit (cost=0.06..95.89 rows=1000 width=21) (actual time=0.016..0.796 rows=1000 loops=1) -> Index Scan using index_version_downloads_not_processed on version_downloads (cost=0.06..1175.30 rows=12264 width=21) (actual time=0.016..0.732 rows=1000 loops=1) Index Cond: (id > 0) Planning time: 0.123 ms Execution time: 0.854 ms ```
1 parent e4c3f4d commit 9cf06d1

File tree

2 files changed

+8
-0
lines changed
  • migrations/2018-04-13-170549_properly_index_version_downloads

2 files changed

+8
-0
lines changed
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
CREATE INDEX index_version_downloads_processed
2+
ON version_downloads (processed)
3+
WHERE processed = FALSE;
4+
DROP INDEX index_version_downloads_not_processed;
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
CREATE UNIQUE INDEX index_version_downloads_not_processed
2+
ON version_downloads (id)
3+
WHERE processed = FALSE;
4+
DROP INDEX index_version_downloads_processed;

0 commit comments

Comments
 (0)