[Collections] CLI 'remove' results in "database is locked" error #3225
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Motivation: Using the `package-collection` CLI, the `remove` command results in "database is locked" error even though the collection is removed successfully. This is because a `SQLitePackageCollectionsStorage` is created at the beginning of each command run, and in the initializer the time-consuming `populateTargetTrie` is called. When the command finishes running, it triggers `SQLitePackageCollectionsStorage` to be closed, which in turn closes the SQLite connection, but this is done while `populateTargetTrie` is still running and thus causes the "database is locked" error. Modifications: - Initializing `PackageCollections` can be expensive and should only be done once in a command - Add and set `isShuttingDown` flag when `SQLitePackageCollectionsStorage.close` is called so `populateTargetTrie` knows that it should stop. - Try `db.close` in a second attempt after allowing time for database operations to react to `isShuttingDown` flag - `populateTargetTrie` should memoize its result Result: No "database is locked" error.
yim-lee
requested review from
abertelrud,
friedbunny,
neonichu and
tomerd
as code owners
|
@swift-ci please smoke test |
tomerd
reviewed
Jan 26, 2021
| guard case .success = semaphore.wait(timeout: .now() + .milliseconds(300)) else { | ||
| throw StringError("Failed to close database") | ||
| } | ||
| } |
There was a problem hiding this comment.
in general the approach seems fine, but the delay here could be fragile - if the DB operation is longer than 200/300ms it will still fail with "db is locked", we could potentially do a recursive "exponential backoff" style thing here which would be safer. alternatively we could try to eliminate the issue using "sqlite3_interrupt" before trying to close again
There was a problem hiding this comment.
Exponential backoff: 0a70a65
|
@swift-ci please smoke test |
|
@swift-ci please smoke test |
tomerd
approved these changes
Jan 27, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation:
Using the
package-collectionCLI, theremovecommand results in "database is locked" error even though the collection is removed successfully. This is because aSQLitePackageCollectionsStorageis created at the beginning of each command run, and in the initializer the time-consumingpopulateTargetTrieis called. When the command finishes running, it triggersSQLitePackageCollectionsStorageto be closed, which in turn closes the SQLite connection, but this is done whilepopulateTargetTrieis still running and thus causes the "database is locked" error.Modifications:
PackageCollectionscan be expensive and should only be done once in a commandisShuttingDownflag whenSQLitePackageCollectionsStorage.closeis called sopopulateTargetTrieknows that it should stop.db.closein a second attempt after allowing time for database operations to react toisShuttingDownflagpopulateTargetTrieshould memoize its resultResult:
No "database is locked" error.