Skip to content

Make import command resumabe #242

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

mkilmanas
Copy link
Contributor

Pull Request

Make it possible to resume import from a mid-point after unsuccessful previous import

Related issue

Fixes #241

What does this PR do?

  • Makes import resumable:
    • Introduce --skip-batches parameter
    • Enforce entity sorting by primary key (consistent, fast)
    • Display the number of processed entities
    • Add test case
  • Other small improvements:
    • Clarify wording of the message telling the number of indexed records
  • Update existing tests for modified messages

PR checklist

Please check if your PR fulfills the following requirements:

  • Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
  • Have you read the contributing guidelines?
  • Have you made sure that the title is accurate and descriptive of the changes?

Copy link
Member

@brunoocasali brunoocasali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mkilmanas thanks for your contribution. I didn't find anything that could be improved!

So I'll merge it and release a new version since according to your issue you're having problems in production!

@brunoocasali
Copy link
Member

bors merge

@brunoocasali brunoocasali added the enhancement New feature or request label Apr 13, 2023
@bors bors bot merged commit 747f035 into meilisearch:main Apr 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make import process resumable
2 participants