Skip to content

Adding segment, totalSegments parameters in dynamodb-enhanced client to support parallel scan #2614

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 4, 2021

Conversation

asanthan-amazon
Copy link
Contributor

Adding segment, totalSegments parameters in dynamodb-enhanced client to support parallel scan.

Motivation and Context

This change is required to support parallel scan on dynamo-db table using enhanced-client.
Feature Request: #1851

Description

It allows sdk user to specify 'segment' and 'totalSegments' parameter in 'ScanEnhancedRequest' model.

Testing

I added unit tests wherever possible. DynamoDB local does not support parallel scan. So, the API flow was tested manually using integration test wherein I ran test to hit actual DDB in parallel using 5 threads and ensured the final outcome retrieved all the items after scan completes on all 5 threads.

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • [ x] New feature (non-breaking change which adds functionality)

Checklist

  • [x ] I have read the CONTRIBUTING document
  • [x ] Local run of mvn install succeeds
  • [x ] My code follows the code style of this project
  • [x ] My change requires a change to the Javadoc documentation
  • [x ] I have updated the Javadoc documentation accordingly
  • [x ] I have read the README document
  • [x ] I have added tests to cover my changes
  • [x ] All new and existing tests passed
  • [ x] A short description of the change has been added to the CHANGELOG
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • [x ] I confirm that this pull request can be released under the Apache 2 license

asanthan-amazon and others added 2 commits July 21, 2021 13:54
…o support parallel scan

Note: DynamoDB local does not support parallel scan. So, this change was tested manually using integration test.
@sonarqubecloud
Copy link

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

50.0% 50.0% Coverage
4.5% 4.5% Duplication

Copy link
Contributor

@bmaizels bmaizels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM and a logical extension of the request (doesn't break the library conceptual precepts). Shame that we can't write a local DDB test for it. Only other thing I'd say is that if we didn't have a test to cover equals and hashcode for ScanEnhancedRequest, that is on us, but since you're touching it it'd be nice if you could add them to improve the code.

@asanthan-amazon
Copy link
Contributor Author

Hi SDK Team,

Any update on merging this change to next release ?

@millems
Copy link
Contributor

millems commented Sep 17, 2021

Thanks for your contribution, will merge as soon as possible.

@asanthan-amazon
Copy link
Contributor Author

Thanks for your contribution, will merge as soon as possible.

Hi, Can you please let me know when this will be merged ?

@asanthan-amazon
Copy link
Contributor Author

Thanks for your contribution, will merge as soon as possible.

Hi, Can you please let me know when this will be merged ?

Hi, I see some merge commits. But, not sure if it is actually merged. Can you please confirm ?

@millems
Copy link
Contributor

millems commented Oct 4, 2021

I will merge today for a release tomorrow.

@millems millems merged commit f489645 into aws:master Oct 4, 2021
@sonarqubecloud
Copy link

sonarqubecloud bot commented Oct 4, 2021

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

50.0% 50.0% Coverage
4.4% 4.4% Duplication

aws-sdk-java-automation added a commit that referenced this pull request Jul 7, 2023
…6192379d2

Pull request: release <- staging/284b25b7-50f5-4e55-b011-8026192379d2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants