Skip to content

SyncResponseHandlerAdapter: Prevent overflow #1542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Dec 17, 2019
Merged

Conversation

rschmitt
Copy link
Contributor

SyncResponseHandlerAdapter requests Long.MAX_VALUE from its
Publisher, not just initially upon subscription but also on every
#onNext call. This means that every Subscription implementation that
does not take steps to prevent overflow (or which throws an exception if
overflow is detected) will eventually see the number of outstanding
requests go negative, and will stop consuming data. Larger responses
require more calls to #onNext and are therefore more likely to
encounter this issue.

The fix is simply to request new messages individually, so that the
number of outstanding requests always hovers around Long.MAX_VALUE.

I confirm that this pull request can be released under the Apache 2 license.

SyncResponseHandlerAdapter requests `Long.MAX_VALUE` from its
`Publisher`, not just initially upon subscription but also on every
`#onNext` call. This means that every `Subscription` implementation that
does not take steps to prevent overflow (or which throws an exception if
overflow is detected) will eventually see the number of outstanding
requests go negative, and will stop consuming data. Larger responses
require more calls to `#onNext` and are therefore more likely to
encounter this issue.

The fix is simply to request new messages individually, so that the
number of outstanding requests always hovers around `Long.MAX_VALUE`.
Copy link
Contributor

@dagnir dagnir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit: Thanks!

@codecov-io
Copy link

Codecov Report

Merging #1542 into master will increase coverage by 0.02%.
The diff coverage is 100%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1542      +/-   ##
============================================
+ Coverage     74.96%   74.98%   +0.02%     
  Complexity      770      770              
============================================
  Files           882      882              
  Lines         27429    27481      +52     
  Branches       2140     2157      +17     
============================================
+ Hits          20561    20607      +46     
- Misses         5897     5907      +10     
+ Partials        971      967       -4
Flag Coverage Δ Complexity Δ
#unittests 74.98% <100%> (+0.02%) 770 <0> (ø) ⬇️
Impacted Files Coverage Δ Complexity Δ
...core/internal/http/async/AsyncResponseHandler.java 88.88% <100%> (ø) 0 <0> (ø) ⬇️
...tomization/CodegenCustomizationProcessorChain.java 81.81% <0%> (-8.19%) 0% <0%> (ø)
...on/awssdk/protocols/ion/AwsIonProtocolFactory.java 72.72% <0%> (-5.06%) 0% <0%> (ø)
.../software/amazon/awssdk/core/sync/RequestBody.java 89.65% <0%> (-3.21%) 0% <0%> (ø)
...ftware/amazon/awssdk/codegen/internal/Jackson.java 59.09% <0%> (-2.82%) 0% <0%> (ø)
...on/awssdk/codegen/internal/DocumentationUtils.java 75% <0%> (-2.42%) 0% <0%> (ø)
...db/mappingclient/extensions/WriteModification.java 52% <0%> (-2.17%) 5% <0%> (ø)
...rvices/s3/internal/ConfiguredS3SdkHttpRequest.java 46.15% <0%> (-1.85%) 0% <0%> (ø)
...o/netty/internal/utils/BetterFixedChannelPool.java 52.48% <0%> (-1.66%) 0% <0%> (ø)
...amodb/mappingclient/operations/ConditionCheck.java 41.66% <0%> (-1.2%) 3% <0%> (ø)
... and 33 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 45fe067...2513e76. Read the comment docs.

@dagnir dagnir merged commit 20166e1 into aws:master Dec 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants