Add support for event stream requests over RPC

[rschmitt]

This commit adds support for RPC services that use client->server or
bidirectional event streaming. In the RPC formats (i.e. AWS JSON), the
marshalled request and response objects are sent as the first event in
the request and response stream respectively. This is different from the
REST formats, such as REST JSON, which bind the request and response
fields to non-payload positions (headers, URI, method, status code,
query string) when event streaming is used.

The implementation of this feature is mainly performed by a new global
interceptor, EventStreamInitialRequestInterceptor. The interceptor
detects when an initial-request event needs to be sent and transforms
the async response body by constructing the initial-request message
and prepending it to the event stream. Due to the high precedence of
global interceptors, any subsequent service or override interceptors
that run on the body will see the transformed version. This also applies
to the EventStreamAws4Signer, which will produce a signature over the
initial-request event.

I confirm that this pull request can be released under the Apache 2 license.

[codecov-io]

Codecov Report

Merging #2388 (4edce20) into master (a0b8ddd) will decrease coverage by 0.06%.
The diff coverage is 45.88%.

@@             Coverage Diff              @@
##             master    #2388      +/-   ##
============================================
- Coverage     77.69%   77.63%   -0.07%     
  Complexity      366      366              
============================================
  Files          1245     1247       +2     
  Lines         39294    39369      +75     
  Branches       3106     3112       +6     
============================================
+ Hits          30531    30563      +32     
- Misses         7277     7315      +38     
- Partials       1486     1491       +5     

Flag	Coverage Δ	Complexity Δ
unittests	77.63% <45.88%> (-0.07%)	0.00 <0.00> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ	Complexity Δ
...erceptor/EventStreamInitialRequestInterceptor.java	0.00% <0.00%> (ø)	0.00 <0.00> (?)
...sdk/core/client/handler/ClientExecutionParams.java	95.45% <33.33%> (-4.55%)	0.00 <0.00> (ø)
...json/internal/marshall/JsonProtocolMarshaller.java	95.45% <60.00%> (-3.57%)	0.00 <0.00> (ø)
...ssdk/core/internal/async/AsyncStreamPrepender.java	64.28% <64.28%> (ø)	0.00 <0.00> (?)
...dk/codegen/poet/client/specs/JsonProtocolSpec.java	91.62% <66.66%> (-0.40%)	0.00 <0.00> (ø)
...sform/protocols/EventStreamJsonMarshallerSpec.java	83.33% <80.00%> (ø)	0.00 <0.00> (ø)
.../awscore/client/handler/AwsClientHandlerUtils.java	90.32% <100.00%> (+0.15%)	0.00 <0.00> (ø)
...sdk/protocols/json/BaseAwsJsonProtocolFactory.java	100.00% <100.00%> (ø)	0.00 <0.00> (ø)
...ternal/marshall/JsonProtocolMarshallerBuilder.java	93.33% <100.00%> (+1.02%)	0.00 <0.00> (ø)
...ore/interceptor/SdkInternalExecutionAttribute.java	100.00% <100.00%> (ø)	0.00 <0.00> (ø)
... and 7 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9f190d7...4edce20. Read the comment docs.

[dagnir]

@rschmitt Awesome! Appreciate the contribution. We will need schedule some time on our side to review this so there will be a delay, but we'll have a look soon.

[dagnir]

I think for this change, it makes sense to add some codegen reftests, just to make sure we're actually generating things like .withInitialRequestEvent(true) for the json services. I think you can just a JSON-RPC specific test here

aws-sdk-java-v2/codegen/src/test/java/software/amazon/awssdk/codegen/poet/client/PoetClientFunctionalTests.java

Line 27 in a402b40

public class PoetClientFunctionalTests {

[dagnir]

I think ideally, we'd leverage the marshallers to marshall the request directly to a suitable initial-request instead of "remarshalling" in the interceptor, but I'm fine with this approach as well since the marshaller route would require a bit more work in the codegen; we could save that for a second iteration in the future.

[dagnir]

Integ tests seem to be failing. Can you take a look?

Locally, I'm seeing software.amazon.awssdk.services.s3.S3PresignerIntegrationTest fail, but there may be others. You can run them using

mvn clean install -P integration-tests

[rschmitt]

Integ tests seem to be failing. Can you take a look?

Locally, I'm seeing software.amazon.awssdk.services.s3.S3PresignerIntegrationTest fail, but there may be others. You can run them using

mvn clean install -P integration-tests

I wasn't immediately able to reproduce this particular failure.

[INFO] Running software.amazon.awssdk.services.s3.S3PresignerIntegrationTest
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.305 s - in software.amazon.awssdk.services.s3.S3PresignerIntegrationTest

[rschmitt]

I think for this change, it makes sense to add some codegen reftests, just to make sure we're actually generating things like .withInitialRequestEvent(true) for the json services. I think you can just a JSON-RPC specific test here

aws-sdk-java-v2/codegen/src/test/java/software/amazon/awssdk/codegen/poet/client/PoetClientFunctionalTests.java

Line 27 in a402b40

public class PoetClientFunctionalTests {

It looks like the JSON tests are actually rest-json tests; I need RPC tests, and I'm not sure any exist currently.

[codecov-commenter]

Codecov Report

Merging #2388 (09afe78) into master (4719dcb) will increase coverage by 0.04%.
The diff coverage is 88.57%.

@@             Coverage Diff              @@
##             master    #2388      +/-   ##
============================================
+ Coverage     77.71%   77.76%   +0.04%     
  Complexity      366      366              
============================================
  Files          1247     1249       +2     
  Lines         39471    39569      +98     
  Branches       3088     3099      +11     
============================================
+ Hits          30675    30769      +94     
- Misses         7311     7314       +3     
- Partials       1485     1486       +1     

Flag	Coverage Δ	Complexity Δ
unittests	77.76% <88.57%> (+0.04%)	366.00 <0.00> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ	Complexity Δ
...sdk/core/client/handler/ClientExecutionParams.java	95.45% <33.33%> (-4.55%)	0.00 <0.00> (ø)
...json/internal/marshall/JsonProtocolMarshaller.java	95.61% <60.00%> (-3.45%)	0.00 <0.00> (ø)
...dk/codegen/poet/client/specs/JsonProtocolSpec.java	91.62% <66.66%> (-0.40%)	0.00 <0.00> (ø)
...sform/protocols/EventStreamJsonMarshallerSpec.java	83.33% <80.00%> (ø)	0.00 <0.00> (ø)
...ntstream/EventStreamInitialRequestInterceptor.java	87.09% <87.09%> (ø)	0.00 <0.00> (?)
.../awscore/client/handler/AwsClientHandlerUtils.java	90.32% <100.00%> (+0.15%)	0.00 <0.00> (ø)
...sdk/protocols/json/BaseAwsJsonProtocolFactory.java	100.00% <100.00%> (ø)	0.00 <0.00> (ø)
...ternal/marshall/JsonProtocolMarshallerBuilder.java	93.33% <100.00%> (+1.02%)	0.00 <0.00> (ø)
...ore/interceptor/SdkInternalExecutionAttribute.java	100.00% <100.00%> (ø)	0.00 <0.00> (ø)
...ssdk/core/internal/async/AsyncStreamPrepender.java	100.00% <100.00%> (ø)	0.00 <0.00> (?)
... and 6 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 69546d5...09afe78. Read the comment docs.

[dagnir]

LGTM overall!

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

91.9% Coverage
0.0% Duplication

[dagnir]