Request/Response Fragmentation & Assembly

[benjchristensen]

Here is an example request/response of a single payload fragmented across multiple frames:

SERVER ==> Writes from server->client: Frame[0] => Stream ID: 2 Type: NEXT_COMPLETE Payload: data: "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklm...
CLIENT <== Input from server->client: Frame[0] => Stream ID: 2 Type: NEXT_COMPLETE Payload: data: "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmn...
SERVER ==> Writes from server->client: Frame[0] => Stream ID: 2 Type: NEXT_COMPLETE Payload: data: "ijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstu...
CLIENT <== Input from server->client: Frame[0] => Stream ID: 2 Type: NEXT_COMPLETE Payload: data: "ijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuv...
SERVER ==> Writes from server->client: Frame[0] => Stream ID: 2 Type: NEXT_COMPLETE Payload: data: "qrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabc...
SERVER ==> Writes from server->client: Frame[0] => Stream ID: 2 Type: NEXT_COMPLETE Payload: data: "yzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijk...
CLIENT <== Input from server->client: Frame[0] => Stream ID: 2 Type: NEXT_COMPLETE Payload: data: "qrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcd...
CLIENT <== Input from server->client: Frame[0] => Stream ID: 2 Type: NEXT_COMPLETE Payload: data: "yzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijkl...

Note how only the first starts with "abc..." while the rest are chopped in the middle.

[benjchristensen]

This is a rebase of #60

[benjchristensen]

Status of Fragmentation

• I believe fragmentation and reassembly is working for all responses. However, I do not have completely thorough unit tests for everything yet though. Much more unit testing needs to be done to prove this is all working.
• Fragmentation is not done at all for requests.

Relation to Head-of-line Blocking

• The FragmentedPublisher is now what all writes flow through. This makes it the place for scheduling the writing. This is more-or-less working, with round-robin behavior and batching (currently set to 128). However, I have not tested any behavior related to head-of-line blocking yet.
• One potential problem area is inside nextFramented: https://github.com/ReactiveSocket/reactivesocket-java/pull/63/files#diff-fb8bafc4e452bc47cc32c9968ad28257R221 It is when iterating over a fragmented frame. I have marked a TODO there as a very large Frame could have thousands of frames and it will keep emitting them unless it runs into the requested rate from the transport. That may be okay, but it may not.

Performance

Performance of this causes a non-trivial hit, but it's not surprising, considering what this has to do. It drops on my machine from 2.8m to 1.9m request/response per second, even without fragmentation.

The overhead is the scheduling machinery to handle both the possibility of fragmentation, and the merge behavior (that would have had to exist instead in a DuplexConnection impl) that does the fairness scheduling to prevent head-of-line blocking.

Benchmark                                 Mode  Cnt        Score       Error  Units
ReactiveSocketPerf.requestResponseHello  thrpt    5  1970941.560 ± 37673.830  ops/s

vs before this change

Benchmark                                 Mode  Cnt        Score        Error  Units
ReactiveSocketPerf.requestResponseHello  thrpt    5  2826171.044 ± 221019.846  ops/s

I have not spent enough time to truly do performance optimizations on this.

Flight Recorder of requestResponseHello shows decent memory behavior (with no contention). If we do a perf test that involves contention, memory allocation inside FragmentedPublisher will go up significantly as it will then start allocating queues.

No individual methods stand out, but in aggregate the FragmentedPublisher and related request(n) behaviors do show up:

Design Thoughts

After doing this, we may want to consider the contract for DuplexConnection. It now should only have addOutput called once, so perhaps should be setOutput and refine the contract so implementations can count on only being invoked once since this protocol implementation already handles the merging into a single, serialized stream with scheduling considerations.

I made the change to make FragmentationPublisher be allocated only once for the duration of the connection. This proved important otherwise every single output, even just a for a single request/response frame, would setup, subscribe, and tear down a FragmentedPublisher operator that is effectively a variant of merge/concat.

[benjchristensen]

@stevegury What do you want to do with this PR? These changes are non-trivial and will become a real mess to try and merge as master keeps moving forward.

[stevegury]

That's on my todo list to integrate it.
I know that I master diverge a little bit (mostly code style), but that is a good occassion for me to deeply understand this PR.

[NiteshKant]

@stevegury I believe this PR has diverged a lot to be able to merge?

[stevegury]

Let's keep the branch, so that we can reuse it later as inspiration.