Add support for streaming server side rendering (#1633)

* tmp

* add support for streaming rendered component using renderToPipeableStream

* put ROR scripts after the first rendered chunk

* remove log statements

* add stream_react_component_async helper function

* add helper function to render a whole view

* fix failing jest tests

* linting

* linting

* remove redundant new line when context is not prepended

* rename stream_react_component_async to stream_react_component

* fix error caused by process on browser

* remove new line appended to the page when has no rails context

* fix the problem of not updating the first streaming chunk

* rename stream_react_component_internal to internal_stream_react_component

* add unit tests for rails_context_if_not_already_rendered

* remove :focus tag from rails_context_if_not_already_rendered spec

* make render function returns Readable stream instead of PassThrough

* use validateComponent function instead of manually validating it

* linting

* add some comments

* don't return extra null at he end of the stream

* update CHANGELOG.md

* update docs

* update docs

Author: AbanoubGhadban

CHANGELOG.md

@@ -18,6 +18,13 @@ Please follow the recommendations outlined at [keepachangelog.com](http://keepac
 ### [Unreleased]
 Changes since the last non-beta release.
 
+### Added
+- Added streaming server rendering support:
+  - New `stream_react_component` helper for adding streamed components to views
+  - New `streamServerRenderedReactComponent` function in the react-on-rails package that uses React 18's `renderToPipeableStream` API
+  - Enables progressive page loading and improved performance for server-rendered React components
+  [PR #1633](https://github.com/shakacode/react_on_rails/pull/1633) by [AbanoubGhadban](https://github.com/AbanoubGhadban).
+
 #### Changed
 - Console replay script generation now awaits the render request promise before generating, allowing it to capture console logs from asynchronous operations. This requires using a version of the Node renderer that supports replaying async console logs. [PR #1649](https://github.com/shakacode/react_on_rails/pull/1649) by [AbanoubGhadban](https://github.com/AbanoubGhadban).
 

SUMMARY.md

@@ -17,6 +17,7 @@ Here is the new link:
   + [How React on Rails Works](docs/outdated/how-react-on-rails-works.md)
   + [Client vs. Server Rendering](./docs/guides/client-vs-server-rendering.md)
   + [React Server Rendering](./docs/guides/react-server-rendering.md)
+  + [🚀 Next-Gen Server Rendering: Streaming with React 18's Latest APIs](./docs/guides/streaming-server-rendering.md)
   + [Render-Functions and the RailsContext](docs/guides/render-functions-and-railscontext.md)
   + [Caching and Performance: React on Rails Pro](https://github.com/shakacode/react_on_rails/wiki).
   + [Deployment](docs/guides/deployment.md).

docs/guides/streaming-server-rendering.md

@@ -0,0 +1,212 @@
+# 🚀 Streaming Server Rendering with React 18
+
+React on Rails Pro supports streaming server rendering using React 18's latest APIs, including `renderToPipeableStream` and Suspense. This guide explains how to implement and optimize streaming server rendering in your React on Rails application.
+
+## Prerequisites
+
+- React on Rails Pro subscription
+- React 18 or higher (experimental version)
+- React on Rails v15.0.0-alpha.0 or higher
+- React on Rails Pro v4.0.0.rc.5 or higher
+
+## Benefits of Streaming Server Rendering
+
+- Faster Time to First Byte (TTFB)
+- Progressive page loading
+- Improved user experience
+- Better SEO performance
+- Optimal handling of data fetching
+
+## Implementation Steps
+
+1. **Use Experimental React 18 Version**
+
+First, ensure you're using React 18's experimental version in your package.json:
+
+```json
+"dependencies": {
+  "react": "18.3.0-canary-670811593-20240322",
+  "react-dom": "18.3.0-canary-670811593-20240322"
+}
+```
+
+> Note: Check the React documentation for the latest release that supports streaming.
+
+2. **Prepare Your React Components**
+
+You can create async React components that return a promise. Then, you can use the `Suspense` component to render a fallback UI while the component is loading.
+
+```jsx
+// app/javascript/components/MyStreamingComponent.jsx
+import React, { Suspense } from 'react';
+
+const fetchData = async () => {
+  // Simulate API call
+  const response = await fetch('api/endpoint');
+  return response.json();
+};
+
+const MyStreamingComponent = () => {
+  return (
+    <>
+      <header>
+        <h1>Streaming Server Rendering</h1>
+      </header>
+      <Suspense fallback={<div>Loading...</div>}>
+        <SlowDataComponent />
+      </Suspense>
+    </>
+  );
+};
+
+const SlowDataComponent = async () => {
+  const data = await fetchData();
+  return <div>{data}</div>;
+};
+
+export default MyStreamingComponent;
+```
+
+```jsx
+// app/javascript/packs/registration.jsx
+import MyStreamingComponent from '../components/MyStreamingComponent';
+
+ReactOnRails.register({ MyStreamingComponent });
+```
+
+3. **Add The Component To Your Rails View**
+
+```erb
+<!-- app/views/example/show.html.erb -->
+
+<%=
+  stream_react_component(
+    'MyStreamingComponent',
+    props: { greeting: 'Hello, Streaming World!' },
+    prerender: true
+  )
+%>
+
+<footer>
+  <p>Footer content</p>
+</footer>
+```
+
+4. **Render The View Using The `stream_view_containing_react_components` Helper**
+
+Ensure you have a controller that renders the view containing the React components. The controller must include the `ReactOnRails::Controller`, `ReactOnRailsPro::Stream` and `ActionController::Live` modules.
+
+```ruby
+# app/controllers/example_controller.rb
+
+class ExampleController < ApplicationController
+  include ActionController::Live
+  include ReactOnRails::Controller
+  include ReactOnRailsPro::Stream
+
+  def show
+    stream_view_containing_react_components(template: 'example/show')
+  end
+end
+```
+
+5. **Test Your Application**
+
+You can test your application by running `rails server` and navigating to the appropriate route.
+
+
+6. **What Happens During Streaming**
+
+When a user visits the page, they'll experience the following sequence:
+
+1. The initial HTML shell is sent immediately, including:
+   - The page layout
+   - Any static content (like the `<h1>` and footer)
+   - Placeholder content for the React component (typically a loading state)
+
+2. As the React component processes and suspense boundaries resolve:
+   - HTML chunks are streamed to the browser progressively
+   - Each chunk updates a specific part of the page
+   - The browser renders these updates without a full page reload
+
+For example, with our `MyStreamingComponent`, the sequence might be:
+
+1. The initial HTML includes the header, footer, and loading state.
+
+```html
+<header>
+  <h1>Streaming Server Rendering</h1>
+</header>
+<template id="s0">
+  <div>Loading...</div>
+</template>
+<footer>
+  <p>Footer content</p>
+</footer>
+```
+
+2. As the component resolves, HTML chunks are streamed to the browser:
+
+```html
+<template hidden id="b0">
+  <div>[Fetched data]</div>
+</template>
+
+<script>
+  // This implementation is slightly simplified
+  document.getElementById('s0').replaceChildren(
+    document.getElementById('b0')
+  );
+</script>
+```
+
+## When to Use Streaming
+
+Streaming SSR is particularly valuable in specific scenarios. Here's when to consider it:
+
+### Ideal Use Cases
+
+1. **Data-Heavy Pages**
+   - Pages that fetch data from multiple sources
+   - Dashboard-style layouts where different sections can load independently
+   - Content that requires heavy processing or computation
+
+2. **Progressive Enhancement**
+   - When you want users to see and interact with parts of the page while others load
+   - For improving perceived performance on slower connections
+   - When different parts of your page have different priority levels
+
+3. **Large, Complex Applications**
+   - Applications with multiple independent widgets or components
+   - Pages where some content is critical and other content is supplementary
+   - When you need to optimize Time to First Byte (TTFB)
+
+### Best Practices for Streaming
+
+1. **Component Structure**
+   ```jsx
+   // Good: Independent sections that can stream separately
+   <Layout>
+     <Suspense fallback={<HeaderSkeleton />}>
+       <Header />
+     </Suspense>
+     <Suspense fallback={<MainContentSkeleton />}>
+       <MainContent />
+     </Suspense>
+     <Suspense fallback={<SidebarSkeleton />}>
+       <Sidebar />
+     </Suspense>
+   </Layout>
+
+   // Bad: Everything wrapped in a single Suspense boundary
+   <Suspense fallback={<FullPageSkeleton />}>
+     <Header />
+     <MainContent />
+     <Sidebar />
+   </Suspense>
+   ```
+
+2. **Data Loading Strategy**
+   - Prioritize critical data that should be included in the initial HTML
+   - Use streaming for supplementary data that can load progressively
+   - Consider implementing a waterfall strategy for dependent data

jest.config.js

@@ -1,4 +1,5 @@
 module.exports = {
   preset: 'ts-jest/presets/js-with-ts',
   testEnvironment: 'jsdom',
+  setupFiles: ['<rootDir>/node_package/tests/jest.setup.js'],
 };

lib/react_on_rails/helper.rb

@@ -91,6 +91,64 @@ def react_component(component_name, options = {})
       end
     end
 
+    # Streams a server-side rendered React component using React's `renderToPipeableStream`.
+    # Supports React 18 features like Suspense, concurrent rendering, and selective hydration.
+    # Enables progressive rendering and improved performance for large components.
+    #
+    # Note: This function can only be used with React on Rails Pro.
+    # The view that uses this function must be rendered using the
+    # `stream_view_containing_react_components` method from the React on Rails Pro gem.
+    #
+    # Example of an async React component that can benefit from streaming:
+    #
+    # const AsyncComponent = async () => {
+    #   const data = await fetchData();
+    #   return <div>{data}</div>;
+    # };
+    #
+    # function App() {
+    #   return (
+    #     <Suspense fallback={<div>Loading...</div>}>
+    #       <AsyncComponent />
+    #     </Suspense>
+    #   );
+    # }
+    #
+    # @param [String] component_name Name of your registered component
+    # @param [Hash] options Options for rendering
+    # @option options [Hash] :props Props to pass to the react component
+    # @option options [String] :dom_id DOM ID of the component container
+    # @option options [Hash] :html_options Options passed to content_tag
+    # @option options [Boolean] :prerender Set to false to disable server-side rendering
+    # @option options [Boolean] :trace Set to true to add extra debugging information to the HTML
+    # @option options [Boolean] :raise_on_prerender_error Set to true to raise exceptions during server-side rendering
+    # Any other options are passed to the content tag, including the id.
+    def stream_react_component(component_name, options = {})
+      unless ReactOnRails::Utils.react_on_rails_pro?
+        raise ReactOnRails::Error,
+              "You must use React on Rails Pro to use the stream_react_component method."
+      end
+
+      if @rorp_rendering_fibers.nil?
+        raise ReactOnRails::Error,
+              "You must call stream_view_containing_react_components to render the view containing the react component"
+      end
+
+      rendering_fiber = Fiber.new do
+        stream = internal_stream_react_component(component_name, options)
+        stream.each_chunk do |chunk|
+          Fiber.yield chunk
+        end
+      end
+
+      @rorp_rendering_fibers << rendering_fiber
+
+      # return the first chunk of the fiber
+      # It contains the initial html of the component
+      # all updates will be appended to the stream sent to browser
+      rendering_fiber.resume
+    end
+
     # react_component_hash is used to return multiple HTML strings for server rendering, such as for
     # adding meta-tags to a page.
     # It is exactly like react_component except for the following:
@@ -330,6 +388,16 @@ def load_pack_for_generated_component(react_component_name, render_options)
 
     private
 
+    def internal_stream_react_component(component_name, options = {})
+      options = options.merge(stream?: true)
+      result = internal_react_component(component_name, options)
+      build_react_component_result_for_server_streamed_content(
+        rendered_html_stream: result[:result],
+        component_specification_tag: result[:tag],
+        render_options: result[:render_options]
+      )
+    end
+
     def generated_components_pack_path(component_name)
       "#{ReactOnRails::PackerUtils.packer_source_entry_path}/generated/#{component_name}.js"
     end
@@ -361,6 +429,33 @@ def build_react_component_result_for_server_rendered_string(
       prepend_render_rails_context(result)
     end
 
+    def build_react_component_result_for_server_streamed_content(
+      rendered_html_stream: required("rendered_html_stream"),
+      component_specification_tag: required("component_specification_tag"),
+      render_options: required("render_options")
+    )
+      content_tag_options_html_tag = render_options.html_options[:tag] || "div"
+      # The component_specification_tag is appended to the first chunk
+      # We need to pass it early with the first chunk because it's needed in hydration
+      # We need to make sure that client can hydrate the app early even before all components are streamed
+      is_first_chunk = true
+      rendered_html_stream = rendered_html_stream.transform do |chunk|
+        if is_first_chunk
+          is_first_chunk = false
+          html_content = <<-HTML
+            #{rails_context_if_not_already_rendered}
+            #{component_specification_tag}
+            <#{content_tag_options_html_tag} id="#{render_options.dom_id}">#{chunk}</#{content_tag_options_html_tag}>
+          HTML
+          next html_content.strip
+        end
+        chunk
+      end
+
+      rendered_html_stream.transform(&:html_safe)
+      # TODO: handle console logs
+    end
+
     def build_react_component_result_for_server_rendered_hash(
       server_rendered_html: required("server_rendered_html"),
       component_specification_tag: required("component_specification_tag"),
@@ -404,20 +499,22 @@ def compose_react_component_html_with_spec_and_console(component_specification_t
       HTML
     end
 
-    # prepend the rails_context if not yet applied
-    def prepend_render_rails_context(render_value)
-      return render_value if @rendered_rails_context
+    def rails_context_if_not_already_rendered
+      return "" if @rendered_rails_context
 
       data = rails_context(server_side: false)
 
       @rendered_rails_context = true
 
-      rails_context_content = content_tag(:script,
-                                          json_safe_and_pretty(data).html_safe,
-                                          type: "application/json",
-                                          id: "js-react-on-rails-context")
+      content_tag(:script,
+                  json_safe_and_pretty(data).html_safe,
+                  type: "application/json",
+                  id: "js-react-on-rails-context")
+    end
 
-      "#{rails_context_content}\n#{render_value}".html_safe
+    # prepend the rails_context if not yet applied
+    def prepend_render_rails_context(render_value)
+      "#{rails_context_if_not_already_rendered}\n#{render_value}".strip.html_safe
     end
 
     def internal_react_component(react_component_name, options = {})
@@ -520,6 +617,9 @@ def server_rendered_react_component(render_options)
                                                js_code: js_code)
       end
 
+      # TODO: handle errors for streams
+      return result if render_options.stream?
+
       if result["hasErrors"] && render_options.raise_on_prerender_error
         # We caught this exception on our backtrace handler
         raise ReactOnRails::PrerenderError.new(component_name: react_component_name,