update descriptions of nonblocking APIs

wkliao · wkliao · commit c0e5bb7f590c · 2024-09-05T15:35:36.000-05:00
diff --git a/docs/source/tutorial/compare_netcdf4.rst b/docs/source/tutorial/compare_netcdf4.rst
@@ -2,56 +2,76 @@
 =================================
 Comparing with netCDF4-python API
 =================================
-.. warning::
 
-   Under construction.
-
-PnetCDF-python inherits many features from netCDF4-python, making the transition from the former to the latter a seamless process
-for most netCDF4-python applications. However, there are some exceptions to this as listed below. On the other hand, PnetCDF-python supports
-some new APIs not available in netCDF4-python.
+PnetCDF-python programming in a way is very similar to netCDF4-python.
+However, there are some differences as listed below.
 
 Supported File Formats
 --------------------------
 
- NetCDF4-python supports NETCDF4 formats(HDF5) in addition to classic netCDF formats(NETCDF3_CLASSIC, NETCDF3_64BIT_OFFSET, NETCDF3_64BIT_DATA). However,
- PnetCDF-python library only supports netCDF classic file formats, which means all netCDF4-dependent features are **not** supported, including user-defined types,
- compression, hirarchical structure, etc.
+ NetCDF4-python supports NETCDF4 format (HDF5) in addition to classic netCDF
+ formats (NETCDF3_CLASSIC, NETCDF3_64BIT_OFFSET, NETCDF3_64BIT_DATA). However,
+ PnetCDF-python library only supports netCDF classic file formats, which means
+ all netCDF4-dependent features are **not** supported, including user-defined
+ types, compound data types, compression, hierarchical structure, etc.
 
 Difference in Programming Model
 --------------------------------
 
  Data/Define Mode
-  NetCDF4-python library automatically switches between data and define mode for the user by calling ``redef`` and ``enddef`` internally within the define-mode
-  operation functions, which is **not** implemented in Pnetcdf-python. A manual call to :meth:`File.redef` is compulsory to activate define mode before swtiching
-  to define mode opearations, following the C library convention. Similarly, :meth:`File.enddef` is required before switching to data mode operations. This design is based on considerations of
-  the following aspects:
+  NetCDF4-python library automatically switches between data and define mode
+  for the user by calling ``redef`` and ``enddef`` internally within the
+  define-mode operation functions. For performance reason, this is **not**
+  adopted in Pnetcdf-python. A manual call to :meth:`File.redef` is compulsory
+  to re-enter the define mode, following the C library convention. Similarly,
+  :meth:`File.enddef` is required before switching to data mode operations.
+  This design is based on considerations of the following aspects:
+
+  - Minimize overheads during consecutive define operations: Automatically
+    wrapping all define functions with :meth:`File.redef` and
+    :meth:`File.enddef` could introduce significant overhead between
+    consecutive define operations. The netCDF4-python approach results in
+    unnecessary data/define mode switches, impacting performance.
 
-  - Minimize overheads during consecutive define operations: Automatically wrapping all define functions with :meth:`File.redef` and :meth:`File.enddef` could introduce
-    significant overhead between consecutive define operations. This approach results in unnecessary data/define mode switches, impacting performance.
-  - Avoid potential hanging when performing independent I/O: if :meth:`File.enddef` is automatically embeded in all data mode operation functions, the program will hang when
-    partial processes are performing independent I/O(while others don't) because :meth:`File.enddef` is a collective call which requires all processes to participate.
+  - Avoid potential hanging when performing independent I/O: if
+    :meth:`File.enddef` is automatically embedded in all data mode operation
+    functions, the program will hang when partial processes are performing
+    independent I/O (while others don't) because :meth:`File.enddef` is a
+    collective call which requires all processes to participate.
 
  Independent/Collective I/O Mode
-  There are two types of parallel I/O, independent I/O and collective I/O supported both in PnetCDF-python and netCDF4-python. NetCDF4-python toggles back and forth
-  between the two types at variable-level. However, PnetCDF-python manages this at file-level through :meth:`File.begin_indep` and :meth:`File.end_indep`. The default I/O mode
-  is collective I/O in PnetCDF-python.
+  There are two types of parallel I/O operations, independent I/O and
+  collective I/O supported both in PnetCDF-python and netCDF4-python.
+  NetCDF4-python toggles back and forth between the two types at
+  variable-level. However, PnetCDF-python manages this at file-level through
+  :meth:`File.begin_indep` and :meth:`File.end_indep`. The default I/O mode is
+  collective I/O in PnetCDF-python.
 
 
 Alternative Reads and Writes Methods
 ------------------------------------------
 
- For reading from and writing to netCDF4 variables, PnetCDF-python provides alternative methods in addition to numpy-like indexer syntax. The :meth:`Variable.get_var` and
- :meth:`Variable.put_var` methods are faithfull python-implementations of the put/get_var families from the original PnetCDF-C library. By overloading the input arguments,
- these methods can fulfill specific I/O needs to the target variable depending on the requirements of the applications: the entire variable, a single data value, an
- (subsampled) array of values, a mapped array or a list of subarrays. These methods require an array argument as read/write buffer, which is a prerequisite non-blocking
- I/O as introduced below.
+ For reading from and writing to netCDF4 variables, PnetCDF-python provides
+ alternative methods in addition to numpy-like indexer syntax. The
+ :meth:`Variable.get_var` and :meth:`Variable.put_var` methods are faithful
+ python-implementations of the put/get_var families from the original PnetCDF-C
+ library. By overloading the input arguments, these methods can fulfill
+ specific I/O needs to the target variable depending on the requirements of the
+ applications: the entire variable, a single data value, a subarray of values,
+ a mapped array or a list of subarrays. These methods require an array argument
+ as read/write buffer, which is a prerequisite non-blocking I/O as introduced
+ below.
 
- For the example program, see ``examples/get_vara.py``.
+ An example program can be found in ``examples/get_vara.py``.
 
-Non-blocking I/O
+Nonblocking I/O
 ------------------------------------------
- In additional to blocking read/writes, PnetCDF also offers the nonblocking API that enables users to initiate multiple requests without actually doing I/O and subsequently
- flush them altogether. This approach is designed to enhance performance by merging small I/O requests and maximizing I/O efficiency. This features is faithfully
- preserved in PnetCDF-python API.
+ In additional to blocking read/write APIs, PnetCDF also offers the nonblocking
+ APIs that lets users to post multiple requests and subsequently flush them
+ altogether. This feature allows PnetCDF to improve performance by aggregating
+ small I/O requests and maximizing I/O bandwidth utilization. This feature
+ implemented in PnetCDF-C library is faithfully preserved in PnetCDF-python
+ nonblocking APIs.
+
+ An example program can be found in ``examples/nonblocking/nonblocking_write.py``.
 
- For the example program, see ``examples/nonblocking/nonblocking_write.py``.
diff --git a/docs/source/tutorial/non_blocking.rst b/docs/source/tutorial/non_blocking.rst
@@ -1,134 +1,175 @@
 .. currentmodule:: pnetcdf
 ==============================
-Non-blocking Reads and Writes
+Nonblocking Reads and Writes
 ==============================
 
-.. warning::
-
-   Under construction. 
-
- 
- 
-Alternative to blocking read/writes, PnetCDF-python nonblocking APIs allow users to first post multiple requests and later flush them altogether 
-in order to achieve a better performance. A common practice is writing (or reading) subarrays to (from) multiple variables, e.g. one or more
-subarrays for each variable defined in the NetCDF file.
+Alternative to blocking read/writes, PnetCDF-python nonblocking APIs allow
+users to first post multiple requests and later flush them altogether in order
+to achieve a better performance. A common practice is writing (or reading)
+subarrays to (from) multiple variables, e.g. one or more subarrays for each
+variable defined in the NetCDF file.
 
 Nonblocking Write
 -------------------
 
- Write requests can be posted by the method call of :meth:`Variable.iput_var`. Same as :meth:`Variable.put_var`, the behavior of :meth:`Variable.iput_var` varies 
- depending on the pattern of provided optional arguments - `index`, `start`, `count`, `stride`, `num` and `imap` as shown below. Note that the method only posts the 
- request, which is not commited until :meth:`File.wait`. The method call returns a request id that can be optionally passed to :meth:`File.wait` to select this request.
+ Write requests can be posted by the method call of :meth:`Variable.iput_var`.
+ Same as :meth:`Variable.put_var`, the behavior of :meth:`Variable.iput_var`
+ varies depending on the pattern of provided optional arguments - `index`,
+ `start`, `count`, `stride`, `num` and `imap` as shown below. Note that the
+ method only posts the request, which is not committed until :meth:`File.wait`.
+ The method call returns a request id that can be optionally passed to
+ :meth:`File.wait` to select this request.
 
- - `data` - Reqeust to write an entire variable
- - `data`, `index` - Reqeust to write a single data value
- - `data`, `start`, `count` - Reqeust to write an array of values
- - `data`, `start`, `count`, `stride` - Reqeust to write a subsampled array of values
- - `data`, `start`, `count`, `imap` - Reqeust to write a mapped array of values
- - `start`, `count`, `num` - Reqeust to write a list of subarrays of values
- 
- Here's a python example to post 10 write requests that write to 10 netCDF variables in the same file. 
+ - `data` - Request to write an entire variable
+ - `data`, `index` - Request to write a single data value
+ - `data`, `start`, `count` - Request to write an array of values
+ - `data`, `start`, `count`, `stride` - Request to write a subarray of values
+ - `data`, `start`, `count`, `imap` - Request to write a mapped array of values
+ - `start`, `count`, `num` - Request to write a list of subarrays of values
+
+ Here's a python example to post 10 write requests that write to 10 netCDF variables in the same file.
 
  .. code-block:: Python
 
     req_ids = []
     write_buff = [randint(0,10, size=(xdim,ydim,zdim)).astype('i4')] * 10
+
     for i in range(num_reqs):
-      v = f.variables[f'data{i}']
-      datam = write_buff[i]
-      # post the request to write the whole variable
-      req_id = v.iput_var(datam)
-      # track the request ID for each write request
-      req_ids.append(req_id)
+        v = f.variables[f'data{i}']
+        datam = write_buff[i]
 
- For the full example program, see ``examples/non_blocking_write.py``.
+        # post a request to write the whole variable
+        req_id = v.iput_var(datam)
+        # track the request ID for each write request
+        req_ids.append(req_id)
+
+    # wait for nonblocking writes to complete
+    errs = [None] * num_reqs
+    f.wait_all(num_reqs, req_ids, errs)
+
+ For the full example program, see ``examples/nonblocking/nonblocking_write.py``.
 
 Nonblocking Read
 ------------------
 
- Read requests can be posted by the method call of :meth:`Variable.iget_var`. Note that unlike :meth:`Variable.get_var`, this method requires a 
- mandatory argument - an empty numpy array reserved to be filled in the future. Again, the method call returns a request id that can be optionally passed to 
- :meth:`File.wait` to select this request. Similar to :meth:`Variable.get_var`, the behavior of :meth:`Variable.iget_var` varies depending on 
- the pattern of provided optional arguments - `index`, `start`, `count`, `stride`, `num` and `imap`. 
+ Read requests can be posted by the method call of :meth:`Variable.iget_var`.
+ Note that unlike :meth:`Variable.get_var`, this method requires a mandatory
+ argument - an empty numpy array reserved to be filled in the future. Again,
+ the method call returns a request id that can be optionally passed to
+ :meth:`File.wait` to select this request. Similar to :meth:`Variable.get_var`,
+ the behavior of :meth:`Variable.iget_var` varies depending on the pattern of
+ provided optional arguments - `index`, `start`, `count`, `stride`, `num` and
+ `imap`.
 
  - `buff` - Request to read an entire variable
  - `buff`, `index` - Request to read a single data value
  - `buff`, `start`, `count` - Request to read an array of values
- - `buff`, `start`, `count`, `stride` - Request to read a subsampled array of values
+ - `buff`, `start`, `count`, `stride` - Request to read a subarray of values
  - `buff`, `start`, `count`, `imap` - Request to read a mapped array of values
  - `buff`, `start`, `count`, `num` - Request to read a list of subarrays of a netCDF variable
- 
- Here's a python example to post 10 read requests that read from 10 netCDF variables in the same file. 
+
+ Here's a python example to post 10 read requests that read from 10 netCDF variables in the same file.
 
  .. code-block:: Python
 
-    req_ids = []
-    # initialize the list of returned array references
+    # initialize the list of references to read buffers
     v_datas = []
-    for i in range(num_reqs):       
-       v = f.variables[f'data{i}']
-       buff = np.empty(shape = v.shape, dtype = v.datatype)# empty numpy array to hold returned variable values
-       req_id = v.iget_var(buff)
-       # track the request ID for each read request
-       req_ids.append(req_id)
-       # store the reference of variable values
-       v_datas.append(buff)
- 
- For the full example program, see ``examples/flexible_api.py``.
+
+    req_ids = []
+    for i in range(num_reqs):
+        v = f.variables[f'data{i}']
+        # allocate read buffer, a numpy array
+        buff = np.empty(shape = v.shape, dtype = v.datatype)
+
+        # post a request to read the whole variable
+        req_id = v.iget_var(buff)
+        # track the request ID for each read request
+        req_ids.append(req_id)
+
+        # store the reference of variable values
+        v_datas.append(buff)
+
+    # wait for nonblocking reads to complete
+    errs = [None] * num_reqs
+    f.wait_all(num_reqs, req_ids, errs)
+
+ For the full example program, see ``examples/nonblocking/nonblocking_read.py``.
 
 Commit Read/Write Requests
 ----------------------------
 
- Pending requests are eventually processed by :meth:`File.wait`. Requests to commited can be specified selectively specified by a request id list. 
- If so, optionally, user can pass in a empty list to collect error statuses of each request, which is useful in request-wise error tracking and debugging.
- Alternatively, user can flush all pending write and/or read requests using module-level NC constants (e.g. `NC_REQ_ALL`) as input parameters. The suffix
- `_all` indicates this is collective I/O in contrast to indepedent I/O (without `_all`).
+ Pending requests are eventually processed by :meth:`File.wait`. Requests to
+ committed can be specified selectively specified by a request id list.  If so,
+ optionally, user can pass in a empty list to collect error statuses of each
+ request, which is useful in request-wise error tracking and debugging.
+ Alternatively, user can flush all pending write and/or read requests using
+ module-level NC constants (e.g. `NC_REQ_ALL`) as input parameters. The suffix
+ `_all` indicates this is collective I/O in contrast to independent I/O
+ (without `_all`).
 
  Here's a python example to commit selected requests:
 
  .. code-block:: Python
 
-    # collective i/o 
+    # when the file is in the collective I/O mode
     req_errs = [None] * num_reqs
     f.wait_all(num_reqs, req_ids, req_errs)
-    # f.wait() # independent i/o
-    # f.wait_all() # commit all requests
-    # f.wait_all(num = NC_PUT_REQ_ALL) # commit all write requests
-    # f.wait_all(num = NC_GET_REQ_ALL) # commit all read requests
 
-Buffered Non-blocking Write
+    # when the file is in the independent I/O mode
+    f.wait(num_reqs, req_ids, req_errs)
+
+    # commit all pending write requests
+    f.wait_all(num = NC_PUT_REQ_ALL)
+
+    # commit all pending read requests
+    f.wait_all(num = NC_GET_REQ_ALL)
+
+Buffered Nonblocking Write
 -----------------------------
 
- One limitation of the above non-blocking write is that users should not alter the contents of the write buffer once the request is posted until the wait API is returned. 
- Any change to the buffer contents in between will result in unexpected error. To alleviate the this limitation, use can post buffered nonblocking write requests using 
- :meth:`Variable.bput_var`. The input parameters and returned values are identical to :meth:`Variable.iput_var`. However, user are free to alter/reuse/delete the write 
- buffer once the requests is postsed. As a prerequisite, the user need to tell PnetCDF the size of memory space required for all future reqests to this netCDF file. This step
- is achieved by :meth:`File.attach_buff`. 
+ One limitation of the above nonblocking write is that users should not alter
+ the contents of the write buffer once the request is posted until the wait API
+ is returned.  Any change to the buffer contents in between will result in
+ unexpected error. To alleviate the this limitation, use can post buffered
+ nonblocking write requests using :meth:`Variable.bput_var`. The input
+ parameters and returned values are identical to :meth:`Variable.iput_var`.
+ However, user are free to alter/reuse/delete the write buffer once the
+ requests is posted. As a prerequisite, the user need to tell PnetCDF the size
+ of memory space required for all future requests to this netCDF file. This step
+ is achieved by :meth:`File.attach_buff`.
+
+ Here's a python example to post a number of write requests and commit them
+ using buffered nonblocking API:
 
- Here's a python example to post a number of write requests and commit them using buffered non-blocking API:
- 
  .. code-block:: Python
 
-    f.enddef()
     data = randint(0,10, size=(xdim,ydim,zdim)).astype('i4')
+
     write_buff = [data] * num_reqs
     # Estimate the memory buffer size of all write requests
     buffsize = num_reqs * data.nbytes
+
     # Attach buffer for buffered put requests
     f.attach_buff(buffsize)
+
     req_ids = []
     for i in range(num_reqs):
        v = f.variables[f'data{i}']
-       # Post the request to write the whole variable
+       # Post a request to write the whole variable
        req_id = v.bput_var(write_buff[i])
        # Track the request ID for each write request
        req_ids.append(req_id)
-   # Free to alter the contents of write_buff here enabled by buffered non-blocking
+
+    # Users can now alter the contents of write_buff here
+
+    # wait for nonblocking, buffered writes to complete
     f.wait_all()
+
+    # Tell PnetCDF to detach the internal buffer
     f.detach_buff()
- 
- For the full example program, see ``examples/non_blocking_write.py``.
 
- Remember to detach the write buffer after write requets are executed.
- 
- 
+ For the full example program, see ``examples/nonblocking/nonblocking_write.py``.
+
+ Remember to detach the write buffer to free up the memory space.
+
+
