revise README.md

examples/Pytorch_DDP/README.md

@@ -1,20 +1,27 @@
-# Example Python programs that use Pytorch Distributed Data Parallel module
+# Example Python programs that use Pytorch DDP module
 
 This directory contains example python programs that make use of Pytorch
-Distributed Data Parallel (DDP) module and MPI to run on multiple MPI processes
-in parallel. Detailed information describing the example programs is provided
-at the beginning of each file.
+Distributed Data Parallel
+([DDP](https://pytorch.org/tutorials/intermediate/ddp_tutorial.html)) module
+and [mpi4pi](https://mpi4py.readthedocs.io/en/stable/) to run on multiple MPI
+processes in parallel. Detailed information describing the example programs is
+provided at the beginning of each file.
 
-## [torch_ddp_skeleton.py](./torch_ddp_skeleton.py) shows how to set up the MPI
-and DDP environment to run a program in parallel.
+* [torch_ddp_skeleton.py](#torch_ddp_skeleton_py) -- a template for using
+  Pytorch DDP
 
-Command usage:
-```sh
-% mpiexec -n 4 python ./torch_ddp_skeleton.py 
-nprocs =  4  rank =  0  device =  cpu
-nprocs =  4  rank =  1  device =  cpu
-nprocs =  4  rank =  2  device =  cpu
-nprocs =  4  rank =  3  device =  cpu
-```
+---
 
+## torch_ddp_skeleton_py
+[torch_ddp_skeleton.py](./torch_ddp_skeleton.py) is a skeleton program showing
+how to set up the MPI and DDP environment to run a program in parallel.
+
+* Command usage and output on screen:
+  ```sh
+  % mpiexec -n 4 python ./torch_ddp_skeleton.py
+  nprocs =  4  rank =  0  device =  cpu
+  nprocs =  4  rank =  1  device =  cpu
+  nprocs =  4  rank =  2  device =  cpu
+  nprocs =  4  rank =  3  device =  cpu
+  ```
 

examples/README.md

@@ -0,0 +1,106 @@
+# PnetCDF-python examples
+
+This directory contains example python programs that make use of PnetCDF to
+perform file I/O. Detailed description of each program and run instructions can
+be found at the beginning of each file.
+
+---
+### Running individual example programs
+
+* Use command `mpiexec` to run individual programs. For example, command
+  line below run `collective_write.py` on 4 MPI processes.
+  ```sh
+  mpiexec -n 4 python collective_write.py [output_dir]
+  ```
+* The optional argument `output_dir` enables the testing program to save the
+  generated output files in the specified directory. Default is the current
+  directory.
+
+---
+### Overview of Test Programs
+
+* [Pytorch_DDP](./Pytorch_DDP)
+  + A directory containing examples that make use of Pytorch Distributed Data
+    Parallel module to run python programs in parallel.
+
+* [collective_write.py](./collective_write.py)
+  + writes multiple 3D subarrays to non-record variables of int type using
+    collective I/O mode.
+
+* [put_vara.py](./put_vara.py)
+  + This example shows how to use `Variable` method put_var() to write a 2D
+    integer array in parallel. The data partitioning pattern is a column-wise
+    partitioning across all processes.
+
+* [get_vara.py](./get_vara.py)
+  + This is the read counterpart of [put_vara.py](./put_vara.py), which shows
+    how to use to `Variable` method get_var() read a 2D 4-byte integer array in
+    parallel.
+
+* [nonblocking_write.py](./nonblocking_write.py)
+  + Similar to `collective_write.py`, it uses nonblocking APIs instead. It
+    creates a netcdf file in CDF-5 format and writes a number of 3D integer
+    non-record variables.
+
+* [nonblocking_write_def.py](./nonblocking_write_def.py)
+  + This is the same as `nonblocking_write.py` expect all nonblocking write
+    requests (calls to `iput` and `bput`) are posted in define mode. It creates
+    a netcdf file in CDF-5 format and writes a number of 3D integer non-record
+    variables.
+
+* [create_open.py](./create_open.py)
+  + This example shows how to use `File` class constructor to create a netCDF
+    file and to open the file for read only.
+
+* [ghost_cell.py](./ghost_cell.py)
+  + This example shows how to use `Variable` method to write a 2D array user
+    buffer with ghost cells.
+
+* [fill_mode.py](./fill_mode.py)
+  + This example shows how to use `Variable` class methods and `File` class
+    methods to set the fill mode of variables and fill values.
+    * `set_fill()` to enable fill mode of the file
+    * `def_fill()` to enable fill mode and define the variable's fill value
+    * `inq_var_fill()` to inquire the variable's fill mode information
+    * `put_vara_all()` to write two 2D 4-byte integer array in parallel.
+
+* [global_attribute.py](./global_attribute.py)
+  + This example shows how to use `File` method `put_att()` to write a global
+    attribute to a file.
+
+* [flexible_api.py](./flexible_api.py)
+  + This example shows how to use `Variable` flexible API methods put_var() and
+    iput_var() to write a 2D 4-byte integer array in parallel.
+
+* [hints.py](./hints.py)
+  + This example sets two PnetCDF hints: `nc_header_align_size` and
+    `nc_var_align_size` and prints the hint values as well as the header size,
+    header extent, and two variables' starting file offsets.
+
+* [transpose2D.py](./transpose2D.py)
+  + This example shows how to use `Variable` method `put_var()` to write a 2D
+    integer array variable into a file. The variable in the file is a
+    dimensional transposed array from the one stored in memory.
+
+* [get_info.py](./get_info.py)
+  + This example prints all MPI-IO hints used.
+
+* [put_varn_int.py](./put_varn_int.py)
+  + This example shows how to use a single call of `Variable` method
+    `put_var()` to to write a sequence of requests with arbitrary array indices
+    and lengths.
+
+* [transpose.py](./transpose.py)
+  + This example shows how to use `Variable` method `put_var()` to write six 3D
+    integer array variables into a file. Each variable in the file is a
+    dimensional transposed array from the one stored in memory. In memory, a 3D
+    array is partitioned among all processes in a block-block-block fashion and
+    in ZYX (i.e.  C) order. The dimension structures of the transposed six
+    arrays are
+    * int ZYX_var(Z, Y, X) ;     ZYX -> ZYX
+    * int ZXY_var(Z, X, Y) ;     ZYX -> ZXY
+    * int YZX_var(Y, Z, X) ;     ZYX -> YZX
+    * int YXZ_var(Y, X, Z) ;     ZYX -> YXZ
+    * int XZY_var(X, Z, Y) ;     ZYX -> XZY
+    * int XYZ_var(X, Y, Z) ;     ZYX -> XYZ
+

examples/collective_write.py

@@ -4,20 +4,21 @@
 #
 
 """
- This example mimics the coll_perf.c from ROMIO. It creates a netcdf file and 
- writes a number of 3D integer non-record variables. The measured write bandwidth
- is reported at the end. 
- To run:
-    % mpiexec -n num_process python3 collective_write.py [test_file_name] [-l len]
+This example mimics the coll_perf.c from ROMIO. It creates a netcdf file and
+writes a number of 3D integer non-record variables. The measured write
+bandwidth is reported at the end.
+To run:
+  % mpiexec -n num_process python3 collective_write.py [test_file_name] [-l len]
 where len decides the size of each local array, which is len x len x len.
 So, each non-record variable is of size len*len*len * nprocs * sizeof(int)
-All variables are partitioned among all processes in a 3D block-block-block 
+All variables are partitioned among all processes in a 3D block-block-block
 fashion.
- Example commands for MPI run and outputs from running ncmpidump on the
- netCDF file produced by this example program:
+
+Example commands for MPI run and outputs from running ncmpidump on the
+netCDF file produced by this example program:
     % mpiexec -n 32 python3 collective_write.py tmp/test1.nc -l 100
     % ncmpidump tmp/test1.nc
-    
+
     Example standard output:
     MPI hint: cb_nodes        = 2
     MPI hint: cb_buffer_size  = 16777216

examples/create_open.py

@@ -4,17 +4,17 @@
 #
 
 """
- This example shows how to use `File` class constructor to create a netCDF file and to 
- open the file for read only.
+This example shows how to use `File` class constructor to create a netCDF file
+and to open the file for read only.
 
- Example commands for MPI run and outputs from running ncmpidump on the
- netCDF file produced by this example program:
-    % mpiexec -n 4 python3  create_open.py /tmp/test1.nc
-    % ncmpidump /tmp/test1.nc
-        netcdf test1 {
-        // file format: CDF-1
-        }
+Example commands for MPI run and outputs from running ncmpidump on the
+netCDF file produced by this example program:
 
+  % mpiexec -n 4 python3  create_open.py /tmp/test1.nc
+  % ncmpidump /tmp/test1.nc
+  netcdf test1 {
+  // file format: CDF-1
+  }
 """
 
 import sys

examples/fill_mode.py

@@ -4,18 +4,18 @@
 #
 
 """
- This example shows how to use `Variable` class methods and `File` class methods 
- to set the fill mode of variables and fill values.
+This example shows how to use `Variable` class methods and `File` class methods
+to set the fill mode of variables and fill values.
  * 1. set_fill() to enable fill mode of the file
  * 2. def_fill() to enable fill mode and define the variable's fill value
  * 3. inq_var_fill() to inquire the variable's fill mode information
  * 4. put_vara_all() to write two 2D 4-byte integer array in parallel.
 
+Example commands for MPI run and outputs from running ncmpidump on the
+netCDF file produced by this example program:
 
- Example commands for MPI run and outputs from running ncmpidump on the
- netCDF file produced by this example program:
-    % mpiexec -n 4 python3 fill_mode.py tmp/test1.nc
-    % ncmpidump tmp/test1.nc
+  % mpiexec -n 4 python3 fill_mode.py tmp/test1.nc
+  % ncmpidump tmp/test1.nc
     netcdf test1 {
     // file format: CDF-1
     dimensions:
@@ -80,7 +80,7 @@ def main():
     parser.add_argument("dir", nargs="?", type=str, help="(Optional) output netCDF file name",\
                          default = "testfile.nc")
     parser.add_argument("-q", help="Quiet mode (reports when fail)", action="store_true")
-    
+
     args = parser.parse_args()
     if args.q:
         verbose = False

examples/flexible_api.py

@@ -4,28 +4,27 @@
 #
 
 """
-/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
-  This example shows how to use `Variable` method put_var() and iput_var() to write a 2D 4-byte
-  integer array in parallel (one is of 4-byte
-  integer byte and the other float type) in parallel. It first defines 2 netCDF
-  variables of sizes
+This example shows how to use `Variable` flexible API methods put_var() and
+iput_var() to write a 2D 4-byte integer array in parallel (one is of 4-byte
+integer byte and the other float type). It first defines 2 netCDF variables of
+sizes
      var_zy: NZ*nprocs x NY
      var_yx: NY x NX*nprocs
- 
-  The data partitioning patterns on the 2 variables are row-wise and
-  column-wise, respectively. Each process writes a subarray of size
-  NZ x NY and NY x NX to var_zy and var_yx, respectively.
-  Both local buffers have a ghost cell of length 3 surrounded along each
-  dimension.
-    To run:
-        % mpiexec -n num_process python3 flexible_api.py [test_file_name]
- 
-  Example commands for MPI run and outputs from running ncmpidump on the
-  output netCDF file produced by this example program:
- 
-     % mpiexec -n 4 python3 flexible_api.py /tmp/test1.nc
- 
-     % ncmpidump /tmp/test1.nc
+
+The data partitioning patterns on the 2 variables are row-wise and column-wise,
+respectively. Each process writes a subarray of size NZ x NY and NY x NX to
+var_zy and var_yx, respectively.  Both local buffers have a ghost cell of
+length 3 surrounded along each dimension.
+
+To run:
+  % mpiexec -n num_process python3 flexible_api.py [test_file_name]
+
+Example commands for MPI run and outputs from running ncmpidump on the
+output netCDF file produced by this example program:
+
+  % mpiexec -n 4 python3 flexible_api.py /tmp/test1.nc
+
+  % ncmpidump /tmp/test1.nc
      netcdf testfile {
      // file format: CDF-5 (big variables)
      dimensions:
@@ -36,7 +35,7 @@
              int var_zy(Z, Y) ;
              float var_yx(Y, X) ;
      data:
- 
+
       var_zy =
        0, 0, 0, 0, 0,
        0, 0, 0, 0, 0,
@@ -58,15 +57,14 @@
        3, 3, 3, 3, 3,
        3, 3, 3, 3, 3,
        3, 3, 3, 3, 3 ;
- 
+
       var_yx =
        0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3,
        0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3,
        0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3,
        0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3,
        0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3 ;
      }
- * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
 """
 
 import sys
@@ -167,7 +165,7 @@ def main():
     buf_zy.fill(-1)
     var_zy.get_var_all(buf_zy, start = starts, count = counts, bufcount = 1, buftype = subarray)
     # print(buf_zy.reshape(array_of_sizes))
-    
+
     # check contents of the get buffer
     for i in range(array_of_sizes[0]):
         for j in range(array_of_sizes[1]):

examples/get_info.py

@@ -4,9 +4,10 @@
 #
 
 """
+This example prints all MPI-IO hints used.
 
- Example commands for MPI run and outputs from running ncmpidump on the
- netCDF file produced by this example program:
+Example commands for MPI run and outputs from running ncmpidump on the
+netCDF file produced by this example program:
     % mpiexec -n 4 python3 get_info.py tmp/test1.nc
     % ncmpidump tmp/test1.nc
         Example standard output:
@@ -29,7 +30,7 @@
     MPI File Info: [15] key =      nc_header_align_size, value = 512
     MPI File Info: [16] key =         nc_var_align_size, value = 512
     MPI File Info: [17] key = nc_header_read_chunk_size, value = 0
- 
+
 """
 
 import sys
@@ -66,10 +67,10 @@ def print_info(info_used):
         value = info_used.Get(key)
         print("MPI File Info: [{:2d}] key = {:25s}, value = {}".format(i, key, value))
 
-    
+
 def main():
     nprocs = size
-    
+
     global verbose
     if parse_help():
         MPI.Finalize()
@@ -80,7 +81,7 @@ def main():
     parser.add_argument("dir", nargs="?", type=str, help="(Optional) output netCDF file name",\
                          default = "testfile.nc")
     parser.add_argument("-q", help="Quiet mode (reports when fail)", action="store_true")
-    
+
     args = parser.parse_args()
     if args.q:
         verbose = False