Examples

1. Processing CSV inputs using regular expression
2. Computation on list of files
3. Rename files in directory
4. Compress files from find command

Processing CSV inputs using regular expression

Suppose we have an input CSV file of http method and URL, this could be used to make parallel calls with curl. Here {method} and {url} are named regular expression capture groups, -j3 is maximum 3 parallel jobs, -t5 is a 5 second timeout:

$ cat >./test.csv <<EOL
GET,http://example.com/endpoint1
PUT,http://example.com/endpoint2
POST,http://example.com/endpoint3
EOL

$ cat test.csv | rust-parallel --regex '(?P<method>.*),(?P<url>.*)' -j3 -t5 curl -X {method} {url}

Computation on list of files

Suppose we have a bash function analyze_file and a list of *.txt files to analyze in current directory. This example uses --jobs 4 to control max parallel jobs, --shell to call a bash function, --progress-bar to display a graphical progress bar, and --timeout-seconds to kill the job if not finished after 5 minutes.

$ analyze_file() {
  # do some expensive analysis of file $1 parameter
}

$ export -f analyze_file

$ rust-parallel --jobs 4 --shell --progress-bar --timeout-seconds $((5*60)) analyze_file ::: *.txt

Rename files in directory

Rename files in current directory from from *.txt to *.csv.

{0} capture group is entire *.txt file name, {1} capture group is prefix of file name before .txt:

$ rust-parallel -r '(.*)\.(.*)' mv {0} {1}.csv ::: *.txt

Compress files from find command

Use find to find all files in current directory and subdirectories. The -0 option works nicely with find -print0 to handle filenames that may have whitespace characters. Call gzip -f -k on each file from find command:

$ find . -type f -print0 | rust-parallel -0 gzip -f -k