Skip to content

Benchmarks

Jump to bottom
Aaron Riekenberg edited this page Apr 8, 2023 · 102 revisions

Test System

  • 2017 MacBook Pro (my personal laptop)
  • 2.8 GHz Quad-Core Intel Core i7
    • 4 physical cores: hw.physicalcpu: 4
    • 2 threads per core, 8 threads total: hw.logicalcpu: 8
    • rust-parallel uses 8 parallel jobs by default on this machine
  • 16GB RAM
  • MacOS 13.2.1

TLDR

  • rust-parallel is more than an order of magnitude faster than GNU Parallel in tests below (~15 times faster)
  • rust-parallel is ~40% faster than command line xargs with 8 processes.

1000 invocations of md5 -s

rust-parallel 0.442 seconds real time elapsed (~40% faster than xargs -P8, ~15x faster than GNU Parallel)

$ rust-parallel -V
rust-parallel 0.5.2

$ time head -1000 /usr/share/dict/words | rust-parallel md5 -s

rust-parallel md5 -s  0.79s user 1.99s system 628% cpu 0.442 total

xargs with 8 processes 0.742 real time elapsed

$ time head -1000 /usr/share/dict/words | xargs -P8 -n1 md5 -s

xargs -P8 -n1 md5 -s  0.65s user 1.71s system 318% cpu 0.742 total

GNU Parallel 6.917 seconds real time elapsed

$ parallel -V
GNU parallel 20230322

$ time head -1000 /usr/share/dict/words | parallel md5 -s

parallel md5 -s  5.18s user 9.06s system 205% cpu 6.917 total

10,000 invocations of md5 -s

rust-parallel 4.290 seconds real time elapsed (~39% faster than xargs -P8, ~16x faster than GNU Parallel)

$ rust-parallel -V
rust-parallel 0.5.2

$ time head -10000 /usr/share/dict/words | rust-parallel md5 -s

rust-parallel md5 -s  7.75s user 19.57s system 636% cpu 4.290 total

xargs with 8 processes 6.993 seconds real time elapsed

$ time head -10000 /usr/share/dict/words | xargs -P8 -n1 md5 -s

xargs -P8 -n1 md5 -s  6.09s user 16.04s system 316% cpu 6.993 total

GNU Parallel 70.69 seconds real time elapsed

$ parallel -V
GNU parallel 20230322

$ time head -10000 /usr/share/dict/words | parallel md5 -s

parallel md5 -s  51.68s user 93.15s system 204% cpu 1:10.69 total
Clone this wiki locally