Powerful GNU parallel, more than a loop

I have been using GNU parallel for many years, which boosts my programming efficiency significantly. Basically, it can repeat the same operation on different inputs easily, like for loop, but more than that -- the repeats run in parallel.

Here is a simple example to copy several files to AWS S3 bucket:

Note that each of input parameters is denoted as '{}' in the command.

To limit how many jobs (each job corresponds to one input) to run in parallel, use the option '-j', say 2 jobs, then add '-j 2'.

To learn more about this command, please check this tutorial https://www.gnu.org/software/parallel/parallel_tutorial.html

To view or add a comment, sign in

Insights from the community

Others also viewed

Explore topics