Open navigation menu

Xargs for Parallel Execution

Uses xargs to build and execute commands from standard input with parallel execution, argument batching, and null-delimiter handling for safe processing.

BashCli Tooling

Code

snippet
#!/usr/bin/env bash
set -euo pipefail

# Basic xargs: pass input as arguments
echo "file1 file2 file3" | xargs mkdir -p

# One argument per command
printf "a\nb\nc\n" | xargs -I{} echo "Processing: {}"

# Parallel execution (-P)
printf "1\n2\n3\n4\n" | xargs -P4 -I{} sh -c "echo Job {} starting; sleep 1; echo Job {} done"

# Null-delimited for safe filenames
find /etc -maxdepth 1 -name "*.conf" -type f -print0 2>/dev/null |
    xargs -0 -I{} echo "Found: {}"

# Batch arguments
printf "1\n2\n3\n4\n5\n6\n" | xargs -n2 echo "Pair:"

# Parallel with progress
process_file() {
    local file=$1
    echo "Processing: $file"
    sleep 0.5
    echo "Done: $file"
}
export -f process_file

find /etc -maxdepth 1 -name "*.conf" -type f 2>/dev/null |
    head -5 |
    xargs -P4 -I{} bash -c "process_file "$@"" _ {}

# Dry run: show commands without executing
printf "file1\nfile2\nfile3\n" | xargs -t -I{} echo "Would process: {}"

# Handle large argument lists
find /usr -type f -name "*.so" 2>/dev/null |
    xargs -n100 -I{} sh -c "echo Processing batch; ls -la {}" 2>/dev/null |
    head -5

# Cleanup
rm -rf file1 file2 file3

Related Examples