Open navigation menu

Text Processing with Sed and Awk

Uses sed for stream editing and awk for field-based processing to transform, filter, and extract data from text files and command output.

BashCli Tooling

Code

snippet
#!/usr/bin/env bash
set -euo pipefail

# Create sample data
cat > /tmp/data.csv << "EOF"
name,age,city,salary
Alice,30,New York,75000
Bob,25,Boston,65000
Charlie,35,Chicago,85000
Diana,28,Seattle,70000
EOF

# sed: substitute and transform
echo "=== Sed Operations ==="
sed "s/Boston/London/" /tmp/data.csv

# Delete lines matching pattern
sed "/^$/d; /^#/d" /tmp/data.csv

# Print specific lines
sed -n "2,4p" /tmp/data.csv

# Multiple commands
sed -e "s/Alice/Alicia/" -e "s/Bob/Robert/" /tmp/data.csv

# awk: field processing
echo "=== Awk Operations ==="
awk -F, "{print \$1, \$4}" /tmp/data.csv

# Filter and format
awk -F, "$3 == "Chicago" {print \$1, "earns \$" \$4}" /tmp/data.csv

# Compute totals
awk -F, "NR > 1 {total += \$4; count++}
         END {printf "Average: \$%.2f\n", total/count}" /tmp/data.csv

# Conditional processing
awk -F, "NR > 1 {"
    "    if (\$4 > 70000) category = "High""
    "    else if (\$4 > 60000) category = "Medium""
    "    else category = "Low""
    "    printf "%-10s %s\n", \$1, category"
"}" /tmp/data.csv

# Group by city
awk -F, "NR > 1 {"
    "    cities[\$3]++"
    "    salaries[\$3] += \$4"
}"
END {"
    "    for (city in cities) {"
    "        printf "%-12s count=%d avg=\$%.0f\n","
    "            city, cities[city], salaries[city]/cities[city]"
    "    }"
"}" /tmp/data.csv

rm -f /tmp/data.csv

Related Examples