Jon's quick and dirty wc, head, tail, (e)grep, awk, and sed tutorial Jon Hart http://www.ccs.neu.edu/home/jhart 10/03/2002 wc -- Prints the number of characters (-c), words (-w) or lines (-l) in a file. Examples: # count number of users $ wc -l /etc/passwd 23 /etc/passwd # count number of words in my ECN test $ wc -w take-home-1.txt 1202 take-home-1.txt # just in case you can't count $ echo "I'm slow" | wc -c 9 =============================================================================== head ---- Prints out the first X characters (-c) or lines (-n) in a file Examples: # show first 3 users $ head -n 3 /etc/passwd root:x:0:0::/root:/bin/zsh bin:x:1:1:bin:/bin: daemon:x:2:2:daemon:/sbin: # show the first 10 random characters $ head -c 10 /dev/urandom m€A8È1Ä ¢G # mix it up a bit. Count the number of characters # in the first 50 lines of my ECN test $ head -n 50 take-home-1.txt | wc -c 2585 =============================================================================== tail ---- Similar to head, but the reverse. Prints out the last X characters (-c) or lines in a file. Check out the -f option. Examples: # show the last 3 users $ tail -n 3 /etc/passwd warchild:x:1000:100::/home/warchild:/bin/zsh sshd:x:1001:102::/home/sshd:/bin/false postfix:x:1002:103::/var/spool/mail:/bin/false # count the number of words between lines 100 and 200 in my ECN test $ head -n 200 take-home-1.txt | tail -n 100 | wc -w 901 =============================================================================== awk --- A programming language. Should you find yourself ever writing extensive scripts/programs in this, seek help. Most useful for parsing out specific fields from a file. The only real option you need is -F which allows you to change the default field separator which defaults to whitespace. Fields are represented using variables: $1, $2, $3, $4, etc... Examples: # print all the known IP addresses for this machine $ awk '{print $1}' /etc/hosts 127.0.0.1 192.168.0.1 192.168.0.2 192.168.0.3 192.168.0.4 192.168.0.5 # get the last 3 groups that were added plus the group members, if any $ tail -n 3 /etc/group | awk -F: '{print "Group: "$1"\tMembers: "$4}' Group: sshd Members: Group: postdrop Members: Group: src Members: warchild =============================================================================== (e)grep ------- Only print stuff that matches a pattern you specify. Useful options are ignore case (-i), inverse match (-v), and recursive search (-r). # Find all bash users $ grep /bin/bash /etc/passwd operator:x:11:0:operator:/root:/bin/bash mysql:x:27:27:MySQL:/var/lib/mysql:/bin/bash gdm:x:42:42:GDM:/var/state/gdm:/bin/bash # find all occurences of the word "telnet" in all of the files in /etc, # recursively $ grep -ir telnet /etc/* # show the shells of all users except "Smelly Bob" $ awk -F: '{print $5 $7}' /etc/passwd | egrep -v "^Smelly Bob" | awk '{print $2}' ================================================================================ sed --- Stream editor. Again, you can get very tricky with sed just like you can in awk. Most common usages including search and replace ('s///') and deletion ('//d'). Examples: # convert everyone to zsh, change the root user's name $ sed -e 's/bash/zsh/' -e 's/root/god/' /etc/passwd god:x:0:0::/root:/bin/zsh bin:x:1:1:bin:/bin: uucp:x:10:14:uucp:/var/spool/uucppublic: operator:x:11:0:operator:/god:/bin/zsh games:x:12:100:games:/usr/games: ftp:x:14:50::/home/ftp: rpc:x:32:32:RPC portmap user:/:/bin/false gdm:x:42:42:GDM:/var/state/gdm:/bin/zsh nobody:x:99:99:nobody:/: warchild:x:1000:100::/home/warchild:/bin/zsh sshd:x:1001:102::/home/sshd:/bin/false postfix:x:1002:103::/var/spool/mail:/bin/false # remove all lines containing bash $ sed '/bash/d' /etc/passwd root:x:0:0::/root:/bin/zsh bin:x:1:1:bin:/bin: daemon:x:2:2:daemon:/sbin: adm:x:3:4:adm:/var/log: lp:x:4:7:lp:/var/spool/lpd: sync:x:5:0:sync:/sbin:/bin/sync shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown halt:x:7:0:halt:/sbin:/sbin/halt