AwkTutorial < Main

Main Web>ProgrammingKnowHow>AwkTutorial (01 Oct 2003, MattWalsh)EditAttach

examples taken from here...

Sample file for examples
examples
Special variables / functions
Redirection
Other separators
My Cluster Scripts

Sample file for examples

   gold     1    1986  USA                 American Eagle
   gold     1    1908  Austria-Hungary     Franz Josef 100 Korona
   silver  10    1981  USA                 ingot
   gold     1    1984  Switzerland         ingot
   tin      1    1809  Campbell's          Soup can
   gold     1    1979  RSA                 Krugerrand
   gold     0.5  1981  RSA                 Krugerrand
   gold     0.1  1986  PRC                 Panda
   silver   1    1986  USA                 Liberty dollar
   gold     0.25 1986  USA                 Liberty 5-dollar piece
   silver   0.5  1986  USA                 Liberty 50-cent piece
   silver   1    1987  USA                 Constitution dollar
   gold     0.25 1987  USA                 Constitution 5-dollar piece
   gold     1    1988  Canada              Maple Leaf

examples

In each case, put the filename after the command...

print every line with `gold` in it. `gold` does not have to be the first thing in the line
`awk '/gold/'` or `awk '/gold/ {print}'` or `awk '/gold/ {print $0}'`

print every line with gold in it. gold does not have to be the first thing in the line

awk '/gold/' or awk '/gold/ {print}' or awk '/gold/ {print $0}'

print when column 3 is < 1980. If a value is not numeric, it doesn't complain

print when column 3 is < 1980. If a value is not numeric, it doesn't complain
`awk '{if ($3 < 1980) print $3, " ",$5,$6,$7,$8}'`

awk '{if ($3 < 1980) print $3, " ",$5,$6,$7,$8}'

`END` is program clause that means after the scanning is done. This prints the number of lines (Number Records) scanned. It did not work for me as a way to, say count the # of lines that say `gold`. If you omit the `END`, you get a line count for each line scanned so far, not just one at the end. You can also do `NF`, which is the Number Fields for a line
`awk 'END {print NR,"coins"}'`

END is program clause that means after the scanning is done. This prints the number of lines (Number Records) scanned. It did not work for me as a way to, say count the # of lines that say gold. If you omit the END, you get a line count for each line scanned so far, not just one at the end. You can also do NF, which is the Number Fields for a line

awk 'END {print NR,"coins"}'

for each line that has `gold`, add the value of column 2 to variable `ounces`. Then multiply by a constant and print at the end
`awk '/gold/ {ounces += $2} END {print "value = $" 425*ounces}'`

for each line that has gold, add the value of column 2 to variable ounces. Then multiply by a constant and print at the end

awk '/gold/ {ounces += $2} END {print "value = $" 425*ounces}'

compute stats for gold and silver, print summary

compute stats for gold and silver, print summary
`awk '/gold/ { num_gold++; wt_gold += $2 } /silver/ {num_silver++; wt_silver += $2 } END {print "# Au / Ag: ", num_gold, " / ", num_silver; print "wt Au / Ag: ", wt_gold, " / ", wt_silver}'`

awk  '/gold/ { num_gold++; wt_gold += $2 } /silver/ {num_silver++; wt_silver += $2 } END {print "#  Au / Ag: ", num_gold, " / ", num_silver; print "wt Au / Ag: ", wt_gold, " / ", wt_silver}'

print when field one...
is gold	`awk '$1 ~ /gold/'`
is not gold	`awk '$1 !~ /gold/'`
is not gold or silver	`awk '$1 !~ /(gold)\|(silver)/'`

matching ranges of lines
`awk 'NR==3'`	match line 3. Don't forget the double equals! `==`
`awk 'NR==3,NR==5'`	match from lines 3 to 5
`awk 'NR==2,/gold/'`	match from line 2 to a line with `gold` in it
`awk '/gold/,/tin/'`	match from `gold` line to `tin` line
`awk '/ingot/,/Panda/'`	match from earliest `ingot` ling to last `panda` line

`awk 'NF = 0'`	match non-empty lines (that have 1 or more field)
`awk 'NF = 0 {++count} END {print count}'`	count number of non-empty lines in a file
`awk '{print ; if (NF = 0) print ""}' infile > outfile`	make a single spaced file double spaced, except don't duplicate already blank lines
`awk '($1 = "tin") \|\| (($1 = "gold") && ($2 < 1))'`	when field 1 is tin, or gold if it's less than an ounce

Special variables / functions

NR	current count of input lines
NF	# fields in current line
FILENAME	current filename
FS	field separator. Can be changed to something other than whitespace
RS	record separator. Default is `newline`
OFS / ORS	output field / record separators (default, space / `newline` )
length	`length` of current line ==awk '{print length ":" $0 }'
substr()
`awk '{c=split($0, s); for( n=1; n<=c; ++n) print s[n] }' $1`	splitting sentences into words
index()

Redirection

`print 3 > "tfile"`	creates a file named `tfile` containing the number "3"
`print 3 >> "tfile"`	appends to file `tfile`
`BEGIN {for (x=1; x<=50; ++x) {printf("%03d\n",x) > "tfile"}; exit}`	creates `tfile` with numbers 1-50

Other separators

awk -F, '{print $1 $3}' uses the comma (,) character as separator

`awk -F, '{print $1 $3}'`	uses the comma (`,`) character as separator

My Cluster Scripts

Prints memory usage for all the nodes (hardest part was getting the hostname to print
cexec 'free | awk "/Mem:/ { printf(\"%8d %8d %8d %s\n\", \$4, \$3, \$2, \"`hostname`\")}"' | awk 'BEGIN {print " Free Used Total\n--------------------------"} !/----/'

Prints memory and cpu percentage usage only on nodes running =xhpl=
cexec "ps -aux | grep xhpl | awk '!/grep/'" | awk 'BEGIN{print " %CPU | %Mem | Machine\n----------------------"} !/----/ {printf(" %2d | %2d | %s\n", $3, $4, $16)}'

-- MattWalsh - 30 Sep 2003

Topic revision: r3 - 01 Oct 2003 - 06:25:43 - MattWalsh

Main

Webs
Main
Sandbox
TWiki

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback