MySQL DBA - Tips and Techniques Magazine

6 Oct 2014

Use Streams, Pipes, Redirects, Grep and Cut!

BASIC  - Cat, Split, Diff


               cat - concatenate files and display

               wc -l      (line count)

cat file* | wc -l   (count of total line in file*)

split -l 10 file2      (splits file2 into 10 lines each in files named xaa , xab )

diff xaa xab   (shows difference between two files)


STREAMS (std out, stderr, stdin) & REDIRECTS

cat file* > newfile.txt        (  > redirect of stdout)

echo "new entry" >> newfile.txt   (appends)


ls missingfile 2>  error.log   ( stderr goes to error.log)

ls missingfile 2>  /dev/null  (stderr goes to nothing)


cat file1 file2 nofile > mystdoutput 2>&1   (stderr goes to stdout file)


set -o     (shows options for shell)

set -o noclobber    (cant overwrite existing file)

set +o noclobber (turn it off)


PIPES (use output of one command as input to another command)

ls /etc/ |grep cron |grep daily > grep .txt

ls /etc |sort -f |less   (sort and display into less editor)





using grep for pattern matching - regular expressions


grep hello testfile.txt     …pulls out all hello words in our file

grep ^hello file  …all the hellos at the beginning of the line

grep hello$ file …lines end with hello


grep [hpok] file   - returns all lines with any of these chars in it

can use ^[] and []$ too

[a-g] or [1-9]

grep -f grepinput  file.txt    …use file grepinput as the pattern to look for in file.txt

grep -lr cron /etc    …list all files  in /etc/ recrusively that have cron in it.


EGREP - extended grep ..use it when we are using extra commands like |, .*

egrep is the same as writing  grep -e

egrep   'hello.*world' file.txt    (.* is and)

egrep -i      (ignore case)

'hello|world'   - search for lines with hello, or world or both

egrep -v    …returns everything that doesn't match

FGREP  - look for literal pattern, no extended patterns

fgrep is same as writing grep -F


fgrep hello$ file.txt    …find the lines with hello$  


CUT - take some of stdout

eg want to grab all usernames on the system

cut -f 1 -d: /etc/passwd   (fi = field 1,  d: = deliminator is :  )



               SED - stream editor


               sed 's/old/new/w newfile.txt'  oldfile.txt


               sed '0,/old/s/old/new' oldfile.txt    ..look for first occurance of old and only substitute that


               eg remove html tages

               sed 's/<[^]*>//' team      eg subsititute beginning with <, but not immeditaly followed by ^> , replace with blank


               TEE command - read from stdin and write to stdout


               eg want to show on screen and to file(s) at same time

ls | tee newfile1 newfile2

ls | tee -a newfile   …append


No comments:

Post a Comment