Use Streams, Pipes, Redirects, Grep and Cut!

BASIC  - Cat, Split, Diff

 

               cat - concatenate files and display

               wc -l      (line count)

cat file* | wc -l   (count of total line in file*)

split -l 10 file2      (splits file2 into 10 lines each in files named xaa , xab )

diff xaa xab   (shows difference between two files)

 

STREAMS (std out, stderr, stdin) & REDIRECTS

cat file* > newfile.txt        (  > redirect of stdout)

echo "new entry" >> newfile.txt   (appends)

 

ls missingfile 2>  error.log   ( stderr goes to error.log)

ls missingfile 2>  /dev/null  (stderr goes to nothing)

 

cat file1 file2 nofile > mystdoutput 2>&1   (stderr goes to stdout file)

 

set -o     (shows options for shell)

set -o noclobber    (cant overwrite existing file)

set +o noclobber (turn it off)

 

PIPES (use output of one command as input to another command)

ls /etc/ |grep cron |grep daily > grep .txt

ls /etc |sort -f |less   (sort and display into less editor)

 

GREP, EGREP and FGREP

 

GREP

using grep for pattern matching - regular expressions

 

grep hello testfile.txt     …pulls out all hello words in our file

grep ^hello file  …all the hellos at the beginning of the line

grep hello$ file …lines end with hello

 

grep [hpok] file   - returns all lines with any of these chars in it

can use ^[] and []$ too

[a-g] or [1-9]

grep -f grepinput  file.txt    …use file grepinput as the pattern to look for in file.txt

grep -lr cron /etc    …list all files  in /etc/ recrusively that have cron in it.

 

EGREP - extended grep ..use it when we are using extra commands like |, .*

egrep is the same as writing  grep -e

egrep   'hello.*world' file.txt    (.* is and)

egrep -i      (ignore case)

'hello|world'   - search for lines with hello, or world or both

egrep -v    …returns everything that doesn't match

FGREP  - look for literal pattern, no extended patterns

fgrep is same as writing grep -F

 

fgrep hello$ file.txt    …find the lines with hello$  

 

CUT - take some of stdout

eg want to grab all usernames on the system

cut -f 1 -d: /etc/passwd   (fi = field 1,  d: = deliminator is :  )

 

 

               SED - stream editor

               eg

               sed 's/old/new/w newfile.txt'  oldfile.txt

 

               sed '0,/old/s/old/new' oldfile.txt    ..look for first occurance of old and only substitute that

 

               eg remove html tages

               sed 's/<[^]*>//' team      eg subsititute beginning with <, but not immeditaly followed by ^> , replace with blank

 

               TEE command - read from stdin and write to stdout

 

               eg want to show on screen and to file(s) at same time

ls | tee newfile1 newfile2

ls | tee -a newfile   …append