Unix basic command and shell

script for DBA and SYSADMIN

DIRECTORY STRUCTURE IN UNIX

Directory Contains
bin/ Required Boot-time binaries
Boot configuration files for the OS
boot/
loader and kernel image
dev/ Device files
System configuration files and
etc/
scripts
home/ User/Sub branch directories
Main OS shared libraries and kernel
lib/
modules
Storage directory for “recovered”
Lost+found/
files
mnt/ Temporary point to connect devices to
Pseudo directory structure containing
information about the kernel,
proc/
currently running processes and
resource allocation
Linux (non-standard) home directory
root/ for the root user. Alternate location
being the / directory itself
System administration binaries and
sbin/
tools
tmp/ Location of temporary files
Difficult to define – it contains
almost everything else including
usr/ local binaries, libraries,
applications and packages (including
X Windows)
Variable data, usually machine
var/ specific. Includes spool directories
for mail and news

GENERAL UNIX COMMANDS

TO SEE WHERE COMMANDS ARE LOCATED

echo $PATH

/bin:/usr/bin:/usr/local/bin:$HOME/bin

which ls

SEEING HELP COMMAND

date –help

info date

man date

whatis date

Extended Documents:=/usr/share/doc

TO CHANGE CONSOLE

Ctrl+alt+(F1-F7)

CHANGING DIRECTORY
cd /home/deb/work

cd ..[One level up]

LISTING DIRECTORIES AND FILES

ls /

ls –al [All hidden files]

ls –l [long listing]

ls –R[recurses through sub-directories]

ls -d[Listing directories only]

ls -ltr|grep ^d[Listing directories only]

List of command options

Option Action

-a list hidden files

-d list the name of the current directory

-F show directories with a trailing ’/’ executable

files with a trailing ’*’

-g show group ownership of file in long listing

-i print the inode number of each file

-l long listing giving details about files and direc-tories

-R list all subdirectories encountered

-t sort by time modified instead of name

COPYING FILES
cp source destination

cp –P source destination [Preserves permissions]

cp –R source destination [Recursively copies the files]

cp ~helper/tmp/for_john tmp/comments[To copy a file from

another user’s directory using a relative

pathname.]

cp /usr/lib/more.help .[To copy a file from another directory

to the current working

directory, preserving the file name:]

MOVING AND RENAMING

mv file1 file2 [ Renaming]

mv source destination [ Move]

REMOVING FILES AND DIRECTORIES

rm chap[1-2][wildcard is used]

rm -i chap1[Removing interactively]

rm -rf[Deleting forefully file or directories]

rmdir deb[removing empty directory]

TO CHECK FILE TYPES

file deb.sh

Displaying the first few lines of a file

head /usr/local/doc/vi.txt[To display the first 10 lines of a

file]

head -5 *.xdh[To display the first 5 lines of several files]

Displaying the last part of a file

tail fred[To display the last 10 lines of a text file]

tail -50c help.txt[To display the last 50 characters in a

file]

tail -c -512 foo[Copies last 512 bytes from foo]

tail -c +512 foo[Copies everything after skippiong 512 bytes]

HANDLING PRINTERS

lp deb.lis[Printing file]

lp -d laser deb.lis[If there are more printers]

lp -n3 deb.lis[Printing 3 copies]

lpstat[To view print queue]

cancel laser[To cancel printing of current printer]

cancel pr1-320[Cancel job with request id pr1-320]

lpq[To see print que job number]

lprm 31[Removes job number 31]

lpr -p laser deb.lis[Prints on printer laser]

lpstat -c[To see pending jobs]

COUNTING LINES
wc deb.txt

wc -c[No. of characters]

wc -l[No. of lines]

COMPRESSING FILES

compress deb.txt

uncompress deb.txt

gzip deb

gunzip deb.z

USING TAR COMMAND

# display contents of a file

tar tvf myfile.tar

# display contents of a diskette (Solaris)

volcheck
tar tvf /vol/dev/rdiskette0/unnamed_floppy

# copy files to a tar file

tar cvf myfile.tar *.sql

# format floppy, and copy files to it (Solaris)

fdformat -U -b floppy99
tar cvf /vol/dev/rdiskette0/floppy99 *.sql

# append files to a tar file

tar rvfn myfile.tar *.txt

# extract files from a tar filem to current dir

tar xvf myfile.tar

SORTING

sort /etc/passwd
sort in order based on the whole line

sort -r /etc/passwd
reverse the order of the sort

sort +2n -t: /etc/passwd

sort on third field, where field delimiter is : (skip the
first two fields)

sort +2n -t: -n /etc/passwd

same sort but treat the field as numbers not ASCII characters

sort –o telnos telnos(sorts telnos and replaces its content

with the sorted output)

DUPLICATE ROW BY UNIQUE COMMAND

uniq names
remove duplicate lines from names and display them on the
screen

uniq names uniq.names

remove duplicates lines from names and put them into
uniq.names

uniq -d names
display all duplicate lines

Using symbolic links

ln -s source linkname

ln -s reports/reportA publications/my_report

TO DETERMINE DISK USAGE

# display disk free, in KB

df -kt

# display disk usage, in KB for directory

du -k mydir

# display directory disk usage, sort by largest first

du -ahx .|sort -rh|head -20

FIND Command

Find files by text search

$ find ~ -name “*.txt” -print

To find all ‘*.txt’ files in the current directory you are in

and any sub-directories:

$ find . -name “*.txt” -print

To find all filenames with at least one upper case character,

in your current directory

and any sub-directories:

$ find . -name “[A-Z]*” -print

To find files in /etc directory that begin with host, and any
other characters after

that:

$ find /etc -name “host*” -print

To find all files in your $HOME directory:

$ find ~ -name “*” -print or find . -print

To bring the system to zero per cent response time, start at

the root level and drill

through all directories listing ‘all’ files. If you want to

remain on good terms with

your sys admin be very careful with this option!

$ find / -name “*” -print

Find all files that begin with two lower case characters,
followed by two numbers,

followed by .txt. The following find command could return a

file called ax37.txt.

$ find . -name “[a-z][a-z][0–9][0–9].txt” -print

Find files by perm mode

To find files with a permission of 755 which is read, write
and executable by

owner and read and execute by everyone else,

$ find . -perm 755 -print

To find files that are read, write and executable for everyone
(watch out for these) use

this find command. Put a dash in front of the octal number.

The perm stands for

permissions and 007 is based on the notation you use for the
chmod (absolute) mode.

$ find . -perm -007 -print

Ignoring directories

To display the files in the apps directory when you do not

want find to search

in /apps/bin:

$ find /apps -name “/apps/bin” -prune -o -print

Find files by user and nouser

To find files by owner, you supply the actual login name of

that user. To find files

owned by user dave in your $HOME directory:

$ find ~ -user dave -print

To find files owned by uucp in /etc:

$ find /etc -user uucp -print

To find files of a user that has been deleted, use the -nouser
option. This

finds files that have no valid userid in the /etc/passwd file.

You do not have to

supply a user name with this option; find will go off and do
all the work. To find

all files not owned by a current user in the /home directory

tree:

$ find /home -nouser -print

Find files by group and nogroup

Like the user and nouser option, find has the same options for
groups. To find all

files with the group membership of ‘accts’ that are in the

/apps directory:

$ find /apps -group accts -print

To find all files that have no valid group membership just

supply the -nogroup
option. Here find starts at the root of the file system.

$ find / -nogroup -print

Find files by modification times

To find all files that have been modified in the last five
days:

$ find / -mtime -5 -print

To find files in /var/adm directory that have not been

modified in the last three

days:

$ find /var/adm -mtime +3 -print

Finding files that are newer or older in days or minutes

$ find . -newer age.awk ! -newer belts.awk -exec ls -l {} \;

Delete old trace files from DIAG location of Oracle Database

find /u01/app/oracle -type f -name “*.trc” -mtime +1 -exec rm

{} \;

find /u01/app/oracle -type f -name “*.trm” -mtime +0.5 -exec

rm {} \;

find /u01/app/oracle -type f -name “*.aud” -mtime +0.5 -exec

rm {} \;
VI EDITOR

List of insert commands

To do this … Command

Insert text after the cursor a

Insert text before the cursor i

Append text at the end of the current line A

Insert text at the start of the current line I

Open a new line above the current line o

Open a new line below the current line O

Recovering files after a system crash

vi -r help.xdh

Viewing a file

view filename

Moving the cursor along a line

Press the ESC key to enter command mode before using these
com-mands.

To move to Do this …
next character l

previous character h

next word w

next n words wn

previous word b

previous n words bn

end of current word e

start of current line 0 (zero)

end of current line $

Moving the cursor between lines

Press the ESC key to enter command mode before using these
com-mands.

To move to Do this …

next line down j

(same column)

start of next line down +

previous line k

(same column)

start of previous line

Deleting characters

Press the ESC key to enter command mode before using these
com-mands.
To delete Do this

current character x

previous character dh

Deleting words and lines

Press the ESC key to enter command mode before using these
com-mands.

To delete Do this

current word dw

previous word db

entire line dd

to end of line d$

to start of line d0 (zero)

next n lines ndd

To get back a word or line that you have just deleted enter
the command:

Searching for text

Press the ESC key to enter command mode before using these
com-mands.

To search Do this …

forward for a pattern /pattern

backward for a pattern ?pattern

repeat previous search n

repeat previous search in N

reverse direction

Examples of replacing text

To replace one word in the current line with another:

:s/that/which

To replace every instance of one word in the current line with

another:

:s/destroy/delete/g

This replaces every occurrence of the word “destroy” with the

word “delete” in the current line.

To replace every occurrence of a word throughout the file,

with

another:

:g/insert/s//add/g

This replaces every occurrence of “insert” with “add”.

To replace ^M from text file

:%s/^v^M/ /g

UNIX Scheduler and Process

CRONTAB

1st column Minutes 1–59

2nd column Hour 1–23 (0 is midnight)

3rd column Day_of_month 1–31

4th column Month 1–12

5th column Weekday 0–6 (0 is Sunday)

6th column Command to run

Here are some examples of crontab entries:

30 21 * * * /apps/bin/cleanup.sh

runs a script called cleanup.sh in /apps/bin at 9:30 every

night.

Cron and crontab 33

45 4 1,10,22 * * /apps/bin/backup.sh

runs a script called backup.sh in /apps/bin at 4:45 a.m., on

the 1st, 10th and 22nd

of each month.

10 1 * * 6,0 /bin/find -name “core” -exec rm {} \;

runs a find command at 1:10 a.m. only on Saturday and Sunday.

0,30 18-23 * * * /apps/bin/dbcheck.sh

runs a script called dbcheck.sh in /apps/bin at every 30

minutes past the hour,

between 18:00 and 23:00.

0 23 * * 6 /apps/bin/qtrend.sh

runs a script called qtrend.sh in /apps/bin at 11:00 p.m.

every Saturday.

The general format of the crontab is:

Crontab [-u user] -e -l -r

where:

-u is the user login name

-e edits the crontab file

-l lists the crontab file

-r deletes the crontab file

Submitting a command to the background

$ find /etc -name “srm.conf” -print >find.dt 2>&1 &

Checking the process using ps

$ ps x|grep 28305

28305 p1 S 0:00 sh /root/ps1

28350 p1 S 0:00 grep 28305

Killing a background job

kill -signal [process_number]

$ kill 28305

$ kill -9 28305

Submitting a job using nohup

nohup command >myout.file 2>&1

$ nohup ps1 &

CONTROLLING ACCESS TO FILE SYSTEM

1.To display the permissions on a single file:

ls -l file1

-rw-r–r– 2 unixjohn 3287 Apr 8 12:10 file1

2.This displays the following information about the file

file1.
-rw-r–r– – access permissions

2 – number of links to this file

unixjohn – owner

3287 – size in bytes

Apr 8 12:10 – date and time last modified

3.There are three types of permissions:

r – read the file or list files in the directory

w – write to the file or directory

x – execute the file or search the directory

4.Each of these permissions can be set for any one of three

types of

user:

u – the user who owns the file (usually you)

g – members of the group the owner belongs to

o – all other users

The access permissions for all three types of user can be

given as a

string of nine characters:

user group others

r w x r w x r w x

5.Changing permission

1. To give yourself permission to execute a file that you

own:

chmod u+x file1

This gives you execute permission for the file file1.

2. To give members of your group permission to read a file:

chmod g+r file2

This gives the group permission to read the file file2.

3. To give read permission to everyone for a particular

type of file:

chmod a+r *.pub

This gives everyone permission to read all files with the

extension

.pub.

4. To give the group write and execute permission:

chmod g+wx $HOME/SCCS

This gives all members of the group permission to place files

in

5.Setting access permissions numerically

There is a shorthand way of setting permissions by using octal

num-bers.

Read permission is given the value 4, write permission the

value

2 and execute permission 1.

r w x

4 2 1
These values are added together for any one user category:

1 = execute only

2 = write only

3 = write and execute (1+2)

4 = read only

5 = read and execute (4+1)

6 = read and write (4+2)

7 = read and write and execute (4+2+1)

So any access permission can be expressed as three digits. For

exam-ple:

user group others

chmod 640 file1 rw- r– —

chmod 754 file1 rwx r-x r–

chmod 664 file1 rw- rw- r–

6.Creating user and groups

useradd -u 21- -g dba -c “THE RDBMS” -d /home/oracle -s

/bin/ksh -m oracle

groupadd -g 241 dba

Informations are stored in /etc/passwd annd /etc/shadow

7.Setting limits on file size

ulimit 234567

8.Modifying and removing users

usermod -s /bin/bash oracle

userdel oracle

9.Changing password

passwd

# change group to staff for this file

chgrp staff myfile

# change owner to jsmith for this file

chown jsmith myfile

Shell input and output

ECHO

$ echo “What is your name :\c”

$ read name

You can put variables within echo statements as well as escape

codes. In this

example, the terminal bell is rung, your $HOME directory is

displayed, and the

command tty is evaluated by the shell.

$ echo “\007your home directory is $HOME, you are connected on

‘tty‘”

your home directory is /home/dave, you are connected on

/dev/ttyp1
If it’s LINUX then . . .

You have to put a ‘-n’ after the echo to suppress the new
line:

$ echo -n “What is your name :”

You have to put a ‘-e’ after echo for the escape code to work:

$ echo -e “\007your home directory is $HOME, you are connected

on

‘tty‘”

your home directory is /home/dave, you are connected on

/dev/ttyp1

READ

read variable1 variable2 . . .

$ read name

Hello I am superman

$ echo $name

Hello I am superman

$ read name surname

John Doe
$ echo $name $surname

John Doe

$ pg var_test

#!/bin/sh

# var_test

echo “First Name :\c”

read 51

read name

echo “Middle Name :\c”

read middle

echo “Last name :\c”

read surname

If it’s LINUX then . . .

Remember to use the ‘-n’ echo option.

$ pg var_test

#!/bin/sh

# var_test

echo “First Name :\c”

read name

echo “Middle Name :\c”

read middle

echo “Last name :\c”

read surname

PIPES

$ who | awk ‘{print $1″\t”$2}’

matthew pts/0

louise pts/1

$ df -k | awk ‘{print $1}’| grep -v “Filesystem”

$ df -k | awk ‘{print $1}’| grep -v “Filesystem”|sed

s’/\/dev\///g’

hda5

hda8

hda6

hdb5

hdb1

hda7

hda1

TEE
The tee command acts the way it is pronounced. It takes the
input and sends

one copy to the standard output and another copy to a file. If

you want to see

your output and save it to a file at the same time, then this
is the command for

you.

The general format is:

tee -a files

$ who | tee who.out

louise pts/1 May 20 12:58 (193.132.90.9)

matthew pts/0 May 20 10:18 (193.132.90.1)

cat who.out

louise pts/1 May 20 12:58 (193.132.90.9)

matthew pts/0 May 20 10:18 (193.132.90.1)

$ find etc usr/local home -depth -print | cpio -ovC65536 -O \

$ echo ” myscript is now running, check out any errors…in

myscript.log” | tee -a myscript.log

$ myscript | tee -a myscript.log

$ echo “stand-by disk cleanup starting in 1 minute” |
tee/dev/console

$ sort myfile | tee -a accounts.log

$ myscript | tee -a accounts.log

Standard input, output and errors

Redirecting standard output

$ cat passwd | awk -F: ‘{print $1}’| sort 1>sort.out

$ ls -l | grep ^d >>files.out

$ ls account* >> files.out

Redirecting standard input

sort < names.txt

sort <names.txt > names.out

$ cat >> myfile <<MAYDAY

> Hello there I am using a $TERM terminal

> and my user name is $LOGNAME

> bye…
> MAYDAY

$ pg myfile

Hello there I am using a vt100 terminal

and my user name is dave

Redirecting standard error

To redirect standard errors you specify the file descriptor

‘2’. Let’s first look at an

example, since they always explain better. Here grep searches

for a pattern called

‘trident’ on a file called missiles.

$ grep “trident” missiles

grep: missiles: No such file or directory

grep reports no such file, and sends its errors to the

terminal which is the default.

Let’s now send all errors to a file, in fact to the system

dustbin, ‘/dev/null’.

$ grep “trident” missiles 2>/dev/null

Combining standard output and error

$ cat account_qtr.doc account_end.doc 1> accounts.out 2>
accounts.err

Merging standard output and standard error

When merging output and errors, the shell evaluates the

command from left to

right, which is really all you need to know when putting

mergers together. Here’s an

example.

$ cleanup >cleanup.out 2>&1

In the above example the script cleanup directs all output (>)
to a file called

cleanup.out, and all errors (2), are directed to (>) the same
place as the output

(&1), which is cleanup.out.

$ grep “standard” * > grep.out 2>&1

In the above example all output from the grep command is put
into the output file

grep.out. When you use here documents, you will probably need
to capture all the

output to a file in case errors are encountered. To do this

you need to use 2>&1 as

part of the command. Here’s how to do it:

$ cat>> filetest 2>&1 <<MAYDAY

> This is my home $HOME directory

> MAYDAY

Command execution order

Using &&

Here’s the general format of &&:

command1 && command2

The operation of these commands is fairly straightforward. The

command on

the left of the && (command1) must be returned true

(successfully executed) if the

command on the right (command2) is to be executed; or to put

it another way, ‘If

this command works’ && ‘then execute this command’.

$ cp justice.doc justice.bak && echo “if you are seeing this

then cp was

OK”

if you are seeing this then cp was OK

$ mv /apps/bin /apps/dev/bin && rm -r /apps/bin

In the above example, the directory /apps/bin is to be moved
to /apps/dev/bin;

if this fails then the deletion of the directory /apps/bin

will not happen.

In the following example, a file is to be sorted with the

output going to a file

called quarter.sort; if this is successful, the file will be

printed.

$ sort quarter_end.txt > quarter.sorted && lp quarter.sorted

Using ||

command1 || command2

The operation of the || is slightly different. If the command

on the left of the ||

(command1) fails then execute the command on the right of the

|| (command2); or

to put it another way, ‘If this command fails’ || ‘then

execute this command’.

Here’s another simple example that illustrates the use of the

||:

$ cp wopper.txt oops.txt || echo “if you are seeing this cp

failed”

cp: wopper.txt: No such file or directory

if you are seeing this cp failed

The copy has failed so now the command on the right-hand side
of the || is

executed.

A more practical example is this. I want to extract the first

and fifth fields

from an accounts file and stick the output into a temp file.
If the extraction does not

work, I want to be mailed.

$ awk ‘{print$1,$5}’ acc.qtr >qtr.tmp || echo “Sorry the

payroll

extraction didn’t work” | mail dave

You don’t always have to use system commands; here I run the
script comet on a file

called month_end.txt. If the script bombs out, the shell

should terminate (exit).

$ comet month_end.txt || exit.

Using GREP

Searching more than one file

If I wanted to search for the string “sort” in all my .doc

files in my directory I

would do this:
$ grep “sort” * .doc

Or I could search through all the files that have the word
“sort it”.

$ grep “sort it” *

Line matches

$ grep -c “48” data.f

$ 4

Line numbers

$ grep -n “48” data.f

1:48 Dec 3BC1997 LPSX 68.00 LVX2A 138

2:483 Sept 5AP1996 USP 65.00 LVX2C 189

5:484 Nov 7PL1996 CAD 49.00 PLV2C 234

6:483 May 5PA1998 USP 37.00 KVM9D 644

Do not match

$ grep -v “48” data.f

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

219 Dec 2CC1999 CAD 23.00 PLV2C 68

216 Sept 3ZL1998 USP 86.00 KVM9E 234

Getting an exact match

$ grep ’48\>’ data.f

48 Dec 3BC1997 LPSX 68.00 LVX2A 138

Being case-sensitive

$ grep -i “sept” data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

216 sept 3ZL1998 USP 86.00 KVM9E 234

Pattern ranges

$ grep ’48[34]’ data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

484 nov 7PL1996 CAD 49.00 PLV2C 234

483 may 5PA1998 USP 37.00 KVM9D 644

Don’t match at the beginning of a line

$ grep ‘^[^48]’ data.f

219 dec 2CC1999 CAD 23.00 PLV2C 68

216 sept 3ZL1998 USP 86.00 KVM9E 234

Trapping upper and lower cases

$ grep ‘[Ss]ept’ data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

216 sept 3ZL1998 USP 86.00 KVM9E 234

Matching any characters

$ grep ‘K…D’ data.f

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

483 may 5PA1998 USP 37.00 KVM9D 644

$ grep ‘[A-Z][A-Z]..C’ data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

219 dec 2CC1999 CAD 23.00 PLV2C 68

484 nov 7PL1996 CAD 49.00 PLV2C 234

Date searching
$ grep ‘5..199[6,8]’ data.f

483 Sept 5AP1996 USP 65.00 LVX2C 189

483 may 5PA1998 USP 37.00 KVM9D 644

$ grep ‘[0-9]\ {3\}[8]’ data.f

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

483 may 5PA1998 USP 37.00 KVM9D 644

216 sept 3ZL1998 USP 86.00 KVM9E 234

Combining ranges

Staying with the use of the [ ] brackets to extract

information, suppose we want to

get city codes where the first character could be any number,
the second character

between 0 and 5 and the third between 0 and 6. If we use this

pattern we’ll get the

following output:

$ grep ‘[0-9][0-5][0-6]’ data.f

48 Dec 3BC1997 LPSX 68.00 LVX2A 138

483 Sept 5AP1996 USP 65.00 LVX2C 189

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

219 dec 2CC1999 CAD 23.00 PLV2C 68

484 nov 7PL1996 CAD 49.00 PLV2C 234

483 may 5PA1998 USP 37.00 KVM9D 644

216 sept 3ZL1998 USP 86.00 KVM9E 234

Well, we certainly got a lot of information back. What we want

is included but we

also got other records we didn’t want. However, looking at our

pattern, all records

that have been returned are the correct ones according to our
rules. We need to

specify that the pattern must start at the beginning of each

line. We can use the ^ for

that.

$ grep ‘^[0-9][0-5][0-6]’ data.f

216 sept 3ZL1998 USP 86.00 KVM9E 234

Occurrences in a pattern

If we want to extract any row that has a number 4 repeated at

least twice, we could

use this:

$ grep ‘4\{2,\}’ data.f

483 may 5PA1998 USP 37.00 KVM9D 644

$ grep ‘9\{3,\}’ data.f

219 dec 2CC1999 CAD 23.00 PLV2C 68

If you want to search for only so many occurrences then take

the comma out.

This will search for only two occurrences of the number 9.

$ grep ‘9\{2\}’ data.f

88 The grep family

There may be a need to match between, say, two and six

occurrences of a

number or maybe a letter. This will match between two and six
occurrences of the

number 8 that ends with 3:

$ grep ‘6\{2,6}3’ myfile

83 – no match

888883 – match

8884 – no match

88883 – match

Blank lines
$ grep ‘^$’ myfile

Matching special characters

If you wish to search for characters that have special

meanings, like one of the

following, $ . ‘ ” * [ ] ^ ( ) | \ + ?, then you must place a

\ in front of them. Suppose

you want to search for all lines that contain a period (.),
you would do this:

$ grep ‘\.’ myfile

Searching for ip addresses

Part of my job is looking after our DNS servers, which means

maintaining a lot of

ip addresses that cover different networks. Our address ip

file can contain over 200

addresses. Sometimes I want to look at just, say, the

‘nnn.nnn’ network addresses,

and forget about the rest that have only two digits in the
second part, i.e. nnn.nn..

To extract all these nnn.nnn. addresses, use

[0-9]\{3\}\.[0-0\[3\}\. This expression is

saying any number repeated three times followed by a period,

any number repeated

three times followed by a period.

$ grep ‘[0-9]\{3\}\.[0-0\[3\}\.’ Ipfile

Pattern matching with wildcards

Let’s take a look at the use of wildcards in using grep.

Suppose we have a file like

this:

$ pg testfile

looks

likes

looker

long

Here is what is displayed when using the following grep

pattern:

$ grep ‘l.*s’ testfile

looks

likes

$ grep ‘l.*k.’ testfile

looks

likes

$ grep ‘ooo*’ testfile

looks

If you want to find a word only at the end of a line, try

this:
$ grep ‘device$’ *

That will search all files for lines that have the word
‘device’ at the end of each line.

Class names 91

All this is saying is that you have typed a filename that does
not exist. If we

use the -s switch in grep we can silence these error messages.

$ grep -s “louise” /etc/password

Using grep on a string

grep is not only reserved for files; you can also use grep on
strings. All you need to

do is echo the string then pipe it through to grep.

$ STR=”Mary Joe Peter Pauline”

$ echo $STR | grep “Mary”

Mary Joe Peter Pauline

egrep

egrep stands for expression or extended grep depending on who

you listen

Egrep accepts the full range of regular expressions. One of

the nice features of egrep is that you can store your strings
in a file and pass them into egrep. We do this with the -f
switch. If we create a file called grep strings and then type
484 and 47 into it:

$ pg grepstrings

484

47

$ egrep -f grepstrings data.f

this would match all records with 484 or 47 in them. The -f

switch really becomes

useful if you want to match a lot of patterns, and typing them

in on the command

line becomes awkward.

If we want to search for store codes 32L or 2CC we can use the
bar sign (|),

which means one or the other or both (all), separated by the

bar sign.

$ egrep ‘(3ZL|2CC)’ data.f

47 Oct 3ZL1998 LPSX 43.00 KVM9D 512

219 dec 2CC1999 CAD 23.00 PLV2C 68

216 sept 3ZL1998 USP 86.00 KVM9E 234

You can use as many bars as you want. If we wanted to see if

users louise,

matty or pauline were logged into the system we could use the
who command and

pipe the output through to egrep.

$ who | egrep (louise|matty|pauline)

louise pty8

matty tty02

pauline pty2

You can also exclude certain strings using the caret sign (^).
If I wanted to see who

was on the system, but I did not want to know if users matty
and pauline were on I

could do this:

$ who | egrep -v ‘^(matty|pauline)’

If you want to search a directory listing for files that were

called shutdown,

shutdowns, reboot or reboots, this is easily accomplished with

egrep.

$ egrep ‘(shutdown | reboot) (s)?’ *

USING AWK

$ pg grade.txt

M.Tansley 05/99 48311 Green 8 40 44

J.Lulu 06/99 48317 green 9 24 26

P.Bunny 02/99 48 Yellow 12 35 28

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

Printing all records

$ awk ‘{print $0}’ grade.txt

Printing individual records

$ awk ‘{print $1,$4}’ grade.txt

Printing report headers

$ awk ‘BEGIN {print “Name Belt\n——————————–“}

{print $1″\t”$4}’ grade.txt

Name Belt

—————————————

M.Tansley Green

J.Lulu green

P.Bunny Yellow

J.Troll Brown-3

L.Tansley Brown-3

Printing report trailers

$ awk ‘BEGIN {print “Name\n——-“} {print $1} END {“end-of-
report”}’

grade.txt

Matching

$ awk {if($4~/Brown/) print $0}’ grade.txt

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

$ awk ‘$0 ~ /Brown/’ grade.txt

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

Exact match

$ awk ‘{if($3~/48/) print $0}’ grade.txt

M.Tansley 05/99 48311 Green 8 40 44

J.Lulu 06/99 48317 green 9 24 26

P.Bunny 02/99 48 Yellow 12 35 28

J.Troll 07/99 4842 Brown-3 12 26 26

Not matched
$ awk ‘$0 !~ /Brown/’ grade.txt

M.Tansley 05/99 48311 Green 8 40 44

J.Lulu 06/99 48317 green 9 24 26

P.Bunny 02/99 48 Yellow 12 35 28

We could have targeted just the belt grade field ‘field-4’ and
done the test this way:

$ awk ‘{if($4!~/Brown/) print $0}’ grade.txt

M.Tansley 05/99 48311 Green 8 40 44

J.Lulu 06/99 48317 green 9 24 26

P.Bunny 02/99 48 Yellow 12 35 28

Less than

$ awk ‘{if ($6 < $7) print $0 “$1 Try better at the next
comp”}’

grade.txt

M.Tansley Try better at the next comp

J.Lulu Try better at the next comp

Any characters

$ awk ‘$1 ~/^ . . . a/’ grade.txt

M.Tansley 05/99 48311 Green 8 40 44

L.Tansley 05/99 4712 Brown-2 12 30 28

Match either

$ awk ‘$0~/(Yellow|Brown)/’ grade.txt

P.Bunny 02/99 48 Yellow 12 35 28

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

AND

$ awk ‘{if ($1==”P.Bunny” && $4==”Yellow”)print $0}’ grade.txt

P.Bunny 02/99 48 Yellow 12 35 28

OR

$ awk ‘{if ($4==”Yellow” || $4~/Brown/) print $0}’ grade.txt

P.Bunny 02/99 48 Yellow 12 35 28

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

awk built-in variables

ARGC The number of command-line arguments

ARGV The array of command-line arguments

ENVIRON Holds the current system environment variables in the

array

FILENAME The name of the current file awk is scanning

FNR The record number in the current file

FS Sets the input field separator; same as the command-line -F

option

NF Number of fields in the current record

NR The number of records read so far

OFS Output field separator

ORS Output record separator

RS Controls the record separator

NF, NR and FILENAME

$ awk ‘{print NF,NR,$0}END{print FILENAME}’ grade.txt

7 1 M.Tansley 05/99 48311 Green 8 40 44

7 2 J.Lulu 06/99 48317 green 9 24 26

7 3 P.Bunny 02/99 48 Yellow 12 35 28

7 4 J.Troll 07/99 4842 Brown-3 12 26 26

7 5 L.Tansley 05/99 4712 Brown-2 12 30 28

grade.txt
$ awk ‘{if (NR >0 && $4~/Brown/)print $0}’ grade.txt

J.Troll 07/99 4842 Brown-3 12 26 26

L.Tansley 05/99 4712 Brown-2 12 30 28

Assigning input fields to field variable names

$ awk ‘{name=$1;belts=$4; if(belts ~/Yellow/)print name” is

belt

“belts}’ grade.txt

P.Bunny is belt Yellow

Comparing fields with values

$ awk ‘{if($6 < 27)print$0}’ grade.txt

J.Lulu 06/99 48317 green 9 24 26

J.Troll 07/99 4842 Brown-3 12 26 26

$ awk ‘BEGIN {BASELINE=”27 “}{if($6 < BASELINE)print$0}’

grade.txt

J.Lulu 06/99 48317 green 9 24 26

J.Troll 07/99 4842 Brown-3 12 26 26

SED COMMAND

[root@debasis cls]# cat deb

debasis 1

debraj 1

debajyoti 1

debasis 1

[root@debasis cls]# sed ‘s/debasis/deb/g’ deb

deb 1

debraj 1

debajyoti 1

deb 1

[root@debasis cls]# sed ‘s/[^ ]*/M&/’ deb

Mdebasis 1

Mdebraj 1

Mdebajyoti 1

Mdebasis 1

[root@debasis cls]# sed ‘s/[^ ]*/&M/’ deb

debasisM 1
debrajM 1

debajyotiM 1

debasisM 1

[root@debasis cls]# sed ‘s/[^ ]*/&M/’ deb

debasisM 1

debrajM 1

debajyotiM 1

debasisM 1

[root@debasis cls]# sed ‘/debasis/ s/1/2/’ deb

debasis 2

debraj 1

debajyoti 1

debasis 2

[root@debasis cls]# sed ‘1 s/1/2/’ deb

debasis 2

debraj 1

debajyoti 1

debasis 1
[root@debasis cls]# sed ‘1,2 s/1/2/’ deb

debasis 2

debraj 2

debajyoti 1

debasis 1

[root@debasis cls]# sed -e ‘s/debasis/deb/’ -e

‘s/debraj/deba/’ deb

deb 1

deba 1

debajyoti 1

deb 1

[cls@debasis cls]$ sed ‘

> /debraj/ s/1/2/

> /debajyoti/ s/1/3/’ deb

debasis 1

debraj 2

debajyoti 3

debasis 1

[cls@debasis cls]$ sed ‘/debraj/ d’ deb

debasis 1
debajyoti 1

debasis 1

[cls@debasis cls]$ sed ‘1,3 d’ deb

[cls@debasis cls]$ sed ‘/^deb/ d’ deb

[cls@debasis cls]$ sed ‘/1$/ d’ deb

[cls@debasis cls]$ sed ‘/debasis/ s/1/2/;/debraj/ d’ deb

debasis 2

debajyoti 1

debasis 2

[cls@debasis cls]$ sed ‘$a\

> stop!!!\

> I am lost’ deb

debasis 1

debraj 1

debajyoti 1

debasis 1

stop!!!
I am lost

[cls@debasis cls]$ sed ‘3a\

stop!!!\

I am lost’ deb

debasis 1

debraj 1

debajyoti 1

stop!!!

I am lost

debasis 1

[cls@debasis cls]$ sed ‘3i\

stop!!!\

I am lost’ deb

debasis 1

debraj 1

stop!!!

I am lost

debajyoti 1

debasis 1
[cls@debasis cls]$ sed ‘/debasis/ c\

> no use’ deb

no use

debraj 1

debajyoti 1

no use

ENVIRONMENT AND SHELL

Local variables

$variable_name=value or ${variable_name = value}

Displaying a variable

$ GREAT_PICTURE=”die hard”

$ echo ${GREAT_PICTURE}

die hard

$ DOLLAR=99

$ echo ${DOLLAR}

99

$ LAST_FILE=ZLPSO.txt
$ echo ${LAST_FILE}

ZLPSO.txt

Clearing a variable

unset variable_name

$ PC=enterprise

$ echo ${PC}

enterprise

$ unset PC

$ echo ${PC}

Displaying all local shell variables

$ set

PWD=/root

SHELL=/bin/sh

SHLVL=1

TERM=vt100

UID=7
USER=dave

dollar=99

great_picture=die hard

last_file=ZLPSO.txt

Using variables to hold arguments for system commands

$ SOURCE=”/etc/passwd”

$ DEST=”/tmp/passwd.bak”

$ cp ${SOURCE} ${DEST}

Making a variable read-only

$ TAPE_DEV=”/dev/rmt/0n”

$ echo ${TAPE_DEV}

/dev/rmt/0n

$ readonly TAPE_DEV

$ TAPE_DEV=”/dev/rmt/1n”

sh: TAPE_DEV: read-only variable

SHELL SCRIPTING

File status tests

-d This is a directory

-f This is a regular file

-L This is a symbolic link

-r This file is readable

-s This file has a size greater than zero, not empty

-w This file is writeable

-u This file has the suid bit set

-x This is executable

We will use both test methods to test if the file scores.txt

is writeable. We will use

the last status command to test it. Remember a zero status is

OK, anything else is

an error.

$ ls -l scores.txt

-rw-r–r– 1 dave admin 0 May 15 11:29 scores.txt

$ [ -w scores.txt ]

$ echo $?

$ test -w scores.txt

$ echo $?

0
cutting columns(-c)

cut -c 1-20 deb [show columns from c1-c10]

cut -c1 /etc/passwd

get the first character from every line

cut -c1,5,10-20 /etc/passwd

get the first, fifth character and every character between 10
and 20

cut -d: -f2 /etc/passwd

get the second field

cut -d: -f3- /etc/passwd

get all fields from the third on

cut –f1,3- telneos > deb

cut –d’ ‘ –f2- file (delimiter as space)

cut –f1,3 telnos address>telnos.all(selecting particular

columns from two files and pasting in one)

colrm 8 12 twinkle (deleting 8 and 12 column)

pasting columns

paste adc efg>>abcefg

paste –d: abc efg>>abcefg

paste telos – > telnos.new

using cut and paste to reorganize a file

cut –f1,3 telnos > temp

cut –f4- telnos>temp2

cut –f2 telos|paste temp-temp2>telnos.new

Using logical operators with tests

-a Logical AND, true, if both sides of the operator are true

-o Logical OR, true, if either sides of the operator can be

true

! Logical NOT, true, if the condition is false

-rw-r–r– 1 root root 0 May 15 11:29 scores.txt

-rwxr-xr– 1 root root 0 May 15 11:49 results.txt

The following example tests whether both files are readable.

$ [ -w results.txt -a -w scores.txt ]

$ echo $?

Testing strings

= The two strings are equal

!= The two strings are not equal

-z This string is null

-n This string is not null

To test if the environment variable EDITOR is empty:

$ [ -z $EDITOR ]

$ echo $?

No, it isn’t. Is it set to vi?

$ [ $EDITOR = “vi” ]

$ echo $?

Testing numbers

-eq The two numbers are equal

-ne The two numbers are not equal

-gt The first number is greater than the second number

-lt The first number is less than the second number

-le The first number is less than or equal to the second

number

-gt The first number is greater than or equal to the second

number
$ NUMBER=130

$ [ “$NUMBER” -eq “130” ]

$ echo $?

Using expr

$ expr 900 + 600

1500

$ expr 30 / 3

10

$ expr 30 / 3 / 2

Incrementing counters

Expr does the incrementing of values when using loops. First

the loop is initialized

to zero. Then one is added to the variable loop. The use of

the quotes means

command substitution, which basically means take the output

from the command
(expr) and put it in the variable loop.

$ LOOP=0

$ LOOP=‘expr $LOOP + 1‘

Pattern matching

$ VALUE=accounts.doc

$ expr $VALUE : October 8, ‘.*’

12

Control flow structures

$ pg iftest

#!/bin/sh

# iftest

# this is a comment line, all comment lines start with a #

if [ “10” -lt “12” ]

then

# yes 10 is less than 12

echo “Yes, 10 is less than 12”

fi
Testing values of variables

$ pg iftest2

#!/bin/sh

# if test2

echo -n “Enter your name :”

read NAME

# did the user just hit return ????

if [ “$NAME” = “” ] ; then

echo “You did not enter any information”

fi