Introduction to the Unix File System

Operating System | Unix File System

Unix file system is a logical method of organizing and storing large amounts of
information in a way that makes it easy to manage. A file is a smallest unit in which the
information is stored. Unix file system has several important features. All data in Unix is
organized into files. All files are organized into directories. These directories are
organized into a tree-like structure called the file system.
Files in Unix System are organized into multi-level hierarchy structure known as a
directory tree. At the very top of the file system is a directory called “root” which is
represented by a “/”. All other files are “descendants” of root.

Directories or Files and their description –

 / : The slash / character alone denotes the root of the filesystem tree.
 /bin : Stands for “binaries” and contains certain fundamental utilities, such as ls or
cp, which are generally needed by all users.
 /boot : Contains all the files that are required for successful booting process.
 /dev : Stands for “devices”. Contains file representations of peripheral devices and
pseudo-devices.
 /etc : Contains system-wide configuration files and system databases. Originally
also contained “dangerous maintenance utilities” such as init,but these have
typically been moved to /sbin or elsewhere.
 /home : Contains the home directories for the users.
 /lib : Contains system libraries, and some critical files such as kernel modules or
device drivers.
 /media : Default mount point for removable devices, such as USB sticks, media
players, etc.
 /mnt : Stands for “mount”. Contains filesystem mount points. These are used, for
example, if the system uses multiple hard disks or hard disk partitions. It is also
often used for remote (network) filesystems, CD-ROM/DVD drives, and so on.
 /proc : procfs virtual filesystem showing information about processes as files.
 /root : The home directory for the superuser “root” – that is, the system
administrator. This account’s home directory is usually on the initial filesystem, and
hence not in /home ..in case specific maintenance needs to be performed, during
which other filesystems are not available.
 /tmp : A place for temporary files. Many systems clear this directory upon startup;
it might have tmpfs mounted atop it, in which case its contents do not survive a
reboot, or it might be explicitly cleared by a startup script at boot time.
 /usr : Originally the directory holding user home directories,its use has changed. It
now holds executables, libraries, and shared resources that are not system critical,
like the X Window System, KDE, Perl, etc.
 /usr/bin : This directory stores all binary programs distributed with the operating
system not residing in /bin, /sbin or (rarely) /etc.
 /usr/include : Stores the development headers used throughout the system.
Header files are mostly used by the #include directive in C/C++ programming
language.
 /usr/lib : Stores the required libraries and data files for programs stored within /usr
or elsewhere.
 /var : A short for “variable.” A place for files that may change often – especially in
size, for example e-mail sent to users on the system, or process-ID lock files.
 /var/log : Contains system log files.
 /var/mail : The place where all the incoming mails are stored. Users (other than
root) can access their own mail only. Often, this directory is a symbolic link to
/var/spool/mail.
 /var/spool : Spool directory. Contains print jobs, mail spools and other queued
tasks.
 /var/tmp : A place for temporary files which should be preserved between system
reboots.
Types of Unix files – The UNIX files system contains several different types of files :

1. Ordinary files – An ordinary file is a file on the system that contains data, text, or
program instructions.

 Used to store your information, such as some text you have written or an image
you have drawn. This is the type of file that you usually work with.
 Always located within/under a directory file.
 Do not contain other files.
 In long-format output of ls -l, this type of file is specified by the “-” symbol.
2. Directories – Directories store both special and ordinary files. For users familiar with
Windows or Mac OS, UNIX directories are equivalent to folders. A directory file contains
an entry for every file and subdirectory that it houses. If you have 10 files in a directory,
there will be 10 entries in the directory. Each entry has two components.
(1) The Filename
(2) A unique identification number for the file or directory (called the inode number)
 Branching points in the hierarchical tree.
 Used to organize groups of files.
 May contain ordinary files, special files or other directories.
 Never contain “real” information which you would work with (such as text). Basically, just
used for organizing files.
 All files are descendants of the root directory, ( named / ) located at the top of the tree.
In long-format output of ls –l , this type of file is specified by the “d” symbol.
3. Special Files – Used to represent a real physical device such as a printer, tape drive
or terminal, used for Input/Ouput (I/O) operations. Device or special files are used for
device Input/Output(I/O) on UNIX and Linux systems. They appear in a file system just
like an ordinary file or a directory.
On UNIX systems there are two flavors of special files for each device, character
special files and block special files :
 When a character special file is used for device Input/Output(I/O), data is
transferred one character at a time. This type of access is called raw device
access.
 When a block special file is used for device Input/Output(I/O), data is transferred in
large fixed-size blocks. This type of access is called block device access.
For terminal devices, it’s one character at a time. For disk devices though, raw access
means reading or writing in whole chunks of data – blocks, which are native to your
disk.
 In long-format output of ls -l, character special files are marked by the “c” symbol.
 In long-format output of ls -l, block special files are marked by the “b” symbol.

4. Pipes – UNIX allows you to link commands together using a pipe. The pipe acts a
temporary file which only exists to hold data from one command until it is read by
another.A Unix pipe provides a one-way flow of data.The output or result of the first
command sequence is used as the input to the second command sequence. To make a
pipe, put a vertical bar (|) on the command line between two commands.For
example: who | wc –l
5. Sockets – A Unix socket (or Inter-process communication socket) is a special file
which allows for advanced inter-process communication. A Unix Socket is used in a
client-server application framework. In essence, it is a stream of data, very similar to
network stream (and network sockets), but all the transactions are local to the
filesystem.
In long-format output of ls -l, Unix sockets are marked by “s” symbol.
6. Symbolic Link – Symbolic link is used for referencing some other file of the file
system.Symbolic link is also known as Soft link. It contains a text form of the path to the
file it references. To an end user, symbolic link will appear to have its own name, but
when you try reading or writing data to this file, it will instead reference these operations
to the file it points to.

chmod command in Linux with examples

In Unix-like operating systems, the chmod command is used to change the access
mode of a file.
The name is an abbreviation of change mode.
Syntax :
chmod [reference][operator][mode] file...
The references are used to distinguish the users to whom the permissions apply i.e.
they are list of letters that specifies whom to give permissions. The references are
represented by one or more of the following letters:
Reference Class Description
u owner file's owner
g group users who are members of
the file's group
o others users who are neither the
file's owner nor members of
the file's group
a all All three of the above, same as ugo
The operator is used to specify how the modes of a file should be adjusted. The
following operators are accepted:
Operator Description
+ Adds the specified modes to the
specified classes

- Removes the specified modes from

the specified classes

= The modes specified are to be made

the exact modes for the specified
classes
The modes indicate which permissions are to be granted or removed from the specified
classes. There are three basic modes which correspond to the basic permissions:
r Permission to read the file.
w Permission to write (or delete) the file.
x Permission to execute the file, or, in
the case of a directory, search it.
Types of permissions which we will be changing using chmod command :
In linux terminal, to see all the permissions to different files, type ls -l command which
lists the files in the working directory in long format. The figure below shows an example
to use ls -l and its output :
- rw- rw- r-- mik mik assgn1_client.c
- rw- rw- r-- mik mik assgn1_server.c
d rwx rwx r-x mik mik EXAM
- rw- rw- r-- mik mik raw.c
- rwx r-x r-x mik mik header.sh
... so on...
 The very first column represents the type of the file i.e. is it a normal file or a
directory where d represents a directory and – represents a normal file.
 The first set three letters after the file type tell what the Owner of the file, have
permissions to do. For example: In assgn1_client.c, has owner’s permission as rw-
, which means the owner mik can only read(r) and write(w) the file but cannot
execute(x)
 Note: The 3rd and 4th columns represents the name of the owner of the file and
the group to which the owner belongs respectively.
 The next three letters after the user’s permission are the group’s permissions.
For example: header.sh has group permissions as r-x, which means Other people
in the mik group can not write(w) the header.sh script but can only read(r) or
execute(x) it.
cp command in Linux with examples::
cp stands for copy. This command is used to copy files or group of
files or directory. It creates an exact image of a file on a disk with
different file name. cp command require at least two filenames in its
arguments.
Syntax:
cp [OPTION] Source Destination
cp [OPTION] Source Directory
cp [OPTION] Source-1 Source-2 Source-3 Source-n Directory

First and second syntax is used to copy Source file to Destination file or
Directory.
Third syntax is used to copy multiple Sources(files) to Directory.

OPTIONS:
1. -i(interactive): i stands for Interactive copying. With this option system first warns the
user before overwriting the destination file. cp prompts for a response, if you
press y then it overwrites the file and with any other option leave it uncopied.
$ cp -i a.txt b.txt
cp: overwrite 'b.txt'? y

$ cat b.txt
GFG
2. -b(backup): With this option cp command creates the backup of the destination file
in the same folder with the different name and in different format.

‘ls’ command in Linux :: ls is a Linux shell command that lists

directory contents of files and directories.Some practical examples of
ls command are shown below.
1. Open Last Edited File Using ls -t
ls -t : It sorts the file by modification time, showing the last edited file first. head -1 picks
up this first file.To open the last edited file in the current directory use the combination of
ls and head commands as shown below.
2.$ ls -l : To show long listing information about the file/directory.

3.ls -lh (h stands for human readable form) : To display file size in easy to read format.
i.e i.e M for MB, K for KB, G for GB.

pwd command in Linux with Examples

pwd stands for Print Working Directory. It prints the path of the working directory,
starting from the root.
pwd is shell built-in command(pwd) or an actual binary(/bin/pwd).
$PWD is an environment variable which stores the path of the current directory.
This command has two flags.
pwd -L: Prints the symbolic path.
pwd -P: Prints the actual path.
SIMPLE FILTER
In Unix and Unix-like operating systems, a filter is a program that gets most of its data from
its standard input (the main input stream) and writes its main results to its standard output (the main
output stream). Auxiliary input may come from command line flags or configuration files, while
auxiliary output may go to standard error. The command syntax for getting data from a device or file
other than standard input is the input operator ( < ). Similarly, to send data to a device or file other
than standard output is the output operator ( > ). To append data lines to an existing output file, one
can use the append operator ( >> ). Filters may be strung together into a pipeline with the pipe
operator (" | "). This operator signifies that the main output of the command to the left is passed as
main input to the command on the right.
Common Unix filter programs are: cat, cut, grep, head, sort, uniq, and tail.

pr (Unix)
pr is a command used to paginate or columnate files for printing. It can also be used to compare two
files side by side, as an alternative to diff.
On Unix-like operating systems, the pr command
formats a file to make it look better
when printed.This document covers
the GNU/Linux version of pr. pr paginates
or columnates FILE(s) for printing. 2.The -
t option is implied if PAGE_LENGTH is less
than or equal to 10. If no FILE is specified,
or when FILE is a dash ("-"), pr reads
fromstandard input instead. Syntax

pr [OPTION]... [FILE]...

OPTIONS ::

-c, --show-control-chars Use caret notation (^G) and octal backslash notation.
-d, --double-space Double-space the output.
-D, --date-format=FORMAT Use FORMAT for the header date.
-e[CHAR[WIDTH]],--expand- Expand input character CHAR (by default, the tab char
tabs[=CHAR[WIDTH]] (by default, 8) spaces.
cut command in Linux with examples
The cut command in UNIX is a command for cutting out the sections from each line of
files and writing the result to standard output. It can be used to cut parts of a line
by byte position, character and field. Basically the cut command slices a line and
extracts the text. It is necessary to specify option with command otherwise it gives error.
If more than one file name is provided then data from each file is not precedes by its
file name.
Syntax:
cut OPTION... [FILE]...
Options and their Description with examples:
1. -b(byte): To extract the specific bytes, you need to follow -b option with the list of
byte numbers separated by comma. Range of bytes can also be specified using the
hyphen(-).
2. -c (column): To cut by character use the -c option. This selects the characters given
to the -c option. This can be a list of numbers separated comma or a range of numbers
separated by hyphen(-). Tabs and backspaces are treated as a character. It is
necessary to specify list of character numbers otherwise it gives error with this option.
Syntax:
$cut -c [(k)-(n)/(k),(n)/(n)] filename
Here,k denotes the starting position of the character and n denotes the ending position
of the character in each line, if k and n are separated by “-”
Paste Command Examples in Unix / Linux Tutorials
Paste command is one of the useful commands in unix or linux operating system. The paste
command merges the lines from multiple files. The paste command sequentially writes the
corresponding lines from each file separated by a TAB

The syntax of the paste command is

paste [options] files-list

The options of paste command are:

-d : Specify of a list of delimiters.

-s : Paste one file at a time instead of in parallel.
--version : version information
--help : Help about the paste command.

ADVANCED FILTER
A filter is a program that reads standard input, performs an operation upon it
and writes the results to standard output.
For this reason, it can be used to process information in powerful ways such
as restructuring output to generate useful reports, modifying text in files and
many other system administration tasks.

1.Grep cmd:; The grep filter searches a file for a particular pattern of characters, and
displays all lines that contain that pattern. The pattern that is searched in the file is
referred to as the regular expression (grep stands for globally search for regular
expression and print out).
Syntax:

grep [options] pattern [files]

Options Description
-c : This prints only a count of the lines that match a pattern
-h : Display the matched lines, but do not display the filenames.
-i : Ignores, case for matching
-l : Displays list of a filenames only.
-n : Display the matched lines and their line numbers.
egrep command in Linux with examples
egrep is a pattern searching command which belongs to the family of grep functions. It
works the same way as grep -E does. It treats the pattern as an extended regular
expression and prints out the lines that match the pattern. If there are several files with
the matching pattern, it also displays the file names for each line.
Syntax:
egrep [ options ] 'PATTERN' files
Options: Most of the options for this command are same as grep.
 -c: Used to counts and prints the number of lines that matched the pattern and not
the lines.
@-v: It prints the lines that does not match with the pattern.
-i: Ignore the case of the pattern while matching.
-l: Prints only the names of the files that matched. It does not mention the
matching

Sed Command in Linux/Unix with examples

SED command in UNIX is stands for stream editor and it can perform lot’s of function on
file like, searching, find and replace, insertion or deletion. Though most common use of
SED command in UNIX is for substitution or for find and replace.
 SED is a powerful text stream editor. Can do insertion, deletion, search and
replace(substitution).
 SED command in unix supports regular expression which allows it perform
complex pattern matching.
Syntax:
sed OPTIONS... [SCRIPT] [INPUTFILE...]
join Command in Linux
The join command in UNIX is a command line utility for joining lines of two files on a
common field.
Suppose you have two files and there is a need to combine these two files in a way that
the output makes even more sense.For example, there could be a file containing names
and the other containing ID’s and the requirement is to combine both files in such a way
that the names and corresponding ID’s appear in the same line. join command is the
tool for it. join command is used to join the two files based on a key field present in
both the files. The input file can be separated by white space or any delimiter.
Syntax:
$join [OPTION] FILE1 FILE2

Options for join command:

1. -a FILENUM : Also, print unpairable lines from file FILENUM, where FILENUM is 1 or
2, corresponding to FILE1 or FILE2.
2. -e EMPTY : Replace missing input fields with EMPTY.
3. -i - -ignore-case : Ignore differences in case when comparing fields.
4. -j FIELD : Equivalent to "-1 FIELD -2 FIELD".
5. -o FORMAT : Obey FORMAT while constructing output line.
6. -t CHAR : Use CHAR as input and output field separator.

Example : Let us assume there are two files file1.txt and file2.txt and we want to
combine the contents of these two files.
// displaying the contents of first file //
$cat file1.txt
1 AAYUSH
2 APAAR
3 HEMANT
4 KARTIK

// displaying contents of second file //

$cat file2.txt
1 101
2 102
3 103
4 104