Unix Training

Welcome To The Unix Training : Welcome To The Unix Training

Slide 2 : Operating System : UNIX features and benefits : Strictly speaking, the term Unix Operating System refers only to those operating systems that have passed the Unix standards test of the X-Open Foundation, which owns the Unix trademark. X-Open acquired the trademark and standards from AT&T, the original creator of Unix. In practice, Unix refers to any operating system that provides the programs, commands, and programming interfaces normally found on official Unix. There are two main variants of Unix, which have roots in the same original source code from the 1970s and 1980s: System V (AT&T) and Berkeley Unix, whose last version is called 4.4BSD. There is also one important Unix work-alike-- Linux -- that looks like Unix to most users and programmers, but has a totally separate origin and shares no source code with official Unix. Also, note that the core of the new MacOS X is straight Berkeley Unix.

Slide 3 : UNIX standardization : The Unix standards that evolved after 1985 were quite successful at this — so much so that they serve as valuable documentation of the API of modern Unix implementations. In fact, real-world Unixes follow published standards so closely that developers can (and frequently do) lean more on documents like the POSIX specification than on the official manual pages for the Unix variant they happen to be using. In fact, on the newer open-source Unixes (such as Linux), it is common for operating-system features to have been engineered using published standards as the specification.

Slide 4 : The structure of UNIX : The Unix operating environment is organized into three layers. The innermost level of Unix is the kernel. This is the actual operating system, a single large program that always resides in memory. Sections of the code in this program are executed on behalf of users to do needed tasks, like access files or terminals. Strictly speaking, the kernel is Unix. The next level of the Unix environment is composed of programs, commands, and utilities. In Unix, the basic commands like copying or removing files are implemented not as part of the kernel, but as individual programs, no different really from any program you could write. What we think of as the commands and utilities of Unix are simply a set of programs that have become standardized and distributed. There are hundreds of these, plus many additional utilities in the public domain that can be installed.

Slide 5 : The final level of the Unix environment, which stands like an umbrella over the others, is the shell. The shell processes your terminal input and starts up the programs that you request. It also allows you to manipulate the environment in which those programs will execute in a way that is transparent to the program. The program can be written to handle standard cases, and then made to handle unusual cases simply by manipulating its environment, without having to have a special version of the program.

Slide 6 : Establishing communication : Starting the UNIX session The traditional Unix user interface is a simple command-line shell. A text-only terminal window with a fixed number of lines (usually 24) and columns (usually 80) is controlled by the shell program. The shell prints a prompt when it is ready to receive commands. You use the keyboard to type in a command. Then the shell parses and processes the command and executes it for you. The command prints its output, if any, to the same terminal window. When the command is done, the shell prompts you again to enter another command. In general, the terminal window has no graphics capability, although there are specialized graphics terminals and some Unix programs know how to display (usually monochrome) images on them.

Slide 7 : # Using basic UNIX commands # Understanding the UNIX command syntax # Using the UNIX "help" facility - the manual pages Logging in, activating the user interface and logging out : Absolute basics : These are the quickies, which we need to get started; we will discuss them later in more detail. Command Meaning ls Displays a list of files in the current working directory, Like the dir command in DOS cd directory change directories passwd change the password for the current user file filename display file type of file with name filename cat textfile throws content of textfile on the screen pwd display present working directory exit or logout leave this session man command read man pages on command info command read Info pages on command apropos string search the whatis database for strings

Slide 8 : General purpose utilities: banner : out dated cal - displays a calendar SYNOPSIS : cal [-smjy13 ] [[ month ] year ] DESCRIPTION Cal displays a simple calendar. If arguments are not specified, the current month is displayed. The options are as follows: -1 : Display single month output. (This is the default.) -3 : Display prev/current/next month output. -s : Display Sunday as the first day of the week. (This is the default.) -m : Display Monday as the first day of the week. -j : Display Julian dates (days one-based, numbered from January 1). -y : Display a calendar for the current year.

Slide 9 : date - print or set the system date and time   SYNOPSIS : date [OPTION]... [+FORMAT] date [-u|--utc|--universal] [MMDDhhmm[[CC]YY][.ss]]   DESCRIPTION Display the current time in the given FORMAT, or set the system date. NAME who - show who is logged on SYNOPSIS : who [OPTION]... [ FILE | ARG1 ARG2 ] DESCRIPTION -a, --all : same as -b -d --login -p -r -t -T -u -b, --boot : time of last system boot

Slide 10 : tty - print the file name of the terminal connected to standard input SYNOPSIS : tty [OPTION]...   DESCRIPTION : Print the file name of the terminal connected to standard input. -s, --silent, --quiet uname - print system information SYNOPSIS uname [OPTION]... DESCRIPTION -a, --all : print all information, in the following order: -s, --kernel-name : print the kernel name -n, --nodename : print the network node hostname -r, --kernel-release : print the kernel release -v, --kernel-version : print the kernel version

Slide 11 : -m, --machine : print the machine hardware name -p, --processor : print the processor type -i, --hardware-platform : print the hardware platform -o, --operating-system : print the operating system passwd - update a user's authentication tokens(s)

Slide 12 : echo - display a line of text SYNOPSIS echo [OPTION]... [STRING]... DESCRIPTION NOTE: your shell may have its own version of echo which will supercede the version described here. Please refer to your shell's documentation for details about the options it supports. tput, reset - initialize a terminal or query terminfo database bc - An arbitrary precision calculator language script - make typescript of terminal session SYNOPSIS : script [-a ] [-c COMMAND ] [-f ] [-q ] [-t ] [file ]

Slide 13 : File system & Navigation   Examining UNIX files and directories Moving through the file system Using full and partial paths understanding directories - HOME, present working directory pathnames (absolute & relative), change, create and moving directory various file listing options

Slide 14 : General overview of the Unix file system A simple description of the UNIX system, also applicable to Linux, is this: "On a UNIX system, everything is a file; if something is not a file, it is a process."

Slide 15 : Sorts of files Most files are just files, called regular files; they contain normal data, for example text files, executable files or programs, input for or output from a program and so on. While it is reasonably safe to suppose that everything you encounter on a Linux system is a file, there are some exceptions. Directories: files that are lists of other files. Special files: the mechanism used for input and output. Most special files are in /dev, we will discuss them later. Links: a system to make a file or directory visible in multiple parts of the system's file tree. We will talk about links in detail. (Domain) sockets: a special file type, similar to TCP/IP sockets, providing inter-process networking protected by the file system's access control. Named pipes: act more or less like sockets and form a way for processes to communicate with each other, without using network socket semantics. The -l option to ls displays the file type, using the first character of each input line: ~/Documents> ls –l -rw-rw-r-- 1 jaime jaime 41472 Feb 21 17:56 Linux.doc drwxrwxr-x 2 jaime jaime 4096 Feb 25 11:50 course

Slide 16 : File types in a long list Symbol Meaning - Regular file d Directory l Link c Special file s Socket p Named pipe b Block device Absolute and relative paths A path, which is the way you need to follow in the tree structure to reach a given file, can be described as starting from the trunk of the tree (the / or root directory). In that case, the path starts with a slash and is called an absolute path, since there can be no mistake: only one file on the system can comply.

Slide 17 : Moving through the file system: Partition layout and types : There are two kinds of major partitions on a Unix system: data partition: normal Linux system data, including the root partition containing all the data to start up and run the system; and swap partition: expansion of the computer's physical memory, extra memory on hard disk. Mount points : All partitions are attached to the system via a mount point. The mount point defines the place of a particular data set in the file system. Usually, all partitions are connected through the root partition. On this partition, which is indicated with the slash (/), directories are created. These empty directories will be the starting point of the partitions that are attached to them. df –h Filesystem Size Used Avail Use% Mounted on /dev/hda8 496M 183M 288M 39% / /dev/hda1 124M 8.4M 109M 8% /boot /dev/hda5 19G 15G 2.7G 85% /opt /dev/hda6 7.0G 5.4G 1.2G 81% /usr /dev/hda7 3.7G 2.7G 867M 77% /var fs1:/home 8.9G 3.7G 4.7G 44% /.automount/fs1/root/home

Slide 18 :

Slide 19 : Subdirectories of the root directory /bin Common programs, shared by the system, the system administrator and the users. /boot The startup files and the kernel, vmlinuz. In some recent distributions also grub data. Grub is the GRand Unified Boot loader and is an attempt to get rid of the many different boot-loaders we know today. /dev Contains references to all the CPU peripheral hardware, which are represented as files with special properties. /etc Most important system configuration files are in /etc, this directory contains data similar to those in the Control Panel in Windows /home Home directories of the common users. /initrd (on some distributions) Information for booting. Do not remove! /lib Library files, includes files for all kinds of programs needed by the system and the users. /lost+found Every partition has a lost+found in its upper directory. Files that were saved during failures are here. /misc For miscellaneous purposes.

Slide 20 : /mnt Standard mount point for external file systems, e.g. a CD-ROM or a digital camera. /net Standard mount point for entire remote file systems /opt Typically contains extra and third party software. /proc A virtual file system containing information about system resources. More information about the meaning of the files in proc is obtained by entering the command man proc in a terminal window. The file proc.txt discusses the virtual file system in detail. /root The administrative user's home directory. Mind the difference between /, the root directory and /root, the home directory of the root user. /sbin Programs for use by the system and the system administrator. /tmp Temporary space for use by the system, cleaned upon reboot, so don't use this for saving any work! /usr Programs, libraries, documentation etc. for all user-related programs. /var Storage for all variable files and temporary files created by users, such as log files, the mail queue, the print spooler area, space for temporary storage of files downloaded from the Internet, or to keep an image of a CD before burning it.

Slide 21 : change, create and moving directory : Current Working Directory : Each shell and system process has a current working directory(cwd) pwd Displays the absolute path to the shell's cwd File and Directory Names Names may be up to 255 characters All characters are valid, except the forward-slash It may be unwise to use certain special characters in file or directory names Some characters should be protected with quotes when referencing them Names are case-sensitive Example: MAIL, Mail, mail, and mAiL Again, possible, but may not be wise

Slide 22 : Changing Directories cd changes directories To an absolute or relative path: + cd /home/joshua/work + cd project/docs To a directory one level up: + cd .. To your home directory: + cd To your previous working directory: + cd - Copying Files and Directories cp - copy files and directories Usage: cp [options] file destination More than one file may be copied at a time if the destination is a directory: cp [options] file1 file2 dest

Slide 23 : Copying Files and Directories: The Destination If the destination is a directory, the copy is placed there If the destination is a file, the copy overwrites the destination If the destination does not exist, the copy is renamed Moving and Renaming Files and Directories mv - move and/or rename files and directories Usage: mv [options] file destination More than one file may be moved at a time if the destination is a directory: mv [options] file1 file2 destination Destination works like cp

Slide 24 : Creating and Removing Files touch - create empty files or update file timestamps rm - remove files Usage: rm [options] ... Example: rm -i file (interactive) rm -r directory (recursive) rm -f file (force) Creating and Removing Directories mkdir creates directories rmdir removes empty directories rm -r recursively removes directory trees Determining File Content Files can contain many types of data Check file type with file before opening to determine appropriate command or application to use file [options] ...

Slide 25 : Handling files : file attributes - file listing options, directory listing options, file permissions, file mode change At the time a new file is created, it gets a free inode. In that inode is the following information: * Owner and group owner of the file. * File type (regular, directory, ...) * Permissions on the file Section * Date and time of creation, last read and change. * Date and time this information has been changed in the inode. * Number of links to this file (see later in this chapter). * File size * An address defining the actual location of the file data.

Slide 26 : Users : * Every user is assigned a unique User ID number (UID) o UID 0 identifies root * Users' names and UIDs are stored in /etc/passwd * Users are assigned a home directory and a program that is run when they log in (usually a shell) * Users cannot read, write or execute each others' files without permission Groups : * Users are assigned to groups * Each group is assigned a unique Group ID number (gid) * GIDs are stored in /etc/group * Each user is given their own private group o Can be added to other groups for additional access * All users in a group can share files that belong to the group File Security : * Every file is owned by a UID and a GID * Every process runs as a UID and one or more GIDs o Usually determined by who runs the process * Three access categories: o Processes running with the same UID as the file (user) o Processes running with the same GID as the file (group) o All other processes (other)

Slide 27 : Permission Precedence : * If UID matches, user permissions apply * Otherwise, if GID matches, group permissions apply * If neither match, other permissions apply Permission Types : * Four symbols are used when displaying permissions: o r: permission to read a file or list a directory's contents o w: permission to write to a file or create and remove files from a directory o x: permission to execute a program or change into a directory and do a long listing of the directory o -: no permission (in place of the r, w, or x) Examining Permissions : * File permissions may be viewed using ls -l $ ls -l /bin/login -rwxr-xr-x 1 root root 19080 Apr 1 18:26 /bin/login * File type and permissions represented by a 10-character string Interpreting Permissions : -rwxr-x--- 1 andersen trusted 2948 Oct 11 14:07 myscript * Read, Write and Execute for the owner, andersen * Read and Execute for members of the trusted group * No access for all others

Slide 28 : Changing File Ownership : * Only root can change a file's owner * Only root or the owner can change a file's group * Ownership is changed with chown: o chown [-R] user_name file|directory * Group-Ownership is changed with chgrp: o chgrp [-R] group_name file|directory Changing Permissions - Symbolic Method : * To change access modes: o chmod [-R] mode file * Where mode is: o u,g or o for user, group and other o + or - for grant or deny o r, w or x for read, write and execute * Examples: o ugo+r: Grant read access to all o o-wx: Deny write and execute to others Changing Permissions - Numeric Method : * Uses a three-digit mode number o first digit specifies owner's permissions o second digit specifies group permissions o third digit represents others' permissions * Permissions are calculated by adding: o 4 (for read) o 2 (for write) o 1 (for execute) * Example: o chmod 640 myfile

Slide 29 : Standard I/O and Pipes or redirection, filters & pipes * Redirect I/O channels to files * Connect commands using pipes * Use the for loops to iterate over sets of valuesStandard Input and Output * Unix provides three I/O channels to Programs o Standard input (STDIN) - keyboard by default o Standard output (STDOUT) - terminal window by default o Standard error (STDERR) - terminal window by default Redirecting Output to a File * STDOUT and STDERR can be redirected to files: o command operator filename * Supported operators include: o > Redirect STDOUT to file o 2> Redirect STDERR to file o &> Redirect all output to file * File contents are overwritten by default. >> appends.

Slide 30 : Redirecting Output to a File * This command generates output and errors when run as non-root: $ find /etc -name passwd * Operators can be used to store output and errors: $ find /etc -name passwd > find.out $ find /etc -name passwd 2> /dev/null $ find /etc -name passwd > find.out 2> find.errRedirecting STDOUT to a Program (Piping) * Pipes (the | character) can connect commands: command1 | command2 o Sends STDOUT of command1 to STDIN of command2 instead of the screen. o STDERR is not forwarded across pipes * Used to combine the functionality of multiple tools o command1 | command2 | command3... etcRedirecting STDOUT to a Program * less: View input one page at a time: $ ls -l /etc | less

Slide 31 : o Input can be searched with / * mail: Send input via email: $ echo "test email" | mail -s "test" user@example.com *lpr : Send input to a printer $ echo "test print" | lpr $ echo "test print" | lpr -P printer_nameCombining Output and Errors * Some operators affect both STDOUT and STDERR o &>: Redirects all output: $ find /etc -name passwd &> find.all o 2>&1: Redirects STDERR to STDOUT + Useful for sending all output through a pipe $ find /etc -name passwd 2>&1 | less o (): Combines STDOUTs of multiple programs $ ( cal 2007 ; cal 2008 ) | less

Slide 32 : Redirecting to Multiple Targets (tee) * $ command1 | tee filename | command2 * Stores STDOUT of command1 in filename, then pipes to command2 * Uses: o Troubleshooting complex pipelines o Simultaneous viewing and logging of output Redirecting STDIN from a File * Redirect standard input with < * Some commands can accept data redirected to STDIN from a file: $ tr 'A-Z' 'a-z' < .bash_profile o This command will translate the uppercase characters in .bash_profile to lowercase * Equivalent to: $ cat .bash_profile | tr 'A-Z' 'a-z'

Slide 33 : Vi Editorvim: An Advanced Text Editor * Use the three primary modes of vi and vim * Navigate text and enter Insert mode * Change, delete, yank, and put text * Undo changes * Search a document * Save and exitIntroducing vim * Newer version of vi, the standard Unix text editor o Executing vi runs vim by default * gvim: Graphical version of vim o Applications + Programming -> Vi iMproved o Provided by vim-X11 package * Advantages: o Speed: Do more with fewer keystrokes o Simplicity: No dependence on mouse/GUI o Availability: Included with most Unix-like OSes * Disadvantages o Difficulty: Steeper learning curve than simpler editors + Key bindings emphasize speed over intuitiveness

Slide 34 : vim: A Modal Editor * Keystroke behavior is dependent upon vim's "mode" * Three main modes: o Command Mode (default): Move cursor, cut/paste text, change mode o Insert Mode: Modify text o Ex Mode: Save, quit, etc * Esc exits current mode * EscEsc always returns to command mode vim Basics * To use vim, you must at least be able to o Open a file o Modify a file (insert mode) o Save a file (ex mode) Opening a file in vim * To start vi: o vim filename o If the file exists, the file is opened and the contents are displayed o If the file does not exist, vi creates it when the edits are saved for the first time

Slide 35 : Modifying a File Insert Mode * i begins insert mode at the cursor * Many other options exist o A append to end of line o I insert at beginning of line o o insert new a line (below) o O insert new line (above) Saving a File and Exiting vim Ex Mode * Enter Ex Mode with : o Creates a command prompt at bottom-left of screen * Common write/quit commands: o :w writes (saves) the file to disk o :wq writes and quits o :q! quits, even if changes are lost Using Command Mode * Default mode of vim * Keys describe movement and text manipulation commands * Commands repeat when preceded by a number * Example o Right Arrow moves right one character o 5, Right Arrow moves right five characters

Slide 36 : Moving Around Command Mode * Move by character: Arrow Keys, h, j, k, l o Non-arrow keys useful for remote connections to older systems * Move by word: w, b * Move by sentence: ), ( * Move by paragraph: }, { * Jump to line x: xG * Jump to end: G Search and Replace Command Mode * Search as in less o /, n, N * Search/Replace as in sed o Affects current line by default o Use x,y ranges or % for whole file + :1,5s/cat/dog/ + :%s/cat/dog/gi

Slide 37 : Manipulating Text Command Mode Change (replace) Delete (cut) Yank (copy)Line cc dd yyLetter cl dl ylWord cw dw ywSentence ahead c) d) y)Sentence behind c( d( y(Paragraph above c{ d{ y{Paragraph below c} d} y}Undoing Changes Command Mode * u undo most recent change * U undo all changes to the current line since the cursor landed on the line * Ctrl-r redo last "undone" change Visual Mode * Allows selection of blocks of text o v starts character-oriented highlighting o V starts line-oriented highlighting o Activated with mouse in gvim * Visual keys can be used in conjunction with movement keys: o w, ), }, arrows, etc * Highlighted text can be deleted, yanked, changed, filtered, search/replaced, etc.

Slide 38 : Using multiple "windows" * Multiple documents can be viewed in a single vim screen. o Ctrl-w, s splits the screen horizontally o Ctrl-w, v splits the screen vertically o Ctrl-w, Arrow moves between windows * Ex-mode instructions always affect the current window *:help windows displays more window commands Configuring vi and vim * Configuring on the fly o :set or :set all * Configuring permanently o ~/.vimrc or ~/.exrc * A few common configuration items o :set number o :set autoindent o :set textwidth=65 (vim only) o :set wrapmargin=15 o :set ignorecase * Run :help option-list for a complete list