A few years back I studied the LPIC1 material, and in doing so gained a much better and fundamental understanding of how linux works. Prior to that, I had been picking up bits here and there by reading and following guides on the internet, but this explained to me in a way that I understood much better, what each command was doing or how the linux server works. I wanted to post the study notes I took at the time here in case they were of use to you. This is what I took from the material as I was studying it.
This will be a multi post article and I will link to the next from each article.
Section 1. Command Line Basics
Moving Within the Shell
Ctrl + A
Ctrl + E
Ctrl + P
Ctrl + N
Ctrl + B – move back one space
Ctrl + F – move forward one space
Ctrl + R – Recursive Search
Ctrl + S – Forward Search
Ctrl + G – Cancel
Ctrl + D – [Delete] key
Ctrl + K – Delete from cursor to end of line
Ctrl + X + Backspace – delete from cursor to start of line
Ctrl + U – also delete from cursor to start of line
Transpose (Swap) Text
Ctrl + T – swap the letter under the cursor with the one before it
Esc + T – swap the word under the cursor with the one before it
Change Case
Esc + U – change word to UPPERCASE
Esc + L – Change word to lowercase
Esc + C – Change single character to uppercase
(Esc + U, Esc + L – cursor must be at the start of the word as it only affects characters from the cursor to the end of the word)
Start Default Editor
Ctrl + X then Ctrl + E
—————————————
exec When you run a program using exec, rather than running alongside the shell, it replaces the shell. When you close the program, it close the shell too
—————————————
xargs
xargs builds a command from its standard input – pipe the output of a find command to it
|
1 2 3 |
xargs [options] [command [command-arguments] ] $ find ./ -name “*~” | xargs –d “\n” rm |
- Find all files under the current directory ending with a ~ and delete them
- Find outputs each file found on a new line, so the delimiter in xargs is set to \n for new lines (-d “\n”)
- Other delimiters could be spaces, tabs, etc
—————————————
cat
Concatenate – a tool for combining files
|
1 |
$ cat file1.txt file2.txt > file3.txt |
-e (show ends) adds $ to end of each line
-n numbers each line
-b numbers only lines with content – avoids blank lines
-s squeezes concurrent (grouped) blank lines down to just one line
tac displays files with line order reversed, i.e. line 3 before line 2 before line 1
—————————————
join
Combine 2 files by matching entries in one to entries in another. There must be matching (key) entries for this to work
| 123 Bob | 123 listed | join | 123 Bob listed |
| 456 Alice | 456 unlisted | —> | 456 Alice unlisted |
- By default it uses the first field to match across files
- Specify another field using -1 or -2 to specify which file, e.g.
|
1 |
<strong>join -1 2 -2 4 file1.txt file2.txt |
- (from the first file (-1) use the 2nd field; from the 2nd file (-2) use the 4th field
—————————————
paste
Paste merges files, line by line, separating the contents of each file using tabs
|
1 |
paste file1.txt file2.txt |
| 123 Bob | 123 listed | paste | 123 Bob [tab] 123 listed |
| 456 Alice | 456 unlisted | paste | 456 Alice [tab] 456 unlisted |
—————————————
expand
Convert tabs to spaces
-t n Convert all tabs to n number of spaces
|
1 |
$ expand –t 1 file.txt |
—————————————
od
- Display files (text, image, binary, etc) in octal format
- Display them in base 8 format
- Can be used to display in hex, decimal, …
—————————————
sort
| -f | ‐‐ignore-case, usually results are sorted to display uppercase first (ASCII value), but this disables this |
| -m | ‐‐month-sort, sort using 3 letter month abbreviations, i.e. Jan-Dec |
| -n | ‐‐numeric-sort, sort by number |
| -r | ‐‐reverse, sort in reverse order |
| -k | tell it what field you want to sort on (default is the first field) |
—————————————
split
Splits a file up according to the parameters provided. Requires an output filename to be supplied
| -b n | splits a file into multiple files, all of a byte size (n) specified |
| -C n | splits files up but doesn’t break across lines, unless the size of the line is greater (in bytes) than specified (n) |
| -l n | splits a file up using the specified number of lines (n) |
—————————————
unexpand
Converts multiple spaces to tabs
—————————————
uniq
Remove duplicate lines
—————————————
fmt
Reformat paragraphs. Reformat blocks of text so they display properly on a screen, etc
-w specify paragraph width (75 = default)
|
1 |
$ fmt –w 100 essay.txt > formatted_essay.txt |
—————————————
nl
Number lines in text
- Can be done with ‘cat –b’ for simple results
- More specialised results are possible with nl
| -b | Body Numbering Style: using a style format code |
| -h | Header numbering style |
| -f | Footer numbering style |
| -d=code | Set a new page delimiter |
| -p | do not restart numbering at #1 for each page |
| -n ln/rn/rz | number format, where: ln = left justified, no leading zeroes rn = right justified, no leading zeroes rz = right justified, with leading zeroes |
—————————————
-pr
Printing files
|
1 |
$ pr file.txt |
default formatting is for a line printer – 80 character line length, monospaced font
Default includes headers with current date & time, original filename & page number
| ‐‐numcols | generate multi-column output. If lines are too long they are cut or run over into multiple columns |
| -d | ‐‐double-spaced, generate double spaced output |
| -f | ‐‐form-fixed, use form feeds |
| -l n | ‐‐length=’n’ , set page length |
| -h ‘text’ | ‐‐header=’text’, set header text |
| -t | omit the header altogether |
| -o n | ‐‐indent=n, set left and right margins |
| -w n | ‐‐width=n, set the page width |
—————————————
head
view the first few lines of a file (default = 10)
| -c | ‐‐bytes=n, display the first n bytes of a file rather than lines |
| -n | ‐‐lines=n, display the first n lines rather than the default (10) |
—————————————
tail
View the last few lines of a file (default = 10)
| -c | ‐‐bytes=n, view the last n bytes |
| -n | ‐‐lines=n, view the last n lines |
| -f | ‐‐follow, keep tail open and display new lines as the are added. Very useful for viewing log files |
| ‐‐pid=n | give tail the pid of the program to follow, and when the process ends, tail will end also |
—————————————
Use head and tail in conjunction
To view lines 25-30 of a file:
|
1 |
$ head –n 30 file1.txt | tail –n 6 |
—————————————
less
Page through files
- Spacebar & B to move pages downwards and backwards
- Esc – v moves backwards too
- Arrow keys to move up/down by line
- /text to search for text
- Pressing n alone jumps the search forward to next result. Pressing N jumps the search backwards
- ?text searches backwards for text
- g# jumps to line #
- h from within less brings you to less’s internal help dialogue
man pages use less so the above is relevant to navigating these too
—————————————
cut
extract portions of input lines & display them on standard output
| -b | ‐‐bytes=LIST, cut only the specified list of bytes from the input |
| -c | ‐‐characters=LIST, cut specified list of characters from input. Used to give same results as –bytes |
| -f | ‐‐fields=LIST, cut specified list of fields |
– A field is a tab delimited section of a line. You can change this using –d / –delim=char, where char is the character you use to delimit fields
– cut echoes lines that don’t contain delimiters. Use –s / –only-delimited to only show lines which do contain the delimiter
– LIST is a way to specify multiple bytes, characters or fields, specified by a number. Can be a single number (4), a closed range (2-4) or open range (-4)(4-)
|
1 |
$ ifconfig eth0 | grep HWaddr | cut –d “ “-f 11 |
—————————————
wc
|
1 2 |
$ wc file1.txt 2002 24781 169762 file1.txt |
2002 lines (new line characters)
24781 words
169762 bytes
| -l | ‐‐lines, show only line counts |
| -w | ‐‐words, show only word counts |
| -c | ‐‐bytes, show only byte counts |
| -L | ‐‐max-line, show longest line length |
—————————————
REGEX
| Bracket Expressions: | [ ] – these match any one character within the brackets b[io]g = big or bog does not equal biog |
| Range Expressions: | [ – ] – instead of listing every character that matches, these list the start and end points a[2-4]z = a2z, a3z, a4z |
| Any Single Character: | The dot (.) represents anything except a new line a.z = a2z, abz, aQz, … |
| Start and End of Line: | Carat (^) represents start of line Dollar ($) represents end of line |
| Repetition Operators: | Asterisk (*) represents zero or more occurrences Plus (+) represents one or more occurrences Question (?) represents zero or one match only |
. Matches any single character
? Will be matched zero or one time
* Will be matched zero or more (multiple) times
+ Will be matched one or more (multiple) times
| Multiple Possible Strings: | Vertical bar ( | ) seperates two possible matches, i.e. car|truck = either car or truck |
| Brackets: | ( ) these surround subexpressions |
| Escaping: | Use \ to match a special character, i.e. example\.com |
—————————————
grep
search files for text
| -c | ‐‐count, count the number of lines matching the pattern/regex |
| -f | ‐‐file=FILE, take pattern/regex input from FILE rather than from the command line |
| -I | ‐‐ignore-case, do case insensitive search |
| -r | ‐‐recursive, search the current directory and all sub-directories. Can also use rgrep |
| -E | ‐‐extended-regexp, use extended regex |
|
1 2 |
$ grep [-options] regex pattern [files/directories] $ grep –r eth0 /etc/* |
May need to enclose the regex in quotes when using vertical bars & asterisks, in case the shell itself, rather than he grep command, tries to interpret them
|
1 |
$ grep –E “(example\.com | test\.com)+127*” /etc/* |
—————————————
REDIRECTION
Standard output = the screen
| > | Creates a new file from standard output, or overwrites it if not there already |
| >> | Appends standard output to a file, or creates it is not there already |
| 2> | Creates a new file from standard error, or overwrites it if there already |
| &> | Creates a new file with both standard output and standard error text, or overwrites existing file |
| < | Takes specified file as standard input |
| << | Takes text on following lines as standard input, terminating with Ctrl + D |
| <> | Uses specified file as standard input & standard output |
| tee | tee both displays text to standard output (screen) and also saves it to a specified file. Pipe output from a command to it |
|
1 |
$ ls –lh | tee file.txt |
—————————————
sed
– 2 forms:
– sed [options] –f script-file [input-file]
– sed [options] ‘script-text’ [input-file]
– sed modifies contents of files, sending changed file to standard output
– ‘script-text’ on contents of script-file are the commands for sed to run on
– ‘script-text’ uses single quotation marks when typed on the command line
| Command | Addresses | Meaning |
|---|---|---|
| = | 0 or 1 | Display the current line number |
| a\text | 0 or 1 | Append text to the file |
| i\text | 0 or 1 | Insert text into the file |
| r filename | 0 or 1 | Append text file filename into the file |
| c\text | range | Replace the selected range of lines with text |
| s\regex\replacement | Replace text matching regex with replacement | |
| w file | Range | Write the current pattern space to the specified file |
| q | 0 or 1 | Quit the script but print the current pattern square |
| Q | 0 or 1 | Quit the script |
– Addresses are line numbers
– sed can take no addresses (so it works on the entire file), one address (one line) or a range of addresses
– To encode the end of a line, ASCII has different ways
– Unix uses single line feed character, \n
– Dos/Windows uses carriage return, \r, and line feed
—————————————
tr
tr changes individual characters from standard input
|
1 |
$ tr [options] SET1 [SET2] |
– Specify the characters you want replaced in a group, SET1
– Specify the characters you want them replaced by in SET2
– Each character in SET1 is replaced by with the respective character in SET2
|
1 |
$ tr L l <file.txt |
‘Linux’ becomes ‘linux’
– SET2 can be shorter than SET1, in which case tr substitutes the last letter in SET2 for the missing letter(s) in SET1
|
1 |
$ tr ABCDE w <file1.txt |
Changes them all to ‘w’
-t –truncate-set1, truncate SET1 down to the size of SET2
-d Make tr delete the characters specified in SET1; SET2 is unneeded in this case
– Can use shortcuts:
[:alnum:] – all alpha & numeric
[:upper:] – all uppercase
[:lower:] – all lowercase
[:digit:] – all digits
– Specify range of characters by using -, i.e. A-M for the letters A to M inclusive