Bash Tips and Tricks

[james147]

For the original see my blog here

Here is a hopefully useful collection of tips and tricks I have found while using the Bourne Again Shell (aka bash). I have to admit that I have not used bash as my main shell in quite a while now so there might be some mistakes here and there (please do tell my if you find any). Why write about bash then? Well it is still a very popular shell and a lot of the stuff for bash works in a lot of other shells, including my current shell zsh (more posts to come on zshs improvements over bash when I have another spare moment).

Notes

There are a number of settings in this post, unless specified otherwise you can type them into an active bash shell session and they will take effect straight away in that session and for that session only (you can reset the shell be closing and opening a new shell). If you decide you like the settings you can apply them at bash start-up by adding them to your bashrc file, normally located at ~/.bashrc. Note that settings you add to ~/.bashrc will not take effect straight away but will be applied on any new bash session started you can source you bashrc file to reapply any settings set inside it by 'sourcing' the bashrc file with:

source ~/.bashrc
. ~/.bashrc

You only need to run one of these as they both do the same thing. Also note that you do not have to source the ~/.bashrc file, you can source any file.

Sourcing vs Running a Script

As mentioned in the notes you can source ~/.bashrc to reapply any settings in it to the current shell, this also works for any bash script. So what is the difference between sourcing and just running a script? It is all to do with what happens to the parent shell, when you run a script a new bash shell is created to run the script and distorted after it exits, leaving your current session unchanged. But when you source a bash script this new shell is not started and the script can modify you current session. This is most useful when the bash script sets an environment variable that you want access to, for example if you have the following file and contents:

[I][B]~/example.sh
[/B][/I]#!/bin/bash
 export VAR="Var is set"

and you run the following in bash

$ [B]~/example.sh
[/B]$ [B]echo "VAR: $VAR"
[/B]VAR:

you can see that VAR is empty, but if you source the file

$ [B]. ~/.example.sh
[/B]$ [B]echo "VAR:[/B] $VAR"
VAR: Var is set

see how the variable VAR now contains what the script set it to?

Exporting Variables

Normally any variables you define are only available to the shell in which you define them. You can source a script to get the definitions into the parent shell, but you need to export the variable to let any child process use it;

$ [B]VAR1="value 1"
[/B]$ [B]VAR2="value 2"
[/B]$ [B]export VAR2
[/B]$ [B]echo "'$VAR1' and '$VAR2"
[/B]'value1' and 'value2'
$ [B]bash[/B]     [I]# This starts a new shell as a child of the previous one
[/I]$ [B]echo "'$VAR1' and '$VAR2"
[/B]'' and 'value2'
$ [B]exit[/B]      [I]# Return to the parent shell
[/I]$ [B]echo "'$VAR1' and '$VAR2"
[/B]'value1' and 'value2'

This is very important for any variable you set in your bashrc file that you want other processes to use, such as the PATH variable (see below).

Shortcut Keys

Bash has many shortcut keys, but so can the terminal emulator you are using. Some useful bash shortcuts (and should work no matter what terminal emulator you are using) are:
Ctrl+U - Clear to the beginning of the line (quicker then pressing backspace repeatedly)
Ctrl+K - Clear to the end of the line
Ctrl+W - Delete the previous word
Ctrl+L - Clear the screen
Ctrl+A - Go to the start of the line
Ctrl+E - Go to the end of the line
Ctrl+C - send sigint to the current process
Ctrl+D - send eof to the current process (useful you logging out of bash and other interactive shells)
Ctrl+Z - send sigtstp to the current process causing it to suspend (type fg to start it again - more on this in the Jobs section)
Some useful terminal emulator shortcuts are (these are from konsole, but might work with other emulators)
Ctrl+Shift+C - copy
Ctrl+Shift+V - paste
Ctrl+Shift+T - new tab
Ctrl+Shift+M - hide/show the menu bar (a kde thing)
Shift+Left Arrow - go to next tab (probably konsole only)
Shift+Right Arrow - go to previous tab
Ctrl+Shift+Arrow - Move tab left/right

Quotes

Both double and single quotes can be used to contain arguments with spaces in them, but there is a slight difference between them when it comes to shell expansion. Basically the single quote will not expand any variables inside it, which is useful when the string contains characters used in the shell expansion. The double quote will expand anything it can.

$ [B]echo "$USER"
[/B]me
$ [B]echo '$USER'
[/B]$USER

Note that any quote inside another quote of the other type is treated as a normal character;

$ [B]echo " 'Hi there' "
[/B] 'Hi there'
$ [B]echo ' "Hi there" '
[/B] "Hi there"

Using output in commands

You can use foo $(bar) to append the output of bar to the command foo. What actually happens if bash runs the command bar first, and when it returns replaces the $(bar) with the output of the bar before executing foo.

$ [B]sudo apt-get purge $(deborphan)
[/B]

Which will run deborphan to find unused packages and then apt-get purge will run with the list of packages deborphan returned.
Or you can store the output of bar in the variable VAR with VAR=$(bar)

$ [B]LS="$(which ls)"
[/B]$ [B]echo "ls is located at: $LS"
[/B]ls is located at /usr/bin/ls

You will also see around the internet the use of the backtick ` for the same purpose. Effectively they do the same thing, but $() is much easier to nest and therefore less error prone so is preferred over the backtick.

$ [B]echo $(ls -l $(which bash))
[/B]rwxr-xr-x 1 root root 738008 Nov 3 01:31 /bin/bash*
$[B] echo [U]`ls -l `[/U]which bash[U]``
[/U][/B]total 0 -rw-r--r-- 1 user users 0 Nov 21 17:16 Documents[I]which bash
[/I]

The first command correctly executes the ls -l on the bash binary, but the second command falls over as it runs ls -l first on the current directory and appends which bash to the end of the result.

Aliasing

Aliases ar a short label that can be given to a command that when executed will run the command associated with them. They are very useful for changing the default flags on arguments or to shorten very long commands. Here are some useful aliases you can define.

alias ls='ls --color=auto' # Enable color output
alias la='ls -a'           # Will expand to ls --color=auto -a due to the alias above
alias upgrade='sudo apt-get update && sudo apt-get upgrade'
alias cp="cp -i"           # confirm before overwriting something
alias df='df -h'           # human-readable sizes
alias free='free -m'       # show sizes in MB
alias cd..='cd ..'         # Fix common typo
alias ..='cd ..'[B]The PATH[/B]

The path variable is a very useful varible in bash, it defines all the paths that executable are stored and normally contains the some or all of the following paths:
/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin
Each entry is separated by a colon : and each directory is searched for from left to right until the executable is found, so if the PATH is set to the above and /usr/local/bin contains an executable called 'ls' then that is executed instead of the one in /usr/bin.
You can add paths to the start or then end of the variable, but generally you will want your custom paths searched before other paths so it is best to add them to the front. This can be simply done by appending the new path to the start of the variable;

$ [B]PATH=/home/user/bin:$PATH
[/B]$ [B]echo $PATH
[/B]/home/user/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin

Bash History

Bash history deserves a post all of its own so what is listed here is only a taster of what is possible, I might be persuaded to go into more depth on a new post but for now here are the basics.
As you probably know you can press the up and down arrows to scroll through the history one line at a time, but there are much faster ways to get to the line you want.

Reverse search

Press Ctrl+R and start typing the command you want to search for, it will list the last match found and when you have the command you want press enter to execute it. For example if you enter

$ [B]echo "A long line that you want to repeat later"
[/B]A long line that you want to repeat later
$ [B]echo "Another line you do not care about"
[/B]A nother line you do not care about

and then press <Ctrl+R>rep you end up with the following

(reverse-i-search)`[B]rep[/B]': echo "A long line that you want to repeat later"

then just hit enter to rerun the listed command.
History commands

I find these far more useful than the above but mostly due to some added features in zsh which makes using them much easier. Note that the following commands also print out what it is going to run just before it runs it, doesn't give you much of a change to stop it but at least lets you know what damage you did (This is one area zsh improves on)

!! - Repeat the last command

$ [B]echo "A long line that you want to repeat"
[/B]A long line that you want to repeat
$ [B]!!
[/B]echo "A long line that you want to repeat later"
A long line that you want to repeat later

This is very useful when you want to prefix the command with something such as sudo, for all those times you get "access denied" just because you forgot those four simple characters Note that this works for all the other commands as well and you can have any number of them on one line.

$ [B]apt-get update
[/B]Permissions denied
$ [B]sudo !!
[/B]sudo apt-get update
password:....

!string - This repeats the last line the begins with "string", this is very useful for repeating a long command that you used a while ago and don't want to have to type it out again.

$ [B]echo "A long line that you want to repeat later"
[/B]A long line that you want to repeat later
$ [B]ls -a
[/B]... directory list ...
$ [B]!ec
[/B]echo "A long line that you want to repeat later"
A long line that you want to repeat later

!?string? - Repeat the last command with "string" anywhere inside it, this is basically the same as the reverse search mentioned above.

$ [B]echo "A long line that you want to repeat later"
[/B]A long line that you want to repeat later
$ [B]echo "Another line you do not care about"
[/B]Another line you do not care about
$ [B]!?rep?
[/B]echo "A long line that you want to [U]rep[/U]eat later"
A long line that you want to repeat later

!$ - The last argument of the last command
This is useful for saving you typing those long file names

$ [B]ls /path/to/some/file
[/B]/path/to/some/file
$ [B]cat !$
[/B]cat /path/to/some/file
This is the contents of some/file

There are many other commands you can use which you can read more about by entering man history.

Linking commands

One of the great features of bash is the ability to chain commands together to make even more powerful commands.

Pipe

The first and most common way is the pipe command |, this takes the standard input of one command and sends to the standard input of another

$ [B]ls | wc -l
[/B]14

This command lists all the files and directories in the current directory and then counts how many there are

Redirection

Another common thing to do is redirect the standard input or output to/from a file

$ [B]echo "Some text" > afile
[/B]$ [B]cat afile
[/B]Some text
$ [B]echo "Some replacement text" > afile
[/B]$ [B]cat afile
[/B]Some replacement text
$ [B]echo "Some additional text" >> afile
[/B]$ [B]cat afile
[/B]Some replacement text
Some additional text

Notice how > replaces the contents and >> appends the contents.

And, Or, Next

Commands in bash can be separated by a ';' so to run two commands once after another you can enter

$ [B]sleep 10; echo "Finally"
[/B](10 seconds of nothing)
Finally

but sometimes you do not want the second command to run if the first failed or vice versa this is where && and || come in handy.

$ [B]true && echo True
[/B]True
$ [B]false && echo False
[/B]$ [B]true || echo True
[/B]$ [B]false || echo False
[/B]False

These are very useful in stopping a chain of commands if one fails;

$ [B]sudo apt-get update && sudo apt-get upgrade[/B]

This will run the upgrade if and only if the update succeeds.

I find || more useful in scripts such as

[I][B]update.sh
[/B][/I]#!/bin/bash
apt-get update || exit 1
apt-get upgrade || exit 1

This script will exit and return 1 if either of the apt-get commands fails.

Jobs

Bash has a powerful job handling system that allows you to run multiple processes from the same terminal (or script). This section will how you how you can use this system.

There are many way you can background a process in bash, this simplest is to append & to the end of the command:

$ [B](sleep 10[/B][B] && echo Hello) &
[/B][1] 4324
$ [B]echo Hi there
[/B]Hi there
... 10 seconds later ...
Hello
[1]+ Done ( sleep 10 && echo Hello )

(command) executes both commands in a subshell, so that they are both influenced by the & (rather than just backgrounding the echo line after running the sleep).

The second line contains some useful information, [1] is the job number of the command, you can use this number with any of the job commands to refer to this job. 4324 is the process id of the command.

After the command completes you will get the last line to indicate this. Note that this line will appear the next time you submit a command rather than straight away so you will need to hit enter after the command completes to see it. You will also notice that the commands will output over what you where typing, it is safe to ignore these characters as they are not part of what you typed they just make it hard to see what you are doing.

You can suspend the current job by pressing Ctrl+Z, this is useful for stopping a long running process to run a shorter process of higher priority, or to start more background tasks you can also type bg to background the suspended process so you can continue to work if you forgot to background it with &.

You can also enter fg to foreground a running job so you can see when it has finished.

$ [B]sleep 10 && echo hello
[/B][B]^Z
[/B][1]+ Stopped sleep 10
$ [B]bg
[/B][1]+ sleep 10 &
$ [B]echo "Hi there"
[/B]Hi there
$ [B]fg
[/B]sleep 10
Hello

[andystmartin]

Some minor clarifications:

> Shortcut Keys
> Bash has many shortcut keys, but so can the terminal emulator you are using. Some useful bash shortcuts (and should work no matter what terminal emulator you are using) are:
...
> Ctrl+C - send sigterm to the current process
> Ctrl+Z - send sigint to the current process causing it to suspend (type fg to start it again - more on this in the Jobs section)

I believe Ctrl-C sends SIGINT and I am fairly certain Ctrl-Z sends SIGTSTP.
Also, you can get vi-style command line editing by executing "set -o vi" from the command line or putting it in your .bashrc.

[james147]

Some minor clarifications:

> Shortcut Keys
> Bash has many shortcut keys, but so can the terminal emulator you are using. Some useful bash shortcuts (and should work no matter what terminal emulator you are using) are:
...
> Ctrl+C - send sigterm to the current process
> Ctrl+Z - send sigint to the current process causing it to suspend (type fg to start it again - more on this in the Jobs section)

I believe Ctrl-C sends SIGINT and I am fairly certain Ctrl-Z sends SIGTSTP.
Also, you can get vi-style command line editing by executing "set -o vi" from the command line or putting it in your .bashrc.

Fixed thanks

[kubicle]

The "Redirection" section might benefit from including stdin and stderr redirections (stderr is especially useful when logging things), as it currently only includes/explains stdout redirection:
http://www.tldp.org/LDP/abs/html/io-redirection.html
http://www.gnu.org/software/bash/man...l#Redirections