Bash

http://wiki.bash-hackers.org/syntax/pe#substring_removal
http://www.davidpashley.com/articles/writing-robust-shell-scripts/

language

summary

((foo > 7))  # Right!
[[ $foo -gt 7 ]] # Also right!  [[ ... ]] is not recommended for Arithmetic, avoid this
  • compare
[[ $foo = bar && $bar = foo ]]       # right! recommended way
[[ -f $file1 && ( -d $dir1 || -d $dir2) ]]

[ bar = "$foo" ] && [ foo = "$bar" ] # also Right!
[ bar = "$foo" -a foo = "$bar" ]     # Not portable, avoid this
good practices
  • always check cd results
cd /foo && bar
 cd /foo || exit 1
 bar
 baz
 bat ... # Lots of commands.

if we need special out put

cd /net || { echo "Can't read /net. Make sure you've logged in to the Samba network, and try again."; exit 1; }
  • sadsa

multiple commands

cmd1 ; cmd2
cmd1;cmd2

that is equal to

$ cmd1
$ cmd2
group commands
subshell
find ... -type d | while read subdir; do
       (cd "$subdir" && whatever && ...)
     done

Forcing a SubShell here causes the cd to occur only in the subshell; for the next iteration of the loop, we're back to our
normal location, regardless of whether the cd succeeded or failed. it is better than

 find ... -type d | while read subdir; do
   cd "$subdir" && whatever && ... && cd -
 done

how to echo

there are three formats

 echo welcome xyz
 echo "welcome xyz"
echo 'welcome xyz'

single quote will treat content as literal, so you don't need to escape. for double quotes, you will need to escape with '\'.
for example, echo "welcome xyz\!" to escape !

- When using echo without quotes, we cannot use a semicolon (;) as it acts as a delimiter between commands in the bash shell.
echo hello;hello takes echo hello as one command and the second hello as the second command.

- When using echo with single quotes, the variables (for example, $var will not be expanded) inside the quotes will not be interpreted by Bash, but will be displayed as is.

further details : http://mywiki.wooledge.org/Quotes

colored output

Color codes are used to represent each color. For example, reset=0, black=30, red=31, green=32, yellow=33, blue=34, magenta=35, cyan=36, and white=37.
In order to print colored text, enter the following:

echo -e "\e[1;31m This is red text \e[0m"

Here \e[1;31 is the escape string that sets the color to red and \e[0m resets the color back. Replace 31 with the required color code.

For a colored background, reset = 0, black = 40, red = 41, green = 42, yellow = 43, blue = 44, magenta = 45, cyan = 46, and white=47, are the color code that are commonly used.
In order to print a colored background, enter the following:

echo -e "\e[1;42m Green Background \e[0m"

printf vs echo

Another command for printing in the terminal is the printf command. printf uses the same arguments as the printf command in the C programming language. For example:

$ printf "Hello world"

printf takes quoted text or arguments delimited by spaces. We can use formatted strings with printf. We can specify string width, left or right alignment, and so on. By default,
printf does not have newline as in the echo command (By default, echo has a newline appended at the end of its output text. This can be avoided by using the -n flag.).
We have to specify a newline when required

printf "%-5s %-10s %-4.2f\n" 3 Jeff 77.564

read

# read 2 characters into a variable
read -n 2 var

# read a password in non-echoed fashion
read -s var

# display a message before read
read -p "Enter your inpur :" var  

# read a var that is typed within 2 seconds time out
read -t 2 var

# read input with a different delimiter character to end the inpute
read -d ":" var

variable

note there is no space around =

var=value

If value does not contain any white space characters (like a space), it need not be enclosed in quotes, else it must be enclosed in single or double quotes.

to use a variable, there are two forms:

echo $var
echo ${var}

the 2nd form is recommended , because array is accessed like ${myarray[0]}

find length of a variable's value
length=${#var}
word splitting

right ways

[ "$foo" = bar ]
  1. because they don't undergo word splitting with [[ … ]], and even blank variables will be handled correctly. On the other hand,
  2. quoting them won't hurt anything either.
[[ bar = "$foo" ]]

the right-hand side should always be quoted:

if [[ $foo = "$bar" ]]
[ x"$foo" = xbar ]
pattern matching
if [[ $foo =~ 'some RE' ]]

The quotes around the right-hand side of the =~ operator cause it to become a string, rather than a
RegularExpression. If you want to use a long or complicated regular expression and avoid lots of
backslash escaping, put it in a variable:

re='some RE'
if [[ $foo =~ $re ]]

array

define array

array_var=(1 2 3 4 5 6)

array_var[0]="test1"
array_var[1]="test2"
array_var[2]="test3"

access array

echo ${array_var[$index]}
echo ${array_var[*]  # print all values
echo ${array_var[@]  # also print all values

Print the length of an array (the number of elements in an array), as follows:

$ echo ${#array_var[*]
associated array

define associated arrays

declare -A fruits_value
fruits_value=([apple]='100dollars' [orange]='150 dollars')
echo "Apple costs ${fruits_value[apple]}"
declare -A fruits_value
fruits_value[apple]='100dollars'
fruits_value[orange]='150 dollars'
echo "Apple costs ${fruits_value[apple]}"

list array indexes

echo ${!fruits_value[*]} 
echo ${!fruits_value[@]}

output is

orange apple

for loop of list

list of be string or sequence.

# string
myvar="name,pasword,location"

# list
for img in *.jpg *.png
do
...
done

# sequence
echo {a..z}

for case -in sensitive, we can do
for img in *.[jJ][pP][gG]
sample for loop

for i in {a..z}; do 
actions; 
done;
  • The for loop can also take the format of the for loop in C. For example:
for((i=0;i<10;i++))
{
  commands; # Use $i
}

refer: http://mywiki.wooledge.org/ArithmeticExpression

export variables

PATH="$PATH:/home/user/bin"
export PATH

notice there is no 4 before PATH for export

or

export PATH="$PATH:/home/user/bin"

get command output

cmd_output=`ls | cat -n`
cmd_output="$(cd ./xyz ; ls | cat -n)"    # avoid this, not portable

note the usage of double quota, otherwise the subshell 's output will strip \n and spacing.

subshell

Subshells are separate processes. A subshell can be defined using the ( )operators as follows:

pwd;
(cd /bin; ls);
pwd;

When some commands are executed in a subshell none of the changes occur in the current shell; changes are restricted to the subshell.
For the above example, when the current directory in a subshell is changed using the cd command, the directory change is not reflected in the main
shell environment.

testing

if  [ $var -eq 0 ]; then 
echo "True"; 
fi

can also be written as the following to avoid usage of many braces:

if  test $var -eq 0 ; then 
echo "True"; 
fi
string compare
Two strings can be compared to check whether they are the same as follows;
  if [[ $str1 = $str2 ]]: Returns true when str1 equals str2, that is, the text contents of str1 and str2 are the same. Note that a space is provided after 
and before =. If space is not provided,  it is not a comparison, but it becomes an assignment statement.
  if [[ $str1 == $str2 ]]: It is alternative method for string equality check 

We can check whether two strings are not the same as follows:

  if [[ $str1 != $str2 ]]: Returns true when str1 and str2 mismatches We can find out the alphabetically smaller or larger string as follows:
  if [[ $str1 > $str2 ]]: Returns true when str1 is alphabetically greater than str2
  if [[ $str1 < $str2 ]]: Returns true when str1 is alphabetically lesser than str2
  if [[ -z $str1 ]]: Returns true if str1 holds an empty string
  if [[ -n $str1 ]]: Returns true if str1 holds a non-empty string
math
count1=4;
count2=5;

# notice there is no $ prefix before variable name
let result=count1+count2
let count1++
let count2--
let count1+=2
let count1=count1+2
let count2-=2
let count2=count2+6

alternative fashions

  • use $[]
# notice there is space after [ or before ]
result=$[ count1 + count2 ]
result=$[ $count1 + $count2 ]
  • use $(())

one may use the (( )) syntax to enforce an arithmetic context

result=$(( count1 + count2 ))

# note no $ prefix
a=$((a+7))         # POSIX-compatible version of previous code.
if test $((a%4)) = 0; then ...
fi

if ((a > 5)); then 
  echo "a is more than 5"; 
fi
  • use expr
result=`expr $count1 + 3`
result=$(expr $count1 + 3)
mathematical conditions
-eq Equal 
-ne not equal to
-gt: Greater than
-lt: Less than
-ge: Greater than or equal to
-le: Less than or equal to
multiple logical conditions

It is easier to combine multiple conditions using the logical operators && and || as follows:

if [ -n $str1 ] && [ -z $str2 ] ;
then
commands;
fi

Another approach is putting all conditions in a single [] and use -a or -o
Multiple test conditions can be combined as follows:

[ $var1 -ne 0 -a $var2 -gt 2 ]  # using AND -a
[ $var -ne 0 -o var2 -gt 2 ] # OR -o

refer here for [[ ... ]]

[[ -f $file1 && ( -d $dir1 || -d $dir2) ]]
 [ -f "$file1" -a \( -d "$dir1" -o -d "$dir2" \) ]
shortcut

We can use logical
operators to make them shorter as follows:

[ condition ] && action; # action executes if condition is true.
[ condition ] || action; # action executes if condition is false.
file system testing

We can test different filesystem related attributes using different condition flags as follows:

[ -f $file_var ]: Returns true if the given variable holds a regular filepath or filename.
[ -x $var ]: Returns true if the given variable holds a file path or filename which is executable.
[ -d $var ]: Returns true if the given variable holds a directory path or directory name.
[ -e $var ]: Returns true if the given variable holds an existing file.
[ -c $var ]: Returns true if the given variable holds path of a character device file.
[ -b $var ]: Returns true if the given variable holds path of a block device file.
[ -w $var ]: Returns true if the given variable holds path of a file which is writable.
[ -r $var ]: Returns true if the given variable holds path of a file which is readable.
[ -L $var ]: Returns true if the given variable holds path of a symlink.

how to test a broken symlink

[[ -e "$broken_symlink" || -L "$broken_symlink" ]]

functions

define functions
  • approach 1
fname()
{
  statements;
}
  • approach 2, avoid this. not very compatible.
function fname()
{
  statements;
}
call functions
fname ;
fname arg1 arg2 ;
arguments
fname()
{
  echo $1, $2; #Accessing arg1 and arg2
  echo "$#";  # print number of arguments
  echo "$@"; # Printing all arguments as list at once
  echo "$*"; # Similar to $@, but arguments taken as single entity, it is used rarely
  return 0; # Return value
}
read return value from function and command
fname ;
echo $?;

please note we also use $? immediately after last running execution statement to get the exit status.

export function

a function is exported so that the scope of the function can be extended to subprocesses, as follows:

export -f fname

file

read from file

exec 3<input.txt
cat <&3

write to file

exec 4>output.txt
echo "abcd" >&4

append to file

exec5>>output.txt
echo "abcd">&5

misc tips

bash pitfalls

bash faq

debug

  • use set
#!/bin/bash
for i in {1..6}; do
  set -x
  echo $i
  set +x
done

set -x: Displays arguments and commands upon their execution
set +x: Disables debugging Shell Something Out
set –v: Displays input when they are read
set +v: Disables printing input

  • another approach
#!/bin/bash
function DEBUG()
{
  [ "$_DEBUG" == "on" ] && $@ || :
}

for i in {1..10}; do
  DEBUG echo $i
done

then er can turn on debug to run it:

_DEBUG=on ./script.sh

In Bash the command ':' tells the shell to do nothing.

  • 3rd approach

change from #!/bin/bash to #!/bin/bash –xv to enable debugging without any additional flags

read from a text block enclosed within a script

# This is what you were trying to do:
  cat <<EOF
  Hello world
  How's it going?
  EOF

you can't use echo with EOF

# Or, use quotes which can span multiple lines (efficient, echo is built-in):
  echo "Hello world
  How's it going?"
# Or use printf (also efficient, printf is built-in):
  printf %s "\
  Hello world
  How's it going?
  "

IFS

An Internal Field Separator (IFS) is an environment variable that stores delimiting characters. It is the
default delimiter string used by a running shell environment.

read csv file

data="name,sex,rollno,location"
#To read each of the item in a variable, we can use IFS.
oldIFS=$IFS
IFS=,
for item in $data; do
  echo Item: $item
done
IFS=$oldIFS

evaluate execution time

#!/bin/bash
#Filename: time_take.sh
start=$(date +%s)
commands;
statements;
end=$(date +%s)
difference=$(( end - start))
echo Time taken to execute commands is $difference seconds.

read content from file or stream

http://mywiki.wooledge.org/BashFAQ/024
because each pipe line is running in a different shell, the right way to update linecnt is
grouping the commands to be executed in the current shell context via { } grouping.

http://www.gnu.org/software/bash/manual/html_node/Command-Grouping.html#Command-Grouping

 # POSIX
  linecnt=0
  cat /etc/passwd |
  {
      while read -r line ; do
         linecnt=$((linecnt+1))
      done
      echo "total number of lines: $linecnt"
  }

http://mywiki.wooledge.org/BashFAQ/001

  • read from file
while read -r line
    do
        echo "$line"
    done < "$file"
  • read from variable
while read -r line; do
        echo "$line"
    done <<< "$var"

process files

for current directory files

for i in *.mp3; do  # Better! and...
   some command "$i" # ...see Pitfall #2 for more info.
done

This reads one filename at a time from the find command and renames the file, replacing spaces with underscores.

find . -name "*.mp3" | while read fn; do
   mv "$file" "${file// /_}"
 done
find . -print0 | while IFS= read -r -d $'\0' file; do
   mv "$file" "${file// /_}"
done

check grep result

if grep foo myfile >/dev/null; then
 ...
fi

If the grep matches a line from myfile, then the exit code will be 0 (true), and the then part will be executed.
Otherwise, if there is no matching line, the grep should return a non-zero exit code.

NOTE!!! we do not need [ …] for the grep. as [ is a command. It takes arguments, and it produces an exit code.
If you want to make a decision based on the output of a grep command, you do not need to enclose it in parentheses,
brackets, backticks, or any other syntax mark-up! Just use grep as the COMMANDS after the if
http://mywiki.wooledge.org/BashPitfalls#for_i_in_.24.28ls_.2A.mp3.29

change the content of a file

sed 's/foo/bar/g' file > tmpfile && mv tmpfile file

check for super user

if [ $UID -ne 0 ]; then
  echo "Non root user. Please run as root.";
else
  echo "Root user";
fi

get pid of a process

http://mywiki.wooledge.org/BashPitfalls#for_i_in_.24.28ls_.2A.mp3.29

$ ps -C gedit | awk '{print $1}' | tail -n1
pgrep gedit

glob & parameter expansion

http://mywiki.wooledge.org/glob
http://mywiki.wooledge.org/BashFAQ/073

find

redirection

ping $ip -c 2 &> /dev/null ;

  if [ $? -eq 0 ];
  then
    echo $ip is alive
  fi

&> /dev/null

is used to redirect both stderr and stdout to /dev/null so that it won't be printed on the terminal.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License