Wednesday, April 9, 2008

Sample Shell scripts Examples:

Shell scripts use to automate your daily tasks and make life peaceful
===================================================================================== A simple 3 line shell script to check if the doent exists then backup the logfile with date suffix and create the new logfile. if the file exists then do nothing
#!/usr/bin/ksh
if [[ ! -f /home/jack/app/donotrun.tmp ]] ; then
mv /tmp/app.log /tmp/app.log.`date +%Y%m%d%H%M`

touch /tmp/app.log
fi
____________________________________________________________________
Examples 1 .

  • Sample code to automate FTP in shell script.
    ############ Create the file needed for FTP ################
    count=`find ${PDFPUB} -type f wc -l` #To checking how many files are there to transfer
    echo open $FTP_HOST > FINPUB_FTP.temp # host name stored in variable
    echo "user ${FINPUBUSER} ${FINPUBPASS}" >> FINPUB_FTP.temp
    echo binary >> FINPUB_FTP.temp
    echo "cd /tmp/FINPUB" >> FINPUB_FTP.temp
    echo "lcd /home/mpansare/pdfpub" >> FINPUB_FTP.temp
    echo prompt >> FINPUB_FTP.temp
    echo "mput *" >> FINPUB_FTP.temp
    echo bye >> FINPUB_FTP.temp
    ############ FTP the file ################
    echo "" >>$LOGFILE
    ftp -nv -i <>>$LOGFILE
    rm FINPUB_FTP.temp
    ------------------------------------------------------------------------------------------------- ftp options:
    -v = Verbose option forces ftp to show all responses from the remote server, as well as report on data transfer statistics.
    -n = Restrains ftp from attempting ``auto-login'' upon initial connection.
    -i = Turns off interactive prompting during multiple file transfers.
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ########## Determine if FTP was successful ################
    grep "226 Transfer complete." $LOGFILE
    return_code=$?
    grepcount=`grep -c "226 Transfer complete" $LOGFILE` # counts how many files were transfered properly

    # If else block
    if [[ $count -eq $grepcount ]]; then
    echo "FTP file transfer counts match" >> $LOGFILE
    else
    echo "FTP files transfer donot match. Some files may not have transfer" >> $LOGFILE
    fi

=====================================================================================

Example 2.

  • Script to check diskusage that runs thru cron every hour to report and alert if disk space exceeds 95%

#!/usr/bin/ksh
#############################################################
# Script Name:checkDiskUsage.sh

###########################################################
dater=`date +%Y%m%d`
sScriptName=checkDiskUsage
sLogfile=/dev/${sScriptName}.${dater}.log
sSpool=/tmp/${sScriptName.$$}.tmp
echo "${sScriptName} starting on `date`" >> ${sLogfile}
iSpoolCmd=`df -k egrep -v "/procFilesystem" > "${sSpool}"`
if [[ $iSpoolCmd -ne 0 ! -s ${sSpool} ]];then
echo "Error: Could not retrieve disk usage statistics. Aborting!" >> $sLogfile
exit 255
fi
while read -r myline; do
sDir=`echo "${myline}" awk '{print $7}'`
sPUsage=`echo "${myline}" awk '{print $4}'`
sUsage=`echo "${sPUsage}" tr -d %`
#echo ${sDir}, ${sUsage}

if [[ ${sUsage} -ge 95 ]];then
echo "WARNING : Usage on ${sDir} has reached ${sPUsage}" >> ${sLogfile}
elif [[ ${sUsage} -ge 90 && ${sUsage} -lt 95 ]];then
echo "NEEDS ATTENTION : Usage on ${sDir} has reached ${sPUsage}" >> ${sLogfile}
fi
done < ${sSpool} echo "${sScriptName} ending on `date`" >> ${sLogfile}
rm ${sSpool} >/dev/null
exit 0

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------

=====================================================================================

Example 3.

  • The dailylogbackup script is used to backup the log files on dailybasis with date suffix This script is called with a PARM file which lists all the logfiles to be backed up (on a daily basis).# Input parms:

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------

#!/usr/bin/ksh
#############################################################
# ScriptName: dailyBkupLogFile.sh
# Purpose: c
# Input parms:
# Usage: dailyBkupLogFile.sh
# Parent script:
#################################################################
# Define a path where the script will log all actions
mylogfile=/dev/Logs/dailyBkupLogFile.log
# Check for proper command line arguments
if [ $# -ne 1 ] ; then
echo "Incorrect Usage. Correct Usage: $0 "
exit 1
fi
# Check if the PARM file really exists
parmfile=$1
if [ ! -f $parmfile ] ; then
echo "Error: C
ould not find the specified PARM file. Please verify the PARM file exists."
exit 1
fi
echo "$0 started on `date`" >> $mylogfile
# Iterate through the list of logfiles supplied in the PARM file
while read -r sLine ; do
if [ -f $sLine ] ; then
#create dated name
newfile="$sLine.`date +%Y%m%d`"
#flag and counter to tack on the end
i=0
#if the newfile exists, set flag
if [ -f $newfile ] ; then
i=1
#loop to see how many we need to do
while [ -f $newfile.$i ]; do
((i+=1)) ; done

fi

#if there were previous logs, append the number
if [ $i -ne 0 ] ; then
mvcmd="mv $sLine $newfile.$i"
else
mvcmd="mv $sLine $newfile"
fi
# Backup the logfile by performing move/rename with current date appended
$mvcmd
if [ $? -eq 0 ] ; then
echo "Backing up logfile $sLine : SUCCESS" >> $mylogfile
else
echo "Backup command failed: $mvcmd" >> $mylogfile
fi
# Create a new logfile with the same original filename
touchcmd="touch $sLine"
$touchcmd
if [ $? -eq 0 ] ; then
echo "Creating logfile $sLine : SUCCESS" >> $mylogfile
else
echo "Logfile create failed: $touchcmd" >> $mylogfile
fi
else
echo "Error: Could not find logfile - Skipping backup of $sLine." >> $mylogfile
fi
done < $parmfile echo "$0 completed on `date`" >> $mylogfile
exit 0

Saturday, April 5, 2008

Unix Shell Scripting.

Shell Programming
This section describes the fundamentals of bash shell programming and covers the following topics:
· Creating and running shell programs
· Using shell variables
· The importance of quotes
· The test command
· Conditional statements
· Iteration statements
· Functions

  • Creating and Running Shell Programs
    Shell programs, or scripts, are just text files that contain one or more shell commands. Shell scripts are similar to batch files in DOS, but are much more powerful. These scripts can be used to simplify repetitive tasks, to replace two or more commands that are always executed together with a single command, to automate the installation of other programs, to write simple interactive applications, and various other useful tasks. The general Linux philosophy for providing functionality is to link several small, discrete commands together to accomplish more complicated tasks. Shell scripting and piping are the most common ways to do this.
    Since bash is for most intents and purposes the standard shell for Linux, only bash shell programming will be discussed here. In bash, the pound symbol (#) signifies a comment to be ignored when it is the first character on a line.
    Once the script exists, there are several ways to execute it. One way to accomplish this is to make the file executable. This is done by entering the command "chmod +x remount". This command changes the permissions of the file so that it is now executable. You can now run your new shell program by typing (./scriptname) on the command line. reasons your
  • Using Variables
    As is the case with almost any language, the use of variables is very important in shell programs. You saw some of the ways in which shell variables can be used in the introductory shell sections. Two of the variables that were introduced were the PATH variable and the PS1 variable. These are examples of built-in shell variables, or variables that are defined by the shell program you are using. This section describes how you can create your own variables and use them in simple shell programs.
    Assigning a Value to a Variable
    You can assign a value to a variable simply by typing the variable name followed by an equal sign and the value you want to assign to the variable. For example, if you wanted to assign a value of 5 to the variable count, you would enter the following command "count=5". With the bash syntax for setting a variable, you must make sure that there are no spaces on either side of the equal sign. Notice that you do not have to declare the variable as you would if you were programming in C or Pascal. This is because the shell language is a non-typed interpretive language. This means that you can use the same variable to store character strings that you use to store integers. You would store a character string into a variable in the same way that you stored the integer into a variable. For example "name=Garry " is also completely valid.
    Accessing the Value of a Variable
    Once you have stored a value into a variable, how do you get the value back out? You do this in the shell by preceding the variable name with a dollar sign ($). If you wanted to print the value stored in the count variable to the screen, you would do so by entering the command "echo -n $count". If you omitted the $ from the preceding command, the echo command would display the word count on-screen. Also, here "-n" is an option for echo not to append a carriage return to the end of the line.
    Positional Parameters and Other Built-In Shell Variables
    The shell has knowledge of a special kind of variable called a positional parameter. Positional parameters are used to refer to the parameters that were passed to a shell program on the command line or a shell function by the shell script that invoked the function. When you run a shell program that requires or supports a number of command-line options, each of these options is stored into a positional parameter. The first parameter is stored into a variable named 1, the second parameter is stored into a variable named 2, and so forth. These variable names are reserved by the shell so that you can't use them as variables you define. To access the values stored in these variables, you must precede the variable name with a dollar sign ($) just as you do with variables you define.
    Example Script: reverse
    The following shell program expects to be invoked with two parameters. The program takes the two parameters and prints the second parameter that was typed on the command line first and the first parameter that was typed on the command line second.
    #program reverse, prints the command line parameters out in reverse order#!/bin/bashecho "$2" "$1"
    Table 1: Built-in shell variables.
    Variable Use
    $# = Stores the number of command-line arguments that were passed to the shell program.
    $? = Stores the exit value of the last command that was executed.
    $0 = Stores the first word of the entered command (the name of the shell program).
    $* = Stores all the arguments that were entered on the command line ($1 $2 ...).
    "$@" = Stores all the arguments that were entered on the command line, individually quoted ("$1" "$2" ...).
    The Importance of Quotation Marks
    The use of the different types of quotation marks is very important in shell programming. Both kinds of quotation marks and the backslash character are used by the shell to perform different functions. The double quotation marks (""), the single quotation marks (''), and the backslash (\) are all used to hide special characters from the shell. Each of these methods hides varying degrees of special characters from the shell. Remember that everything in this section also applies to the bash command line, so for example you could use a backslash to use a space in the name of a file.
    Double Quotes
    The double quotation marks are the least powerful of the three methods. When you surround characters with double quotes, all the whitespace characters are hidden from the shell, but all other special characters are still interpreted by the shell. This type of quoting is most useful when you are assigning strings that contain more than one word to a variable. For example, if you wanted to assign the string hello there to the variable greeting, you would type the following command:greeting="Hello there"
    This command would store the string "Hello there" in the variable "greeting" as one word. If you typed this command without using the quotes, you would not get the results you wanted. bash would not understand the command and would return an error message.
    Single Quotes
    Single quotes are the most powerful form of quoting. They hide all special characters from the shell. This is useful if the command that you enter is intended for a program other than the shell. Because the single quotes are the most powerful, you could have written the previous example using single quotes. You might not always want to do this. If the string being assigned to the greeting variable contained another variable, you would have to use the double quotes. For example, if you wanted to include the name of the user in your greeting, you would type the following command: greeting="Hello there $LOGNAME"
    This would store the string "Hello there " and the value of $LOGNAME into the variable greeting. The LOGNAME variable is a shell variable that contains the username of the person who is logged in to the system. If you tried to write this command using single quotes it wouldn't work, because the single quotes would hide the dollar sign from the shell and the shell wouldn't know that it was supposed to perform a variable substitution.
    Backslash
    Using the backslash is the third way of hiding special characters from the shell. Like the single quotation mark method, the backslash hides all special characters from the shell, but it can hide only one character at a time, as opposed to groups of characters. You could rewrite the greeting example using the backslash instead of double quotation marks by using the following command: greeting=Hello\ There
    In this command, the backslash hides the space character from the shell, and the string "Hello there" is assigned to the variable "greeting".
    Backslash quoting is used most often when you want to hide only a single character from the shell. This is usually done when you want to include a special character in a string. For example, if you wanted to store the price of a box of computer disks into a variable named disk_price, you would use the following command: disk_price=\$5.00
    The backslash in this example would hide the dollar sign from the shell. If the backslash were not there, the shell would try to find a variable named 5 and perform a variable substitution on that variable. Assuming that no variable named 5 were defined, the shell would assign a value of .00 to the disk_price variable. This is because the shell would substitute a value of null for the $5 variable. The disk_price example could also have used single quotes to hide the dollar sign from the shell.
    Back Quotes
    The back quote marks (") perform a different function. They are used when you want to use the results of a command in another command. For example, if you wanted to set the value of the variable contents equal to the list of files in the current directory, you would type the following command: contents='ls'
    This command would execute the ls command and store the results of the command into the contents variable. As you will see in later, this feature can be very useful when you want to write a shell program that performs some action on the results of another command.
    The test Command
    A command called test is used to evaluate conditional expressions. You would typically use the test command to evaluate a condition that is used in a conditional statement or to evaluate the entrance or exit criteria for an iteration statement. The test command has the following syntax: test expression
    or [ expression ]
    Several built-in operators can be used with the test command. These operators can be classified into four groups: integer operators, string operators, file operators, and logical operators.
    The test command's integer operators.
    Operator Meaning
    int1 -eq int2 - Returns True if int1 is equal to int2.
    int1 -ge int2 - Returns True if int1 is greater than or equal to int2.
    int1 -gt int2 - Returns True if int1 is greater than int2.
    int1 -le int2 - Returns True if int1 is less than or equal to int2.
    int1 -lt int2 - Returns True if int1 is less than int2.
    int1 -ne int2 - Returns True if int1 is not equal to int2.

    The test command's string operators.
    Operator Meaning
    str1 = str2 - Returns True if str1 is identical to str2.
    str1 != str2 - Returns True if str1 is not identical to str2.
    str - Returns True if str is not null.
    -n str - Returns True if the length of str is greater than zero.
    -z str - Returns True if the length of str is equal to zero.
    The test command's file operators.
    Operator Meaning
    -d filename - Returns True if file, filename is a directory.
    -f filename - Returns True if file, filename is an ordinary file.
    -r filename - Returns True if file, filename can be read by the process.
    -s filename - Returns True if file, filename has a nonzero length.
    -w filename - Returns True if file, filename can be written by the process.
    -x filename - Returns True if file, filename is executable.
    The test command's logical operators.
    Command Meaning
    ! expr - Returns True if expr is not true.
    expr1 -a expr2 - Returns True if expr1 and expr2 are true.
    expr1 -o expr2 - Returns True if expr1 or expr2 is true.
    Conditional Statements
    The bash shell has two forms of conditional statements. These are the if statement and the case statement. These statements are used to execute different parts of your shell program depending on whether certain conditions are true.
    The if Statement
    bash supports nested if...then...else statements. These statements provide you with a way of performing complicated conditional tests in your shell programs. The syntax of the if statement is shown here:

if [ expression ] then;

commands

elif [ expression2 ]then;

commands

else

commands

fi
The elif and else clauses are both optional parts of the if statement. Also note that bash use the reverse of the statement name in most of their complex statements to signal the end of the statement. In this statement the fi keyword is used to signal the end of the if statement. The elif statement is an abbreviation of else if. This statement is executed only if none of the expressions associated with the if statement or any elif statements before it were true. The commands associated with the else statement are executed only if none of the expressions associated with the if statement or any of the elif statements were true.
The case Statement
The case statement enables you to compare a pattern with several other patterns and execute a block of code if a match is found. The shell case statement is quite a bit more powerful than the case statement in Pascal or the switch statement in C. This is because in the shell case statement you can compare strings with wildcard characters in them, whereas with the Pascal and C equivalents you can compare only enumerated types or integer values. The syntax for the case statement is the following:

case string1

instr1) commands;;

str2) commands;;

*) commands;;

esac
The string string1 is compared to str1 and str2. If one of these strings matches string1, the commands up until the double semicolon (;;) are executed. If neither str1 nor str2 matches string1, the commands associated with the asterisk are executed. This is the default case condition because the asterisk matches all strings.
The following code is an example of a bash case statement. This code checks to see if the first command-line option was -i or -e. If it was -i, the program counts the number of lines in the file specified by the second command-line option that begins with the letter i. If the first option was -e, the program counts the number of lines in the file specified by the second command-line option that begins with the letter e. If the first command-line option was not -i or -e, the program prints a brief error message to the screen.

case $1 in

-i) count='grep ^i $2 wc -l' echo "The number of lines in $2 that start with an i is $count" ;;

-e) count='grep ^e $2 wc -l' echo "The number of lines in $2 that start with an e is $count" ;;

*) echo "That option is not recognized";;

esac
Iteration Statements
The shell languages also provide several iteration or looping statements.
The for Statement
The for statement executes the commands that are contained within it a specified number of times. bash has two variations of the for statement. The first form of the for statement that bash support has the following syntax:

for var1 in list

do

commands

done
In this form, the for statement executes once for each item in the list. This list can be a variable that contains several words separated by spaces, or it can be a list of values that is typed directly into the statement. Each time through the loop, the variable var1 is assigned the current item in the list, until the last one is reached. The second form of for statement has the following syntax: for var1do statementsdone
In this form, the for statement executes once for each item in the variable var1. When this syntax of the for statement is used, the shell program assumes that the var1 variable contains all the positional parameters that were passed in to the shell program on the command line. Typically this form of for statement is the equivalent of writing the following for statement: for var1 in "$@"do statementsdone
The following is an example of the for statement. This example takes as command-line options any number of text files. The program reads in each of these files, converts all the letters to uppercase, and then stores the results in a file of the same name but with a .caps extension. for filedo tr a-z A-Z < $file >$file.capsdone
The while Statement
Another iteration statement offered by the shell programming language is the while statement. This statement causes a block of code to be executed while a provided conditional expression is true. The syntax for the while statement is the following:

while expression

do

statements

done
The following is an example of the while statement. This program lists the parameters that were passed to the program, along with the parameter number. count=1while [ -n "$*" ]do echo "This is parameter number $count $1" shift count='expr $count + 1'done
As you will see later the shift command moves the command-line parameters over one to the left.
The until Statement
The until statement is very similar in syntax and function to the while statement. The only real difference between the two is that the until statement executes its code block while its conditional expression is false, and the while statement executes its code block while its conditional expression is true. The syntax for the until statement is:

until expression

do

commands

done
The same example that was used for the while statement can be used for the until statement. All you have to do to make it work is negate the condition. This is shown in the following code: count=1until [ -z "$*" ]do echo "This is parameter number $count $1" shift count='expr $count + 1'done
The only difference between this example and the while statement example is that the -n test command option (which means that the string has nonzero length) was removed, and the -z test option (which means that the string has zero length) was put in its place. In practice the until statement is not very useful, because any until statement you write can also be written as a while statement.
The shift Command

The shift command moves the current values stored in the positional parameters to the left one position. For example, if the values of the current positional parameters are:$1 = -r $2 = file1 $3 = file2 and you executed the shift command the resulting positional parameters would be as follows: $1 = file1 $2 = file2 You can also move the positional parameters over more than one place by specifying a number with the shift command. This is a very useful command when you have a shell program that needs to parse command-line options. This is true because options are typically preceded by a hyphen and a letter that indicates what the option is to be used for. Because options are usually processed in a loop of some kind, you often want to skip to the next positional parameter once you have identified which option should be coming next. For example, the following shell program expects two command-line options, one that specifies an input file and one that specifies an output file. The program reads the input file, translates all the characters in the input file into uppercase, then stores the results in the specified output file. while [ "$1" ]do if [ "$1" = "-i" ]; then infile="$2" shift 2 elif [ "$1" = "-o" ] then outfile="$2" shift 2 else echo "Program $0 does not recognize option $1" fidone tr a-z A-Z < $infile > $outfile
Functions
The shell languages enable you to define your own functions. These functions behave in much the same way as functions you define in C or other programming languages. The main advantage of using functions as opposed to writing all of your shell code in line is for organizational purposes. Code written using functions tends to be much easier to read and maintain and also tends to be smaller, because you can group common code into functions instead of putting it everywhere it is needed.
The syntax for creating a function in is the following:

fname ()

{ shell commands}
Once you have defined your function using one of these forms, you can invoke it by entering the following command: fname [parm1 parm2 parm3 ...]
Notice that you can pass any number of parameters to your function. When you do pass parameters to a function, it sees those parameters as positional parameters, just as a shell program does when you pass it parameters on the command line.
The following shell program contains several functions, each of which is performing a task associated with one of the command-line options. This example illustrates many of the topics covered in this section. It reads all the files that are passed on the command line and—depending on the option that was used—writes the files out in all uppercase letters, writes the files out in all lowercase letters, or prints the files. upper () { shift for i do tr a-z A-Z < $1 > $1.out rm $1 mv $1.out $1 shift done;} lower () { shift for i do tr A-Z a-z < $1 > $1.out rm $1 mv $1.out $1 shift done;} print () { shift for i do lpr $1 shift done;} usage_error () { echo "$1 syntax is $1 " echo "" echo "where option is one of the following" echo "p to print frame files" echo "u to save as uppercase" echo "l to save as lowercase"; } case $1in p -p) print $@;; u -u) upper $@;; l -l) lower $@;; *) usage_error $0;;esac;}

Tuesday, April 1, 2008

Conditional Expressions in Shell Scripting

A conditional expression is used with the [[ compound command to test attributes of files and to compare strings. Word splitting and filename generation are not performed on the words between [[ and ]]. See "The Test Statement" for details of [[.
Each expression can be constructed from one or more of the following unary or binary expressions:

Condition Description :

-a file True if file exists.
-b file True if file exists and is a block special file.
-c file True if file exists and is a character special file.
-d file True if file exists and is a directory.
-f file True if file exists and is an ordinary file.
-g file True if file exists and is has its setgid bit set.
-h file True if file exists and is a symbolic link.
-k file True if file exists and is has its sticky bit set.
-n string True if length of string is non-zero.
-o option True if option named option is on.
-p file True if file exists and is a fifo special file or a pipe.
-r file True if file exists and is readable by current process.
-s file True if file exists and has size greater than zero.
-t file_des True if file descriptor number file_des is open and associated with a terminal device.
-u file True if file exists and is has its setuid bit set.
-w file True if file exists and is writable by current process.
-x file True if file exists and is executable by current process. If file exists and is a directory, the current process has permission to search in the directory.
-z string True if length of string is zero.
-H file True if file exists and is a hidden directory
-L file True if file exists and is a symbolic link.
-O file True if file exists and is owned by the effective user ID of this process.
-G file True if file exists and its group matches the effective group ID of this process.
-S file True if file exists and is a socket.
file1 -nt file2 True if file1 exists and is newer than file2.
file1 -ot file2 True if file1 exists and is older than file2.
file1 -ef file2 True if file1 and file2 exist and refer to the same file.
str = pattern True if string, str, matches pattern. str != pattern True if string, str, does not match pattern.
str1 <> str2 True if str1 comes after str2 based on the ASCII value of their characters.
exp1 -eq exp2 True if exp1 is equal to exp2. exp1 -ne exp2 True if exp1 is not equal to exp2. exp1 -lt exp2 True if exp1 is less than exp2.
exp1 -gt exp2 True if exp1 is greater than exp2.
exp1 -le exp2 True if exp1 is less than or equal to exp2.
exp1 -ge exp2 True if exp1 is greater than or equal to exp2.
A compound expression can be constructed from these primitives by using any of the following, listed in decreasing order of precedence.

Compound Description

(expression) True, if expression is true. The round-brackets are used to group expressions. ! expression True if expression is false. expr1 && expr2 True, if expr1 and expr2 are both true. && means Logical-AND expr1 expr2 True, if either expr1 or expr2 is true. means Logical-OR
The Test Statement:
The syntax for the test statement should be in the form:
$ test c ..or.. $ [[ condition ]]
Where condition is one or more valid conditional statements. The syntax, [[ , represents the command word, 'test', but must always be closed with ]]. Because [[ is, in effect, a command word, it must be followed by at least one white-space character (space or TAB) and the closing brackets, ]] , must also be preceded by at least one white-space character.
The following test statement uses the Bourne shell compatible syntax.
$ [ condition ]
Note that the character, [, represents to command word, test and must be followed by at least one SPACE or TAB character. The closing square bracket must be preceded by at least one SPACE or TAB character.
The following test statement uses the Korn shell syntax.
$ [[ condition ]]
Again, that the characters, [[, represents to command word, test and must be followed by at least one SPACE or TAB character. The closing square brackets must be preceded by at least one SPACE or TAB character.
The AND and OR statements differ between the Bourne and the Korn shell syntax.
Bourne shell syntax:
$ [ condition1 -a condition2 ] # ( 1 AND 2) $ [ condition1 -o condition2 ] # ( 1 OR 2)
Korn shell syntax:
$ [[ condition1 && condition2 ]] # ( 1 AND 2) $ [[ condition1 condition2 ]] # ( 1 OR 2)
The -a and -o clauses can not be used inside the Korn shell-specific, [[...]], test statement.
Numeric Conditional Testing:
Numeric conditional testing can be carried out within double round-brackets. The following test statements are unique to the Korn shell.
$ ((condition))
The opening double round brackets need not be followed by a SPACE or TAB character. Nor do the closing double round brackets neet to be preceeded by a SPACE or TAB.
Numeric conditions can be:
Condition Description
((x==$y)) Test if x is equal to y. ((x!=$y)) Test if x is not equal to y. ((x>$y)) Test if x is greater than y. ((x<$y)) Test if x is less than y. ((x>=$y)) Test if x is greater than or equal to y. ((x<=$y)) Test if x is less than or equal to y. If numeric variables are used within the test condition, the variable name does not to be preceded by a dollar ($) character if the variable name is to the left of the equals (=) character. The following two test conditions would be treated as being identical: (($var==3)) Test if content of variable,var, is equal to 3. ((var==3)) Test if content of variable, var, is equal to 3. Note: Any special shell variable, such as # (number of arguments) or ? (return status value) do need to be preceeded by the $ character. (($#==3)) would be a valid test condition, whereas((#==3)) would not! Logic Testing (if, then, ..., fi): The logical state of a condition can be used to determine whether one or more command lines should be executed. The simplest form of locical branching is performed using the if conditional statement. The structure of an if statement can be any of the following. Note: The word "then" is a reserved word and must be treated as a command in its own right. If it is to appear on the same command line as the if statement, it must be preceded by a semi-colon (to separate the two commands). if condition_statement then ... series of commands to be executed ... fi is the same as if condition_statement ; then ... series of commands to be executed ... fi For readability, it is probably better to have the word, then, on a line of its own. if statements can be nested, as shown below. Each if must be closed by the command word fi (the word `if' reversed). if condition_statement then command line 1 command line 2 if condition_statement2 then commands_for_nested_if fi command line 3 command line 4 fi Logic Testing (if, then,... elif ... ,else ...): Alternative command sequences can be executed based on the TRUE or FALSE state of a condition test. Shown below are more forms for the if statement. if condition_statement then commands to be executed if condition is TRUE else commands to be executed if condition is FALSE fi Multiple conditions can be applied. For example: if condition_statement1 then commands to be executed if condition1 is TRUE elif condition_statement2 then commands to be executed if condition2 is TRUE elif condition_statement3 then commands to be executed if condition3 is TRUE else commands executed if preceding conditions are FALSE fi Note that each elif is followed by the separated command word, then. The entire statement is closed down by just one occurrence of the word fi. Any of the conditional statements, described on the preceding pages could be used where the words, condition_statement, appear in the examples shown above. Logic Testing (case ... esac): As an alternative to using if then ... fi, the Korn shell supports the use of the case statement. The syntax for the case statement is: case value in pattern1) commands to execute when value matches pattern1 ;; pattern2) commands to execute when value matches pattern2 ;; ... patternn) commands to execute when value matches patternn ;; esac Where value could be either: A variable's contents (substituted on the command line) or A single value returned by a command substitution. When value matches one of the patterns, no further attempt will be made to match any other patterns within the case statement. The pattern can be any shell pattern matching structure. Multiple, alternative patterns can be used when separated by the pipe () character. Pattern Matching within case ... esac: The patterns which can be used within case ... esac are: Pattern Description
abcd) The literal string of characters, `abcd'.
ab) The literal character, `a' or `b'.
a*) The letter `a' followed by zero or more (of any) characters
a*A*) The letter `a' or the letter `A' followed by zero or more occurrences of any character
[aA]*) The letter `a' or the letter `A' followed by zero or more occurrences of any character
[!aA]*) A string of 1 or more occurences of any character where the first character is not the letter `a' or `A'.
a????) A four-character string of characters where the first character is the letter `a'.
\)) A literal bracket `)'.
*) Any number (zero or more) of any character.
The last pattern, shown above, would normally be used as the last, "catch all", pattern match. For example:
case $var in
ab) echo "var contains the letter a or b" ;;
[AB]) echo "var contains the letter A or B" ;;
*) echo "var does not contain a, A, b or B" ;;
esac

Monday, March 31, 2008

Autosys - Job Scheduling

Autosys Job Management for UNIX

This guide will be helpfull in giving basic idea about what autosys is, defining jobs to AutoSys, monitoring and managing jobs.

Introduction to Autosys: AutoSys is an automated job control system for scheduling, monitoring, and reporting. These jobs can reside on any AutoSys-configured machine that is attached to a network.
An AutoSys job is any single command, executable, script, or Windows batch file. Each AutoSys job definition contains a variety of qualifying attributes, including the conditions specifying when and where a job should be run.


Defining Jobs :
There are the two methods you can use to create job definitions:
■ Using the AutoSys Graphical User Interface (GUI).
■ Using the AutoSys Job Information Language (JIL) through a command-line interface.

Autosys Jobs:

  • Job Types and Structure :
    There are three types of jobs: command, file watcher, and box.
    As their names imply, command jobs execute commands, box jobs are containers that hold other jobs (including other boxes), and file watcher jobs watch for the arrival of a specified file.
    In the AutoSys environment, the box job (or box) is a container of other jobs. A box job can be used to organize and control process flow. The box itself performs no actions, although it can trigger other jobs to run. An important feature of this type of job is that boxes can be put inside of other boxes.

Default Box Job Behavior:
Some important rules to remember about boxes are

  1. Jobs run only once per box execution.
  2. Jobs in a box will start only if the box itself is running.
  3. As long as any job in a box is running, the box remains in RUNNING state; the box cannot complete until all jobs have run.
  4. By default, a box will return a status of SUCCESS only when all the jobs in the box have run and the status of all the jobs is "success.
  5. By default, a box will return a status of FAILURE only when all jobs in the box have run and the status of one or more of the jobs is "failure."
  6. Unless otherwise specified, a box will run indefinitely until it reaches a status of SUCCESS or FAILURE.
  7. Changing the state of a box to INACTIVE (via the sendevent command) changes the state of all the jobs in the box to INACTIVE.

  • Job States and Status :
    AutoSys keeps track of the current state, or status, of every job. The value of a job’s status is used to determine when to start other jobs that are dependent on the job. The job status is displayed in the job report generated by the autorep command, and in the job report you can view in the Job Activity Console

Following are the status of Autosys jobs:

  1. INACTIVE : The job has not yet been processed. Either the job has never been run, or its status was intentionally altered to “turn off” its previous completion status
  2. ACTIVATED :The top-level box that this job is in is now in the RUNNING state, but the job itself has not started yet.
  3. STARTING : The event processor has initiated the start job procedure with the Remote Agent.
  4. RUNNING : The job is running. If the job is a box job, this value simply means that the jobs within the box may be started (other conditions permitting). If it is a command or file watcher job, the value means that the process is actually running on the remote machine.
  5. SUCCESS : The job exited with an exit code equal to or less than the “maximum exit code for success.” By default, only the exit code “0” is interpreted as “success.” If the job is a box job, this value means that all the jobs within the box have finished with the status SUCCESS (the default), or the “Exit Condition for Box Success” evaluated to true
  6. FAILURE : The job exited with an exit code greater than the “maximum exit code for success.” By default, any number greater than zero is interpreted as “failure.” AutoSys issues an alarm if a job fails
  7. TERMINATED : The job terminated while in the RUNNING state. A job can be terminated if a user sends a KILLJOB event or if it was defined to terminate if the box it is in failed. If the job itself fails, it has a FAILURE status, not a TERMINATED status. A job may also be terminated if it has exceeded the maximum run time (term_run_time attribute, if one was specified for the job), or if it was killed from the command line through a UNIX kill command. AutoSys issues an alarm if a job is terminated.
  8. RESTART : The job was unable to start due to hardware or application problems, and has been scheduled to restart.
  9. QUE_WAIT : The job can logically run (that is, all the starting conditions have been met), but there are not enough machine resources available.
  10. ON_HOLD : This job is on hold and will not be run until it receives the JOB_OFF_HOLD event.
  11. ON_ICE : This job is removed from all conditions and logic, but is still defined to AutoSys. Operationally, this condition is like deactivating the job. It will remain on ice until it receives the JOB_OFF_ICE event.


The difference between "on hold" and "on ice" is that when an "on hold" job is taken off hold, if its starting conditions are already satisfied, it will be scheduled to run, and it will run. On the other hand, if an "on ice" job is taken "off ice," it will not start, even if its starting conditions are already satisfied. This job will not run until its starting conditions reoccur.
The other major distinction is that jobs downstream from the job that is "on ice" will run as though the job succeeded. Whereas, all dependent jobs do not run when a job is on "on hold"—nothing downstream from this job will run.

  • Starting Parameters :
    AutoSys determines whether to start or not to start a job based on the evaluation of the starting conditions (or starting parameters) defined for the job. These conditions can be one or more of the following:
    ■ Date and time scheduling parameters are met (it is or has passed the specified date and time).
    ■ Starting Conditions specified in the job definition evaluate to true.
    ■ For jobs in a box, the box must be in the RUNNING state.
    ■ The current status of the job is not ON_HOLD or ON_ICE.
    Every time an event changes any of the above conditions, AutoSys finds all the jobs that may be affected by this change, and determines whether or not to start them.

sample jil code / Writing jil code:

jil = Job information language
When using JIL to create a job definition, you enter the jil command to display the JIL prompt.

Runs the Job Information Language (JIL) processor to add, update, and delete AutoSys jobs, machines, monitors, and reports. Also used to insert one-time job override definitions.

-------------------------------------------------------------------------------------------------

/* ----------------- SAP_UAT_MU03_C ----------------- */
insert_job: SAP_UAT_MU03_C job_type: c

command: /local/SAP/processCheckUAT.sh

machine: MU03-UAT

owner: admin@MU03-UAT

permission: gx,wx,mx,me

days_of_week: all

start_times: "15:00, 14:00"

description: "Job used for Run testing of process"

alarm_if_fail: 1

max_exit_success: 1
-------------------------------------------------------------------------------------------------
The above example is a simple jil code of autosys which we have to write if we want any particular job to be thru autosys.

To Insert a new JIL code :
issue command "jil"
bash-3.00$ jiljil>>1>
"The following prompt will appear" copy paste the jil code u have made example of jil code below...........
At the end the "C" or "B" determines if the job is box job or child job.
if the jil is inserted properly successfull message will come if any errors are there the jil code contains some errors..
if successfull exit;
2. Other way is to create a temp file named temp.jil which contains jil code on autosys server: give command.

bash-3.00$ jil<>

Functional Listing of AutoSys Commands :

This section lists which AutoSys commands to use for specific tasks. All commands are for both Windows and UNIX, unless otherwise specified.

  • Accessing Sybase : xql
  • Checking System Status : autoflags
    autoping
    autosyslog
    chase chk_auto_up
  • Converting cron to JIL (UNIX Only) : cron2jil
  • Defining AutoSys Jobs or Machines : jil
  • Defining Calendars : autocal, autocal_asc
  • Reporting Job Status : autorep
    autostatus
  • Starting AutoSys (UNIX Only) : eventor
  • Stopping AutoSys : sendevent

Using Autorep command:

Function
Reports information about a job, jobs within boxes, machines, and machine status. Also reports information about job overrides and global variables.
Syntax
autorep {-J job_name -M machine_name -G global_name} [-s -d -q -o over_num] [-r run_num]

autorep -J (job name here)

This will display a list of jobs with complete details with box/jobname, last/latest run date & time, status, exit code, etc.
Viewing JIL code for any Autosys job

autorep -J (job name here) -q

To obtain the underlying JIL (Job Interaction Language) source code for any Autosys job, run command:

To obtain the information of previous runs

autorep -J (job name here) -r (No of runs back) example : autorep -J (job name here) -r 1

would generate a report for the job run one runs back

-------------------------------------------------------------------------------------------------
Status Abbreviations
The following table lists the abbreviations used in the ST (status) column of the autorep report, and gives the status for each abbreviation.

AC - ACTIVATED
FA - FAILURE
IN - INACTIVE
OH - ON_HOLD
OI - ON_ICE
QU - QUE_WAIT
RE - RESTART
RU - RUNNING
ST - STARTING
SU - SUCCESS
TE - TERMINATED

-------------------------------------------------------------------------------------------------

sendevent:

sendevents to AutoSys for a variety of purposes, including starting or stopping AutoSys jobs, stopping the Event processor, and putting a job on hold. This command is also used to set AutoSys global variables or cancel a scheduled event.

sendevent is normally used with "-E" & -J option


-J job_name : Specifies the name of the job to which the specified event should be sent. This option is required for all events except STOP_DEMON, COMMENT, ALARM, or SET_GLOBAL

-E event :Specifies the event to be sent. This option is required. Any one of the following events may be specified:

  • STARTJOB
  • KILLJOB
  • DELETEJOB
  • FORCE_STARTJOB
  • JOB_ON_ICE
  • JOB_OFF_ICE
  • JOB_ON_HOLD
  • JOB_OFF_HOLD
  • CHANGE_STATUS
  • STOP_DEMON
  • CHANGE_PRIORITY
  • COMMENT
  • ALARM
  • SET_GLOBAL
  • SEND_SIGNAL

Following are the example of sendevent command frequently used.

____________________________________________________________________

To start or force start a job manually using sendevent :

sendevent –E FORCE_STARTJOB -J "Job Name Here"

sendevent -E STARTJOB -J "Job Name Here"

To put jobs on OFF ICE or ON ICE :

sendevent -E OFF_ICE -J "Job Name Here"

sendevent -E ON_ICE -J "Job Name Here"

______________________________________________________________

autostatus: Reports the current status of a specific job, or the value of an AutoSys global variable. Ex: autostatus -J job_name, -S instance

Perl Regular Expressions

Perl regular expressions quick start
This sections covers the very basics of understanding, creating and using regular expressions ('regexes') in Perl.

Simple word matching:

The simplest regex is simply a word, or more generally, a string of characters. A regex consisting of a word matches any string that contains that word:
"Hello World" =~ /World/; # matches
In this statement, World is a regex and the // enclosing /World/ tells perl to search a string for a match. The operator =~ associates the string with the regex match and produces a true value if the regex matched, or false if the regex did not match. In our case, World matches the second word in "Hello World", so the expression is true. This idea has several variations. Expressions like this are useful in conditionals:
print "It matches\n" if "Hello World" =~ /World/;
The sense of the match can be reversed by using !~ operator:
print "It doesn't match\n" if "Hello World" !~ /World/;
The literal string in the regex can be replaced by a variable:
$greeting = "World";
print "It matches\n" if "Hello World" =~ /$greeting/;
If you're matching against $_, the $_ =~ part can be omitted:
$_ = "Hello World";
print "It matches\n" if /World/;
Finally, the // default delimiters for a match can be changed to arbitrary delimiters by putting an 'm'
out front:
"Hello World" =~ m!World!; # matches, delimited by '!'
"Hello World" =~ m{World}; # matches, note the matching '{}'
"/usr/bin/perl" =~ m"/perl"; # matches after '/usr/bin',
# '/' becomes an ordinary char
Regexes must match a part of the string exactly in order for the statement to be true:
"Hello World" =~ /world/; # doesn't match, case sensitive
"Hello World" =~ /o W/; # matches, ' ' is an ordinary char
"Hello World" =~ /World /; # doesn't match, no ' ' at end
perl will always match at the earliest possible point in the string:
"Hello World" =~ /o/; # matches 'o' in 'Hello'
"That hat is red" =~ /hat/; # matches 'hat' in 'That'

Not all characters can be used 'as is' in a match. Some characters, called metacharacters, are
reserved for use in regex notation.The metacharacters are
{}[]()^$.*+?\
A metacharacter can be matched by putting a backslash before it:
"2+2=4" =~ /2+2/; # doesn't match, + is a metacharacter
"2+2=4" =~ /2\+2/; # matches, \+ is treated like an ordinary +
'C:\WIN32' =~ /C:\\WIN/; # matches
"/usr/bin/perl" =~ /\/usr\/bin\/perl/; # matches
In the last regex, the forward slash '/' is also backslashed,because it is used to delimit the regex.
Non-printable ASCII characters are represented by escape sequences.Common examples are \t for a tab, \n for a newline, and \r for a carriage return. Arbitrary bytes are represented by octal escape sequences, e.g., \033, or hexadecimal escape sequences, e.g., \x1B:
"1000\t2000" =~ m(0\t2) # matches
"cat" =~ /\143\x61\x74/ # matches, but a weird way to spell cat
Regexes are treated mostly as double quoted strings, so variable substitution works:
$foo = 'house';
'cathouse' =~ /cat$foo/; # matches
'housecat' =~ /${foo}cat/; # matches
With all of the regexes above, if the regex matched anywhere in the string, it was considered a match. To specify where it should match, we would use the anchor metacharacters ^ and $. The anchor ^ means match at the beginning of the string and the anchor $ means match at the end of the string, or before a newline at theend of the string. Some examples:
"housekeeper" =~ /keeper/; # matches
"housekeeper" =~ /^keeper/; # doesn't match
"housekeeper" =~ /keeper$/; # matches
"housekeeper\n" =~ /keeper$/; # matches
"housekeeper" =~ /^housekeeper$/; # matches

Using character classes:
A character class allows a set of possible characters, rather than just a single character, to match at a particular point in a regex. Character classes are denoted by brackets [...], with the set of characters to be possibly matched inside. Here are some examples:
/cat/; # matches 'cat'
/[bcr]at/; # matches 'bat', 'cat', or 'rat'
"abc" =~ /[cab]/; # matches 'a'
In the last statement, even though 'c' is the first character in the class, the earliest point at which the regex can match is 'a'.
/[yY][eE][sS]/; # match 'yes' in a case-insensitive way
# 'yes', 'Yes', 'YES', etc.
/yes/i; # also match 'yes' in a case-insensitive way
The last example shows a match with an 'i' modifier, which makes the match case-insensitive.

Character classes also have ordinary and special characters, but the sets of ordinary and special
characters inside a character class are different than those outside a character class. The special
characters for a character class are -]\^$ and are matched using an escape:
/[\]c]def/; # matches ']def' or 'cdef'
$x = 'bcr';
/[$x]at/; # matches 'bat, 'cat', or 'rat'
/[\$x]at/; # matches '$at' or 'xat'
/[\\$x]at/; # matches '\at', 'bat, 'cat', or 'rat'

The special character '-' acts as a range operator within character classes, so that the unwieldy
[0123456789] and [abc...xyz]become the svelte [0-9] and [a-z]:
/item[0-9]/; # matches 'item0' or ... or 'item9'
/[0-9a-fA-F]/; # matches a hexadecimal digit
If '-' is the first or last character in a character class, it is treated as an ordinary character.
The special character ^ in the first position of a character class denotes a negated character class, which matches any character but those in the brackets. Both [...] and [^...] must match a character, or the match fails. Then
/[^a]at/; # doesn't match 'aat' or 'at', but matches
# all other 'bat', 'cat, '0at', '%at', etc.
/[^0-9]/; # matches a non-numeric character
/[a^]at/; # matches 'aat' or '^at'; here '^' is ordinary
Perl has several abbreviations for common character classes:
\d is a digit and represents
[0-9]
\s is a whitespace character and represents
[\ \t\r\n\f]
\w is a word character (alphanumeric or _) and represents
[0-9a-zA-Z_]
\D is a negated \d; it represents any character but a digit
[^0-9]
\S is a negated \s; it represents any non-whitespace character
[^\s]
\W is a negated \w; it represents any non-word character
[^\w]
The period '.' matches any character but "\n"
The \d\s\w\D\S\W abbreviations can be used both inside and outside of character classes. Here are some in use:
/\d\d:\d\d:\d\d/; # matches a hh:mm:ss time format
/[\d\s]/; # matches any digit or whitespace character
/\w\W\w/; # matches a word char, followed by a
# non-word char, followed by a word char
/..rt/; # matches any two chars, followed by 'rt'
/end\./; # matches 'end.'
/end[.]/; # same thing, matches 'end.'
The word anchor \b matches a boundary between a word character and a non-word character
\w\W or \W\w:
$x = "Housecat catenates house and cat";
$x =~ /\bcat/; # matches cat in 'catenates'
$x =~ /cat\b/; # matches cat in 'housecat'
$x =~ /\bcat\b/; # matches 'cat' at end of string
In the last example, the end of the string is considered a word boundary.
Matching this or that
We can match different character strings with the alternation metacharacter ''. To match dog or
cat, we form the regex dogcat. As before, perl will try to match the regex at the earliest possible
point in the string. At each character position, perl will first try to match the first alternative, dog. If dog doesn't match, perl will then try the next alternative, cat. If cat doesn't match either, then the match fails and perl moves to the next position in the string. Some examples:
"cats and dogs" =~ /catdogbird/; # matches "cat"
"cats and dogs" =~ /dogcatbird/; # matches "cat"
Even though dog is the first alternative in the second regex, cat is able to match earlier in the string.
"cats" =~ /ccacatcats/; # matches "c"
"cats" =~ /catscatcac/; # matches "cats"
At a given character position, the first alternative that allows the regex match to succeed will be the one that matches. Here, all the alternatives match at the first string position, so the first matches.
Grouping things and hierarchical matching
The grouping metacharacters () allow a part of a regex to be treated as a single unit. Parts of a
regex are grouped by enclosingthem in parentheses. The regex house(catkeeper) means
match house followed by either cat or keeper. Some more examples are
/(ab)b/; # matches 'ab' or 'bb'
/(^ab)c/; # matches 'ac' at start of string or 'bc' anywhere
/house(cat)/; # matches either 'housecat' or 'house'
/house(cat(s))/; # matches either 'housecats' or 'housecat' or
# 'house'. Note groups can be nested.
"20" =~ /(1920)\d\d/; # matches the null alternative '()\d\d',
# because '20\d\d' can't match
Extracting matches
The grouping metacharacters () also allow the extraction of the parts of a string that matched. For each grouping, the part thatmatched inside goes into the special variables $1, $2, etc.They can be used just as ordinary variables:


# extract hours, minutes, seconds
$time =~ /(\d\d):(\d\d):(\d\d)/; # match hh:mm:ss format
$hours = $1;
$minutes = $2;
$seconds = $3;
In list context, a match /regex/ with groupings will return the list of matched values ($1,$2,...).
So we could rewrite it as
($hours, $minutes, $second) = ($time =~ /(\d\d):(\d\d):(\d\d)/);
If the groupings in a regex are nested, $1 gets the group with the leftmost opening parenthesis, $2 the next opening parenthesis, etc. For example, here is a complex regex and the matching variables indicated below it:
/(ab(cdef)((gi)j))/;
1 2 34
Associated with the matching variables $1, $2, ... are the backreferences \1, \2, ... Backreferences are matching variables that can be used inside a regex:
/(\w\w\w)\s\1/; # find sequences like 'the the' in string
$1, $2, ... should only be used outside of a regex, and \1, \2, ... only inside a regex.

Matching repetitions
The quantifier metacharacters ?, *, +, and {} allow us to determine the number of repeats of a
portion of a regex we consider to be a match. Quantifiers are put immediately after the character, character class, or grouping that we want to specify. Theyhave the following meanings:
a? = match 'a' 1 or 0 times
a* = match 'a' 0 or more times, i.e., any number of times
a+ = match 'a' 1 or more times, i.e., at least once
a{n,m} = match at least n times, but not more than m times.
a{n,} = match at least n or more times
a{n} = match exactly n times
Here are some examples:
/[a-z]+\s+\d*/; # match a lowercase word, at least some space, and
# any number of digits
/(\w+)\s+\1/; # match doubled words of arbitrary length
$year =~ /\d{2,4}/; # make sure year is at least 2 but not more
# than 4 digits
$year =~ /\d{4}\d{2}/; # better match; throw out 3 digit dates
These quantifiers will try to match as much of the string as possible, while still allowing the regex to match. So we have
$x = 'the cat in the hat';
$x =~ /^(.*)(at)(.*)$/; # matches,
$1 = 'the cat in the h'
# $2 = 'at'


The first quantifier .* grabs as much of the string as possible while still having the regex match. The
second quantifier .* hasno string left to it, so it matches 0 times.
More matching
There are a few more things you might want to know about matching operators. In the code
$pattern = 'Seuss';
while (<>) {
print if /$pattern/;
}
perl has to re-evaluate $pattern each time through the loop. If $pattern won't be changing, use
the //o modifier, to only perform variable substitutions once. If you don't want any substitutions at all, use the special delimiter m'':
@pattern = ('Seuss');
m/@pattern/; # matches 'Seuss'
m'@pattern'; # matches the literal string '@pattern'
The global modifier //g allows the matching operator to match within a string as many times as
possible. In scalar context, successive matches against a string will have //g jump from match to
match, keeping track of position in the string as it goes along.You can get or set the position with the
pos() function. For example,
$x = "cat dog house"; # 3 words
while ($x =~ /(\w+)/g) {
print "Word is $1, ends at position ", pos $x, "\n";
}
prints
Word is cat, ends at position 3
Word is dog, ends at position 7
Word is house, ends at position 13
A failed match or changing the target string resets the position. Ifyou don't want the position reset after failure to match, add the //c, as in /regex/gc.
In list context, //g returns a list of matched groupings, or if there are no groupings, a list of matches to the whole regex. So
@words = ($x =~ /(\w+)/g); # matches,
# $word[0] = 'cat'
# $word[1] = 'dog'
# $word[2] = 'house'
Search and replace
Search and replace is performed using s/regex/replacement/modifiers.The replacement is
a Perl double quoted string that replaces in the string whatever is matched with the regex. The
operator =~ is also used here to associate a string with s///. If matching against $_, the $_ =~ can be dropped. If there is a match, s/// returns the number of substitutions made, otherwise it returns false. Here are a few examples:
$x = "Time to feed the cat!";

$x =~ s/cat/hacker/; # $x contains "Time to feed the hacker!"
$y = "'quoted words'";
$y =~ s/^'(.*)'$/$1/; # strip single quotes,
# $y contains "quoted words"
With the s/// operator, the matched variables $1, $2, etc. are immediately available for use in the replacement expression. With the global modifier, s///g will search and replace all currences of the regex in the string:
$x = "I batted 4 for 4";
$x =~ s/4/four/; # $x contains "I batted four for 4"
$x = "I batted 4 for 4";
$x =~ s/4/four/g; # $x contains "I batted four for four"
The evaluation modifier s///e wraps an eval{...} around the replacement string and the
evaluated result is substituted for thematched substring. Some examples:
# reverse all the words in a string
$x = "the cat in the hat";
$x =~ s/(\w+)/reverse $1/ge; # $x contains "eht tac ni eht tah"
# convert percentage to decimal
$x = "A 39% hit rate";
$x =~ s!(\d+)%!$1/100!e; # $x contains "A 0.39 hit rate"
The last example shows that s/// can use other delimiters, such as s!!! and s{}{}, and even
s{}//. If single quotes are used s''', then the regex and replacement are treated as single quoted
strings.
The split operator split /regex/, string splits string into a list of substringsand returns that list. The regex determines the character sequence that string is split with respect to. For example, to split a string into words, use
$x = "Calvin and Hobbes";
@word = split /\s+/, $x; # $word[0] = 'Calvin'
# $word[1] = 'and'
# $word[2] = 'Hobbes'
To extract a comma-delimited list of numbers, use
$x = "1.618,2.718, 3.142";
@const = split /,\s*/, $x; # $const[0] = '1.618'
# $const[1] = '2.718'
# $const[2] = '3.142'
If the empty regex // is used, the string is split into individual characters. If the regex has groupings, then the list produced contains the matched substrings from the groupings as well:
$x = "/usr/bin";
@parts = split m!(/)!, $x; # $parts[0] = ''
# $parts[1] = '/'
# $parts[2] = 'usr'
# $parts[3] = '/'
# $parts[4] = 'bin'

Since the first character of $x matched the regex, split prependedan empty initial element to the
list.

About Me

This site will be helpful for Unix system administrator to automate their tasks using scripts, comments on posts are welcome.