Scripting 101: Monitoring Autosys Jobs - Evolved Form

Finally, here it is! I have this tested last night - on a PROD server [Linux only for some commands might NOT run on Solaris] - and it went well. After making some adjustments, it did behave as expected. Wow, I can't believe it. After a long while, I did it. Not of course without the valuable insights from my mentors - CJ and Mors - who gave their all [I hope!?]. I am a student and beginner in scripting. As for now, I'm still analyzing the logs to maximize performance for I have seen that it wastes so much time in loops. So anyone out there, please help me!

Here are some of what I've observed and future plans:

1. Run via cron; OR run it via "screen"

2. High CPU utilization

3. Integrate mutt in the script [not sure where but I'll try; it's a bit cool, ain't it?]

4. Instead of having an exclusion file, after a job is done, it will remove from the JOBLIST.txt and exit when nothing to read.

**********

#! /bin/bash

# Calling AUTOSYS environment
. /sbcimp/shared/config/CA/Autosys/v4.0/prod/autouser/shellPLN

# Definition and initialization of VARIABLES.
E_NOTFOUND=66
E_MISSING=43
E_DONE=0
EMAIL_ADD=`whoami`
COUNTER=0
NO_JOBS=`cat JOBLIST.txt | wc -l`


# Initialization of EXCLUSION file; this will be used to exclude successful job

> EXCLUDEJOB.txt
/bin/touch EXCLUDEJOB.txt
if [ ! -w EXCLUDEJOB.txt ]
then
/bin/echo "EXCLUDEJOB.txt file does not exist and/or not writeable! Check write permissions on the directory." | /bin/mail -s "EXCLUDEJOB.txt Missing" $EMAIL_ADD
exit $E_NOTFOUND
fi


# This is a mailer function for successful jobs; once the job ended successfully, it will write to EXCLUDEJOB.txt before sending a mail
SU_MAILER() {
/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | awk '{sub (/:/,""); print $1$2$3}' >> EXCLUDEJOB.txt

/bin/mail -s "FOS Job Monitor Update" $EMAIL_ADD >>EOF

****** Feed Status ******

Job ended SU-ccessfully!

$(/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -d -J $JOBNAME)

***** End of Status *****

EOF

} # End of SU_MAILER function

FA_MAILER() {

/bin/mail -s "FOS Job Monitor Alert" $EMAIL_ADD >> EOF

***** PLEASE check Job Status *****

$(/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -d -J $JOBNAME)

$(/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -q -J $JOBNAME)

***** End of Status ****

EOF

} # End of FA_MAILER function

JOB_CHECKER() {
DONE_JOB=`grep $JOBNAME EXCLUDEJOB.txt`
ONQUEUE_JOB=`/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | awk '{sub (/:/,""); print $1$2$3}'`

if [ "$DONE_JOB" = "$ONQUEUE_JOB" ]
then
continue
else
JOB_TIMESTAMP=`/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | awk '{sub (/:/,""); print $3}'`

if [ "$DATE_STAMP" = "$JOB_DATE" ] && [ "$TIME_STAMP" -ge "$JOB_TIMESTAMP" ]
then
if /sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | grep -w SU 2>&1 > /dev/null
then

SU_MAILER
COUNTER=$[$COUNTER+1]

elif /sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | egrep -w "ST|AC|RU" 2>&1 > /dev/null
then
break
else

FA_MAILER
COUNTER=$[$COUNTER+1]

fi
fi
fi
} # End of JOB_CHECKER function

# This is basically the main function of the script; it will continue to run until all jobs listed in the JOBLIST.txt are successful
while [ "$COUNTER" -lt "$NO_JOBS" ]
do
for JOBNAME in `cat JOBLIST.txt`
do
JOB_TZ=`/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME -q | grep timezone | awk '{print $2}' | cut -d/ -f2`

if [ -z "$JOB_TZ" ]
then
/bin/echo "Timezone is NOT defined in $JOBNAME properties. Please check." | /bin/mail -s "Missing TIMEZONE" $EMAIL_ADD
continue
else

JOB_DATE=`/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | awk '{print $2}'`

case $JOB_TZ in
London)
DATE_STAMP=`TZ=GMT date +%m/%d/%Y`
TIME_STAMP=`TZ=GMT date +%X | cut -d: -f1-2 | awk '{sub (/:/,""); print}'`;;

Zurich)
DATE_STAMP=`TZ=GMT-1 date +%m/%d/%Y`
TIME_STAMP=`TZ=GMT-1 date +%X | cut -d: -f1-2 | awk '{sub (/:/,""); print}'`;;

Eastern)
DATE_STAMP=`TZ=GMT+5 date +%m/%d/%Y`
TIME_STAMP=`TZ=GMT+5 date +%X | cut -d: -f1-2 | awk '{sub (/:/,""); print}'`;;

Singapore|HongKong)
DATE_STAMP=`TZ=GMT-8 date +%m/%d/%Y`
TIME_STAMP=`TZ=GMT-8 date +%X | cut -d: -f1-2 | awk '{sub (/:/,""); print}'`;;

Tokyo)
DATE_STAMP=`TZ=GMT-9 date +%m/%d/%Y`
TIME_STAMP=`TZ=GMT-9 date +%X | cut -d: -f1-2 | awk '{sub (/:/,""); print}'`;;

Sydney)
DATE_STAMP=`TZ=GMT-11 date +%m/%d/%Y`
TIME_STAMP=`TZ=GMT-11 date +%X | cut -d: -f1-2 | awk '{sub (/:/,""); print}'`;;

*)
/bin/echo "What else could go wrong? Notify the script owner... Stressful." | /bin/mail -s "FOlSe Alert. Something is missing..." $EMAIL_ADD
exit $E_MISSING;;
esac
fi

JOB_CHECKER

done # End of FOR-loop

# Sleep will be added here for about 5 minutes

sleep 300

done # End of WHILE-loop

exit $E_DONE