Mabuhay

Hello world! This is it. I've always wanted to blog. I don't want no fame but just to let myself heard. No! Just to express myself. So, I don't really care if someone believes in what I'm going to write here nor if ever someone gets interested reading it. My blogs may be a novel-like, a one-liner, it doesn't matter. Still, I'm willing to listen to your views, as long as it justifies mine... Well, enjoy your stay, and I hope you'll learn something new because I just did and sharing it with you.. Welcome!
Showing posts with label function. Show all posts
Showing posts with label function. Show all posts

Wednesday, November 18, 2009

Perl-icious: Function that reads from a config file

I was asked by my colleague to make a function he'll use for one of our scripts - to make our lives easier which I don't think won't happen unless THEY get rid of the night effin' shift! Anyway, it took me sometime to figure this out (I'm learning ok?!) and then I sought the help of a resident guru BE (his initial you idiot!).

Background:

I was to read the NetAgent configuration file and determine if the server is an MDH or P2PS (this is about RMDS). Another was from the RMDS config file to do the same, which means I got to make two.

Apologies but I cannot post config files here. But I believe you can easily understand how it process things.

Here is the code - as a standalone:

sub inNetAgentConfig {

my $host = shift;
my $conf_file = "/path/to/config.file";

open FH, $conf_file or die "Error: $!\n";
$/ = "ATTRIBUTES"; # set the line delimiter to ATTRIBUTES instead of default value of newline; see special variables
my @array = <FH>;

if ( grep {/$host/} $conf_file ) {
foreach my $line ( grep {/$host/} @array ) {
if ( $line =~ /GROUP=(\w+)/ ) {
return $1;
}
}
} else {
return "Non-existent";
}
}


Another function that I made:


sub inRmdsConfig {

my $host = shift;
my $rmds_file = "/path/to/config.file";

open FILE, $rmds_file or die "Error: $!\n";
while (<FILE>) {
chomp ( $_ );

if ( $_ =~ /^$host\*(\w+)\*serverId.?/ ) {
return $1;
} else {
return "Non-existent";
}
}

Wednesday, April 01, 2009

Scripting 101: Monitoring Autosys Jobs - Changes

When I got to the office today, I started working on the planned changes for the script. 'Though not all, I was able to find a way to implement number 4 from my early list. So here it goes.

Let me establish first the fact that this is primarily designed to address the current setup of jobs that we're monitoring. We already know when it'll be executed so the list is pre-arranged according to time of execution. From which I have considered the changes that I did. And as always, this is a work in progress so bear with me.

First change that I did was to add a couple of lines in the JOB_CHECKER function after the SU_MAILER.

cp -p JOBLIST.txt TEMP.out
sed '1d' TEMP.out > JOBLIST.txt

It deletes job 1 from the JOBLIST.txt as not to waste time when doing a FOR-loop for jobs already marked as SU. It's kinda lame really but am looking for better way to cleanup this mess.

The second one is changing the condition in JOB_CHECKER function.

if [ "$DONE_JOB" = "$ONQUEUE_JOB" ]

was changed to [and DONE_JOB is removed]:

if grep -w $ONQUEUE_JOB EXCLUDEJOB.txt

Third is on WHILE-loop. The script exits when there's nothing left [to read] in the JOBLIST.txt. Hence, COUNTER is no longer needed.

while [ "$NO_JOBS" -gt "0" ]
...

It's still consuming a lot of CPU and I haven't figure this one out yet. To allow "others", after a FOR-loop, I put the script into sleep.

And lastly, a safety net was added in the SU_MAILER function. It'll erase duplicate entries from the exclusion list.

SU_MAILER() {
/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | awk '{sub (/:/,""); print $1$2$3}' >> EXCLUDEJOB.txt

awk '!x[$0]++' EXCLUDEJOB.txt > EXCLUDEJOB.txt

I hope I'm on track on improving my scripting skills; always looking forward for a new challenge.

********
This part was added on 15 April, 2009 at 21:33 SST.

The variable $NO_JOBS was removed. Instead, its value was placed in the WHILE-loop. It was late when I found out that it is NOT being updated. Also, there are times when some of the jobs listed may be invalid for one reason or another. With this, condition was added as well. So the code will be:

while [ `cat JOBLIST.txt | wc -l` -gt "0" ]
do
for JOBNAME in `cat JOBLIST.txt`
do

if /sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep "Invalid Job Name" 2>&1 > /dev/null
then
cp -p JOBLIST.txt VALJOB.out
sed -e "\|^$JOBNAME\$|d" VALJOB.out > JOBLIST.txt
continue
fi

...
done

exit $E_DONE

SU_MAILER () is still in the works. I still can't find how to delete duplicate entries while leaving one of them. The "awk" part that previously used was scrapped. It's NOT up to the job. Of course, it's my fault. ;)

I really hope I'll have the flame to continue on this for I started on my new job and is being "forced" to take on Perl which seems great. Now I know that it ain't a scripting language but rather on the grey area between interpreted and compiled. Still a long wayyyyyy. See'ya round.

Scripting 101: Monitoring Autosys Jobs - Evolved Form

Finally, here it is! I have this tested last night - on a PROD server [Linux only for some commands might NOT run on Solaris] - and it went well. After making some adjustments, it did behave as expected. Wow, I can't believe it. After a long while, I did it. Not of course without the valuable insights from my mentors - CJ and Mors - who gave their all [I hope!?]. I am a student and beginner in scripting. As for now, I'm still analyzing the logs to maximize performance for I have seen that it wastes so much time in loops. So anyone out there, please help me!

Here are some of what I've observed and future plans:
1. Run via cron; OR run it via "screen"
2. High CPU utilization
3. Integrate mutt in the script [not sure where but I'll try; it's a bit cool, ain't it?]
4. Instead of having an exclusion file, after a job is done, it will remove from the JOBLIST.txt and exit when nothing to read.

**********
#! /bin/bash

# Calling AUTOSYS environment
. /sbcimp/shared/config/CA/Autosys/v4.0/prod/autouser/shellPLN

# Definition and initialization of VARIABLES.
E_NOTFOUND=66
E_MISSING=43
E_DONE=0
EMAIL_ADD=`whoami`
COUNTER=0
NO_JOBS=`cat JOBLIST.txt | wc -l`


# Initialization of EXCLUSION file; this will be used to exclude successful job
> EXCLUDEJOB.txt
/bin/touch EXCLUDEJOB.txt
if [ ! -w EXCLUDEJOB.txt ]
then
/bin/echo "EXCLUDEJOB.txt file does not exist and/or not writeable! Check write permissions on the directory." | /bin/mail -s "EXCLUDEJOB.txt Missing" $EMAIL_ADD
exit $E_NOTFOUND
fi


# This is a mailer function for successful jobs; once the job ended successfully, it will write to EXCLUDEJOB.txt before sending a mail
SU_MAILER() {
/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | awk '{sub (/:/,""); print $1$2$3}' >> EXCLUDEJOB.txt

/bin/mail -s "FOS Job Monitor Update" $EMAIL_ADD >>EOF
****** Feed Status ******

Job ended SU-ccessfully!
$(/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -d -J $JOBNAME)

***** End of Status *****
EOF
} # End of SU_MAILER function


FA_MAILER() {
/bin/mail -s "FOS Job Monitor Alert" $EMAIL_ADD >> EOF
***** PLEASE check Job Status *****

$(/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -d -J $JOBNAME)
$(/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -q -J $JOBNAME)

***** End of Status ****
EOF
} # End of FA_MAILER function


JOB_CHECKER() {
DONE_JOB=`grep $JOBNAME EXCLUDEJOB.txt`
ONQUEUE_JOB=`/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | awk '{sub (/:/,""); print $1$2$3}'`

if [ "$DONE_JOB" = "$ONQUEUE_JOB" ]
then
continue
else
JOB_TIMESTAMP=`/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | awk '{sub (/:/,""); print $3}'`

if [ "$DATE_STAMP" = "$JOB_DATE" ] && [ "$TIME_STAMP" -ge "$JOB_TIMESTAMP" ]
then
if /sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | grep -w SU 2>&1 > /dev/null
then

SU_MAILER
COUNTER=$[$COUNTER+1]

elif /sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | egrep -w "ST|AC|RU" 2>&1 > /dev/null
then
break
else

FA_MAILER
COUNTER=$[$COUNTER+1]

fi
fi
fi
}
# End of JOB_CHECKER function

# This is basically the main function of the script; it will continue to run until all jobs listed in the JOBLIST.txt are successful

while [ "$COUNTER" -lt "$NO_JOBS" ]
do
for JOBNAME in `cat JOBLIST.txt`
do
JOB_TZ=`/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME -q | grep timezone | awk '{print $2}' | cut -d/ -f2`

if [ -z "$JOB_TZ" ]
then
/bin/echo "Timezone is NOT defined in $JOBNAME properties. Please check." | /bin/mail -s "Missing TIMEZONE" $EMAIL_ADD
continue
else

JOB_DATE=`/sbcimp/run/tp/CA/Autosys/v4.0/bin/autorep -J $JOBNAME | grep $JOBNAME | awk '{print $2}'`

case $JOB_TZ in
London)
DATE_STAMP=`TZ=GMT date +%m/%d/%Y`
TIME_STAMP=`TZ=GMT date +%X | cut -d: -f1-2 | awk '{sub (/:/,""); print}'`;;

Zurich)
DATE_STAMP=`TZ=GMT-1 date +%m/%d/%Y`
TIME_STAMP=`TZ=GMT-1 date +%X | cut -d: -f1-2 | awk '{sub (/:/,""); print}'`;;

Eastern)
DATE_STAMP=`TZ=GMT+5 date +%m/%d/%Y`
TIME_STAMP=`TZ=GMT+5 date +%X | cut -d: -f1-2 | awk '{sub (/:/,""); print}'`;;

Singapore|HongKong)
DATE_STAMP=`TZ=GMT-8 date +%m/%d/%Y`
TIME_STAMP=`TZ=GMT-8 date +%X | cut -d: -f1-2 | awk '{sub (/:/,""); print}'`;;

Tokyo)
DATE_STAMP=`TZ=GMT-9 date +%m/%d/%Y`
TIME_STAMP=`TZ=GMT-9 date +%X | cut -d: -f1-2 | awk '{sub (/:/,""); print}'`;;

Sydney)
DATE_STAMP=`TZ=GMT-11 date +%m/%d/%Y`
TIME_STAMP=`TZ=GMT-11 date +%X | cut -d: -f1-2 | awk '{sub (/:/,""); print}'`;;

*)
/bin/echo "What else could go wrong? Notify the script owner... Stressful." | /bin/mail -s "FOlSe Alert. Something is missing..." $EMAIL_ADD
exit $E_MISSING;;
esac
fi

JOB_CHECKER

done # End of FOR-loop

# Sleep will be added here for about 5 minutes
sleep 300

done
# End of WHILE-loop

exit $E_DONE

World Clock