Mabuhay

Hello world! This is it. I've always wanted to blog. I don't want no fame but just to let myself heard. No! Just to express myself. So, I don't really care if someone believes in what I'm going to write here nor if ever someone gets interested reading it. My blogs may be a novel-like, a one-liner, it doesn't matter. Still, I'm willing to listen to your views, as long as it justifies mine... Well, enjoy your stay, and I hope you'll learn something new because I just did and sharing it with you.. Welcome!

Monday, August 18, 2008

Fibre channel host port is OFFLINE

This may not be your regular error but this is quite useful especially for those who have SAN implementation in their infra [SC21948063]. When we did receive a call from the customer[?], we were compelled to investigate on the raised issue. Basically, I myself, ain't familiar with this. But since this calls for investigation, I accepted the ticket without second thought. Ahhh, ticket! Well I checked first if the box went on a maintenance. So, I did: `uptime`, `date`, `who -r`. Then I came to check for the syslog [HP-UX]; and HW scan [`ioscan -fnCfc`]. Well [why do I use this word too often?!? Well, I don't know either!], there are interesting entries that I found.

[root@server001:/home/user001]
# uptime
11:22am up 323 days, 15:45, 3 users, load average: 0.51, 0.37, 0.31

[root@server001:/home/user001]
# date
Mon Aug 18 11:22:05 EDT 2008

[root@server001:/home/user001]
# who -r
. run-level 4 Sep 29 19:37 4 0 S

[root@server001:/root]
# cat /home/user001/fc_20080818.log
Info for /dev/fcd0

Vendor ID is = 0x001077
Device ID is = 0x002312
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x0012ba
PCI Mode = PCI-X 133 MHz
ISP Code version = 3.2.162
ISP Chip version = 3
Topology = PTTOPT_FABRIC
Link Speed = 2Gb
Local N_Port_id is = 0x454a00
Previous N_Port_id is = 0x454a00
N_Port Node World Wide Name = 0x50060b00003966a5
N_Port Port World Wide Name = 0x50060b00003966a4
Switch Port World Wide Name = 0x204a006069e2147e
Switch Node World Wide Name = 0x1000006069e2147e
Driver state = ONLINE
Hardware Path is = 0/2/1/0
Maximum Frame Size = 2048
Driver-Firmware Dump Available = NO
Driver-Firmware Dump Timestamp = N/A
Driver Version = @(#) libfcd.a HP Fibre Channel ISP 23xx Driver B.11.11.01

/ux/kern/kisu/FCD/src/common/wsio/fcd_init.c:Jul 16 2003,18:50:14

Info for /dev/fcd1

Vendor ID is = 0x001077
Device ID is = 0x002312
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x0012ba
PCI Mode = PCI-X 133 MHz
ISP Code version = 3.2.162
ISP Chip version = 3
Previous Topology = UNINITIALIZED
Link Speed = UNKNOWN
Local N_Port_id is = None
Previous N_Port_id is = None
N_Port Node World Wide Name = 0x50060b00003966a7
N_Port Port World Wide Name = 0x50060b00003966a6
Switch Port World Wide Name = 0x0000000000000000
Switch Node World Wide Name = 0x0000000000000000
Driver state = AWAITING_LINK_UP
Hardware Path is = 0/2/1/1
Maximum Frame Size = 2048
Driver-Firmware Dump Available = NO
Driver-Firmware Dump Timestamp = N/A
Driver Version = @(#) libfcd.a HP Fibre Channel ISP 23xx Driver B.11.11.01

/ux/kern/kisu/FCD/src/common/wsio/fcd_init.c:Jul 16 2003,18:50:14

Info for /dev/fcd2

Vendor ID is = 0x001077
Device ID is = 0x002312
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x0012ba
PCI Mode = PCI-X 66 MHz
ISP Code version = 3.2.162
ISP Chip version = 3
Topology = PTTOPT_FABRIC
Link Speed = 2Gb
Local N_Port_id is = 0x464a00
Previous N_Port_id is = None
N_Port Node World Wide Name = 0x50060b00003966a9
N_Port Port World Wide Name = 0x50060b00003966a8
Switch Port World Wide Name = 0x204a006069e2145e
Switch Node World Wide Name = 0x1000006069e2145e
Driver state = ONLINE
Hardware Path is = 0/4/2/0
Maximum Frame Size = 2048
Driver-Firmware Dump Available = NO
Driver-Firmware Dump Timestamp = N/A
Driver Version = @(#) libfcd.a HP Fibre Channel ISP 23xx Driver B.11.11.01

/ux/kern/kisu/FCD/src/common/wsio/fcd_init.c:Jul 16 2003,18:50:14

Info for /dev/fcd3

Vendor ID is = 0x001077
Device ID is = 0x002312
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x0012ba
PCI Mode = PCI-X 66 MHz
ISP Code version = 3.2.162
ISP Chip version = 3
Previous Topology = UNINITIALIZED
Link Speed = UNKNOWN
Local N_Port_id is = None
Previous N_Port_id is = None
N_Port Node World Wide Name = 0x50060b00003966ab
N_Port Port World Wide Name = 0x50060b00003966aa
Switch Port World Wide Name = 0x0000000000000000
Switch Node World Wide Name = 0x0000000000000000
Driver state = AWAITING_LINK_UP
Hardware Path is = 0/4/2/1
Maximum Frame Size = 2048
Driver-Firmware Dump Available = NO
Driver-Firmware Dump Timestamp = N/A
Driver Version = @(#) libfcd.a HP Fibre Channel ISP 23xx Driver B.11.11.01

/ux/kern/kisu/FCD/src/common/wsio/fcd_init.c:Jul 16 2003,18:50:14

Info for /dev/fcd4

Vendor ID is = 0x001077
Device ID is = 0x002312
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x0012ba
PCI Mode = PCI-X 66 MHz
ISP Code version = 3.2.162
ISP Chip version = 3
Previous Topology = UNINITIALIZED
Link Speed = UNKNOWN
Local N_Port_id is = None
Previous N_Port_id is = None
N_Port Node World Wide Name = 0x50060b0000396885
N_Port Port World Wide Name = 0x50060b0000396884
Switch Port World Wide Name = 0x0000000000000000
Switch Node World Wide Name = 0x0000000000000000
Driver state = AWAITING_LINK_UP
Hardware Path is = 0/5/2/0
Maximum Frame Size = 2048
Driver-Firmware Dump Available = NO
Driver-Firmware Dump Timestamp = N/A
Driver Version = @(#) libfcd.a HP Fibre Channel ISP 23xx Driver B.11.11.01

/ux/kern/kisu/FCD/src/common/wsio/fcd_init.c:Jul 16 2003,18:50:14

Info for /dev/fcd5

Vendor ID is = 0x001077
Device ID is = 0x002312
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x0012ba
PCI Mode = PCI-X 66 MHz
ISP Code version = 3.2.162
ISP Chip version = 3
Topology = PTTOPT_FABRIC
Link Speed = 2Gb
Local N_Port_id is = 0x686900
Previous N_Port_id is = None
N_Port Node World Wide Name = 0x50060b0000396887
N_Port Port World Wide Name = 0x50060b0000396886
Switch Port World Wide Name = 0x2069006069e207b2
Switch Node World Wide Name = 0x1000006069e207b2
Driver state = ONLINE
Hardware Path is = 0/5/2/1
Maximum Frame Size = 2048
Driver-Firmware Dump Available = NO
Driver-Firmware Dump Timestamp = N/A
Driver Version = @(#) libfcd.a HP Fibre Channel ISP 23xx Driver B.11.11.01

/ux/kern/kisu/FCD/src/common/wsio/fcd_init.c:Jul 16 2003,18:50:14


[root@server001:/home/user001]
# cat syslog.out
Aug 17 14:00:02 server001 vmunix: 0/2/1/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:02 server001 vmunix: 0/2/1/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:02 server001 vmunix: 0/4/2/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:02 server001 vmunix: 0/4/2/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:02 server001 vmunix: 0/5/2/0: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:02 server001 vmunix: 0/5/2/0: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:06 server001 EMS [4474]: ------ EMS Event Notification ------ Value: "CRITICAL (5)" for Resource:

"/adapters/events/ql_adapter/0_2_1_1" (Threshold: >= " 3") Execute the following command to obtain event details:

/opt/resmon/bin/resdata -R 293208071 -r /adapters/events/ql_adapter/0_2_1_1 -n 293208079 -a
Aug 17 14:00:06 server001 EMS [4474]: ------ EMS Event Notification ------ Value: "CRITICAL (5)" for Resource:

"/adapters/events/ql_adapter/0_2_1_1" (Threshold: >= " 3") Execute the following command to obtain event details:

/opt/resmon/bin/resdata -R 293208071 -r /adapters/events/ql_adapter/0_2_1_1 -n 293208079 -a
Aug 17 14:00:07 server001 EMS [4474]: ------ EMS Event Notification ------ Value: "CRITICAL (5)" for Resource:

"/adapters/events/ql_adapter/0_4_2_1" (Threshold: >= " 3") Execute the following command to obtain event details:

/opt/resmon/bin/resdata -R 293208087 -r /adapters/events/ql_adapter/0_4_2_1 -n 293208080 -a
Aug 17 14:00:07 server001 EMS [4474]: ------ EMS Event Notification ------ Value: "CRITICAL (5)" for Resource:

"/adapters/events/ql_adapter/0_4_2_1" (Threshold: >= " 3") Execute the following command to obtain event details:

/opt/resmon/bin/resdata -R 293208087 -r /adapters/events/ql_adapter/0_4_2_1 -n 293208080 -a
Aug 17 14:00:07 server001 EMS [4474]: ------ EMS Event Notification ------ Value: "CRITICAL (5)" for Resource:

"/adapters/events/ql_adapter/0_5_2_0" (Threshold: >= " 3") Execute the following command to obtain event details:

/opt/resmon/bin/resdata -R 293208092 -r /adapters/events/ql_adapter/0_5_2_0 -n 293208081 -a
Aug 17 14:00:07 server001 EMS [4474]: ------ EMS Event Notification ------ Value: "CRITICAL (5)" for Resource:

"/adapters/events/ql_adapter/0_5_2_0" (Threshold: >= " 3") Execute the following command to obtain event details:

/opt/resmon/bin/resdata -R 293208092 -r /adapters/events/ql_adapter/0_5_2_0 -n 293208081 -a
Aug 17 14:00:10 server001 vmunix: 0/2/1/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:10 server001 vmunix: 0/2/1/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:10 server001 vmunix: 0/5/2/0: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:10 server001 vmunix: 0/5/2/0: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:10 server001 vmunix: 0/4/2/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:10 server001 vmunix: 0/4/2/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:14 server001 vmunix: 0/2/1/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:14 server001 vmunix: 0/2/1/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:14 server001 vmunix: 0/4/2/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:14 server001 vmunix: 0/4/2/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:14 server001 vmunix: 0/5/2/0: Fibre channel host port is OFFLINE, can not scan
Aug 17 14:00:14 server001 vmunix: 0/5/2/0: Fibre channel host port is OFFLINE, can not scan
Aug 17 18:00:20 server001 vmunix: 0/2/1/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 18:00:20 server001 vmunix: 0/2/1/1: Fibre channel host port is OFFLINE, can not scan
Aug 17 18:00:20 server001 vmunix: 0/5/2/0: Fibre channel host port is OFFLINE, can not scan
Aug 17 18:00:20 server001 vmunix: 0/5/2/0: Fibre channel host port is OFFLINE, can not scan
Aug 17 18:00:20 server001 vmunix: Fibre channel host port is OFFLINE, can not scan
Aug 17 18:00:20 server001 vmunix: Fibre channel host port is OFFLINE, can not scan
Aug 18 11:20:50 server001 vmunix: 0/4/2/1: Fibre channel host port is OFFLINE, can not scan
Aug 18 11:20:50 server001 vmunix: 0/4/2/1: Fibre channel host port is OFFLINE, can not scan
Aug 18 11:20:50 server001 vmunix: 0/5/2/0: Fibre channel host port is OFFLINE, can not scan
Aug 18 11:20:50 server001 vmunix: 0/5/2/0: Fibre channel host port is OFFLINE, can not scan
Aug 18 11:20:50 server001 vmunix: 0/2/1/1: Fibre channel host port is OFFLINE, can not scan
Aug 18 11:20:50 server001 vmunix: 0/2/1/1: Fibre channel host port is OFFLINE, can not scan

[root@server001:/home/user001]
#


If you can see from the logs the line that contains this: Driver state = AWAITING_LINK_UP. As per suggestion [which resolved the issue btw] from http://forums12.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1219074300410+28353475&threadId=1165601, this could be a port issue:

The "AWAITING_LINK_UP" state describes the problem. Check with your SAN team about the status of the switch port (F-port) to which the HBA is connected.

To which the caller affirm that the FC was improperly connected. But still, there could be some other reason for this. I'll wait for the response/action...

Also, if you can see from the syslog, you can run this for more info:

[root@server001:/home/user001]
# /opt/resmon/bin/resdata -R 293208092 -r /adapters/events/ql_adapter/0_5_2_0 -n 293208081 -a

Wednesday, August 06, 2008

FTP and IP address

Two issues that struck me today: something to do with FTP and the other, IP address.

1. FTP: A requestor is asking if the files transferred from a source regardless of the permissions will be "converted" to the permissions as defined in the destination server/location. Which led me to some digging.

So basically, FTP permissions can be set. But not without affecting all users, i.e., globally as this is done via inetd. Or you can check these:
  1. http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=913770&admit=109447626+1217982179357+28353475
  2. http://forums11.itrc.hp.com/service/forums/questionanswer.do?admit=109447626+1217991636938+28353475&threadId=603250
A preview:

Sridhar Bhaskarla May 26, 2004 18:47:14 GMT
--------------------------------------------------------------------------------
Hi,
Set the umask in the ftp session itself. For ex.,
ftp>umask 022
ftp>put file1 file2
-Sri


Griselda Sanchez May 26, 2004 18:59:24 GMT
--------------------------------------------------------------------------------
Thanks Sri for your advice, but Is there any way to apply this permanent?

Mel Burslan May 26, 2004 19:04:19 GMT

--------------------------------------------------------------------------------
other than what Sri said, you can not set some ftp sessions' umask to something while keeping the others at something else.
in
/etc/inetd.conf, you will find a line similar to this,

ftp stream tcp nowait root /usr/lbin/ftpd ftpd -L 002

where the 002 at the tail end, is the default umask for files put to the server via ftp. you can change this value to set the umask but it will effect every user.

Michael Tully May 26, 2004 19:06:46 GMT

--------------------------------------------------------------------------------
There is no way to set it permanently, other than enforcing these things from within a script wrapper.

Sridhar Bhaskarla May 26, 2004 19:26:20 GMT
--------------------------------------------------------------------------------
Hi,
As far as I know, you can do it either for all the users by configuring ftpd in inetd.conf with -u option or for a directory using ftpaccess file.
Look at 'man ftpaccess' on directory level permissions. Search for "upload".
-Sri

Bill Hassell May 26, 2004 22:15:26 GMT
--------------------------------------------------------------------------------
There is no way to change this for a single user. ftpaccess can allow or disallow the use of umask, but not preset the mask. You'll need to script the ftp transfer and includxe the required permission changes in your script. This is easy in Unix, a real pain with Macs and PCs, especially if the end user insists on using a fancy GUI interface.

But, I guess, it would much better to set things within the source thru umask [set it via .profile].

2. IP address [SC#21810036]: This has to be done on a Solaris box. An IP - virtual - (although two were requested), needs to be "un-bind" from the NIC. Here are the some of the info I got:

[root@server:/root]
# ifconfig -a ["pipe"] grep -e 192.44.122.154 -e 192.44.122.226
....
hme0:26: flags=1000842 mtu 1500 index 2
inet 192.44.122.226 netmask fffffe00 broadcast 192.44.123.255
....
[root@server4013:/root]
# ping 192.44.122.226
no answer from 192.44.122.226

[root@server4013:/root]

# netstat -r

Routing Table: IPv4

Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ------ ---------
192.44.123.0 server1.com U 1 84389 hme0:4
6192.44.123.0 server2.com U 1 0 hme0:47
192.44.123.0 server3.com U 1 0 hme0:48
192.44.123.0 server4.com U 1 0 hme0:49
192.44.123.0 192.44.123.79 U 1 0 hme0:50
192.44.123.0 server5.com U 1 0 hme0:51
192.44.122.0 server4013.com U 1 940909 hme0
.............
default 192.44.122.4 UG 11260332localhost localhost UH 417021843 lo0

Routing Table: IPv6
Destination/Mask Gateway Flags Ref Use If
--------------------------- --------------------------- ----- --- ------ -----
fe80::/10 fe80::a00:20ff:fef9:81d4 U 1 0 hme0
ff00::/8 fe80::a00:20ff:fef9:81d4 U 1 0 hme0
default fe80::a00:20ff:fef9:81d4 U 1 0 hme0
localhost localhost UH 1 1736 lo0

[root@server4013:/root]
#


I also got some info from this site: http://www.tek-tips.com/faqs.cfm?fid=4083

But how?!?

The answer: It was a futile effort. Nothing much to read about.

"only one Interface was there : 192.44.122.226
removed the interface"

What can get from this?!? SO, I have to know this by hook or by crook or by research. And I found this: http://www.experts-exchange.com/OS/Unix/Solaris/Q_21713340.html


jephilc:

Hi

1) There are two ways to remove logical interfaces...
Either do this for each one you want to remove (as root)

# ifconfig hme0:1 down unplumb

or if you want to specify the address you can do this:

# ifconfig hme0 removeif 172.16.17.241

[ The above shows two ways of removing the 241 address.

2) To see what type of network interface you've got for example, for an ultra system run this as root

# /usr/platform/sun4u/sbin/prtdiag -v

You'll see a line like this for an hme interface

0 PCI-1 33 1 network-SUNW,hme

Check dmesg for messages after a boot which will tell you the speed the interface initialized and whether it was full or half duplex etc.

3) Use dmesg or look in /var/adm/messages to tell you at what speed and duplex setting the interface is running at

4) To hardcode the interface at a specific speed and duplex, you use the ndd command as root, but most of the time the interface will probably be set to autonegotiate. The autonegotiate facility is extremely useful especially if you have legacy systems on your network that might be running at 10 Mbps as your system will automatically talk to them OK. If you turn off this facility, and you are running legacy network interfaces, you could lose connectivity to them or cause errors, so it would be wise to check this first.

First check whether the general autoneg capability is set to 1 by running

# ndd /dev/hme adv_autoneg_cap

If it's set to 1 then you need to turn it off by doing:

# ndd -set /dev/hme adv_autoneg_cap 0

then if you want to manually set it to 100 full duplex run this:

# ndd -set /dev/hme adv_100fdx_cap 1

and make sure the other settings are set to 0. See the other available settings by listing the possible variables with this command:

# ndd /dev/hme \?

The ones marked (read and write) are the ones you can set.

Good luck

John




01.28.2006 at 02:45PM PST, ID: 15814641


jephilc:

Hi again

Apologies but I forgot to mention the /etc/hostname.hme0 file.

If the entries had been made permanent for the logical interfaces, then you will need to edit this file to remove the relevant interface entries. This will be needed so that the interfaces are permanently removed and do not re-appear when you reboot. In my original posting, I was assuming that you wanted to do the operation on the fly.

If rebooting isn't a problem, then all you really need to do in order to remove the interfaces is to edit /etc/hostname.hme0, remove the entries and reboot.

Hope this helps

John


Well, I hope did...

World Clock