Discussion: Device special file changed for unknown reason.

[Torsten.]

I still think there was something wrong with ioconfig file(s), because all disk devices got new instance numbers and for that reason new device files.

[Ken Grabowski]

It's a binary file that is modified by insf, rmsf, and ioscan.  Not easily played with and pretty much only used to keep data between boots.  While running the data is kept in the kernel data structure. I rarely rule out anything as a possibility, but somebody modifying this file manually is pretty hard to do.

At boot the system does an ioscan!  I've done a lot of SAN migrations and when disks are moved between arrays they wind up with new WWN's and that means their hardware path has changed.  The C value will change, and if the SCSI was modified too, then the T and D values could change too!  This could also happen within the same SAN if the storage team decided to move disks to different control's (FA ports) to balance or "Fan Out", disk load in something like and EMC DMX or vMax style array.

I'm still surprised by the change to the IR Volume, but I think somebody doing something unannounced on the SAN or fabric is more likely than somebody playing with or stepping on the ioconfig file. 

[Torsten.]

There are procedures to modify ioconfig to change exactly these values. (e.g. to make them the same among several servers).

I don't think there where changes in the SAN, because all the h/w paths are still the same.

I really wonder why the IR Volume is no longer instance 1 ...

[stephen peng]

Guys,

It is a real discussion!:)

Maybe I should post full output of ioscan -fnCdisk, and you can notice that instance number of DVD changed also. It looked like IR volume disappeared for a while.

By the way, there was a MCA before I aware this weird thing. I run errdump mca at EFI Shell> prompt, and paste the output to a file as an input to Machine Check Analyzer, but the tools panicked after I start analyze. What wrong did I do in producing MCA file?

Regards

Stephen

[stephen peng]

Guys,

Liked I said before, I was curious when I found that the OS could still boot up under such circumstance, and unliked usual, vgdisplay -v vg00 showed in attachment, and the OS seemed ok.

c1t0d0 was not there, but vg00 could still activated, it really conflicted my understanding of LVM; it is not so hard to understand why os could boot(EFI located /stand, right?), but it is really hard to explain, for me, whether there was REAL relationship between vg00 and the PV that consist it.

regards

Stephen.

[Ken Grabowski]

EFI is not in /stand, /stand is in vg00 which is in an EFI partition.  You give setboot the hardware address of the EFI disk(s) , for example "Primary bootpath : 0/0/12/1/0.10.15.0.0.0.1".

Exactly how the boot translates that into the disk that's in vg00 in spite of lvmtab having a different device path, is beyound me.  But I assume since it already has the hardware address it doesn't have to worry about it.

[Torsten.]

This is not OK:

Cur PV                      1     
Act PV                      0     

I'm sure the system has booted with original configuration, then during boot the change happened.

Let's see

# vgdisplay -v vg00

# strings /etc/lvmtab

Check the syslog file for related information.

[stephen peng]

Torsten,

Check syslog.log and event.log as attachment.

There was an MCA indeed, but nothing wrong with vg00 and filesystem, in fact, I could rlogin the box, so I thought network service was OK. since I could read syslog, so the filesystems should be OK; there were some FIBRE syslog which I could not interpret.

strings /etc/lvmtab showed:

/dev/vg00
/dev/dsk/c1t0d0s2

It was really beyond my understanding.

regards

Stephen

[Torsten.]

execute:

/opt/resmon/bin/resdata -R 175636482 -r /system/events/ipmi_fpl/ipmi_fpl -n 175636481 -a

/opt/resmon/bin/resdata -R 175636482 -r /system/events/ipmi_fpl/ipmi_fpl -n 175636482 -a

vg00 lists still the old device file - you should correct this asap.

[stephen peng]

Torsten,

I am not on site now, and I captured those syslog and event.log or other information before I noticed the stranges. I corrected the mismatch thing right after I found it, and all I want to know is WHY.

For resdata output, you could check out event.log, and I am pretty sure that were some MCA notices. I am not that agreed with you about the "OK" thing cause I don't see any "NOT OK" thing except that vg00's mismatch, and you could also check the rc.log before I did the correction.

Regards

Stephen