Extremely slow boot

[Chad Scheer]

I have an N class, 8CPUx16GB with 4 2GB fibre cards. It is attached to 2TB useable Clariion CX600 storage. Last night I had to reboot it and it took ONE HOUR AND FIVE MINUTES! All this time was spend activating the VG's. I have dug through the system, but can not find anything obvious to make it take so long to activate the VG's on boot. Any ideas on how to put the boot in debug mode or how to trouble shoot the slow boot process?

[Steven E. Protter]

The volume groups are on the san.

I would suggest you have the fiber network checked for chatter and packet flooding.

I think you have already identified the problem area, which is volume group activation, so making the boot more interactive won't tell you much.

/etc/rc.log may have some clues to a problem, examine it closely for problems.

SEP

[TwoProc]

Does an ioscan show any missing hardware? I'm wondering if you've got a card out and are now using alternate paths for all/some/many of your volume groups.

If not, I've seen this once when I had a fiber card that wasn't bad enough for it to be failed, but it only gave service intermittenly. This showed as a regular "pulsing" of I/O that was viewable in Glance. All I/O went big, then to zero at regular intervals (10 sec or so intervals). Once the suspect card was identified, the problem went away instantly as soon the fiber cable was unplugged from the card.

[Victor BERRIDGE]

Hi,
I went throught this once or twice...
Once it was a FC card, the next it was the fiber connection (JBIC)....
The only way to diagnose is to use fcmsutil
fcmsutil /dev/td1 stat|pg
fcmsutil /dev/td1 clear_stat
fcmsutil /dev/td1 test 0x0a0f00 200 200
fcmsutil /dev/td1 get remote all
ioscan -fkc fc etc...

And this for all your HBAs

Good luck

All the best
Victor

[Chris Magargee_2]

Chad -

Are you running EMC PowerPath or Navisphere? There appears to be a bug with the latest PowerPath where VG activation can take forever depending on the numbers of LUNs.

I had an rp4440 running 11.11 with the latest PowerPath and it took nearly around 45 minutes for two VGs with 6 and 8 25GB LUNs respectively off a Clariion.

It is supposed to fixed in their next release end of Q3/05.

Thanks!
Chris

[Chad Scheer]

we are running PP 4.31 on this box.

[Mark Fisher_4]

This is related to another thread. I have the same problem. It takes 80 - 220 seconds per VG to activate. EMC came out with a patch for Powerpath, 4.3.1.2, but it did not fix it for me. I would lg a ticket with EMC so we can turn up the heat on them to fix this.

[Alan Zdunic]

Hi All,

Does anyone have any more news from EMC or HP on this matter, as I have same problem,withClariion CX700 and
rp4440 hp-ux 11.11 v1 with EMC PoweraPath 4.3.1_b040 installed.

Also another thing HP-UX SAM does not see all Clariion disks, out of 5 Clariion disks only sees 2, which is very odd. ioscan sees all 5 and powerpath â powermtâ displays 5 disks.
Symmetric DMX 1000 is all OK on both counts.

I was told by EMC to downgrade my FC card (FC -TACHYON-TL) driver from B.11.11.12 to B.11.11.09 as .12 is not certified by EMC.

Now I'm not too keen to go through this exercise unless it is absolutely necessary and it will fix a problem, but no one gives me any guarantees.

[Raj D.]

Hi Chad,

You may upgrade the Powerpath to powerpath 1.6.6.6.10.7

Also try downgrading the FC driver to B.11.11.09 .

We are having this:
FC-TACHYON-TL B.11.11.09 PCI/HSC FibreChannel;Supptd HW=A6684A,A6685A,A5158A,A6795A
FCMassStorage B.11.11 Fibre Channel Mass Storage

with Powerpath 1.6.6.6.10.7 ,
with EMC DMX 2000 , and works fine , reboot takes 25 min..

Cheers,
Raj.

[Mark Fisher_4]

Alan, I worked w/ EMC on this and it appears only PP v4.3.0 works. All variations of 4.3.1 do not work and have the slow vg activation issue. You MUST completely deinstall all versions of powerpath and then install v4.3.0. Also, EMC does not have a fix in the near future for 4.3.1.

I also ran into the clarion disks not showing up in SAM. Search for and article titled "SAM (Disk Devices) and ioscan does not match" that explains how to edit the /usr/sam/lib/C/pd_devinfo.tx file.

[Raj D.]

Sorry , in last posting I was given the powermt version wrongly. Its Powerpath 3.0.3

Well we are having version
EMCpower HP.3.0.0_b118 Symmetrix PowerPath for 11.x systems
EMCpower_patch301 HP.3.0.1_b002 Symmetrix PowerPath for 11.x systems
EMCpower_patch303 HP.3.0.3_b003 PowerPath patch 3.0.3

with DMX 2000 , and works fine.

It seems , Upgrading to Powerpath v4.3.0 can make improvement..the bootup and overall..

hth,
Raj.

[Alan Zdunic]

Hi All,

Thanks Mark and Raj.

Removing PP patch 4.3.1 and just leaving PP 4.3.0. worked for me.

Also Mark thanks for the info on pd_devinfo.tx, I can now see all clariion disks through SAM, which makes things a bit easier, though I couldn't find DKKBRC00006641 doc.

Once again thanks to both of you for your help.

Rgds
Alan