ServiceGuard + Mirror-UX + Sybase

[Fernando Boza]

I have the following:
2 server with HP-UX 11.11 in cluster with ServiceGuard.
Both servers are attached to two storages: Storage1 and Storage2
LUNs on Storage1 are mirrored to LUNs in Storage2 with Mirror-UX.
I have PV-LInk.... there are not multipath software.
Sybase is installed in LUNs.



In maintenance event... I turn off Storage2.
The operating system and cluster continue accesing to Storage1 for all logical volumens.... but Sybase lost conection to clients.


any idea for solve this issue

[TTr]

Check the sybase logs for any specific errors. If sybase is defined on the LVOLs and not the LUNs/disks devices, sybase is probably not the problem
> ...I turn off Storage2

How do you do that?

Check if serviceguard detects the mirror splitting or any unpresenting of the LUNs in Storage2 and is shutting down the database package.

[Patrick Wallek]

>>but Sybase lost conection to clients

Anything in any Sybase logs? Errors? Warning? Anything?

The system **should** have kept running normally, but I suppose it is possible that Sybase was looking for something on Storage2 specifically and when it couldn't find it, it stopped.

[Emil Velez]

make sure the package that starts sybase and activates the package activates it with vgchange -a e -q n

or when you do a test did any filesystems go away ?

[sujit kumar singh]

Hi
can please tell what was the package status at that time, assuming that the ckuster had been up.

Apart from the Sybase logs what do the syslog and the package control log do suugest?

Was there any Resource being monitored by the package Which got triggered by keeping the Storage 2 Down?

PLease give the syslogs and the package Control logs for more Details and also Check if the package Conf file points about some resources

Regards
Sujit

[Stephen Doud]

What did /var/adm/syslog/syslog.log capture when the package halted?

Is a monitor (service) configured for the package? Use 'cmviewconf | grep service' to review this.

When will the service exit? (A package service usually has an associated script which stays running until a particular event, such as the exit of an application's process(es) occurs).

Does the package control log give any more clues?

[Fernando Boza]

Attach syslog.log.... the event ocurred at 10:04

[Fernando Boza]

I attach sybase logs

[Fernando Boza]

I attach SYBASE.cfg

[Fernando Boza]

I attach pkgbase.cntl.log

[TTr]

There is a PV-links path change in the logs. I don't know what causes that, itf it is the forcefull disconnect of storage2 that means that you don't use LVM mirroring across the two storage systems. The LUN paths may be crosslinked on the two storages. The package probably fails over due to the errors in the LVM.

The sybase config file that you posted is the generic one that comes with the installation. I don't know if that is the actual file that you are using or there is another config file that is named sybase_instance_name.cfg that may have the actual settings in it.

Are you using raw volumes with sybase? If so, unless you use sybase mirroring then sybase can crash as soon as there is a PV-link path failover. Note that LVM PV-link switching is only safe if you do file i/o.

There are too many interconnected componets here that you need to identify the dependencies among them.

[sujit kumar singh]

@TTr,

the path malfunctioning perhaps as the author sometime back had said had been caused by switching off one of the storages as i think so.(correct me if i am wrong)

I hope this gives you clue to the Path Inactivation.

regards
sujit

[sujit kumar singh]

@TTr,


apologies for my haste ...

Regards
sujit

[Fernando Boza]

Sybase is not on RAW disks... is in LV (with file systems and raw)

After turn off storage2, I down package and up package and all is correct still with storage2 turned off.

[TTr · Accepted solution]

> Sybase is not on RAW disks... is in LV (with file systems and raw)

What do you mean by "with filesystems and raw"?
I asked if sybase is using raw volumes. That means in addition to raw disks it can also use raw logical volumes as raw devices such as /dev/vgsyb/rlvdata1. In this case the sybase is susceptible to PV-link path switching and can hang if there is a path switching by PV-links.

[Fernando Boza]

ohhh... in this case

my Sybase use raw logical volumes

How avoid this susceptibity of sybase to PV-link path switching? any patch or software?

P.D: I have direct conect to storage

[TTr]

There are no patches for this. The only way to protect from this is use sybase mirroring for each of the sybase internal raw files. This is similar to LVM mirroring but this is happening inside sybase. You should talk to your sybase tech support about a formal explanation on this and about a recommended approach using sybase mirroring.

[TTr]

Depending on your sybase version you can also consider using regular OS files for sybase data and logs. Sybase traditionally prefered using raw devices but I think after version 12.5 they were getting good with using regualr files.