topic Re: NFS problem causing Oracle database to hang in Operating System - HP-UX

NFS problem causing Oracle database to hang

Madhus — Fri, 22 Jun 2007 04:40:26 GMT

Hi,

Network connectivity to the remote data center lost. Because of that NFS file system mounted from the server located in that data center was not accessible. Though this NFS file system is not used by Oracle database anyway databases hosted on this server stopped responding. Once we got NFS file system back database started responding. Is there anything common between Oracle databse and NFS which is causing the problem?

Re: NFS problem causing Oracle database to hang

Luk Vandenbussche — Fri, 22 Jun 2007 05:47:38 GMT

There must be some action under normal circumstance on the nfs filesystem.

Can you check the process under normal circumstances

fuser -cu /nfsfilesystem

Re: NFS problem causing Oracle database to hang

spex — Fri, 22 Jun 2007 06:25:07 GMT

Hello Madhus,

> Though this NFS file system is not used by
> Oracle database anyway databases hosted on
> this server stopped responding.

I doubt this statement is true. Double-check your database configuration for any reference to the NFS filesystem.

PCS

Re: NFS problem causing Oracle database to hang

Madhus — Fri, 22 Jun 2007 07:10:11 GMT

Hi All,

To confim again, oracle databases are not using NFS file system at all. The file system is mend for putting installation binaries. fuser -cu /filesystem did not result anything (hung) as the file system was not accessible.

Is there any thing common between NFS and oracle? NFS uses rpc, does oracle also uses rpc for anything? Can it be that rpc hung while trying for NFS and which inturn caused oracle databases not to respond

Re: NFS problem causing Oracle database to hang

Martin Johnson — Fri, 22 Jun 2007 13:12:09 GMT

I agree that you can have problems without oracle acccessing the NFS file system. Any process that accesses the NFS will hang (such as "bdf"), which can also get other processes to hang.

HTH
Marty

Re: NFS problem causing Oracle database to hang

Dave Olker — Sat, 23 Jun 2007 16:48:26 GMT

Hi Madhus,

Are you positive there is nothing in the Oracle environment that references the NFS mount? I'm talking about PATH variables, environment variables, etc.? Anything that would cause the Oracle process to somehow reference the down NFS filesystem would potentially lead to a hang situation if the server is down.

I've seen plenty of cases where applications that were not using an NFS filesystem at all would hang when the server went down simply because an environment variable used by the application or by the user who launched the application, somehow referenced the down NFS mount point.

Regards,

Dave

Re: NFS problem causing Oracle database to hang

Bill Hassell — Sat, 23 Jun 2007 19:17:37 GMT

Unless you are willing to do everything necessary to make the NFS server MORE reliable than your server, I would not use NFS on a production system. Oracle may not be using the NFS mountpoint but there are likely several other tools and applications that are hanging, bdf and df are hanging and of course, any new logins will hang as the login code checks all mountpoints.

Re: NFS problem causing Oracle database to hang

Dave Olker — Sat, 23 Jun 2007 20:00:08 GMT

One other point here:

> Network connectivity to the remote data center lost.
> Because of that NFS file system mounted from the server
> located in that data center was not accessible.

How do you know the Oracle database going non-responsive was related to the NFS mount point in the first place? It's not like the NFS server went down and your application hung, your entire connection to the remote data center was down. Could there be another connection between the local system and the remote data center that would explain your Oracle database hang?

> Once we got NFS file system back database started responding.

I assume this means once the connection between the local and remote data centers was restored things started working again. While this would certainly explain the NFS mount point working again, this might also mean any other network connection between these sites was also restored.

Is there anything specifically tying the Oracle hang with the NFS mount point?

Regards,

Dave

Re: NFS problem causing Oracle database to hang

skt_skt — Sun, 24 Jun 2007 17:47:36 GMT

other than the NFS mount you mentioned any other data is accessed across this locations. Example if the data centers act as primary and DR sites and if any DB level data replication is in progress and an immediate n/w disconnectivity could cause a DB hung too.