Networking
cancel
Showing results for 
Search instead for 
Did you mean: 

NFS asynchronous Vs. synchronous writes

mhm
Advisor

NFS asynchronous Vs. synchronous writes

Dears,

We want to implement an archive solution for the databases (2 servers) and the application servers (2 servers), all of them are HP-UX 11.23, Itanium servers. We will mount a NFS exported share from NAS storage. my concerns revolve around what best mode to use (if NAS supported it) asynchronous or synchronous?
As far as I know, synchronous is the default and its preferred because of the data integrity, also as what is written here about "Safe Asynchronous Writes":
http://forums11.itrc.hp.com/service/forums/questionanswer.do?admit=109447626+1265529583347+28353475&threadId=1139502
it seems that the asynchronous mode on NFS V.3 resolved the data corruption problems in case the NFS server goes down while the clients are writing to it.

The other concern is to use boid or not? if yes, is there any recommended number other than the default one?
I'm planning to start with the default value, then trying to increase and decrease the value while monitoring to achieve the best value?

Note: Its an archiving solution, and its about 3TB space, the DB's going to use it once a week but I'm pretty sure they will use it heavily.

BR
Mohammad
11 REPLIES
Dennis Handly
Acclaimed Contributor

Re: NFS asynchronous Vs. synchronous writes

>resolved the data corruption problems in case the NFS server goes down while the clients are writing to it. ... Note: It's an archiving solution,

Since it's an archiving solution, why do you care if the server goes down? Just as long as you can back up your database before that happens or be able to restart.
mhm
Advisor

Re: NFS asynchronous Vs. synchronous writes

We can't take that risk, so every time we face a network connection problem or the server hangs we should restart again, as long as we can configure something better, why is not?
Dennis Handly
Acclaimed Contributor

Re: NFS asynchronous Vs. synchronous writes

>as long as we can configure something better, why is not?

Speed may be more important.
And depending on how you are copying the data, if the server goes down, you may not have a restart capability anyway.
Kapil Jha
Honored Contributor

Re: NFS asynchronous Vs. synchronous writes

biod starts num_nfsiod asynchronous block I/O daemons. These daemons are used on the NFS client to improve I/O performance by caching
asynchronous read-ahead and write-behind requests.

So biod would run with async and I suppose this would be the better way in your case.

Regarding number of biod i think below link would give you some idea
http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp?topic=/com.ibm.aix.prftungd/doc/prftungd/num_necessary_biod_threads.htm

BR,
Kapil+
I am in this small bowl, I wane see the real world......
rick jones
Honored Contributor

Re: NFS asynchronous Vs. synchronous writes

Indeed, the explicit support for client control of "write to cache" and "flush" in NFS v3 pretty much addresses what the previous HP-UX async mounts were meant to address. Modulo Dave Olker chiming in to correct me I would say that with NFS v3 you don't need to worry about enabling the old async write stuff.

The number of nfsds one "needs" depends on how many concurrent NFS requests you need the server to service.

The number of biods on any given client will depend on how many concurrent NFS requests it needs/wants to have outstanding to the server at one time.

Those will all depend on things like the link bandwidth between the client(s) and the server, and the latency.
there is no rest for the wicked yet the virtuous have no pillows
Steven E. Protter
Exalted Contributor

Re: NFS asynchronous Vs. synchronous writes

Shalom Mohammad,


You pay your money you take your choice.

asynchronous writes get you better performance. But if there is a sudden stop on your system you are much more likely to lose written data.

No OS upgrade or new version of NFS can eliminate the risk of data loss.

That risk increases with how busy your system is.

The good news is Oracle is well written and can recover you almost to the last transaction even with a sudden loss of power with no backup.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Michael Steele_2
Honored Contributor

Re: NFS asynchronous Vs. synchronous writes

Hi

I currently have 24 NFS file systems with async writes mounted from 3 servers. When one of the master's go down then the 3 masters have to be reexported and 24 clients have to be remounted. It takes a minute or two with a script and it is perform as often as once a month. During patching for example.

I have never heard of any loss of data complaints from any SAP or oracle admins or DBA's, nor any complaints from the users.

The key is to re-export first and then remount. And like I said above, it only takes a minute or two with a script.
Support Fatherhood - Stop Family Law
Dave Olker
HPE Pro

Re: NFS asynchronous Vs. synchronous writes

Rick is correct. NFS v3 uses safe async writing by default, which means the client will keep a copy in local cache of any data it sends to the server until the server notifies it the data is on stable storage.

The default value of 16 biods is a good starting point. I would start there and monitor performance.

I'd also recommend tuning the 11.23 systems using the tuning guidelines in Appendix A of the KRPC guide found here:

http://docs.hp.com/en/14765/v2963139.pdf

Tunable Default Recommended
nfs_async_read_avoidance_enabled 0 1
nfs_fine_grain_fs_lock 0 2
nfs_new_lock_code 0 1
nfs_new_rnode_lock_code 0 1
nfs_wakeup_one 0 2
nfs3_new_acache 0 1


Note: I am not recommending tuning the nfs_exportfs_rwlock parameter as there has been a recent report of problems involving this tunable on NFS servers.

Regards,

Dave
mhm
Advisor

Re: NFS asynchronous Vs. synchronous writes

Many thanks to all of you guys,

I tested the the NAS share with the basic timex and dd commands as the following:

I issued the command:
timex dd if=/dev/zero of=/myshare/test bs=1024k count=10240

First time, I kept biod number on the default (16), it gave me 2m 57 seconds
Second time, I increased the biod number to 32, it gave me 2m 23s.
Third time, I increased the biod number to 63, it gave me 2m 55s.
Fourth time, I made biod=0, it gave me 5m 22s.

So, I decided to go with the second configuration. For now, am still monitoring to see if I can enhance the throughput.

I'll update you if there is any new news.

BR
Mohammad
Shailendran V Naidu
Frequent Advisor

Re: NFS asynchronous Vs. synchronous writes

Thats some research ......
biod is supposed to be the NFS client deamon, can you mount NFS shares with no biods running !? Never heard of it.

Anyone can post more details ?
Dave Olker
HPE Pro

Re: NFS asynchronous Vs. synchronous writes

Biods are an *optional* daemon for the NFS client. If they are running they will handle the asynchronous read and write requests on the applications' behalf. If there are no biods running then the reading or writing application will perform these over-the-wire calls in its own process context.

Try it for yourself. Kill all the biods and see if you can still access/read/write to your NFS filesystems.

In the vast majority of cases biods are helpful to application performance. There have been very few cases in the past where specific application workloads would run faster with no biods on the system. One major underlying cause was a "thundering herd" condition in the biod code that we've since identified and resolve (the fix is in the latest patches and using the kctune parameter set I described in a previous post in this thread).

Again, biods are optional, not required. Easy enough to prove by killing them. However, in nearly every situation you'll get better performance with them than without them.

All of this is specific to 11i v2 and prior, since 11i v3 doesn't use biods. :)

Let me know if you have any other questions about this.

Regards,

Dave