3PAR NFS performance slowness with Centos 7

[Mortuza]

We are using 3PAR 8450. All SSD. 

Under file persona, 1 VFS is created and shared as NFS. This storage is using for VoD service. 

Host servers OS is Centos-7 and NFS share mounted in NFS V4.

But from the day one data processing performance getting low. 

I've checked witth HPE storage support. By log analysis HPE didn't fine any problem on storage other than there is increment in RX drop packets ( INCOMING ). HPE says there is no problem on storage HW and configuration level performance. 

Is anyone faced this kind of isssue before ?

Can this issue occur with NFS V4 in Centos-7 ? 
What can I do to or check to find exact point of problem ?

[Bertram Stoeckler]

Hello,

- Can you please provide the 3PAR and Filepersona version ? 

cli% showversion 

cli% showfs

- Can you also please provide the output of the following command, run at the NFS-client:

nfsstat -m

- How far are NFS-client and the 3PAR away from each other? Same room? Same building? Same site? Same country/continent?

To provide some answers to the NFSv4 and Centos7 question.

There is no known issue in regards to interoperability between those.

In genereal, NFSv4 is NOT faster than NFSv3. There are special cases where especially the NFSv4-delegation feature can dramatically improve the performance, but this is not very often the case.

If your environment allows to use NFSv3, ( no firewall blocking ports other than 2049 ), then you should test with NFSv3 as it is sometimes faster than NFSv4.X. (To mount the NFS-share with a NFSv3 use mount option: "-o vers=3".)

If you are running FilePersona 1.6.X, then the new supported and most likly used default will be NFSv4.1, instead of NFSv4.0.

NFSv4.1 can be much slower than NFSv4.0 or NFSv3. due to the session-overhead that is mandatory in NFSv4.1.

It can, on the other hand, be faster due to the way how file-delgation is done ( better RPC backchannel support, improved blocking-lock-notification)

All that depends on the workload that is used. 

[Mortuza]

Hi @Bertram Stoeckler 

Thanks for your reply. 

BL-VOD-STORAGE-01 cli% showversion
Release version 3.3.1 (MU5)
Patches: P126,P132,P135,P136

Component Name Version
CLI Server 3.3.1 (P132)
CLI Client 3.3.1
System Manager 3.3.1 (P136)
Kernel 3.3.1 (MU5)
TPD Kernel Code 3.3.1 (MU5)
TPD Kernel Patch 3.3.1 (P132)
BL-VOD-STORAGE-01 cli% showfs
Node FSNode State Active InCluster ----Version----- ---N:S:P--- BondMode MTU
0 Yes Running Yes Yes 1.5.7.1-20190830 0:2:3,0:2:4 1 1500
1 Yes Running No Yes 1.5.7.1-20190830 1:2:3,1:2:4 1 1500
-------------------------------------------------------------------------------
2 total

- Can you also please provide the output of the following command, run at the NFS-client:

nfsstat -m

- Right now I can't access NFS-client. I will give it within couple of hours.

- How far are NFS-client and the 3PAR away from each other? Same room? Same building? Same site? Same country/continent?

-Same building. Same DC.

Additional:

Network

15:59:53 07/07/2021

                    --------------------Bytes--------------------- ----------------IOs----------------

          Node/Port              r               w               t           r           w           t

                  0    13272427389     16534673955     29807101344    67736895    55160920   122897815

              0:2:3     5528955269        18512814      5547468083    46858245      288231    47146476

              0:2:4     5283398785          730332      5284129117    45117557       11178    45128735

                  1    20295583384     20922405993     41217989377    82205643    79825661   162031304

              1:2:3     5310696286          586085      5311282371    45249873        9522    45259395

              1:2:4 14425869091580 115003611271328 129429480362908 11662189181 21270508459 32932697640

------------------------------------------------------------------------------------------------------

            total 6    33568010773     37457079948     71025090721   149942538   134986581   284929119

NFS

15:59:53 07/07/2021 -Client RPC-- ----Server RPC----- -------V3-------- -----------V4------------

               Node Calls Retrans      Calls Badcalls Read Write Create      Read    Write Create

                  0     0       0        216        0    0     0      0         0        0      0

                  1     0       0 2287873305        0    0     0      0 434605973 54803345 183007

-------------------------------------------------------------------------------------------------

            total 2     0       0 2287873521        0    0     0      0 434605973 54803345 183007

[sbhat09]

Hello Mortuza,

I wonder if you are still facing the issue? Please let me know if you need assistance.

Regards,
Srinivas Bhat