NAS 2000s slow writing files to disk

[CA1154852]

We just got a NAS 2000s with 1.6 TB of space, RAID 5, running Windows 2003 storage server connected to a 1 gig port on the router. We used all the defaults when setting it up and have found it slow to write files larger then 100 megs to disk. On our old file server it look 60 seconds, now it is 4-5 minutes from an XP clients. I just got service pack one but I don't think that is going to help. Any ideas on how to increase performance?

[CA1118839]

Hi Jason,
MANY questions, but first a bit of information. I am guessing there is an MSA30 attached due to 1.6TB. This will use the onboard RAID controller (5i) that has 64MB cache. That is not a very large cache to be working with for large file performance, but for file sizes of less than 1GB (2000s ships w/ 1GB RAM) performance should be good. As soon as the file size exceeds the amount of system RAM (Windows System Cache), performance will drop expectedly.

-You mention 1 gig port on the router. Are you really linked right up to a routing device or is it just a switch? Routing will affect performance (slightly, though).

-You write about 100MB files taking 1 minute normally, but 4-5 minutes with the 2000s. 100MB/min is pretty terrible performance for gigabit any way you look at it. 100bT connections are capable of about 500MB/min with all of the ethernet overhead taken into consideration. 1000bT connections are theoretically capable of 5GB/min (simply 10x of 100bT). In our testing we have yet to fully saturate a 1000bT network connection like a 100bT. With synthetic benchmark tests it is easy to do, but we have done real-world stuff. The most we have been able to get on a 1000bT link is 3-4GB/min from several clients. Also, a single SCSI disk is capable of well over 100MB/min transfer rate (real world). 120MB/min would be just 2MB/sec and that is easy to do with a single SCSI or IDE disk (especially taking into consideration that system cache - remember, RAM - will buffer transfers).

-There is something else going on - networking would be a guess from the limited info in the post. Easy test would be to take an XP client and use a crossover cable to connect directly to the 2000s. The XP client should be able to push well over 100MB/min and the 2000s should be able to handle anything a single XP client can throw at it. It would be a shock if the performance from even a single client to the 2000s is not 100s of MB/min (maybe even near 100bT saturation - about 500MB/min)

Please supply some more info and let us know.

[CA1154852]

I have loaded SP1 for WSS. I did performance checks for logicial disks. Average Disk Read Queue Length Low=0 Average=.12 Min=0 Max=.446. For Average Disk Write Queue Length. Low=0 Average=.01 Min=0 and Max .26. For Average Disk Queue Length. Low=0 Average=.03 Min=0 Max=03. When I connect my XP laptop to the Extreme black diamond or other servers that are connected to the Black Diamond, the transfer speeds for read/write are all under 1 minute for 100 MB file. As soon I connect to an Extreme 48 port Switch, the speeds go back to 5 minutes to write and less then 30 seconds to read. I am not a network person but why it that this server is the only one slowing down compared to my old DL 380 with Windows 2000?

[CA1154852]

Our old file server is a DL 380 with 14 disks on the disk array and they are 36 GB SCSI disks. The array is set to Expand Priority of Low, Rebuild Priority of Medium and 100% Read and 0% Write and a Surface scan of 15. On the new NAS 2000s I have 14 disks on the disk array and they are 146 GB SCSI drives. The array, Array 5i controller, is set for Expand Priority of Low, Rebuild Priority of Low, and 25% Read and 75% write, and Surface Scan Delay of 15 seconds. What is the Surface Scan Delay do? I thought Write setting is writing data to disk and read is reading data from disk is not not correct? I was doing the network teaming, fault settings, but I turned that off.

[CA1157836]

Hello Jason,

I've been working with Keith on this one. I'm not totally clear about what works and what doesn't. Your second to last message makes it sound like the Extreme 48 is part of the problem and that only the 2000s has problems when connected to it? Or is it when the laptop is connected to the 48 that *only* the 2000s is slow (I get the perception from your messages that the 2000s *does* perform well, but not in all configurations). Just a few points to offer you before I request your clarifications. First, it'd be easy to configure a share on the local OS drive to test against. This is only 2 disks that are mirrored (lower performance), but should be able to provide over 100MB/min performance. If performance is still slow, we still need to troubleshoot to root cause. If performance improves, then we can suspect the 5i, 5i settings, background initialization or something else 5i specific. Second, we run 100%/0% for the read/write cache settings on the 2000s because the cache is not battery backed. In the event of a power failure you could lose/corrupt data. The 25/75 you have configured should, by all means, improve write performance, though. Please clarify and we'll work on an answer as quick as possible

[CA1154852]

Ken

Here is what I have tested. On the Extreme Black Diamond I setup a port to be on the same VLAN as our servers or a different VLAN setup for DHCP and I can read and write under 1 minute easy to the Raid 5 drives. When I connect to an any Extreme 48 switch that is connected by fiber to the Extreme Black Diamond the responce time is slow writing, 3.21 minutes for a 92 meg file verse 1 minute or less to the old DL 380 file server running Windows 2000. I used ethereal to get dumps from writing to new and old file server. All the numbers look better for the new file server then the old one execpt SMB "Write andX". That seems to be the bottleneck. My Network packet guys are out of the office till Monday for me to ask them to look at the dump files. I am not a network guy so I really don't know what that means and how to fix it. I really appricate all your help.
Jason

[CA1154852]

Just to be clear.

The Nas 2000s is connected to the Black Diamond cooper gig port. I have been using my XP Dell laptop connected to either the Black Diamond or Extreme Summit 48 switch, 10/100 ports.

I will try creating a share on the mirrored drive to see what that does.

[CA1157836]

Sorry about the delay. I've been doing investigation and testing regarding your problem. First, in our test lab we use many ProCurve switches/models. We've also got 3COM and Cisco products as well. I've tested a scenario similar to yours, but with a Cisco Catalyst 6506 (comparable to Black Diamond line) and various 1000bT switches uplinked (like your 48). No problem found. Can you try another client other than the Dell laptop just for another datapoint? Can you get some other network switch or hub (100bT or better), use it in place of the 48 and verify the problem persists? If it only happens with the Black Diamond and Summit 48 uplinked, then I'll see if I can contact Extreme Networks directly (I've poked around their website for info with no luck). We've also got some Extreme switches here, but not the same models (Summit only I think). Any work from your network folks?

[CA1154852]

Sorry for the delay in my responce back. Just to recap: I am having slowness when writing to disk, 3x slower than any other Windows server. I am not having problems reading files from the server. I have turned off all the extras, Volume Shadow Copy, filtering, virus scan, and nothing changed. I had the NIC Team set for failover but I have turned that off. I was connected a gig cooper port on an Extreme Black Diamond and I have changed ports to a 100 meg connect, with the NIC hard and the port set to 100/full and changed cables. Nothing changed. I loaded service pack 1 and all the latest MS patches. Nothing changed. We installed Ethereal on the box and found we are getting check some errors on the server. One issue we thought of is that the maximum packet is 15000 and on the NAS it is over that 15400, or thing thing like that. I got a new DL 320 so I decided to try a transfer to do a transfer from it which is connected to the same switch as my desktop. We did not see the slow down doing a write but did see checksum errors. We are thoughts are the NIC card or drivers are bad. The drivers should be current from patching and service pack one. So we are asking HP field techs to replace the card. We also are looking at installing a Intel NIC card just to see if the errors stop from doing that.