Monitoring fabric traffic?

[chud67]

I have some backups that are running slow over fabric. They run fast over regular network, but not over fabric. I think it is the configuration of the backup, and not the fabric, that is at fault...but I have to prove it.

How do you all monitor traffic across your fabric? Do you just use the performance monitor within Brocade switch viewer?
What about EVAperf? If so, how?

Any other suggestions?

[Jeff Barnicki]

Not sure about your specific environment, but we use Work Load Analyzer, which is part of EMC Control Center (apparently called Ionix now).

It's better for archiving past performance and not real time.

For real time, I just use the Performance Monitor on brocades' the web GUI.

So, your backups that go over iSCSI are fast, but the ones over the fabric are slow? I mean, are these the same boxes that you're backing up over the network and fabric, or different boxes?

[chud67]

We have brocade switches.
The backups are going to MSL tape libraries, that are zoned.
Traffic seems to go fast over ethernet, but not always over fibre.

It's the same server having the problem with its backups.

[PP BIJU KRISHNAN · Accepted solution]

You could use DCFM Enterprise to monitor port performance over a period. This is licensed product.

In the recent versions of Fabric OS, the command bottleneckmon has been included to monitor fabric bottlenecks.

Instead of pondering over Fabric performance which is usually not the root cause, check these factors

1. Errors on relevant ports with
porterrshow
fabriclog -s
portstats64show

2. Check if there is a possibility of ISL oversubscription. If its too grave then you will see errors in errshow -r
Try connecting the MSL port to the same switch as the host or maybe the adjacent port for better locality.

3. Check windows error logs for any errors, specifically errors related to the HBA driver or the SCSI driver.

4. You could try zoning the tape drive to just one HBA to test things.

5. Fix the switch port speed and the HBA speed to the desired and possible level.

I can think of these for now. Since backup over network is fine, I would not doubt the EVA for now.

[chud67]

^^^ Thanks for your reply!

[DeafFrog]

Hi Chud ,

this is how i did the previous time :
open the switch console , one of the tabs (if i remem. correctly , it's view tab) you would have the option to monitor the traffic real time,port wise(this will vary based on your fabric set up)
On servers , you could install tcpdump , use command line option of tcpdump to monitor traffic and dump to file and connection from lan card of server and DP server.

start the DP backup.
I rememeber , when i did this test previous time i saw most of the backup traffic on lan card only.

Regards ,

[chris huys_4]

Hi,

Connect via telnet to the relevant brocadeswitches. (know upfront on which port of which sanswitch ,the server is connected, on which port(s) of which servers the fc tapedrives are connected and via which interswitchlinks the sanswitches are connected)

And then see with portperfshow on the different brocadeswitches, how the traffic flows from the server to the fc tapedrives and hopefully together with the information of PP BIJU KRISHNAN response, this will give some ideas. ;)

Greetz,
Chris

[chud67]

Thanks everyone, I appreciate all the feedback.

[chud67]

Hey guys, I like porterrshow that was suggested above, I have been using that.

However I have a question about "fabriclog -s". How do you capture that data? It scrolls continuously when you run the command, so I tried redirecting it to a file, but no luck ("cannot redirect output"). Also, how would you get the data off of the Brocade? I tried to ftp to my Brocade and can't do it.

[chud67]

Never mind, I see that you can pipe it to more and then just copy and paste off to a text file. Thanks guys.