buffer credits sonsumption

[florence mathon lermusi]

Hi,

does anyone knox how to see the buffer credits consumption on a brocade switch port ?

thanks

[RBaker]

If you can read at 2.0625 Gb/s, better might be to look at portstatshow and look for tim_txcrd_z which is time BB credit zero.
I'm assuming here that you think you are having probs. If you need any stuff on BB credits, I've got loads and loads of engineering stuff.

[florence mathon lermusi]

hi,

thanks for the answer.

In fact, i would like to be able to monitor the BB credits consumption "in real time" to be sure for exemple that the number of credits assign to an extended link is sufficiant to optimize my bandwith.

[RBaker]

Are we talking about a DRM type issue, what type of link are you using?

1550 nm/DWDM/1330nm etc.
HSG/EVA ?

as I've got the calculations which can be worked out using performance monitor and a ruler (humour) etc.

In response to real-time then you'll have to interrogate the MIB via management software, or derive from E_Port perfomance using brocade fabric watch.

[florence mathon lermusi]

I thought more about a DWDM solution between "two distant SAN".

What you are saying is that there is no brocade tool which can give us this metrics...

[RBaker · Accepted solution]

I don't think it could poll it quickly enough without wrecking the performance.
Got a bit of time on my hands and since you have a red star, I hope you don't mind:

The science is:
Frame is usually up to 2048 Bytes (2k) for SCSI traffic = 16384 bits, when encoded for transport 8B/10B gives 20480 bits.

SOF, CRC, EOF and the frame header comprise another (1+1+1+6) transmission words which is 9*40= 360 bits

This gives a total of 20840 bits.

Padding (elastic-region) between frames requires a minimum of 6 transmission words or 240 bits of which four may be primitive R_RDY (or BB_Credit) giving an additional 240 bits yielding a grand total of 21080 bits assuming 2125000000 bits per second 100806 frames per second are possible.

Now the primitive R_RDY (40 bits within the stream at 2Gb/s is dealt with by the FC chip at it's framing and signalling layer (FC_2) which is just quick enough to recognise and strip off the primitive. None of this info is passed any higher up the protocol stack and hence does not leave the FC chip into software but if it was then:

100806*4=403225 credits per second might then be assumed to be average hence any real-time measurement of BB_Credits could reliably be assumed to be updated 403225 times every second, would be unreadable, unrecordable and would damage the switch performance so much it would negate any effects of measurement.

It might be useful if you have the equipment setup to adjust the BB_Credit etc. such that the variable Time BB credit zero is 0.


Link Performance

With regards the extended link then I use the following (close) approximation, it's worked for me many times, though they might have updated some stuff in recent firmware. (Might as well tell everyone the secrets)

Distance is measured + 30-50% (fibre is never laid in a straight line (unless you are in the desert)).

Remember Brocade silkworms have a "loom" or switching engine for each four ports. Credits max are around 117 per loom (I forget the exact figure but it's close enough).

Long distance fabric licence (allows up to 55 credits per port) but starve the other three ports on the loom (might have to blank out switch ports).

1550 nM (DWDM) GLM 2 Gb 95MB Bandwidth per channel 16 credits, up to 60KM at 85 KM 27 credits (max without a BROCADE EXTENDED FABRICS license key use fabric configure to increase to 27, remember to do it on both ends) returns this figure to 95MB. At 120 KM 60 credits (max for the Silkworm port) are required.

The bandwidth figure is per link so with DWDM is multiplied by the number of links(within the loom limits so again you might have to use several looms and blank out switch ports). The average I/O size (data rate) and the IO/s (request rate) from the server(s) or storage controller(s) can then be used to determine the number of links required to fit this figure within the bandwidth constraints.

With a BROCADE EXTENDED FABRICS license key:

Configure switch as long distance using Configure fabric Long_distance_fabric=1.

Then for each port required:
portCfgLongDistance port_number

0 Reconfigures port as a regular switch port. 16 Credits
1 Level one long distance, 27 credits
2 Level two long distance, 60 credits

Level 1 equivalent can be achieved using Configure fabric parameters.

[florence mathon lermusi]

Thanks a lot for this interesting answer !