Maximum iSCSI Port Queue Depth

ngky79 — Mon, 14 Jul 2014 05:10:26 GMT

Dear All,

We are getting alot of alerts from VMware VCOPS on volume latency lately.

I suspect that the iSCSI ports' queue depth on our Nimble array is full during certain time of the night. We have a 2 ports CS460.

Is there a way to report the queue depth usage on the array?

What is the max queue depth setting on the array?

Thanks.

Re: Maximum iSCSI Port Queue Depth

smurthy110 — Tue, 15 Jul 2014 05:10:01 GMT

Hi KimYong,

Nimble Uses Queue Depth of 128 per ISCSI connections. The details are explained on the below link. The VCOPS settings could be due to Thresholds but will have our Nimble Support check the system status.

https://connect.nimblestorage.com/message/2375#2375

ESXTop shows up the parameters.

Where does the "128" value come from?

The 128 is actually set by Nimble. iSCSI is better than FC because the target can tell the initiator how much queue depth it has and then the initiator can use that value (or a lower one). In our case we respond with 128. That value is per session so you if you have multiple paths you’ll actually get (#_paths * 128) as the total queue depth for that LUN.

Is "128" the Queue Depth at LUN level?

The queue depth is actually at the connection level. If there are 2 connections/sessions, then it would actually be 256 per LUN/volume.

What is the typical Q-depth at the Array Port in Nimble storage? 128 seems high, is that because Nimble is a Flash storage and LUNs are capable of processing high amount of commands?

The value of 128 was selected so that queue_depths would not generally be a limitation in terms of performance, so yes, because the Nimble arrays are high performance. The cost of more queue depth is also not high in our architecture. It should probably be noted the q-depth is not per port but per iSCSI session/connection.

topic Maximum iSCSI Port Queue Depth in Array Setup and Networking

Maximum iSCSI Port Queue Depth

Re: Maximum iSCSI Port Queue Depth