max_queue_depth setting and oracle

[Charles McCary]

Group,

Hey - I've just discovered the max_queue_depth setting and it's relationship to the EVA we have. I'm playing around with some scenarios and had a thought.

If I make changes to the max queue depth on the luns that our oracle database uses, do changes need to be made within oracle to take advantage of the new max queue depth? Or is this simply how many I/O requests the OS will throw at the lun?

One reason that I ask, and I realize this is not a "normal" database function, but I doubled the queue depth on the luns used during the oracle hot backup and only saw minimal increases in the IO rate. This should have been a good spot to test in my opinion. I was hoping to see the IO rate really jump.


I appreciate your feedback.

thanks,

Charlie

[spex]

Hi Charlie,

My gut feeling is that the new max_queue_depth will take effect immediately. Maybe someone else could confirm.

PCS

[Yang Qin_1]

Yes, if you change max_queue_depth with scsictl command the new value will take effect immediately. However, the value with be set back to original one after the server reboot.

Yang

[Charles McCary]

Sorry...guess I didn't make myself clear.

I understand the the effect is immediate.

My question has to do with oracle and if changes need to be made in the oracle configuration to take advantage of the new max_queue_depth.

Also, if anyone has any ideas on why I didn't see a big jump in IO rate when I changed the luns used by the oracle backup.

thanks,

Charlie

[spex]

Charlie,

That is what I meant by "will take effect immediately". In other words, no changes need to be made to Oracle configuration.

As for the minimal increase in IO, I believe you're past the point of diminishing returns for max_queue_depth.

PCS

[Charles McCary]

Point taken.

How does one go about knowing when they are "past the point of diminishing returns", ie how can I verify this?

[Alzhy]

What was your max_queue_depth anyway and to what value did you change it to?

Our experience with Oracle on large EVA5K LUNs show that a qdepth setting of 64-128 is optimal.

BTW, you will not see the immediate returns of tuning qdepth in a hotbackup scenario... You will see it in terms of how well your database will now be able to scale - i.e. more queries -- leading to overall faster performance...

[Charles McCary]

Hi - it's an eva3000 and all queue depths were set to the default of 8.

I changed them on these 3 luns to 16, as a test.

I'd love to be able to set them all to 64, but I'm running in to the 2048 limit on the EVA. We have some large luns and some smaller.

Do you think just changing the larger luns to 64 would be more effective than changing all the luns to 16 or so?

[spex]

Hi Charlie,

You should be able to glean a recommended value from the EVA documentation. I would perform standard benchmarking to determine where the rate of increased IO performance is at a maximum. That is, set max_queue_depth to a low value, benchmark, increase max_queue_depth, benchmark again, etc.

PCS

[Alzhy]

Charles, depending on how many of your connected servers are active at the same time (or how many of your LUNS will be active at the same time) -- I think it is safe to bump it up to a higher value.

And even if it did reach the 2048 q limit, I don't think it will have serious repercussions to all your connected servers as they will just have an IO Wait on queuing...

Just some thoughts...

[Charles McCary]

Nelson,

HP was pretty conservative about not crossing the 2048 barrier. I understand where you're coming from though.

FYI, we're using securepath and here's an example of one of the luns using spmgr display:

Here's an example of spmgr display output for one of the luns:

TGT/LUN Device WWLUN_ID H/W_Path #_Paths
3/ 6 c13t3d6 6005-08B4-0010-23BC-0000-A000-004A-0000 4
255/255/0/3.6
Controller Path_Instance HBA Preferred? Path_Status
P66C5E2AAQT02B no
c17t3d7 td1 YES Active
c7t3d7 td0 YES Active

Controller Path_Instance HBA Preferred? Path_Status
P66C5E2AAQT016 no
c5t3d7 td0 no Standby
c15t3d7 td1 no Standby

[Charles McCary]

Nelson,

Also, when you say "active", do you mean:

1) has an lvol and fs mounted on it.

or

2) has an lvol and fs mounted on it and is actively being read/written from/to.


I would say all of our luns meet 1 all of the time.

I would say that there are times when 2 is the case as well.

[Alzhy]

Charles, I would say (2)...