Showing results for 
Search instead for 
Do you mean 

Capacity Planning!!!

SOLVED
Go to Solution
Super Advisor

Capacity Planning!!!

Team:

Can someone help with this topic Capacity Planning!!! what would be the role of a capacity planner on hp-ux environment? what he needs to know and some links.

Thanks,

Best Regards,
Joe.
19 REPLIES
Outstanding Contributor

Re: Capacity Planning!!!

Well, that's a pretty deep subject, Joe.

I would think first off, a unix capacity planner would have to be intimately familiar with the environment as it exists today, the capabilities of the existing equipment, the capabilities of new equipment on the marketplace, the norm for existing performance, the acceptability of that performance to users and management, and the strategies for performance tuning. That's a pretty high level take at it. How much more detail are you looking for?


Pete


Pete
Honored Contributor

Re: Capacity Planning!!!

Hi Joe,

Capacity Planning? That person would have to understand your current environment. What hardware you have, the number of CPUs and speed, amount of RAM, amount of disk storage, network connections, backup solutions, etc. They would also need a decent understanding of the applications currently running and anything that is being proposed. They would need to be able to track the current system usages [CPU, memory, disk, etc.] on a regular basis and be able to project trends into the future.

Just my wild swag at it. It sounds good that your place is considering making it a regular job. Around here, it happens when we start running out of a resource, or when we just happen to find out about the next big application that needs a 16-way rp8400 and a terabyte of disk space. Oops!


JP
Exalted Contributor

Re: Capacity Planning!!!

You have to do a survery. First find out what you are using.

Then talk to the persons responsible for the major applications and find out what they want. Then add a factor for the unplanned stuff. New software, new databases and whatnot.

Then pray you have the money and aren't off by the 85% I was off for year 2001.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Honored Contributor

Re: Capacity Planning!!!

What is the role? The role could be hundreds of different things. From design of filesystems, layout, SAN configuration, determining the rate at which your storage will be maxed out, proposing new storage solutions, etc..
Super Advisor

Re: Capacity Planning!!!

Pete:

Thanks for that start. I need to know in what part of hp-ux the "capacity Planner" needs to be strong. will these be enough?

1. HP Certified.
2. Perfomance tuning (Glance, Measureware, sar, vmstat, iostat)
3. LVM
4. Kernel Tuning.
5. NIS.
6. MC Service Guard.
7. Security.
OR did i miss anything? Please feel free to add additional skills required.

Thanks

Joe.
Trusted Contributor

Re: Capacity Planning!!!

Joe,

I have been filling that role for better part of a decade for several years. I am also the system admin on a large v2600 system with 300 users 23.5/7 and burst to 500 with 1000's of impromptu sessions.

First of all the planner needs to know about your applications, and databases.

He also needs to know about the storage platforms. If you have the money send the planner away to several schools for storage, at least EMC and HP, and $ permitting IBM.

We have EMC here. If your training dollars are zippo like many companies' budgets allow none for Techies at that level,

Your best bet is to remember one thing. HP recommends HP, and EMC recommends EMC. Remember that all storage needs to be mirrored at least once. Root drives require HP's mirrordisk/UX product, and let the raid handle the rest of the storage.

I recommend that EMC is the best if you are doing this in a SAN environment. HP if you are going minimal volume. If the serever needs to have more than 10 filesystems go EMC, use PowerPath with multiple fibre HBA's and if not SANed run fibre between the boxes, use BCV's for databases and shut them down long enough to split BCVs and go do backups from there. This requires 3X disk to usable, but eliminates most of bs. If you use EMCs EDM product, the direct connect backup is expensive, but the best with EMC storage.

What you really need is someone with the experience not to pick stupid choices. Avoiding certain problems come with grey hair, because you can guess how it got grey.

Understanding your budgeting process is also important. When a system goes into production things changed. As time goes by, DATA growth slowed to 30%/year. We were expanding the DB at 43% /year , this year for 6 month the growth was 14.75% additional storage. If you have only one shot at money, get a loaded raid.

We have expanded from 60 GB to 1.4 TB on one DB alone since I came here 3 years ago. I was slightly off my estimate of 1.5 TB, but after this next week I will be within 32 GB of being a perfect guesser before 1/1/04.

We needed 3 years growth in our leased sym, and we have 3.4 TB mirrored total so we have split it up in a creative way. All storage is harware mirrored. Some is BCVed 3 WAY we have 2.4 GB in BCV'd storage 800 GB usable and the balance in mirrored storage. BCVs are able to be re-used and re associated and can be used to backup multiple groups of disk.


Good Luck,
Tim

Tim
Honored Contributor

Re: Capacity Planning!!!

Well, I don't see why you would need to be strong in NIS to be a capacity planner.

I would think LVM, VXVM, VXFS, perhaps fibre channel knowledge to setup a SAN would be good.
Trusted Contributor

Re: Capacity Planning!!!

The real Catch is a good admin who likes to do this part of the job, it is my favorite part of the job I currently have, problem is now we have an Enterprise Architecture specialist...
Outstanding Contributor

Re: Capacity Planning!!!

Joe,

That looks like a good list to me. I would stress performance tuning and kernel tuning and de-emphasize security unless it has some bearing on performance. I would also suggest that they be familiar with hardware offerings, current and future.


Pete


Pete
Trusted Contributor

Re: Capacity Planning!!!

Of the list you recomend:

1. HP Certified.
2. Perfomance tuning (Glance, Measureware, sar, vmstat, iostat)
3. LVM
4. Kernel Tuning.
5. NIS.
6. MC Service Guard.
7. Security.
OR did i miss anything? Please feel free to add additional skills required.

I would say for a capacity planner none are relevant. Of an admin all are relevant. My watch says Advanced IT Professional and I have a boatload of certifications. A planner needs none of those skills, but a top admin all of them...

Get out the checkbook, cause that is an expensive critter you are looking for...

Tim
Exalted Contributor

Re: Capacity Planning!!!

I think the skill list is pretty good for running the box.

I specified my first capacity planning for a box before I even knew what HP-UX was. It involved the process I posted before.

A working knowledge of LVM and Mirror/UX is a good thing because there are great benefits from using mirror, both for performance and reliability.

Here are some links and case studies on the subject:

http://whitepapers.informationweek.com/data/rlist?t=987097376_78577485

http://www.ftspower.com/ManServices/Capacity/default.asp

http://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=UTF-8&q=related:whitepapers.informationweek.com/data/rlist%3Ft%3D987097376_78577485

Enjoy your reading.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Honored Contributor

Re: Capacity Planning!!!

Hi Joe

I've been filling a similar but lesser role. As Tim S. has been saying good admin skills are paramount. Further to this they need to be fairly up to date on HP specs, bus speeds, bandwidth & throughput limitations, CPU, tpm-c etc. Unfortunately I know a few people that would satisfy the HP specs side but would be (actually are) rubbish at capacity panning. Why... because they do not know the system very well & above all they do not study or have any interest in the system data AT ALL.

Philosphically a system responds to the load placed on it. In engineering terms it could be thought of as a static analysis problem, with a load being placed on a structure in a particular way. The structure responds by supporting the load (or not as the case may be), system response. So far, very amouphous, glib set of statements, BUT IF YOU do not UNDERSTAND how the load is applied to your system you have VERY LITTLE CHANCE of understanding its response. Unfortunatley HP do not (cannot) do a course on deciding what is the important system load because it is up to YOU to determine what is important & what is not, number of users, number of queries etc. As an example we are a Telecom Co & could measure 100+ things but actually only measure, on various different platforms.
o Number of calls (5 min)
o Number of subscribers (dailily)
o insert rate into critical table (DB query, 5 min)
o Backlog of files to be loaded into critical tables (5 min)
HP do not have a chance of being able to determin the above (but we do use MeasureWare to LOG it & help analyse it). Once I have these loads that quite frankly are just educated guesses & some ksh, perl, sql & dsilog stringed together. I plot them against the system response (CPU% disk% ....)

We are lucky in that this method generates a relatively straight line so summarising it into a formula is the easy bit. Once you have a formula you can try some numerical fudges so X million customers is y call rate which in turn generates z loading etc...

So adding to your list
o DB skills (sql & be aware of the impact it has on other queries & the environment)
o perl, awk, ksh, ....
o Understanding of disk subsystems
o understanding of LAN speeds, latencies ...
o similar for SCSI
o ANALYTICAL & METHODICAL
o Did I mention ANALYTICAL & METHODICAL

Regards

Tim

BTW for the people that studied Engineering Physics Mathematics etc who do not like by static structure analogy, I'm working on a dynamic version, but it is 10 times worse & only really helps with some spiked/stepped loading profiles which are rare(ish)
-
Esteemed Contributor

Re: Capacity Planning!!!

Hello!

Yep it's good, also add
Hardware
Network
Basic compilations
Backups

Caesar
Advisor

Re: Capacity Planning!!!

Joe,

You have received a lot of good advice in this thread. What I have not seen (excuse me if I missed it) was the need for some basic Statistics background. It is very easy to make predictions that "clearly show" the need for lots more of some resource. Unfortunately, it is also very easy to make some assumption that is completely invalid.

One key strategy I have found in many years of doing this is to take *at least* two completely independent prespectives on the question. If they both come up with (roughly) the same numbers, they have a much better chance of being right (given that the assumptions are reasonable).

Another key point is to look at the system *as a whole* to make sure that addressing one bottleneck (and bottleneck analysis itself is a huge topic) does not immediately reveal another. (A classic example is upgrading a CPU or RAM only to find out that the network cannot support the faster system, and you have either spent a lot of money for nothing, or in some cases even made the problem worse).

Good luck.

Jim Hendrick
Super Advisor

Re: Capacity Planning!!!

Jim:

Thanks for the post. Can you expand on your statement please. It is not vey clear.

Thanks

Joe.
Honored Contributor

Re: Capacity Planning!!!

Joe

I think what Jim is alluding to is that a wholeistic aproach is best. There are various methods one could take in reaching a conclusion & hopefully two independant methods may yield a similar results.

1 - Direct measurement of live system, using MeasureWare, application logs, lightweight DB queries etc
2 - Testbed measurements
3 - Empirical test resulst & corolations, such as using tpm-c, tpm-h, SPECINT, SPECFP results to compare relative performances of systems.
4 - Rules of thumb, observations that you or your collegues may have developed
5 - Hardware knowledge,

generally no one of the above approachs will be sucessful in ALL situations. e.g. a new system or new application, can't use 1. Equally, cash may not be forth comming to get a fully speced testbed for 2. The only way to fill the gaps is to try & intelegently use what methods and data are easily available & try to fill in the gaps. You will probably need to be doing, cvertainly simple scaling mathematics, if not full blow statistical corolation.

e.g. You've been told that a current system will need to support double the numbers of users. Immidiately, you think, double number of processors, LAN cards, RAM & storage. Price tag too large. You then need to take a look at the system. You can do this in a couple of ways
1 - simply use MeasureWare/sar/top to look at various key peaks, CPU%, disk%, IO/s, network usage, number of users. thus forming a picture of what resources are being used the most & noting their peaks
2 - As 1 but try to corolate with some loading aspect, say number of users on the system.

the 2nd method is more scientific but requires more effort. 1st method is an overview of system, and you will get a good idea of the highly used resources.

Let us assume disk & CPU are both being used highly, then you would need to get some data together to figure out how to move forward. tpm-c/tpm-h results etc could be used for the CPU. I'll be more specific on the disk front. Assume you are using 20x 10,000 rpm disks, they have an average service time of say 8ms (assume seagate), you have a choice
a - buy more 10,000 rpm disks etc
b - replace existing 10,000 rpm disk with 15,000 rpm disks + more.
Both options will have a price tag attached & someone will need to decide whitch path is appropiate. To do this they need to understand HOW disks deal with IO, HOW the IO will increase & apply this to the problem in hand. I would start with disk service times. 10,000 rpm disk are about 8.0ms service times & 15,000 are about 6ms. This meas 15,000 are 35% (ish) faster than 10,000. Let us assume you believe the IO load will be 3 times larger. This will mean you will need
3x number of 10,000 rpm disks (+40 disks)
2.25x number of 15,000 rpm disk (+45 disks)

THUS you can get your calculator out & figure out the relative capital cost of each. Then put your support hat on & figure out the sysrem outage cost (time).

the above is a very simple example, tying to illustrate how different sets of data can thread together to alow a (sensible) answer to be be generated, or give someone a good foundation to make a decisiuon.

Regards

Tim
-
Honored Contributor

Re: Capacity Planning!!!

To me - the complete Capacity Planner (not just on HPUX) is a seasoned Systems Admin who has to have proven hands on experience and must understand all aspects of:

- Memory utilization
- I/O and bandwidth
- Storage management
- Load and resource management

By default, the senior Systems Admin most often bears the cudgels of capacity planning. Be careful --- new positions such as Enterpise Architects, Infrastructure Architects, etc.. must be scrutinized well -- that the person in that position is truly qualified. I have been contending with so called "Architects" that have either come from obscure backgrounds or are totally reliant on what the vendors say..

The capacity planner must be able to use tools (ie. CA, BMC Predict, etc...) or write his own tools to "model" loads on a system...

Hakuna Matata.
Trusted Contributor

Re: Capacity Planning!!!

I think Jim mentioned stats: definitely, an aptitude to represent data in a meaningful (to management) way is crucial.
I would say a couple of years as a SysAdmin would be OK, preferably on hardware where the planning needs to take place. Then a good intro to SAN and RAID solution from several vendors. A good trend analysis tool would be good (BMC, BEST/1, etc).
Trying is the first step to failure - Homer Simpson
Valued Contributor

Re: Capacity Planning!!!

I just wanted to add a minor point.

Someone was talking about "Structure".

I want to raise the issue of Architecture.
The reason certain architectures survive time and some do not holds some important lessons for a capacity planner.

I like to look at the Enterprise as a layer of structures (mostly connected) with some disjoint pieces.

Recognizing the source of a bottleneck looking at a architecture is also a essential skill of a capacity planner.

Thanks
Paddy

The sufficiency of my merit is to know that my merit is NOT sufficient
//Add this to "OnDomLoad" event