IT Operations Management (ITOM)
Showing results for 
Search instead for 
Do you mean 

Monitoring virtual environments - VM101

‎08-06-2014 09:11 AM - edited ‎02-08-2016 11:53 AM

Monitoring virtual environments is highly important - we have built multiple products focusing on this.

Here is a diagram to help you better understand our products and how our solutions can help you based on your needs.

 

My focus in today’s blog is to explain this picture, as I lead you through this introductory course on how to manage your virtual environments.

 

To start with, let us understand why you need special software to monitor your virtual machines. Why is it not enough to monitoring your virtual machines's guest OS? Why is something more required?

 

To answer that I need to digress a bit here...

Let's not forget the premise for monitoring is to support business continuity. IT and monitoring teams need to ensure that all the business functions must run without failures that impact the business. Monitoring must make sure that there are no failures arising from machine downtime, or poor performing apps and/or operating systems. However, if monitoring is not applied in the right place, and in the right way, there's always possibilities that notifications are not raised in time, alerting potential / occurred problems. We've discussed some ideas in best practices for monitoring blog posts Monitor the job queue, ascertain system bottlenecks & Set threshold values correctly for good monitoring since not every day is Sunday earlier.

 

In the world of virtualization and resource sharing it is important to know how 1 VM affects another VM running on the same host.  This cannot be achieved just with monitoring the guest OS.

 

For virtual machines there are multiple ways and multiple aspects to monitor and some key things to keep in mind.

 

  • The first aspect to monitor is the machine itself - though it is a machine, it is all software (like thermodynamics is all hot air :) ). It is important and appropriate to monitor the machine. How do we have access to this software based machine? That's via the hypervisor/host as well as via management software like VMware vCenter / Microsoft SCOM.  It is important to note that the regular performance counters obtained from the guest OS (say via Windows perfmon) is not the same as performance counters obtained by querying the VMware ESXi host running the VM. Accurate metrics for performance are always obtained when querying the host or the element management software managing the VMs, not the guest OS.
  • The next aspect to monitor is the guest OS - Based on the point above, when is monitoring  the guest OS required? One must monitor within the virtual machine - to know the state of apps running on the VM and to obtain information like disk space left on the system. Is an agent or running agentless monitors always required on the guest? Not always.

          - If you are running crucial apps on the guest, you may choose to go with a full agent with the complete set of monitoring policies from HP OM/OMi or monitors from SiteScope for the app (agent-based or agentless - refer here)

          - With an agent it is possible to monitor and gain information about the system errors and app faults, using log monitoring too

          - If it is a dev-test environment, crash-and-burn lab, and app metrics inside the guest OS are not important, you could go with an approach to not monitor the guest OS

          - There's always approaches like using micro-instrumentation shipping with HP Virtual Performance Viewer (version 1.2 upwards) to obtain information from the guest OS and apps in a non-intrusive way

  • Availability monitoring / Uptime calculations - For node availability checking as well as uptime calculations, it is best to monitor the virtual machine from the host system or management software like VMware vCenter. System time inside a guest OS is not always correct as there can be delays in updating system time on the host process running the VM
  • Due to the way virtualization works and adds overheads to the running apps to interact with hardware, a VM is not always great with IOPS - so it is important to monitor the IO of a virtual machine's guest OS and apps - this has to be done from the storage perspective, in conjunction with the guest OS apps.
  • Reporting software must use a combination of metrics obtained from the hypervisor host as well as within, in order to provide the end-to-end perspective on VM and virtualized application health.

The good news is that this behavior is common to all virtualization whether it is Windows or Linux (x86) or on mission-critical UNIX platforms.

 

 

How can software help with these scenarios?

 

HP SiteScope is agentless monitoring for virtual and physical machines. One typically tends to use this to perform continual monitoring (say every 10 minutes or so) of machines. HP SiteScope is quick to setup and easy to get to work. HP SiteScope also provides a VMware vCenter Solution Template that provides rich metrics for monitoring of VMware hosts. 

 

HP Operations Manager is a systems monitoring tool monitoring virtual and physical systems, and it uses OM agents and SiteScope for monitoring the systems (a.k.a OM managed nodes). HP Operations Manager's Virtualization SPI (VISPI) allows one to monitor virtual systems and virtualization hosts. VISPI gives 24x7 monitoring of virtual environments including monitoring for UNIX virtualization (LPAR, HPVM, Oracle Solaris Zones).

 

HP OMi's Virtualization MP provides all of VISPI functionality for the OMi users. With Monitoring Automation, there's the added benefit of simplified deployment and use above what OMi already offers which makes it an good choice over just HP OM.

 

HP vPV is a  monitoring tool for x86-architecture based virtual and cloud environments with a visual interface to virtual performance, capacity, and health at-a-glance. You can also drill down into specific performance bottlenecks or capacity issues. With vPV 2.0, there is also service discovery and event integration to HP OM (and with that, to Ops-Bridge).

 

HP Service Health Reporter is HP's enterprise data warehouse software with pre-built cross-domain content and reports. You can use HP SHR with most of the software listed above to obtain the data, and report upon.

 

So to break it down in simple logic -



(flowchart alongside - click to view in full size) 

 

  • If you have VMware, Hyper-V, Linux kernel-based virtualization or Openstack on x86 architectures, first choice for monitoring these is HP vPV. vPV offers you higher scale of monitoring (upto 6000 VMs)  and heterogeneity - see figure on top. vPV offers advanced monitoring capabilities, and capacity planning features. vPV does bring in additional overhead on vCenter - this is natural and any application doing this kind of data collection will impact the vCenter.

The next set of points are only if vPV does not work out for your x86-virtualized environments, for any reason. (I would like to know why, please comment if so).

 

  • The next choice for monitoring x86 virtualization is the HP OM VISPI with the approach to do monitoring using HP OA Virtual Appliance - this approach is best suited for teams used to monitoring via HP OM policies, and also interested in monitoring a smaller environment without need for too much drill-down or capacity calculations for virtual machines.
  • If vCenter is not accessible (it is controlled by another team), or on the other hand, if you just have a small set of ESXi/ESX hosts to monitor then you can go with the VISPI still, but use proxy-monitoring via the VMware vMA virtual appliance. NOTE: HP recommends to move towards using the HP operations agent virtual appliance as VMware vMA support may not be available for longer term.
  • If yours is an organization more driven towards using SiteScope for monitoring your systems and applications, SiteScope's  VMware performance monitor as well as the 'VMware Host' and ' VMware Storage' Solution Templates can be used.
  • If you have more than 20 systems, do not go with using multiple vMA devices to monitor these. Use the HP virtual appliance to make your deployment more simplified - 1 HP VA instead of 4 or 5 vMAs.
  • If you use resource pools for DRS in your vCenter, it is recommended to go with the HP virtual appliance,  as using VMware vMA causes duplicate resource pool CIs to get created in the central topology (RtSM) and data warehouse (SHR's PMDB) as represented on the ESXi hosts to which vMA connects for data collection.
  • If you have IBM LPAR based AIX instances, HP VMs or Solaris zones (containers) then you must use the HP OM VISPI to monitor these. For example, if you have x86-based virtual machines as well as AIX LPARs, then you could go for using a combination of tools

         - HP OM VISPI for monitoring the AIX LPAR, derive deep metrics from the OM agent running on one LPAR for each powerpc frame

         - HP vPV for monitoring the x86-based VMs

 

Hope your doubts in this area are now resolved. Feel free to pose further questions.

About the Author

Ramkumar Devanathan

Ramkumar Devanathan (twitter: @rdevanathan) is Product Manager for HPE Cloud Optimizer (formerly vPV). He was previously a member of the IOM-Customer Assist Team (CAT) providing technical assistance to HP Software pre-sales and support teams with Operations Management products including vPV, SHO, VISPI. He has experience of more than 14 years in this product line, working in various roles ranging from developer to product architect.

Comments
thunderdog
on ‎04-13-2015 11:06 PM

Good article, but i was wondering suppose i have 200 VM's on EXSI host then to monitor them

 

1.Do I required to install OM agent manually in all 200 VM's ?? as far as i know it cant be automated using HP-OM only

 

2.Is there any way in which i can view all my VM in virtual environmnet on HP-OM

on ‎05-12-2015 04:57 AM

Thunderdog, thank you for your feedback.

 

The answer to "Do I required to install OM agent manually in all 200 VM's " is No.

 

What do you wish to monitor in the VM? Do you wish to monitor applications running in the VM? If so, you need a monitoring capability that allows you to regularly poll the applications - this can be achieved via SiteScope or the OM agent with some SPIs or other collection and monitoring policies.

 

If all you wish to monitor, is the resource utilization done by the virtual machine, then you can do with just vPV. Note that vPV offers much more than just monitoring. In the newer editions of vPV you can do so much more like forecasting a resource unavailability for a VM, host, datastore, obtain right-size recommendation for VMs, waste reclamation - recovering unused but allocated storage, etc.

 

 

on ‎05-12-2015 04:59 AM

The answer to "Is there any way in which i can view all my VM in virtual environmnet on HP-OM" is Yes.

 

With the vPV - HP OM and HP OMi integrations the virtual environment topology gets forwarded to HP OM  and HP OMi RtSM respectively. With this your virtual machines are part of the monitored object set and of course can be viewed in the management ui.

Arun_S_TQ
on ‎06-17-2015 03:27 PM

Your note on HP's recommendation to move to HP Operations Agent Virtual Appliance, could you point me to a document or something that explains this in a little more detail? I am looking understand the reasons behind it.

 

Also, is it typical to have both the HP Operations Agent Virtual Appliance, and the vMA deployed in one environment or does HP advice against it? We have both of them deployed and we noticed that the values for some of the metrics that these two solutions collect seem to be considerably different, for the same collection time. Due to this we are not able to get a good grasp on the resource utilization.

 

Thanks,

Arun

 

on ‎06-17-2015 10:12 PM

Arun, with the vMA you are directly connecting to the ESXi host, in case of OA-VA we moved towards getting metrics from the vCenter. So it is possible that the metrics may vary based on the time of getting data, and summarization that vCenter does.

 

It is definitely not recommended to load up both vMA and OA-VA for collection, against the same hosts - this unnecessarily increases the CPU cycles for data collection.

 

- ramd

Vijay101
on ‎08-23-2015 02:13 AM

 

In our envionment vendor configure the site scope to connect vcetner and monitoring esxi. the problem is i have 40 ESXi and its configure to establish 40 connection to monitor the 40 ESXI and we are in a need to add more esxi that means more connection from sitescope which could impact normal operation to vcenter, is this a the recommended way or is there a way to configure in such a way that HP site scope establish single connection to vcenter and monitor all 40 ESXI. BMC Patrol does this, it establish a single connection to moinitor all esxi.

 

Thanks 

Vijay

on ‎08-24-2015 02:41 AM

Hi Vijay, with SiteScope there's a connection pool that is used. So multiple connections (esp at the rate of 1 per host) will not be opened to the vCenter. Refer to the SiteScopeMonitorReference.pdf for VMware monitors for more information on this.

 

vPV also uses a connection pool similarly connecting to the vCenter.

Mani989415
on ‎11-10-2015 10:42 AM

We are using OML 9.20 , and HP SIM . What is the best way to  monitor only the Esxi hosts.

My preference is to monitor via Agent base monitoring.

on ‎11-11-2015 08:12 AM

Mani989415,  you cannot (no one can!) place a regular agent on ESXi hosts - this is not supported by VMware, and also consider that ESXi does not have a full OS on which to run the agent. The only access is the ESXi support shell, and that's only for that 'support'. Note that VMware tools can be considered to be some kind of agent, and to avoid any confusion there, that's the only semblance of an agent that is known to run (and supported) on ESXi.

Coming to your other question - one of the better known ways to monitor the ESXi performance is via reading vCenter metrics. vPV does this, and also connects back to the ESXi hosts for real-time (20s) metric collection.

HP SIM probably could connect to the hardware providers on the iLO for the HP Proliant server and would also get you the important fault information for hardware components, as well as all hardware diagnostic data.

If you are with Proliant Gen8/Gen9 boxes you could also go for HP's OneView for vCenter - which will give you an integrated solution for this. vPV integrates with OneView for the entire end-to-end story too.

Hope this helps.

joe7790
on ‎02-22-2016 02:14 AM

Hi,

   I'm new to VPV. May I know what is required to integrate BSM/OMI with VPV ? Does it require addtional integration software , etc ? Please provide me some guidelines on the integration point. Can VPV run on Windows platform ?

 

Thanks.

Joseph

on ‎02-22-2016 06:44 AM

Joe7790, you would integrate vPV (Cloud Optimizer, as it is known now, new name) with OMi / BSM using the integration approach documented in the online help. This only requires the HPE Operations Agent to be installed on the vPV server, a management pack on the OMi side, and deploying policies to the vPV node.

Windows is not supported. Only Linux.

on ‎06-06-2016 03:31 AM

HI Ram,

‎06-26-2016 11:45 AM - edited ‎06-26-2016 11:49 AM

Hi Ram,

In this blog you did talk about monitoring Virtual Machine that is HP OMi Virtualization MP provides all of VISPI functionality for the OMi users. With Monitoring Automation, there's the added benefit of simplified deployment and use above what OMi already offers which makes it an good choice over just HP OM.

Above means need to install OA 12 on VMware and deploy Virtualization MP or need to install OA 12 on Guest OS. Can you please elaborate on this part. As i am aware we cannot monitor VMware using OA12. We have a customer, they want to monitor VMware, they have OMi and do not have Sitescope.

Please let us know what is the best way to Monitor VMware considering only OMi 10.x.

on ‎06-26-2016 12:21 PM

Deepak, there are many q&a above. I hope you went through these.

For VMware environment monitoring, you should employ Cloud Optimizer (vPV). This is integrated with HP OMi and is free to use (no licenses needed in the 'monitoring-only' mode which gives better than VISPI capabilities).

Installing an agent on each guest OS will give you only half the picture in the monitoring space. Consider a VM running on a ESXi host where several other VMs are running. Now if one VM decides to take up a lot of resources all other VMs are potentially affected, since it is in the nature of virtualization to share resources. this can lead to troubles at the service/ application/end-user level, that transactions are slow. The root-cause identification would be hard to do, for this, if you had an agent on each guest.

The right solution is to connect to the hypervisor layer (ESXi/HyperV/KVM/others) and ascertain what's going wrong, what needs to be corrected. This is the aim with which we have built Cloud Optimizer for monitoring the virtualization layer - the hosts (ESXi, etc), the VMs (not just the guest OS in these) and also the storage part. CO not only provides standard monitoring, but since it stores the data in the vertica db and runs capacity analysis on the data, it can provide optimization and efficient recommendations too - this in turn will help improve application performance at the end-user level.

have a look at the following -

http://community.hpe.com/t5/Business-Service-Management/Monitor-your-x86-virtualized-environment-for-free-with-vPV/ba-p/6816690

With CO you can get cpu, memory, disk IOPS, network IO, and also space usage information for the VMs, hosts, storage and this is likely better than what you can get from running agent in the guest OS, sicne this data is obtained from the hypervisor layer (for VMware, we connect to vCenter, which is in turn connected to ESXi hosts)

So, when does one need OM agents in virtualization world?

HPE OM agents (or an agentless monitoring tool such as HPE SiteScope ) are needed to monitor the guest OS, logs, apps on the guest. This cannot be achieved today in CO. So, if you need to know how well your db is performing on a VM, deploy OM agent onto the guest OS and start monitoring with Oracle MP or SQL Server MP policies/aspects. this cannot be replaced.

 

 

on ‎06-26-2016 12:39 PM

Thanks a lot for detailed explaination, it clears all the doubt.

Events
Each Month in 2016
Online
Software Expert Days - 2016
Join us online to talk directly with our Software experts during online Expert Days. Find information here about past, current, and upcoming Expert Da...
Read more
Sep 30
Seattle, WA
OpenStack Days Seattle
OpenStack Days Seattle, September 30, is the largest gathering of OpenStack users and prospective users in the Pacific Northwest region.
Read more
View all