topic Re: Experience with Availability Manager V2.6 in Operating System - OpenVMS

Experience with Availability Manager V2.6

Bart Zorn_1 — Tue, 29 Aug 2006 05:33:11 GMT

This is not a question but a request for comments (an RFC ?).

Although I still am very disappointed that OpenVMS product management does not want to listen to all requests to keep AMDS at least working (that is, supply the minimum needed for newer OpenVMS versions), I have experimented a bit more with Availability Manager.

Now that AM finally has the equivalent of AMDS's system overview, it gets to a state that one could call 'useable'. Of course, there are problems:

- I have one DS15 with 1 Gb memory to run AM on (Java V1.5, FastVM), and a second DS15 also with 1 Gb (my workstation) to display the output. However, this seems to be not enough. Opening menus from subwindows (such as the Node display) takes several seconds. Repainting of windows (expose events) also take several seconds. The first DS15 is running at 65% cpu load all the time and it has little else to do but run AM. What configuration would I need to make it faster?

- When I exit AM on my workstation, CDE crashes. This is annoying, to say the least. DECW$SERVER_0_ERROR.LOG gives me no clue. I may find the time to open a call for this, but I probably have to crash CDE many times more to find out if there is relevant information somewhere.

There is also a positive point: When I open the Group overview window and minimize the main window, the CPU load on the DS15 drops to about 1%. I like the ability of the group window to enlarge the font!

AM is getting better, but it is still no AMDS!

Regards,

Bart Zorn

Re: Experience with Availability Manager V2.6

Bart Zorn_1 — Tue, 29 Aug 2006 07:49:12 GMT

I have to make a correction. The CPU load on the DS15 varies between 20% and 30%, not 65%.

The observed respons times, however, are correct.

Bart

Re: Experience with Availability Manager V2.6

Barry Kierstein — Wed, 30 Aug 2006 12:04:34 GMT

Bart,

CDE crashes? That is a new one, haven't seen the problem here... What does the accounting record for the process say? Sometimes that has a clue. Look for quota exhaustion, etc. and the final status.

Are you running AM GUI on the console of the DS15, or throwing the display from the DS15 to another system? It sounds like if the DS15 is on your desk, you are using the console, but I like to be sure. If you are throwing the display, make sure you have installed the UPDATE patch for OpenVMS.

You might need to adjust a couple of the memory qualifiers. One other customer uses /IHEAP=48M/MHEAP=64M. If this is the case, you will see less CPU time spent on the application. This may help the window delays you mentioned.

Barry

Re: Experience with Availability Manager V2.6

John Gillings — Wed, 30 Aug 2006 17:45:12 GMT

Bart,
Sounds like a bug to me. Please log a case with your local customer support centre.

Re: Experience with Availability Manager V2.6

Bart Zorn_1 — Thu, 31 Aug 2006 02:01:26 GMT

Barry, John, thanks for your reaction!

I am running AM on one DS15, in the datacenter, which has access to the Vlan on which the production systems sit, The display goes to a second DS15, on my desk, which (of course) only has tcp/ip access.

I used the default memory qualifier values until now. Of course, as luck would have it, when I stopped AM to restart it with the new qualifier values, CDE did not crash. I will try to find the accounting record of the last crash.

Now that AM is running with the bigger heap, the CPU load is still high, if not higher than before. I have been looking a few minutes, and according to AM itself the load varies between 50% and 80%.

Would it be useful if I open a call with HP?

Regards,

Bart

Re: Experience with Availability Manager V2.6

Volker Halle — Thu, 31 Aug 2006 03:42:41 GMT

Bart,

if you're running V7.3-2 (or higher) on the DS15, which you use to run Availability Manager Console/Analyzer, you could use System PC sampling to find out, where the CPU is spending its cycles:

$ ANAL/SYS
SDA> PCS ! to get some basic HELP
SDA> PCS LOAD
SDA> PCS START TRACE
... wait some time while running AM ...
SDA> PCS STOP TRACE
SDA> PCS SHOW TRACE/STAT
SDA> PCS UNLOAD
SDA> EXIT

Volker.

Re: Experience with Availability Manager V2.6

Bart Zorn_1 — Thu, 31 Aug 2006 04:01:19 GMT

Thanks, Volker.

I did have a look at the pc samples, and it sits most of its time in JAVA.

But we did not expect anything else, did we?

Regards,

Bart

Re: Experience with Availability Manager V2.6

Bart Zorn_1 — Thu, 31 Aug 2006 08:51:27 GMT

I see no way to identify the correct accounting record, because accounting records do not carry process names.

There are some processes terminated around the time of the crash: some with 00002BD4 (%SYSTEM-F-EXITFORCED), and some with 0001C0F4 (%RMS-F-RER)

Is there anything else I could do to retrieve usefule information?

Regards,

Bart

Re: Experience with Availability Manager V2.6

Barry Kierstein — Fri, 01 Sep 2006 09:45:12 GMT

Bart,

On those accounting records, compare the peak page file and working set with those of the account, in case there is a resource exhaustion problem. You may want to set up a job that does a $ SHOW PROCESS/FULL/ID= periodically on your server process to catch more details. This may catch some problem with CDE.

The amount of CPU time AM uses depends on how much data is being collected and what is being shown on the screen. The 60% seems a bit excessive from what the team has seen here, the 15-30% range is more like it.

Work is being done to streamline the application where bottlenecks are found. Another is that the next version of AM will use the JVM 1.5, and that seems to be about 20% more efficient. Also, the team is working on WAN support, which makes the amount of network traffic much less (data travels in an IP stream).

Barry

Re: Experience with Availability Manager V2.6

Jeffrey Goodwin — Fri, 01 Sep 2006 12:52:28 GMT

I'm also using AM V2.6A and am seeing similar system utilization as Bart. I have a DS10 running AM that is exporting the display to a Windows system via X. CPU utilization may dip down to 60% at times, but I'm typically running around 90%. In addition to the default collections, I'm also collecting Cluster, CPU Mode/Process, and Lock Contention. I have three groups with a total of 15 nodes being collected. I've increased memory usage to the point where it appears AM is mostly running in memory.

-Jeff

Re: Experience with Availability Manager V2.6

Volker Halle — Sat, 02 Sep 2006 01:18:17 GMT

Bart,

did you run

@SYS$COMMON:[AMDS$AM.JRE.LIB]JAVA$CHECK_ENVIRONMENT

as described in chapter 6.4 of the installation guide ? It is to supposed to check user/system parameters and give recommendations.

Volker.

Re: Experience with Availability Manager V2.6

Bart Zorn_1 — Thu, 07 Sep 2006 01:42:28 GMT

Barry,

I found no accounting records which indicate a resource problem.

I am looking forward to the WAN support!

Volker,

I did run the procedure and it does not report anything.

Thanks for your thoughts!

Bart

Re: Experience with Availability Manager V2.6

Barry Kierstein — Thu, 07 Sep 2006 09:49:55 GMT

Bart,

Too bad about the accounting records...

WAN support is the major thrust in the next version of AM, and it is working well so far.

Jeff,

I did an experiment on a DS15 (closest I could get to a DS10) with 14 nodes, your data collections, etc. It showed around 40% CPU utilization. When I ran AM on the local graphics head, it was about 32%, so going over the network does cost. Working on performance, and found some things that help that will be in the next version of AM.

Hope this helps.

Barry

Re: Experience with Availability Manager V2.6

Jeffrey Goodwin — Thu, 07 Sep 2006 13:00:54 GMT

Barry,

Can you give us a rough idea when we might see the version with WAN support released?

-Jeff

Re: Experience with Availability Manager V2.6

Ian Miller. — Fri, 08 Sep 2006 04:56:10 GMT

The VMS roadmap shows Availability Manager V3 scheduled for end of 2007.

Re: Experience with Availability Manager V2.6

Bart Zorn_1 — Mon, 11 Sep 2006 02:20:06 GMT

Barry,

Please let us know when there is something available for field testing!

Thanks,

Bart

Re: Experience with Availability Manager V2.6

Barry Kierstein — Mon, 11 Sep 2006 08:55:32 GMT

All,

The team is determining the final scheduling for V3.0, as well as scoping out user authentication and encryption issues, so will know more later on.

If you all were to use WAN support, what would be your situation:

1) Used inside a corporate environment (on site or virtual) - no security really needed
2) Will need to cross the corporate firewall or some other IP hurdle - some form of security/encryption needed
3) Access from a public IP site to the OpenVMS machines - security/encryption needed
4) Other

Thanks!

Barry

Re: Experience with Availability Manager V2.6

labadie_1 — Mon, 11 Sep 2006 09:02:22 GMT

1) is my answer

Thanks for caring

Re: Experience with Availability Manager V2.6

Ian Miller. — Mon, 11 Sep 2006 09:07:02 GMT

2) is more likely for me as corporate networks I deal with have internal routers, firewalls etc.

Re: Experience with Availability Manager V2.6

Dave Gudewicz — Mon, 11 Sep 2006 10:37:08 GMT

In some cases #1 and in others #2 in our environment.

Dave...