topic Re: rx3600 has 8 CPUs after installing 83I Update V4.0? in Operating System - OpenVMS

rx3600 has 8 CPUs after installing 83I Update V4.0?

Michael Moroney — Wed, 17 Oct 2007 17:12:18 GMT

While I like the idea of a free hardware upgrade that comes included with a patch kit, something's definitely not right here.

The HP rx3600 has 2 Itanium dual core chips which should show up as 4 CPUs on VMS. And SHOW SYSTEM would confirm this, CUR processes would have a CPU numbered 0-3. (running VMS V8.3) I install the VMS83I UPDATE V4.0 kit and reboot, and I happen to notice processes using CPUs 6, 7.
$ SHOW CPU reveals the following:

STARDV> sh cpu

System: STARDV, HP rx3600 (1.59GHz/9.0MB)

CPU ownership sets:
Active 0-7
Configure 0-7

CPU state sets:
Potential 0-7
Autostart 0-7
Powered Down None
Not Present None
Failover None

What's going on here? It has 8 CPUs now? I look at the computer itself, it looks the same as before. I've also seen at least 7 of the CPUs in use when doing a SHOW SYSTEM.

Re: rx3600 has 8 CPUs after installing 83I Update V4.0?

John Gillings — Wed, 17 Oct 2007 18:13:44 GMT

Michael,
Interesting!

What does:

$ ANALYZE/SYSTEM
SDA> CLUE CONFIG

say?

$ SHOW CPU/FULL and
$ SHOW LICENSE/CHARGE

might also be interesting.

Something else worth trying, create a DCL procedure:

$ c=0
$ loop: c=c+1
$ IF c.LT.100000000 THEN GOTO loop
$ LOGOUT/FULL

(or some other suitably large number)

Now submit 8 copies of this procedure at the same time. When they've all completed, compare the total CPU time with the elapsed time.

Re: rx3600 has 8 CPUs after installing 83I Update V4.0?

Hein van den Heuvel — Wed, 17 Oct 2007 21:12:08 GMT

It would seems CPU threads (Hyperthreading) are turned on.

If you go to the EFI Shell>
and try: cpuconfig
What is under the "# of Logical CPUs" column?

Try again after playing with
Shell> cpuconfig threads [ on | off ]

If Hyperthreading is in play, then the cpu-time versus wallclock time picture muddle quit a bit, as it heavily depends on th intensity of memory stalls to exploit overlaps.

According to the 8.3 doc, a $SHOW CPU /BRIEF and /FULL will show hyperthreading is enabled referring to it as cothreads.

Check the 'COTHd:' field in the display?

hth,
Hein.

Re: rx3600 has 8 CPUs after installing 83I Update V4.0?

Michael Moroney — Thu, 18 Oct 2007 15:13:50 GMT

That (hyperthreading) must be it!

Show CPU/BRIEF shows (for each CPU) something like this:

CPU 7 State: RUN CPUDB: 8811E600 Handle: 000064E0
Owner: 000004C8 Current: 000004C8 Partition 0 (STARDV)
Cothd: 3

I ran a batch job like what John G. suggested when things were quiet, and when there was just one instance running I got CPU and wall clock usage about the same:

Charged CPU time: 0 00:02:53.12 Elapsed time: 0 00:02:53.13

but when 8 were running each has a result like this:

Charged CPU time: 0 00:02:55.14 Elapsed time: 0 00:05:50.44

or elapsed time double. This makes sense since it appears that hyperthreading makes one physical CPU act like two virtual CPUs, if I understand it correctly.

What is the advantage of hyperthreading for VMS, if any?

Re: rx3600 has 8 CPUs after installing 83I Update V4.0?

Hein van den Heuvel — Thu, 18 Oct 2007 16:43:06 GMT

At the OpenVMS Bootcamp 2006, in session A901 Clair Grant & Burns Fisher did a 1 hour session titled "Intermediate Hyperthreads on Montecito and OpenVMS".
I'm not going to be able to summarize all that in 2 minutes.

Unfortunatly I don't think I am supposed to share that slideset here and now. I'll poke them to see if they are willing to reply here.

Bruce Ellis presented slides on hyperthreading in Canada using:

http://www.encompasscanada.com/documents/sem-tor07/tor07%20Ellis_Integrity_server_perf.pdf

Hoff has a web page: http://64.223.189.234/node/13

Google for +Hyperthreads +Montecito will give a good few docs to check

Here is my take, with some hands-on experience with Xeon hyperthreading.

- Hyperthreading tries to utilize the waits for memory which can be many ( 80+ ) cycles.
- It _should_ be strictly positive... but it is not.
- The extra scheduling and increased contentions slows things down.

- The scheduler needs to learn that all CPUs are NOT create equal... if thread 0 on core 1 is busy, and no thread is busy on core 2, then the scheduler should not pick CPU 5 (thread 1, core 1) but CPU 2. They 'look the same' but they will not behave the same. Yikes.

- Many OpenVMS SMP solutions already suffer from contention (think spinlocks and MPsync) where you have cycles available, but can not use them just yet. Adding more active threads is threat to the general health.
- Hyperthreading has been shown to give up to 30% improvement for just the right cpu / memory load mix, mostly memory bound where the cpu seems busy 100% in usermode but is really waiting for memory to come in. To get to htat 30%, one often needs active, aggresive, CPU affinity settings.
- OpenVMS applications tend to run less than 20% usermode and the inner modes often involve synchronization and serialization.

- Practically Hyperthreading shows -5% to + 10% gain, more towards 5%.

- Hyperthreading cause 100% confusion

So the risk-reward picture isn't great.

Therefor, OpenVMS took the typical cautious stance and decided to initially disable hyperthreading, but at least they did keep it available.
They may have changed the attitude with the update, but I would have expect that to be well documented (was it?), or maybe something slipped through the cracks?!

Ok... that was a little more than 2 minutes.

Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

Re: rx3600 has 8 CPUs after installing 83I Update V4.0?

Hein van den Heuvel — Fri, 19 Oct 2007 06:18:23 GMT

Clair (Thanks Clair!) reminded me that :

"VMS can't diretly turn on HT. You do it from the console. We have a program sys$test:hthreads which will toggle it but the change doesn't take effect until you reboot."

So... all the kit might have done is trigger a reboot.

If any curious privved user on the box had run sys$test:HTTHREADS any time after the last reboot before that update, then the effect of running that program would only show when rebooting after installing the update!

Hein.

Re: rx3600 has 8 CPUs after installing 83I Update V4.0?

Martin Vorlaender — Fri, 19 Oct 2007 14:31:55 GMT

The TechUpdateDays recently held around Europe also had a session on Hyperthreading. Online e.g. at http://de.openvms.org/TUD2007/Hyperthread_Support_in_OpenVMS_2.pdf

HTH,
Martin

Re: rx3600 has 8 CPUs after installing 83I Update V4.0?

Martin Vorlaender — Fri, 19 Oct 2007 14:33:54 GMT

I just noticed the slides are in german. Sorry - it was the only german presentation given...

cu,
Martin

Re: rx3600 has 8 CPUs after installing 83I Update V4.0?

Michael Moroney — Sat, 20 Oct 2007 08:13:46 GMT

Hi,

I want to mention that a few months ago I did poke around in sys$test: and probably did run sys$test:hthreads.exe. As mentioned offline, it doesn't take effect until reboot, and VMS being VMS, no reboot was needed for months. Update 4, the reason for the reboot, is not to blame.

Re: rx3600 has 8 CPUs after installing 83I Update V4.0?

Michael Moroney — Sat, 20 Oct 2007 08:17:53 GMT

Cause is hthreads.exe not Update 4.