GS1280, 8 CPU, Shuffle topology

[Alexey Borchev]

Hi!
I've discovered that Alpha EV7 can be configured not only with Torus topology, but with shuffle too. (see attachment).
It is more efficient, at least for 8-CPU systems.
1. Every CPU connected not to 3 but to 4 other CPU's.
2. Average number of inter-CPU hops is 12/8 for torus, and 10/8 for shuffle => ~15% performance boost.

- Is it supported by Marvel chips?
- Is it supported by Tru64?
- Is it supported by HP?
- If anybody tried this setup?

[Hein van den Heuvel]

Interesting.

So instead of the loopback module (These modules are basically an interprocessor (IP) cable in etch) going for example from south 5-7 to north 4-6 you would run an ip cable from south 5-7 to norht 0-2.

I quick perusal of some hp internal site does not mention this configuration.

I'll send a friend in Alphaland (Marlboro, MA) a pointer to this topic and hope he will react here.

hth,
Hein.

[Hein van den Heuvel · Accepted solution]

Answer came in:

"We looked at alternative EV7 interconnect schemes a couple of years ago. Some performance advantages were shown, but only for the extreme boundary cases for loads on the systems. EV7 has such low latency and high bandwidth that the practical value of these changes would be small for most real applications. The firmware changes and support complexity argued for having only one consistent interconnect scheme."

So to make this work well, a firmware change would be needed which is not available.

On my next question, whether it would work:

"It probably won't work at all. In any case, definitely not supported."

hth,
Hein. (don't shoot the messenger please :-).

[Alexey Borchev]

OK, it was expected answer.
Shuffle connection scema is evidently much harder for interconnection scemas, routing algorithms and support.
I just can't imagine Shuffle scema for 32 CPU system, if this possible and more efficient then torus.

OK, it was still worth asking the question.

P.S.
I had to put this question in General category.
Maybe, it is reasonable to make new TRU64 forum sub-category - NUMA issues?

[Hein van den Heuvel]

>> I just can't imagine Shuffle scema for 32 CPU system

There would be no point. The shuffle removes a duplicate/double connection between a pair of CPU's (N->S and S->N) and uses it to connect to other CPUs, establishing more short paths. In the 32 CPU case the each cpu is connected to 4 different cpu's already.

>> I had to put this question in General category.

That's fine. Strictly speaking it probably does not belong in Tru64 at all. First and foremost it is an Alpha Systems question. Once the system supports it, it becomes a Tru64 question whether it can handle the distance tables in the proposed config.

Thanks for bringing up the topic though, while there is still interest around, and while there are still knowledgeable engineers available.

Hein.