> I have a two node VMS cluster running OpenVMS 8.3.  [...]

   Not much meat on that bone.  About what kind of hardware are we
talking?  AlphaStation 200/100?  Some fancy multi-GHz-CPU server? 
Memory?  Quotas?

> Traditionally people have logged in via telnet, set their DISPLAY
> variable and then run the program. [...]

   Normaly, LOGIN.COM can be rigged to SET DISPLAY appropriately without
user intervention.

> [...] the telnet service on the VMS boxes seems to stop working rather
> frequently.

   Not a very complete problem description.

> [...] SSH and SSH X forwarding [...]

   Have you tried SSH for the user log-in, but plain-old SET DISPLAY for
the X stuff?  Again, it should be possible to SET DISPLAY automatically
in LOGIN.COM.  RSH, too, can be rigged with proxies to avoid shipping
plain-text passwords around, of course.

> [...] very sluggish [...]

   What investigation have you done?  Is the CPU (are all the CPUs)
busy while this stuff is running?  Memory?  Page-fault rate?  For a
start:

      help monitor

   I seem to recall some documents existing which purported to provide
some guidance toward improved performance on VMS systems.  A Web search
might find some things.

> [...] a Linux machine [...]

   The description of this hardware is no more useful than the earlier
one of the VMS hardware.  I would not be amazed if a modern,
resource-rich x86 system were faster than some decade-old, resource-poor
Alpha system.  But, with my weak psychic powers, I know approximately
nothing about your hardware (or the users' resource quotas, or anything
else, really).

> [...] SSH was starved for decent entropy for encryption purposes, [...]

   I know nothing, but I thought that this sort of thing affected the
initial set-up of an encrypted connection, not the operation of that
connection once it had been established.