topic Re: CMU 7.1 - New Info: xterm crashing in Server Clustering

CMU 7.1 - New Info: xterm crashing

BrentGee — Tue, 26 Nov 2013 21:00:25 GMT

xterm still crashes every time I use VSP and reboot a node. Nothing new here. I have no idea where to start troubleshooting this. Very troubling. However, now, for even more reasons I cannot understand, I can no longer use KDE or Gnome terminals instead of xterm. I know that this stuff isn't supposed to be 100% fool-proof (because of how tricky both those environments are. But here's the thing: I was using Gnome 2.3 as the terminal no problem for about a week. Nothing has changed as far as I can tell. And all of a sudden it just stops working. No error message. Nothing. I select a node, click SSH connection and I can wait for hours and nothing happens. As soon as I edit the configuration file to use xterm again, everything works - except that xterm crashes every time I watch a node reboot.

This is extremely frustrating. Is there nothing I can do to debug this? I've submitted a stack trace and reported all pertanent info that I could think of in my previous post. Does nobody else experience this?

I have 950 nodes to maintain and I sincerely dislike flying blind. There are less convenient work-arounds and I really don't want to switch back to 7.0 but if I can't resolve this issue, then it's pretty much a show stopper.

Once again, any advice would be extremely helpful. Thank you for your time.

Re: CMU 7.1 - New Info: xterm crashing

Dennis Handly — Tue, 26 Nov 2013 21:15:05 GMT

>I've submitted a stack trace and reported all pertinent info that I could think of in my previous post.

Your previous topic:

http://h30499.www3.hp.com/t5/HP-Insight-Cluster-Management/CMU-7-1-xterm-crashing/m-p/6259947

Re: CMU 7.1 - New Info: xterm crashing

BrentGee — Tue, 26 Nov 2013 21:24:26 GMT

That's correct. Do you have any new suggestions for how I can begin to troubleshoot the issue?

Re: CMU 7.1 - New Info: xterm crashing

Dennis Handly — Tue, 26 Nov 2013 22:23:29 GMT

>Do you have any new suggestions for how I can begin to troubleshoot the issue?

Not much other than, get the xterm source and compile with debug info, then debug the bad free or pointer/heap corruption.

Re: CMU 7.1 - New Info: xterm crashing

Dave-SBS — Wed, 27 Nov 2013 17:12:19 GMT

Brent,

When you initially click to open the xterm connection to the node's console, it sounds like the xterm does initially load and shows you the console, correct? You initially get to watch the node shutdown, right?

I have had this problem on and off with xterms launched from CMU GUI with various versions. I have never really troubleshot this, but just launched another xterm. I believe that the xterm would disappear when the node powered off. You can see in the iLO output that there IS no virtual serial port to be connected to because the node is powered down. If you xterm were to survive this momentary lose of power, the console/VSP would return as soon as the node powered up again.

The way I have gotten around this issue of the xterm disappearing is to open an xterm or a terminal outside of CMU, connect to the node's iLO manually, and start the VSP. Doing it this way, I rarely experience the xterm disappearing. Try it this way once and compare to opening the xterm or terminal from CMU. Doing it this way, which I prefer, you can use almost any type of CLI program you want: Xterm, Terminal, Eterm, etc, etc.

Dave Holton

Re: CMU 7.1 - New Info: xterm crashing

BrentGee — Wed, 27 Nov 2013 20:49:07 GMT

Hi Dave:

Thanks very much for your response. I really appreciate this.

> When you initially click to open the xterm connection to the node's console, it sounds like the xterm does initially load

> and shows you the console, correct? You initially get to watch the node shutdown, right?

This is correct. Everything works perfectly during the shutdown. Then I can see the post. Then I can see grub. However, as soon as the init scripts start spitting out information, this is when xterm disappears and a bunch of glibc error messages get spit out to whatever other terminal is connected to the cmu server.

So, with regard to your next paragraph, I don't seem to suffer any issues when the node is shutdown per se.

I too am using a workaround in order to deal with nodes on a one-on-one basis. However, I manage a large cluster of ~1000 nodes and we are in the middle of an OS upgrade and it sure would be nice to view the cluster one full enclosure at a time.

Again, thanks very much for your time.

Re: CMU 7.1 - New Info: xterm crashing

BrentGee — Wed, 27 Nov 2013 20:50:43 GMT

@Dennis Handly wrote:
Not much other than, get the xterm source and compile with debug info, then debug the bad free or pointer/heap corruption.

This is great idea! Thank you Dennis. I believe that I can then modify the cmuserver.conf in order to point to my compliled xterm.

Re: CMU 7.1 - New Info: xterm crashing

BrentGee — Wed, 27 Nov 2013 22:36:16 GMT

I am closing this issue. Compiling the latest xterm (v. 298) just worked out of the box. I did the following:

Downloaded the latest source from:

http://invisible-island.net/xterm/#download

Built the software and stuck all the files in /opt/xterm:

./configure --prefix=/opt/xterm
make
make install

Modified the following line in cmuserver.conf:

# ***** settings for Xterm *****
#CMU_REMOTE_TERMINAL="/usr/bin/xterm"
CMU_REMOTE_TERMINAL="/opt/xterm/bin/xterm"

Restarted cmu and now everything works as a dream. For whatever, the version that installs with centos 6.4 (xterm-253-1) is giving me grief and I no longer need to figure out why. I also no longer care why gnome-terminal stopped working, too.

I am also closing the following thread:

http://h30499.www3.hp.com/t5/HP-Insight-Cluster-Management/CMU-7-1-xterm-crashing/td-p/6259947#.UpZztY10wck

Thanks, everyone, for your help.