topic Re: CMU 7.1 - xterm crashing in Server Clustering

CMU 7.1 - xterm crashing

BrentGee — Tue, 05 Nov 2013 21:35:42 GMT

Anybody else experiencing xterm crashing during the following conditions:

- using VSP to watch a node boot

- at some point during service initialization the screen just disappears and the following error occurs:

*** glibc detected *** /usr/bin/xterm: free(): invalid pointer: 0x0000000001bbea70 ***
======= Backtrace: =========
...

whole bunch of stuff ...

...

======= Memory map: ========

...

whole bunch of libs listed

...

My OS is centos6.4 and the version of xterm, according to rpm -qa, is xterm-253-1.el6.x86_64 -- which I believe is the latest.

Any info would be extremely appreciated.

Re: CMU 7.1 - xterm crashing

Rakshika — Wed, 06 Nov 2013 07:55:44 GMT

Hello,

Please clarify what operation were you performing when you saw xterm disappearing.

where did you see this glibc stack trace?

Can you please send us whole stack trace ?

Regards,

Rakshika Gupta

Re: CMU 7.1 - xterm crashing

BrentGee — Wed, 06 Nov 2013 19:47:09 GMT

Hi:

Thanks very much for your reply. The operations I performed were as follows:

- selected the compute node in question in the CMU GUI

- right-clicked and selected "Virtual Serial Port"

- logged in via the VSP connection and typed reboot

- watched it reboot (saw POST, saw GRUB) right up until the initialization scripts kicked in

- then the window disappeared

- and the following output was spit out to an open terminal connected to the CMU server node:

[root@orca-dev ~]# *** glibc detected *** /usr/bin/xterm: free(): invalid pointer: 0x0000000001600ee0 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3b39c76126]
/usr/bin/xterm[0x42d29d]
/usr/bin/xterm[0x42d868]
/usr/bin/xterm[0x430449]
/usr/bin/xterm[0x438a32]
/usr/bin/xterm[0x415209]
/usr/bin/xterm[0x418ab1]
/usr/bin/xterm[0x418bf2]
/usr/bin/xterm[0x424125]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x3b39c1ecdd]
/usr/bin/xterm[0x4069c9]
======= Memory map: ========
00400000-0045b000 r-xp 00000000 68:02 666713                             /usr/bin/xterm
0065b000-00664000 rw-p 0005b000 68:02 666713                             /usr/bin/xterm
00664000-00665000 rw-p 00000000 00:00 0
00863000-00867000 rw-p 00063000 68:02 666713                             /usr/bin/xterm
015a3000-01629000 rw-p 00000000 00:00 0                                  [heap]
3b39400000-3b39420000 r-xp 00000000 68:02 1572871                        /lib64/ld-2.12.so
3b3961f000-3b39620000 r--p 0001f000 68:02 1572871                        /lib64/ld-2.12.so
3b39620000-3b39621000 rw-p 00020000 68:02 1572871                        /lib64/ld-2.12.so
3b39621000-3b39622000 rw-p 00000000 00:00 0
3b39c00000-3b39d8a000 r-xp 00000000 68:02 1572878                        /lib64/libc-2.12.so
3b39d8a000-3b39f89000 ---p 0018a000 68:02 1572878                        /lib64/libc-2.12.so
3b39f89000-3b39f8d000 r--p 00189000 68:02 1572878                        /lib64/libc-2.12.so
3b39f8d000-3b39f8e000 rw-p 0018d000 68:02 1572878                        /lib64/libc-2.12.so
3b39f8e000-3b39f93000 rw-p 00000000 00:00 0
3b3a000000-3b3a002000 r-xp 00000000 68:02 1595533                        /lib64/libdl-2.12.so
3b3a002000-3b3a202000 ---p 00002000 68:02 1595533                        /lib64/libdl-2.12.so
3b3a202000-3b3a203000 r--p 00002000 68:02 1595533                        /lib64/libdl-2.12.so
3b3a203000-3b3a204000 rw-p 00003000 68:02 1595533                        /lib64/libdl-2.12.so
3b3a400000-3b3a45a000 r-xp 00000000 68:02 672650                         /usr/lib64/libXaw7.so.7.0.0
3b3a45a000-3b3a659000 ---p 0005a000 68:02 672650                         /usr/lib64/libXaw7.so.7.0.0
3b3a659000-3b3a664000 rw-p 00059000 68:02 672650                         /usr/lib64/libXaw7.so.7.0.0
3b3c000000-3b3c002000 r-xp 00000000 68:02 657363                         /usr/lib64/libXau.so.6.0.0
3b3c002000-3b3c202000 ---p 00002000 68:02 657363                         /usr/lib64/libXau.so.6.0.0
3b3c202000-3b3c203000 rw-p 00002000 68:02 657363                         /usr/lib64/libXau.so.6.0.0
3b3c400000-3b3c41d000 r-xp 00000000 68:02 657265                         /usr/lib64/libxcb.so.1.1.0
3b3c41d000-3b3c61d000 ---p 0001d000 68:02 657265                         /usr/lib64/libxcb.so.1.1.0
3b3c61d000-3b3c61e000 rw-p 0001d000 68:02 657265                         /usr/lib64/libxcb.so.1.1.0
3b3c800000-3b3c937000 r-xp 00000000 68:02 657805                         /usr/lib64/libX11.so.6.3.0
3b3c937000-3b3cb37000 ---p 00137000 68:02 657805                         /usr/lib64/libX11.so.6.3.0
3b3cb37000-3b3cb3d000 rw-p 00137000 68:02 657805                         /usr/lib64/libX11.so.6.3.0
3b3d000000-3b3d012000 r-xp 00000000 68:02 657809                         /usr/lib64/libXext.so.6.4.0
3b3d012000-3b3d212000 ---p 00012000 68:02 657809                         /usr/lib64/libXext.so.6.4.0
3b3d212000-3b3d213000 rw-p 00012000 68:02 657809                         /usr/lib64/libXext.so.6.4.0
3b3d400000-3b3d498000 r-xp 00000000 68:02 657208                         /usr/lib64/libfreetype.so.6.3.22
3b3d498000-3b3d697000 ---p 00098000 68:02 657208                         /usr/lib64/libfreetype.so.6.3.22
3b3d697000-3b3d69d000 rw-p 00097000 68:02 657208                         /usr/lib64/libfreetype.so.6.3.22
3b3d800000-3b3d826000 r-xp 00000000 68:02 1572953                        /lib64/libexpat.so.1.5.2
3b3d826000-3b3da25000 ---p 00026000 68:02 1572953                        /lib64/libexpat.so.1.5.2
3b3da25000-3b3da28000 rw-p 00025000 68:02 1572953                        /lib64/libexpat.so.1.5.2
3b3e000000-3b3e034000 r-xp 00000000 68:02 657827                         /usr/lib64/libfontconfig.so.1.4.4
3b3e034000-3b3e234000 ---p 00034000 68:02 657827                         /usr/lib64/libfontconfig.so.1.4.4
3b3e234000-3b3e236000 rw-p 00034000 68:02 657827                         /usr/lib64/libfontconfig.so.1.4.4
3b3e400000-3b3e409000 r-xp 00000000 68:02 657821                         /usr/lib64/libXrender.so.1.3.0
3b3e409000-3b3e609000 ---p 00009000 68:02 657821                         /usr/lib64/libXrender.so.1.3.0
3b3e609000-3b3e60a000 rw-p 00009000 68:02 657821                         /usr/lib64/libXrender.so.1.3.0
3b3f000000-3b3f009000 r-xp 00000000 68:02 657825                         /usr/lib64/libXcursor.so.1.0.2
3b3f009000-3b3f209000 ---p 00009000 68:02 657825                         /usr/lib64/libXcursor.so.1.0.2
3b3f209000-3b3f20a000 rw-p 00009000 68:02 657825                         /usr/lib64/libXcursor.so.1.0.2
3b3f400000-3b3f405000 r-xp 00000000 68:02 657815                         /usr/lib64/libXfixes.so.3.1.0
3b3f405000-3b3f605000 ---p 00005000 68:02 657815                         /usr/lib64/libXfixes.so.3.1.0
3b3f605000-3b3f606000 rw-p 00005000 68:02 657815                         /usr/lib64/libXfixes.so.3.1.0
3b3fc00000-3b3fc18000 r-xp 00000000 68:02 657819                         /usr/lib64/libXmu.so.6.2.0
3b3fc18000-3b3fe18000 ---p 00018000 68:02 657819                         /usr/lib64/libXmu.so.6.2.0
3b3fe18000-3b3fe1a000 rw-p 00018000 68:02 657819                         /usr/lib64/libXmu.so.6.2.0
3b40000000-3b40016000 r-xp 00000000 68:02 1595538                        /lib64/libgcc_s-4.4.7-20120601.so.1
3b40016000-3b40215000 ---p 00016000 68:02 1595538                        /lib64/libgcc_s-4.4.7-20120601.so.1
3b40215000-3b40216000 rw-p 00015000 68:02 1595538                        /lib64/libgcc_s-4.4.7-20120601.so.1
3b40c00000-3b40c04000 r-xp 00000000 68:02 1595549                        /lib64/libuuid.so.1.3.0
3b40c04000-3b40e03000 ---p 00004000 68:02 1595549                        /lib64/libuuid.so.1.3.0
3b40e03000-3b40e04000 rw-p 00003000 68:02 1595549                        /lib64/libuuid.so.1.3.0
3b41000000-3b41007000 r-xp 00000000 68:02 657183                         /usr/lib64/libSM.so.6.0.1

I hope this helps. Please let me know, if you need any additional log info. I will also be running tests to see if I can figure out at which exact point in the boot process the error is triggered.

Thank you again for your help,

Brent.

Re: CMU 7.1 - xterm crashing

BrentGee — Wed, 06 Nov 2013 20:11:35 GMT

Just performed a quick test.

On the CMU server node, I launched xterm from the console (ie. NOT from the CMU GUI interface), I logged to the compute node's management processor (ILO2), typed "vsp" and then once again rebooted the node. This time xterm DID NOT crash, so I don't think that this is a problem with xterm itself.

Are there any settings in cmuserver.conf (with regard to CMU's xterm config) that could be a contributing factor?

Re: CMU 7.1 - xterm crashing

BrentGee — Wed, 06 Nov 2013 20:35:59 GMT

Second quick test. I switched from xterm to gnome-terminal (via edits to the the cmuserver.conf) and I was unable to replicate the crash. The setting switched were the following:

# ***** settings for Gnome 2.30 *****
# NOTE: the '--disable-factory' flag is mandatory for broadcast
#
CMU_REMOTE_TERMINAL_CMD_OPT="-x"
CMU_REMOTE_TERMINAL="/usr/bin/gnome-terminal"
CMU_REMOTE_TERMINAL_DISPLAY_OPT="--display"
CMU_REMOTE_TERMINAL_TITLE_OPT="--title"
CMU_REMOTE_TERMINAL_NAME_OPT="--name"
CMU_REMOTE_TERMINAL_GEOMETRY_OPT="--geometry"
CMU_REMOTE_TERMINAL_MINIMIZE_OPT="none"
CMU_REMOTE_TERMINAL_GEOMETRY_PIXEL_MODE="no"
CMU_REMOTE_TERMINAL_OPTIONS="--window-with-profile=cmu"
CMU_BROADCAST_TERMINAL_OPTIONS="none"

# ***** settings for Xterm *****
#CMU_REMOTE_TERMINAL="/usr/bin/xterm"
#CMU_REMOTE_TERMINAL_DISPLAY_OPT="-display"
#CMU_REMOTE_TERMINAL_TITLE_OPT="-title"
#CMU_REMOTE_TERMINAL_OPTIONS="-fg white -bg black -sb"
#CMU_REMOTE_TERMINAL_GEOMETRY_OPT="-geometry"
#CMU_REMOTE_TERMINAL_GEOMETRY_PIXEL_MODE="no"
#CMU_REMOTE_TERMINAL_CMD_OPT="-e"
#CMU_REMOTE_TERMINAL_NAME_OPT="-n"
#CMU_BROADCAST_TERMINAL_OPTIONS="+sb -fg black -bg white -font 5x8 -rv"
#CMU_REMOTE_TERMINAL_MINIMIZE_OPT="-iconify"

I would happily use gnome-terminal for everything (it seemed quite stable on my setup). However, I can't seem to figure out how to use it for "multiple windows broadcasts." If there was a way to to switch completely to gnome-terminal, I would leave xterm in the dust and consider this issue resolved.

Re: CMU 7.1 - xterm crashing

Rakshika — Fri, 08 Nov 2013 05:12:05 GMT

Hello,

Thanks for details.

For gnome-terminal broadcast, change settings for gnome-terminal in cmuserver.conf;

#CMU_BROADCAST_TERMINAL_OPTIONS="none"

CMU_BROADCAST_TERMINAL_OPTIONS="--disable-factory --window-with-profile=broadcast"

Gnome-terminal is not that stable, it varies a lot between configurations. This setting may work with your configuration.

Regards,

Rakshika Gupta

Re: CMU 7.1 - xterm crashing

BrentGee — Fri, 08 Nov 2013 20:42:12 GMT

Hi:

Thank you for the suggestion. I tried it but, you are right, gnome-terminal is not stable and the windows just never materialized. Therefore, the solution was NOT successful.

Back to square one...

Re: CMU 7.1 - xterm crashing

BrentGee — Wed, 27 Nov 2013 22:37:24 GMT

Work around in place. Compliled my own xterm. Issue is resolved. See here.

http://h30499.www3.hp.com/t5/HP-Insight-Cluster-Management/CMU-7-1-New-Info-xterm-crashing/m-p/6284833/highlight/false#M130