Restarting SAP guest VM hangs on console message Starting SAP Services

[FCopeland]

It has been a while since I posted a question in HP. Last time it was in the "old" itrc. Hope I'm getting this in the correct area with the right protocol of info provided.

Having a problem where the VM guests hang on startup with this message in /etc/rc.log. It's a simple little issue yet pesky.

tail -f /etc/rc.log is showing:

" Starting SAP Services "
" Output from "/sbin/rc3.d/S900sapinit start": "

---------------------

I'm the cautious one who doesn't want to start up any software on a server until the UX boot is complete with the famous:

**************************************************
HP-UX run-level transition completed
Mon Mar 26 15:28:20 2012
**************************************************

We are running entire SAP landscape on HPVMs for 2 years now. We have upgraded from ia64 BL870 i (one) to ia64 BL870 i2 (two) and the problem started at about that same time as new hardware. Yes of course we upgrade with latest OS patches (always 11.iv3  The consultant who stood up the server upgrade and continues to support me on this configuration can't figure it out and hasn't the bandwidth to be bothered with pesky details.

But every time I re-boot the VM OS I get the hang. I've found the only way to clear the problem is to go to the hpvmconsole for the guest server being restarted. At that servers console I always see the following last line in the console display:

"     Starting SAP Services ....................................... TERM = (hp)  "

I press enter at this message in console and within a few moments the restart completes on the guest being restarted. Then I see the famous "...transition completed" message in rc.log.

The key here is I have to interact with the console by pressing the <enter> or carriage return key. This happens pretty consistently in all my guest server reboots, so much so that I have taken to just rebooting from a host server console in our configuration just to save me from having to connect later anyway.That's not a bad idea I understand but never mind that.

We are a small shop with not much VM UX experience to go around. I occasionally need to have someone else reboot a server but if I do they will not wait for the "famous message" and the guest servers console will be stuck with service message hand on them AND possibility of not having a fully booted system before the restart of the general database and SAP software. To clarify most of the part time UX admins in my shop just telnet to the server they are working with. They know nothing of the underlying host servers.

The "Starting SAP Services" message is near the last of the messages from console for startup. Perhaps I'm just being **bleep** here but I'd like for all our servers to completely reboot without console interaction on every occasion.

OS is:

UX 11.31 on VM hosts and Guests

HPVM-Guest            B.04.20.05     Integrity VM Guest

VMKernelSW            B.04.20        Integrity VM Kernel Software

Machine model is "ia64 hp Integrity BL870c i2"

Various levels of SAP software on each guest server all running SAP NetWeaver 7.00 and greater.

Please advise of any patches, new or 8 months recent, which may address this issue. My searches reveal nothing on the matter.

[Dennis Handly]

>It has been a while since I posted a question in HP.

(It looks like you didn't migrate your ITRC account.)

>I always see the following last line in the console display:

"     Starting SAP Services ....................................... TERM = (hp)  "

This looks like S900sapinit is invoking HP's broken .profile?  This looks like the output of tset(1).

One quick fix would be to redirect stdin to /dev/null.

[FCopeland]

OK, so I am not the most unix enlightened here.

In the servers root .profile of the VM guest is this logic for tset:

        if [ "$TERM" = "" ]
        then
                eval ` tset -s -Q -m ':?hp' `
        else
                eval ` tset -s -Q `
        fi

Is the suggestion to change it to the following (?):

        if [ "$TERM" = "" ]
        then
                eval ` tset -s -Q -m ':?hp' ` < /dev/null
        else
                eval ` tset -s -Q `
        fi

[Dennis Handly · Accepted solution]

>In the servers root .profile of the VM guest is this logic for tset:

Yes, this is the broken part.  It doesn't handle batch.  But sh-posix(1) and ksh(1) say they don't read /etc/profile or .profile unless interactive.  So perhaps they aren't broken after all??

>Is the suggestion to change it to the following (?):

No, then it wouldn't work for a root interactive user.

I guess my intention was to have you look into S900sapinit and look for something "strange".

One thing would be "su - some-user-name".  This may cause an interactive shell to be created?

If it isn't obvious where the bad programming is, switch to the scummy C shell in another script?, then you could just add this to somewhere near the top of S900sapinit:

# Close stdin so it doesn't hang trying to do tset(1), somewhere:

eval < /dev/null

If you wanted to track down the bad command/script, you could add near the top of S900sapinit:

set -x

Then see what is printed before it hangs.

[FCopeland]

The veil finally lifts from the apprentices eyes as the warrior slaps him with clarity of knowledge.

Now I understand.

Thanks for the advice. SAP900init is delivered as part of the SAP package. Not quite ready for HP-VM in this respect is it? It is a sh script which will have to be dealt with like a new deck of cards.

Thanks for the advice. Over and out.

[Dennis Handly]

>Not quite ready for HPVM in this respect is it?

I would assume it would fail whether VM or not.

>It is a sh script

Have you tried using "set -x" as I suggested?