Server doing some weird things, HELP!!

[Phillip Popp]

Here is my password file

[Court Campbell]

Oh, you should not have posted the passwd file. Not a smart move. I just wanted to see the shell entries.

[Andrew Young_2]

Hi.

Could you also check that the user has permissions to their /home/ directory and that someone hasn't done a chown on the whole of /home

Also could you do a 'whereis ksh'. Lastly try and run ksh from your root prompt.

Andrew Y

[V. Nyga]

Hi again,

check File or Directory Correct Access

/bin/sh -r-xr-xr-t
/ drwxr-xr-x
/bin drwxr-xr-x
/etc/passwd -r--r--r--

Also maybe /sbin/sh corruption?
Check:
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=8615

Also you could make a google search yourself:
http://www.google.de/search?num=20&hl=de&ie=ISO-8859-1&oe=ISO-8859-1&as_qdr=all&q=no+shell+site%3Aforums1.itrc.hp.com&btnG=Suche&meta=lr%3Dlang_en

HTH
Volkmar

[V. Nyga · Accepted solution]

Hi again,

one short question (or have I overread your answer?)
Have you done 'pwck' as suggested before?

V.

[MarkSyder]

I've just read about a remarkably similar problem where someone had inadvertently changed the permissions on /. / should have 755 permissions: check with

ll -d /

Mark

[Court Campbell]

Nice catch Mark. I actually ran into that situation once. An Oracle DBA was changing perms as root on a test box and did exactly what you described.

[Phillip Popp]

whereis ksh
/usr/bin/ksh

[Andrew Young_2]

Hi

A lot of shells point /bin/ksh and not /usr/bin/ksh.

ls -l /bin

If it gives a lot of files thats good, if it gives /bin -> /usr/bin even better.

Regards

Andrew Y

[Phillip Popp]

OK,
from ll -d / I get drwxr-xr-x 36 root root

from pwck, it says

prodeng::206:8400:Product Eng:/home/prodeng:/bin/ksh
login directory not found

this for all users.

[Andrew Young_2]

Hi Phil

Please run ls -ld /home

Also run ls -l /home

Andrew Y

[Phillip Popp]

here are the output of ls -l /home

[Phillip Popp]

here are the output of ls -l /home

and the other ls -ld /home

[Andrew Young_2]

Hi.

It looks like a number of your /home directories are missing.

Regards

Andrew Y

[Phillip Popp]

ls -l bin links to /usr/bin

[V. Nyga]

Hi again,

we should concentrate to one or two users - for example 'prodeng' - there's no home directory for prodeng!
Where should it be? Where had it been?

V.

[Phillip Popp]

ls -l bin links to /usr/bin

it gives me
lr-xr-xr-t 1 root sys

[Phillip Popp]

Thats the thing, the /home/"individual directories" are there makes no sense. Lets go back to the beginning.

[Andrew Young_2]

Hi Phil

The ls /home shows only one third of them are there.

Can you run ls -lb /home

Regards

Andrew Y

[Robert-Jan Goossens]

Phil,

Keep in mind this weird behaviour could also be caused by bad hardware (cpu, system board, disk etc). The part could be failing (almost dead).

Regards,
Robert-Jan

[V. Nyga]

Hi,

you can rlogin and go the first boot stage (init 1) by typing 'init 1' then 'umount -a'.
Check with 'bdf' if /home is unmounted, then run fsck at the raw device of the home disk.
See 'man fsck' - something like 'fsck /dev/rdsk/c1d0s8'

Your raw device you get with:
'ioscan -fnC disk'

Other possibility than 'init 1' to be able to fsck is to reboot server and boot in single user mode.

HTH
Volkmar

[Phillip Popp]

All,
Thanks for all your support. turns out a back up script that was running was backing up to a server that went down at the same time the script was running. the script changes permissions to directories at the beginning but never completed to repermission them. A lot of permissions got hosed up, the /root /lib all kinds of stuff. Fortunatly, we just hired some gent from Scotland who happened to be visiting our main location, and he was a big help in finding the root cause and the damaged done.

Again, thank you all for your help, this site is such a help to people!!!

Phil

[Patrick Wallek]

Why in the world would your backup script change permissions on directories prior to backing them up, and then change them back?

I don't understand the logic in that, especially for your basic OS directories (/, /usr, /var, etc).

I think you've also just found a good reason NOT to do that.

At this point I would seriously consider redesigning your backup script.

[Yogeeraj_1]

Hi Phil,

Patrick has a good point above. Why should you backup be mingling with your files??

I would also add that you do a full backup immediately.

good luck!
kind regards
yogeeraj

[James R. Ferguson]

Hi Phil:

> A lot of permissions got hosed up, the /root /lib all kinds of stuff.

Yikes, don't run that script again! I'd do some further damage assessment and/or fixup by running 'swverify'. Unfortunately, since this appears to be an out-dated 10.20 release, you are limited to finding the mangled objects:

# swverify \*

If you were running a current 11i release you could find *and* fix the damage with:

# swverify -F \*

Regards!

...JRF...