topic Re: things went bad when do this fuser -kc /var in Operating System - HP-UX

things went bad when do this fuser -kc /var

hpuxhelp — Tue, 31 Dec 2002 20:36:36 GMT

I did that, and now i tried to login on..it won't let me in...from the console.... what 's wrong ?

Re: things went bad when do this fuser -kc /var

hpuxhelp — Tue, 31 Dec 2002 20:38:07 GMT

it kept saying wait for login retry....

Re: things went bad when do this fuser -kc /var

harry d brown jr — Tue, 31 Dec 2002 20:39:58 GMT

You killed every process that had /var open. Which on my 11i machine is about 39 root processes.

You probably need to reboot.

live free or die
harry

Re: things went bad when do this fuser -kc /var

Patrick Wallek — Tue, 31 Dec 2002 20:40:30 GMT

You shouldn't have done that.

You should have just rebooted the machine into single user mode.

The /var filesystem is used by LOTS of OS processes and by issuing the 'fuser -kc' you just killed a lot of VERY IMPORTANT HP-UX processes.

You may have to now do a CTRL-B and at the prompt do either a TOC or an RS to restart the system. If you have another session logged in somewhere, try rebooting the system from there before you do the above.

Re: things went bad when do this fuser -kc /var

John Poff — Tue, 31 Dec 2002 20:41:07 GMT

Hi,

I think you just sent the SIGKILL signal to every process that had the /var filesystem open. Ouch!

What problem were you trying to solve?

JP

Re: things went bad when do this fuser -kc /var

Uday_S_Ankolekar — Tue, 31 Dec 2002 20:43:51 GMT

It looks like you killed everything mounted from /var

Does bdf shows you /var mounted??
If not try mounting /var again and try to login.

Re: things went bad when do this fuser -kc /var

Patrick Wallek — Tue, 31 Dec 2002 20:43:52 GMT

John,

I think he was trying to kill all the processes in /var so he could extend it. See this post:

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x4d95afe90f1cd71190050090279cd0f9,00.html

By the way, do NOT do the 'fuser -kc' command on /tmp. If you do, you are likely to run into some of the same problems you have right now as a lot of root processes access /tmp as well.

Re: things went bad when do this fuser -kc /var

hpuxhelp — Tue, 31 Dec 2002 20:46:30 GMT

i meant to type ku .......is there a way i can do this without have to shutdown....
there must be a way to recover this....
i can't reboot right now..

Re: things went bad when do this fuser -kc /var

Jeff Schussele — Tue, 31 Dec 2002 20:46:50 GMT

Well - that should go right to the top of your
"Things to Never Do Again" list.

Without OnLineJFS, the ONLY way to extend /var is in sigle-user mode. There's simply NO way around this fact.

Rgds,
Jeff

Re: things went bad when do this fuser -kc /var

Patrick Wallek — Tue, 31 Dec 2002 20:48:17 GMT

I really don't think you are going to have a choie in the matter. I think a reboot may be your only option.

If you have a support contract on your machine, Call HP and tell them what has happened. They may be able to help you.

To see what processes are using a mountpoint you should do:

# fuser -cu /mountpoint

Re: things went bad when do this fuser -kc /var

hpuxhelp — Tue, 31 Dec 2002 20:53:58 GMT

yeah..learn my lesson..
but strange things..if I come in as a user and su - root
and it is ok for me to do ..but i can't do the same from the console work ???
now since i am in as root, is there a way i can repair the damage ???

Re: things went bad when do this fuser -kc /var

Martin Johnson — Tue, 31 Dec 2002 20:54:24 GMT

You say you can't reboot right now. You don't have a choice. You have inadvertantly killed over half the OS. Your system can barely function. I would question the integrity of any transactions occurring on the system.

Reboot now!

HTH
Marty

Re: things went bad when do this fuser -kc /var

John Poff — Tue, 31 Dec 2002 20:55:45 GMT

So you are on a 10.20 box? I just did an 'fuser' on my /var and I have 221 processes that have /var open, out of a total of 526 processes. Here are some of the root processes that you probably just killed:

/usr/lbin/cmsnmpd
/usr/sbin/syslogd
/opt/omni/lbin/rds
/usr/sbin/swagentd
/opt/dce/sbin/rpcd
/usr/sbin/netfmt
/usr/sbin/ptydaemon
/usr/sbin/rbootd
/usr/lbin/ntl_reader
/opt/perf/bin/perflbd
/opt/perf/bin/alarmgen
sendmail:
/usr/sbin/cron
/opt/perf/bin/midaemon
/usr/lbin/cmcld
/usr/sbin/envd
/opt/omni/lbin/mmd
/usr/lbin/cmsrvassistd
/opt/perf/bin/rep_server
/opt/perf/bin/ttd
/opt/perf/bin/scopeux
/opt/omni/lbin/crs
/usr/dt/bin/dtlogin
/opt/perf/bin/agdbserver

Now, if you are really adventurous and motivated [and you sure seem to be so far!], and if you have a root connection open to the box, you COULD try to restart these daemons via /sbin/init.d somehow. Maybe.

Or you could just reboot to single user mode and expand the filesystems as you will need to do anyway.

In situations like this I've found it to be quite convenient to trip over the power cord. ;)

JP

Re: things went bad when do this fuser -kc /var

hpuxhelp — Tue, 31 Dec 2002 21:05:03 GMT

i do have access to root now...
how do i start this using the daemon

Re: things went bad when do this fuser -kc /var

John Poff — Tue, 31 Dec 2002 21:10:02 GMT

Alright! That's the spirit! Rebooting is the easy way out. Let's see if we can really recover this system.

Let me login to one of my 10.20 boxes and see what scripts we can run in /sbin/init.d that might get things restarted.

JP

Re: things went bad when do this fuser -kc /var

hpuxhelp — Tue, 31 Dec 2002 21:12:18 GMT

ok...thank you so much.... i hate if i have to reboot this box... i am in dead water...now ...

Re: things went bad when do this fuser -kc /var

John Poff — Tue, 31 Dec 2002 21:16:35 GMT

For starters, let's see if we can get the first three processes going by using /sbin/init.d scripts. Try these:

/sbin/init.d/envd start
/sbin/inid.d/ptydaemon start

See if those two get the envd and ptydaemon daemons going. If so, we might be able to pull this off. I won't claim to know which daemons are the most important, but we'll start with those.

JP

Re: things went bad when do this fuser -kc /var

hpuxhelp — Tue, 31 Dec 2002 21:24:38 GMT

/sbin/init.d/envd start
Environmental daemon started
# /sbin/init.d/ptydaemon start
Starting the ptydaemon

yes..it seems the2 daemon is working at this point..so waht's next??
i am so nervous

Re: things went bad when do this fuser -kc /var

hpuxhelp — Tue, 31 Dec 2002 21:29:33 GMT

i did ps -ef
and i saw that lvmkd is not running...and it doesn't look so good

Re: things went bad when do this fuser -kc /var

John Poff — Tue, 31 Dec 2002 21:42:03 GMT

No lvmkd daemon. That's a bigger problem. You are closer to having to reboot now. I don't think there is a way to restart it manually.

http://www2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000062942884

JP