nfskd is producing a core file

[JUP]

I don't want anything thats got to do with NFS running on my system so I switched all NFS entries off, ie. in /etc/rc.config.d/nfsconf I set NFS_CLIENT=0, NFS_SERVER=0, START_MOUNTD=0 and AUTOMOUNT=0. And in SAM I switched off NFS.
But for some reason /sbin/rc2.d/S400nfs.core is executed and forces nfskd to produce a core file. The strange thing is /sbin/rc3.d/S100nfs.server does a check to see if NFS_SERVER is on before attempting to fo anything, yet rc2.d/S400nfs.core does not.

Any ideas what I can do to stop NFSKD running without just deleting the S400nfs.core file as its too messy and may be needed one day.

Thanks in advance
PA

[Steven E. Protter · Accepted solution]

you don't have to delete the S400nfs.core file.

cd /etc/rc2.d

mv S4000nfs.core s400nfs.core

Then restart.

You need to make sure however yuou aren't running anything like Ignite that needs nfs running.

The machine will need to be booted after these changes.

Alternatively you can do this:

/sbin/init.d/nfs.core stop

This should stop nfs core services if client and server are already stopped.

Then, lets look at the core files and see if there is anything useful in there.

strings core | more

Might see something you can search itrc for.

The first thing I'd look at in this case are NFS patches.

The next thing is what have you recently done to your system to trigger this problem. It could be as simple as a san change, you never know.

SEP

[Con O'Kelly]

Hi

The nfskd daemon is started by nfs.core, it doesn't check any of the files in /etc/rc.config.d for this. (Note this is different to nfsd daemon)

The S400nfs.core is just a link to /sbin/init.d/nfs.core so you can remove the link & just re-create the link if its needed again. This will ensure nfs.core isn't run.
Alternatively you can rename it (eg /sbin/rc2.d/olds400nfs.core)

You may also want to check why nfskd is doing a core dump.

Cheers
Con

[JUP]

thanks for the replies.

Steven, you wrote:

"You need to make sure however yuou aren't running anything like Ignite that needs nfs running."

I do run Ingite-UX for full system backups once a week or so, so do I need to make sure nfs is running so my backups will work ?

I ran strings core | more and got the following:



B.11.00
9000/800
832930670
nfskd
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
$Revision: 92453-07 linker linker crt0.o B.10.60 970821 $
/usr/lib/dld.sl
ERROR: mmap failed for dld
ERROR: mmap failed for TSD
@(#)nfskd daemon X.10ONC+.20
Getting nfskd kernel thread info
Error returned from kernel; errno = %d
nfskd thread #(tid): kthrp flags state pri wchan
-------------------------------------------------------------
nfskd thread #%d(%d): %08x 0x%x %d %d %08x
nfskd proc_addr = 0x%08x
Error returned from kernel; errno = %d
%s must be run as root
sqckrpz:
Spawning nfskd

....
....


Any ideas ?

Thanks in advance
PA

[Steven E. Protter]

If your machine is an Ignite Server, meaning you store Golden images on it or do make_net_recovery operations to it, you need to straighten out the nfs problem or these functions will not work.

I believe in the past I've done Ignite (make_tape_recovery) backups on machines with NFS totally shut down and the tapes tested out all right.

I've solved most of my NFS problems in the past with patches. I just search itrc for nfs patches and look for clues that match the symptoms I'm having.

NFS problems can be triggered by inconsistencies in the network setup, like in /etc/rc.config.d/netconf.

If you have a vg00 only Ignite (make_tape_recovery) tape and it dates before your problem began, you might want to consider booting the machine off of it and restoring your configuration.

You can take an fbackup of the system prior to the restore so any script work and such is preserved.

The important thing now is to not do anything that will make it worse. Consider your steps, collect as much evidence as you can and make the right move.

I'm always under pressure when these things happen. Its Sunday at 6 a.m. and the system must be back working in 12 hours. Stuff like that. I always do an Ignite tape for myself, not the vault prior to patching just in case. I date it, and then check the logs when flakey things happen so I can back out quickly.

I also practice Ignite restores on my sandbox a few times a year, just to keep my skills sharp.

Take a look at the log files, /var/adm/syslog/syslog.log and see what happened to the box. Post anything interesting. If I'm asleep, someone will help you.

SEP

[Con O'Kelly]

Hi

As SEP said if you don't use make_net_recovery you don't need NFS.

A point about nfskd. I quote from Dave Olkers
NFS Performance Tuning for HP-UX paper.
http://docs.hp.com/hpux/onlinedocs/1435/NFSPerformanceTuninginHP-UX11.0and11iSystems.pdf

" nfskd daemon currently serves no useful purpose on HP-UX.......it is created dynamically & can't be killed."

So I guess if you're getting a core dump from nfskd & can't figure out why...I wouldn't worry too much. Though as SEP said it is definitely worth ensuring you are up to date on Patches.

Cheers
Con

[JUP]

Thanks again Steven, you have been a great help.

PA

[JUP]

I thought I can get it working by moving the S400nfs.core file but this did not work.

Is it safe to remove the execute permissions on nfskd. it causes a core dump and I can't work out whats causing it. I don't need NFS so I disabled NFS it through /etc/rc.config.d/nfsconf but the system always tries to run nfs.core whcih in turn runs nfskd.

If I move nfs.core (or it linked version; S400nfs.core) then I can't telnet to the system because this runs rpcbind which is needed.

Any ideas ?
Thanks
PA

[Con O'Kelly]

Hi PA

As far as I'm aware telnet does not require rpcbind to be running. You only need rpcbind running if you are using rpc programs, the main use is for NFS. Are you having problems telneting when you're not running rpcbind?

It is quite OK to rename S400nfs.core or as you've said change the permissions on nfskd.

Cheers
Con

[JUP]

Yes Con I am having trouble telnetting when I rename or remove execute permission from the nfs.core script.

Windows or any other hosts can't telnet to the HP-UX if it does not run nfs.core. So I looked at this script and notices it runs rpcbind. So I thought I'd let this script run but change the permissions on nfskd (which it runs later in the script). This seems to be OK but not a nice way of doing things. There is no core file and nfskd is not running but I can still telnet to the host.

Weird, I know but I think I need a better solution.

Thanks for your reply.
PA

[Steven E. Protter]

telnet only requires the network be up and inetd daemon be running.

If you are having intermittant problems with telnet, then if possible, walk up to the machine and use the console.

This is a further sign that the root cause of the problem is networking.

lanadmin -x 0

do it for every NIC. Some built in cards don't support the command.

Are the results what you expect?

Are you using autonegotiate? Might want to stop. Depending on your model machine and NIC type, you can hard code the speed and duplex settings you want in one of these files:

/etc/rc.config.d/hbptlanconf

If you hard code settings here a reboot is required.

or one that starts with hpgsc. It all depends on your NIC card.

NFS is extremely sensitive to network issues and can do all kinds of nasty things if the LAN is bad.

Check the light on the NIC. Is it light, is it flickering? Should it be flickering?

Lights on the network swtich. Contact the network admin and check for errors.

traceroute to and from the box, see if there are long delays or timeouts.

ping the box from across the network. If the ping times getting steadily longer, could be a bad card.

The problem with NFS issues is it could be anything.

Strange behavior can be triggered by backups in the /etc/rc.config.d directory.

if there is a netconf and a netconf.back they both will get sourced and your machine can get way strange.

Are there backups in there? If so, move them and restart the box.

SEP

[Con O'Kelly]

Hi

Yeah that is really strange. Telnet doesn't need rpcbind to be running. What error messages are you getting from telnet?

What about enabling inetd logging (if its not already enabled) from the command line:
# inetd -l

Then try telneting & see if you get any info in syslog.log about why telnet connections are being refused.

Cheers
Con

[Steven E. Protter]

Here we go!

I found your error message.

Take a look here.

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0xa81e9607df6ed711abdc0090277a778c,00.html

Is your swapspace sufficient?

post

swapinfo -tam

output machien data including physical memory and model.

some less probable threads:
http://search.hp.com/redirect.html?url=http%3A//forums.itrc.hp.com/cm/QuestionAnswer/1,,0x9da0e3ed7640d71190080090279cd0f9,00.html&qt=ERROR%3A+mmap+failed+for+dld+nfs&hit=9

http://search.hp.com/redirect.html?url=http%3A//forums.itrc.hp.com/cm/QuestionAnswer/1,,0x1c98ee3e323bd5118fef0090279cd0f9,00.html&qt=ERROR%3A+mmap+failed+for+dld+nfs&hit=13

http://search.hp.com/redirect.html?url=http%3A//forums.itrc.hp.com/cm/QuestionAnswer/1,,0xe9a4afe90f1cd71190050090279cd0f9,00.html&qt=ERROR%3A+mmap+failed+for+dld+nfs&hit=23

night night for SEP

I'll check this one in the morning.

SEP