useradd bug? "/etc/passwd file does not exist" and "/etc/passwd file busy -- try again"

[Thomas Pawlikaniec]

Dear gurus,

I got a couple TCB HP-UX 11.00 boxes which consistently fail on 'useradd' with a misleading error message. Of course /etc/passwd exists (and users can login normally, too), and useradd works on many other boxes with the same (replicated) /etc/passwd, /etc/group and TCB structure.

root@to173:/# useradd dummy
/etc/passwd file does not exist
root@to173:/#
root@to173:/# lsof /etc/passwd
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
pwgrd 12117 root 11r VREG 64,0x3 82241 4364 /etc/passwd
root@to173:/# /sbin/init.d/pwgr stop
pwgrd stopped
root@to173:/# lsof /etc/passwd
root@to173:/#
root@to173:/# useradd dummy
/etc/passwd file does not exist

pwgrd seems to make no difference: running or not, same error.

There is no leftover 'vipw', or any other process with a handle on /etc/passwd or /etc/group.

However, the existence or not of /etc/.pwd.lock makes a small difference (regardless of where this file came from; I can just use 'touch'):
- the error message changes
- takes a few seconds to return (a timeout?), as opposed to immediate return in the first case.

root@to173:/# lsof /etc/.pwd.lock
root@to173:/# lsof /etc/passwd
root@to173:/# lsof /etc/group
root@to173:/# useradd dummy
/etc/passwd file does not exist <= ERROR #1
root@to173:/#
root@to173:/# rm /etc/.pwd.lock
root@to173:/# useradd dummy
/etc/passwd file busy -- try again <= ERROR #2
root@to173:/#
root@to173:/# touch /etc/.pwd.lock
root@to173:/# useradd dummy
/etc/passwd file does not exist <= ERROR #1, back again

A similar thread: http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=261248 has some interesting suggestions, but no solution.


Any ideas? Thanks!

[Sridhar Bhaskarla]

Hi,

There is one more file that affects /etc/passwd modifications. /etc/ptmp. If you have it, them move it as say /etc/ptmp.old and then try.

Make sure there are no other commands running on the system like usermod etc., that will lock /etc/passwd file.

-Sri

[Thomas Pawlikaniec]

Thanks for the speedy reply, Sri!

I forgot to mention, no /etc/ptmp either...

And, as the lack of output from lsof indicates, there appear to be no processes with open handles on /etc/passwd. However, quite a few processes got an open handle on /tcb/files/auth/system/default, including init itself.

--TP

[Ermin Borovac]

Does pwck return anything interesting?

# /usr/sbin/pwck -s

[john korterman · Accepted solution]

Hi,
try searchin in the tech base, e.g. Document id: A5392162 or USECKBRC00008679

or try these links:
http://www5.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000062950294

http://www5.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000062922674

regards,
John K.

[Sridhar Bhaskarla]

Hi,

You have to have the file .pwd.lock and ensure the permissions are root:sys and 400.

I guess 'pwck' 'grpck' and 'authck' may give some hints.

Also try running 'tusc' along with your useradd command and see if you can find any clues out of it.

-Sri

[Cheryl Griffin]

Do you have an /etc/nsswitch.conf file on the system? If so is there a passwd entry?

[Thomas Pawlikaniec]

Thanks for all the links & suggestions. Most of you pointed in a similar direction (btw: nsswitch.conf points to files, looks ok), and indeed, pwck and authck both show inconsistencies. But the game's not over, as pwck/authck show similar inconsistencies on machines where useradd works! read-on...

On the other hand, the weird behaviour documented above between '/etc/.pwd.lock' and 'useradd' (or usermod) remains obscure.

Most pwck warnings look rather harmless (empty fields), but authck's reported TCB vs. /etc/passwd diffs are more worrisome.

There are some "expected differences": some old-time scripts delete TCB files for expired/deleted accounts on a server, but keep the passwd/group entry to identify (uid, gid) preserved and backed-up files. It did not seem to cause problems before, but those scripts obviously do not use useradd/userdel commands... Does anyone have experience with such a setup?

In fact, similarly-looking TCB inconsistencies exist on machines where useradd works and those where it fails. Any clues on that?

Our servers run a complex mix of critical applications and scripts, and touching such a fragile status quo is always a bit suicidal. I've seen too many horror stories involving the sloppy 'tsconvert -r' + 'tsconvert', (although suggested by HP). And 'pwconv' (basically a wrapper around 'tsconvert -u') is not more reassuring.

So now, I wonder:
1] which TCB inconsistencies actually make useradd fail? and which are harmless?
2] What's the SAFEST way to fix everything authck complains about? is it doable with zero downtime?

Thanks in advance,
-- TP

[Thomas Pawlikaniec]

(off-topic) John, re: the links. I tried searching prior to possting but it never came up with the docs you pointed out. In fact, it seems only forums get searched -- is it possible I am missing some access rights? Yet I could access your links without problem (except on Americas' sub-site), so I guess I was just not searching in the right place? Any hints? Thanks!

[john korterman]

Hi again Thomas,

to go to the tech base:
on this page, click on "maintenance and support for hp products".
On the following page click on "technical knowledge base": there you will then find a good search tool for searching e.g. error messages.

regards,
John K.