/var/adm/wtmp grows unlimitedly and clogs /var

[jfucinos]

Hello,

I have a problem with a 11.00 machine. /var is ~550MB and it gets clogged by /var/adm/wtmp, which grows out of control. Some process might be trying to do zillions of logs on, but it is so fast (70-80 times/minute) that I cannot check the PPID of the PID which is responsible of each of these logon attempts.

Please find a tail of the /var/adm/wtmp file:

>machinename:/var/adm$/usr/sbin/acct/fwtmp < /var/adm/wtmp | tail
>LOGIN dt console 13562 8 0000 0001 1136221510 Jan 2 18:05:10 2006
>LOGIN dt console 13567 6 0000 0001 1136221510 Jan 2 18:05:10 2006
>LOGIN dt console 13567 6 0000 0001 1136221510 Jan 2 18:05:10 2006
>LOGIN dt console 13567 8 0000 0001 1136221511 Jan 2 18:05:11 2006
>LOGIN dt console 13572 6 0000 0001 1136221511 Jan 2 18:05:11 2006
>LOGIN dt console 13572 6 0000 0001 1136221511 Jan 2 18:05:11 2006
>LOGIN dt console 13572 8 0000 0001 1136221512 Jan 2 18:05:12 2006
>LOGIN dt console 13577 6 0000 0001 1136221512 Jan 2 18:05:12 2006
>LOGIN dt console 13577 6 0000 0001 1136221512 Jan 2 18:05:12 2006
>LOGIN dt console 13577 8 0000 0001 1136221512 Jan 2 18:05:12 2006

Any suggestion? Thanks in advance for your feedback.

Cheers.

[Raj D.]

Hi jfucinos ,

You can nullify the file wtmp .
Also check you can do :

/var/adm/syslog/syslog.log
/var/adm/syslog/mail.log
/var/adm/crash/*

To recover space.


You can also check for biggest files on /var filesystem with this:
# cd /var
# ls -lR | sort +4 -5nr | more

To nullify you can use:
# > wtmp

cheers,
Raj.

[Kent Ostby]

Back it up to tape and zero it out before your next reboot:

> /etc/wtmp

[Muthukumar_5]

Do you want to have the wtmp informations for tracking users?

IF so,

cat /var/adm/wtmp >> /wtmp.log

Put this script in crontab to execute this based on timing.

Hope your / file system is having more space. Check with bdf command.

If you don't want those informations then,

> /var/adm/wtmp

in the cron tab with time settings to nullify wtmp entries.

-Muthu

[Arunvijai_4]

Hello,

You can null /var/adm/wtmp out by # >/var/adm/wtmp.

-Arun

[Robert-Jan Goossens_1 · Accepted solution]

Hi jfucinos,

There is a simular problem in the Tech database.

Title: ReflectionX: The /var/adm/wtmp grow with dtlogin messages
Document ID: A5257730

Suggesting to attach a mouse and keyboard to your server.

Best regards,
Robert-Jan

[jfucinos]

Hello everyone for your quick replies. However, my question was aimed at a general solution: I'd like to find the root cause of the growth of /var/adm/wtmp (growing THAT fast cannot be normal ...) and solve it instead of periodically zeroing the file.

Robert, I cannot find the article that you are referring to but it seems to be a good hypothesis. Could you please provide me with a link?

Thanks everybody.

[Sanjay_6]

Hi,

Here is the link to the doc suggested by Robert,

http://www1.itrc.hp.com/service/cki/search.do?category=c0&prevQueryString=&mode=id&searchString=A5257730&searchCrit=allwords&docType=Security&docType=Patch&docType=EngineerNotes&docType=BugReports&docType=Hardware&docType=ReferenceMaterials&docType=ThirdParty&dateRange=all&search.x=14&search.y=10

Hope this helps.

regds

[Pete Randall]

Here is a link to the article Robert-Jan pointed out:

http://www4.itrc.hp.com/service/cki/search.do?category=c0&prevQueryString=&mode=id&searchString=A5257730&search.x=41&search.y=10&searchCrit=allwords&docType=Security&docType=Patch&docType=EngineerNotes&docType=BugReports&docType=Hardware&docType=ReferenceMaterials&docType=ThirdParty&dateRange=all


Pete

[Rick Garland]

The wtmp file can be examined with the 'last' command.

# last | more

Who is trying to do the numerous and rapid logins? Also look at the lastb

# lastb -R | more

Are there a lot of failed logins attempts?

The output can provide some clues to help track down the offending process/user.

[jfucinos]

I am not sure if I am not too bright today or something, but am I missing something?

When I follow those links I get 0 hits:

>Note: We're sorry but your search produced 0 results.
>Please try the following:
>
> * use a different search string
> * change your search criteria selection
> * check your search string for spelling/syntax errors
> (Note: Use all UPPERCASE or all lowercase to maximize matching search results)

[Sanjay_6]

Hi,

You may need a support contract / handle linked to your id to access these doc's.

In this link,

http://www1.itrc.hp.com/service/cki/enterService.do

Choose the option search by Doc ID and then type the doc id of the document you are looking for.

If you do not have a HP support contract, you may not be able to access some content on HP ITRC site.

Hope this helps.

regds

[Robert-Jan Goossens_1]

I think I forgot to add the link :-)

Europe
http://www4.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000080060770

US
http://www2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000080060770

and attached!

Robert-Jan

[Pete Randall]

OK, I'll paraphrase, then:

The article suggests attaching a keyboard and mouse to the system. It presumes that the system is a 700 series workstation and therefore the "father dtlogin process" is attempting to spawn a second dtlogin process - probably on the console (that's my guess) and this second dtlogin is failing and causing the messages in wtmp. Attaching a keyboard and mouse should clear the problem.

Another thought, if you can't attach a keyboard and mouse, would be to periodically clear wtmp via a cron job.


Pete

[Ralf Seefeldt]

Hello,

at a 1st step, I would have a look at the nearest switch. There you can see if or if not those requests are generated from a remote Client. Alternatively, you may use netstat to display networkconections.

At the Server itself, you may run glance or sth. like this (just a absolute basic way to get some info)
for i in 1 2 3 4 5
do
ps -ef > /tmp/ps_${i}
#sleep 1
done

now, you can diff those files. If there are so many conections, you should se some prozesses, that are shortlived. Those should be analyced.

i hope this helps a little.

Bye
Ralf

[jfucinos]

Congrats Robert-Jan, I tried hooking up a keyboard and a mouse and it worked like a charm. You got it! :)

However, I'd be interested in having the box as it was, without any HID attached to it. Does any of you have some clues about which settings I must change to do it without wtmp growing without limits?

Thanks for your help people, you are stunning :)

[Robert-Jan Goossens_1]

Hi,

Do you need CDE running on this server ?

You can disable it with below command.

# man dtconfig
# dtconfig -d

Best regards,
Robert-Jan

[Ted Buis]

You might also want to dtconfig -kill after the dtconfig -d.

[jfucinos]

Thank you very much, people. The problem seems to be solved because CDE seems to be needed in that machine. Hooking a keyb and a mouse up solved it all :)

[jfucinos]

I'll proceed to close this thread. Thanks everyone! :)