NOQUEUE space.

[Jeffrey F. Goldsmith]

I received the following error on my computer this morning. What could cause this error? Also, I checked my system and /var is at 99%. What is /var used for and what information can I delete from it without crashing my system?

Here is the error I received this morning:

Aug 23 07:32:35 diomede sendmail[1033]: NOQUEUE: low on space (have 0, SMTP-DAEMON needs 101 in /var/spool/mqueue)


Here is the information concerning my /var:

/dev/vg00/lvol8 2457600 2441994 14973 99% /var


If I cant find anything to delete from /var then I am going to have to increase it’s size. Do I need to be single user mode to do that?

Thanks,

[Alan Meyer_4]

If you have OnlineJFS, and available space, then you can do it online.

[Rick Garland]

You have outgoing emails backed up on your system. Look in '/var/spool/mqueue'

Do the 'mailq' command. How many are there? Will also provide some reason as to why the mails are not going out.

Once you get the emails sending out then the mqueue directory will decrease in space used.

If you have OnLineJFS, you can increase /var on the fly.

[Tom Danzig · Accepted solution]

Checkand clean/trim:
/var/tmp
/var/mail
/var/adm/cron/log

You can also commit superseded patches:
cleanup -c 1

[saju_2]

Hi


The error came because there was no spcae in /var for sendmail to put the mail queue (I believe is /var/spool/mqueue)

There are different methods to find out what is the spcae consuming files in a FS.

cd /var
find . -size +1000000c
This will show u the files having size more than 1000000 bytes.

or
quot /var will give u which user is holding the maximum FS space

cd /var
du - sk * |sort -n will give the files sizes in sorted order.

Regards
CS

[Raj D.]

Hi Jeffrey ,

1. check maillog file.

/var/adm/maillog , sometimes its become too large , u can copy it to some other place and do a null to this.
# > /var/adm/maillog

2. You can check other big files under /var
# cd /var
# ls -lR | sort +4 -5nr | more

This will give biggest files.

3. and as per the error "have 0, SMTP-DAEMON needs 101 in /var/spool/mqueue" in syslog , you need some space in /var.

Cheers ,

Raj.

[saju_2]

Hi..
I missed the last part..

If u r using online JFS then u can extend the /var FS even without unmounting it..

swlist -l roduct|grep -i jfs
With the above command u can find whether u r using online jfs and if so u can do a lvextend and then fsadm -b M mount point

regards
CS

[Jeffrey F. Goldsmith]

I checked the mail and there isnt any mail at this time. What I did find was a very large file and I dont know where it came from. Do you know if I can delete this file?

/var/adm/wtmp

-rw-rw-r-- 1 adm adm 44732820 Aug 23 10:59 wtmp
root: /var/adm ==>


Thanks,

[Rick Garland]

The wtmp is the file that the 'last' command works from.

Do not delete this file. If you really want to get rid of it null it out 1st.

> wtmp

This will make the file 0 bytes.

If you remove then inodes will still be occupied because it is an open process.

[Jeffrey F. Goldsmith]

Rick, I tried that but was denied because of permissions.

root: /var/adm ==> wtmp
sh: wtmp: Execute permission denied.
root: /var/adm ==>

Do I need to change the owner to root before I can reduce its size? If I can reduce the files size to 0 then that would free up the space I need.

Thanks.

[Rick Garland]

Looking back through the posts, this does not explain the "Aug 23 07:32:35 diomede sendmail[1033]: NOQUEUE: low on space (have 0, SMTP-DAEMON needs 101 in /var/spool/mqueue)" message you received.

How big is the /var filesystem?

If there are no emails to go out, what is the space on /var now?

Was a file deleted earlier? Could this deleted file have been for an open process so the inodes are not freed up?

[Raj D.]

Hi Jeffrey,

/var/adm/wtmp is the file keeps logs of all last logins.

You can make it null , if you dont need the logs , or else keep a copy elsewhere.

To make it null :
# cat /dev/null > /var/adm/wtmp

or simply # > wtmp

Cheers ,
Raj

[Rick Garland]

The root user should be able to do anything.

What is the return from the 'id' command? Is UID=0?

[saju_2]

Hi

Wtmp contains informatin about the user logins, who commmand etc.


From the man page of wtmp it is clear that it should not be deleted.

" Note that wtmp and btmp tend to grow ithout bound, and should be checked regularly. Information that is no longer useful should be removed periodically to prevent it from becoming too large. Also note that wtmp and btmp are not created by the programs that maintain them.Thus, if these files are removed, record-keeping is turned off."

Best was is to trim the file by issuing
>/var/adm/wtmp. This will make it into zero byte file with keeping the file permissions properly.

Regards
CS

[Rick Garland]

Looking closer at your post. Do this command to get rid of the wtmp file. (Do as the root user)

# cd /var/adm
# > wtmp

This will make the file 0 bytes. Now you can delete the file.

What it appears you were doing is trying to execute the file. This is not an executable file. It is a log file for the 'last' command.

[Alex Lavrov.]

The best thing to do:

"cleanup -c 2" or "cleanup -c 1"

Will free up lot of space.

Alex.

[Raj D.]

You can use the -p option with cleanup ,

-p option will preview the cleanup task but do not actually perform the requested action.

Cheers.

[Jeffrey F. Goldsmith]

My thanks to everyone that helped me out with this problem. I was able to clean up a lot of space from committing superceded patches and from reducing log files.

[Jeffrey F. Goldsmith]

I needed to clean out some of the patches on my server by committing the superceded patches. I also reduced the size of several log files.

[Bill Hassell]

/var is *FULL* of files that need to be trimmed on a regular basis. Unless you write a script to run regularly to do this, mail will be the least of your problems. The reason is that /var is the most critical filesystem for virtually all processes on the system. When /var is full, not only email stops (which you've already seen), but logging from a dozen different sources, spooling stops, SAM will probably hang, and most application programs will crash because they can't get any disk space in /var.

To reduce the size of /var/adm/wtmp, you must be root and you can do this:

cat /dev/null > /var/adm/wtmp

NOTE: This file is a list of all login sessions. When you zero the file, you will have no history of logins/logouts. The command: last will show the contents of the special binary file.

Another big file that probably needs trimming: /var/adm/btmp which is a list of failed login attempts. Use the same technique to zero out the file:

cat /dev/null > /var/adm/btmp

And as before, this log is a history, in this case, all failed login attempts are kept here. Both btmp and wtmp must be trimmed regularly.

But wait, there's more! ;-)

/var/adm is full of logfiles. Another logfile location is /var/adm/syslog where you'll find mail.log and syslog.log. The mail.log file can be zeroed out (the cat /dev/null technique above) but syslog should be examined for errors. You may have a bad disk, bada tape, memory problems, processes crashing, etc, etc which are logged in syslog.log. So look at it with vi and make sure everything is OK. Then you can trim the file.

There are numerous other locations to check but I would start by finding the largest directories first:

du -kx /var | sort -rn | head -20

Then look in those directories for large files using:

ll | sort -rnk5 | head -20

If you don't know about certain files, post questions here in the forums. This is a regular task for all Unix administrators.