topic Re: Simple question in Operating System - HP-UX

Simple question

Romildo — Fri, 07 Mar 2003 17:02:33 GMT

Hi,

I have N4000 box with hpux 11.0. This box, runs a several applications and frequently I??ve a file system full ( application and operation system ). The question is ???this frequently file system full on the files system of operation system could cause any problem on my system ????

Thanks,
Romildo

Re: Simple question

Marco Santerre — Fri, 07 Mar 2003 17:06:25 GMT

I guess the right answer to this is "It depends". It depends which file system it is.. what use it has on your system? adn things of that nature. But yes it could.. as much as maybe not..

Re: Simple question

Helen French — Fri, 07 Mar 2003 17:09:51 GMT

If I understood your question correctly, then Yes, if the OS *important* file systems such as root (/), /stand, /tmp, /var, /opt, /home etc are full, then you will have problems.

If your data file systems are full, that may affect your user operations. The best option would be to monitor the FS sizes everyday.

Re: Simple question

John Payne_2 — Fri, 07 Mar 2003 17:17:23 GMT

Romildo,

The FS's that are filling up - are they filling with log files, data files, or both? if your FS is filling with data, could you consider moving the data to it's own FS or extending the one you have? If log files are filling the FS, you might consider creating a FS dedicated to log files. (Call it /var/opt/logs or whatever.) Then point as many applications as possible to log into that FS. If the FS fills, at worst you don't get the logging you want. If you can't get the application to log in a different place, you may be able to link the file to /var/opt/logs and force the app to log in the other FS.

Hope it helps

John

Re: Simple question

Pete Randall — Fri, 07 Mar 2003 17:22:40 GMT

Simple answer: yes. You've got a potential problem. You need to address this either by enlarging the file system in question, modifying the application to stop it from filling the file system, or routine maintenance of the file system.

If you could expand on this, we could offer more precise assistance.

Pete

Re: Simple question

A. Clay Stephenson — Fri, 07 Mar 2003 17:25:51 GMT

At the very least, applications are crashing. Filesystems filling up are problems that should not be ignored. Often the fix is to educate the users. You need to consider what happens when a non-critical application fills a filesystem that is also needed by a critical application. If this is the /var filesystem then yes, filling it can clobber the system. You should consider creating separate mountpoints for /var/tmp, var/spool, and maybe /var/opt.

Re: Simple question

Steven E. Protter — Fri, 07 Mar 2003 17:33:47 GMT

If /usr fills up, no big deal, it is supposed to contain static executables.

If /var hits 100% your system will halt. This is where your system writes logs and it simply can't run if its full.

If / "root" fills up, your system may stop accepting logins. The /etc/group or /etc/passwd file may get clobbered.

Application filesystems may be a problem too.

Lets say you have oracle or some other database writing archive logs(database transaction logs) to an application filesystem. Filesystem fills up. Oracle crashes, no more databse up. That would be a problem if that database powers a 24/7 customer service site or something.

Space management is a big issue. As sysadmin you need to stay on top of it.

SEP

Re: Simple question

Romildo — Fri, 07 Mar 2003 18:37:20 GMT

Hi,
To clariffy the question. The box runs three instances of sybase dataserver using raw devices. Runs batches programs (C) too, that uses flat file on file system of application only. After four months active and various file system full on /, /var and /tmp, the databases servers has problems with shared memory and solution was a system reboot.
The system operation file system was quite dimensioned, but s.. happens.
I oppen hp call and the aswer is that no relation on ???file system full??? ( of operation system??s ) and ???shared memory problems???. Is it true ?

Thanks all,
Romildo

Re: Simple question

A. Clay Stephenson — Fri, 07 Mar 2003 18:51:14 GMT

Depending upon your application there could very well be a relationship between filesystem full and shared memory. IF you have an application that hangs because of lack of filespace or dies without detaching its shared memeory OR (and thius is most likely) you have killed processes with kill -9 then shared memory will not be detached/removed. This is not an OS problem but rather an administrative problem. You need to monitor the disk usage and take pre-emptive step to avoid the problem.

Re: Simple question

Pete Randall — Fri, 07 Mar 2003 19:04:27 GMT

Unless you need to keep these application files around for four months, I would suggest a periodic cleansing with the find commmand:

find /name_of_file_system -type f -mtime +30 -exec rm {} \;

This would search your filesystem for files not touched in 30 days and remove them. If you put this in a cron job that runs every night, or even once a week, you'll have less trouble with things filling up.

Pete

Re: Simple question

Bill Hassell — Fri, 07 Mar 2003 19:15:49 GMT

The 3 filesystems you mentioned (/ /var /tmp) are CRITICAL filesystems and cannot be allowed to fill up. As Clay mentioned, indirectly shared memory cannot be allocated (primarily a limitation of old 32bit applications) if programs are not stopped correctly (kill -9 is definitely the WRONG way to terminate any process).

The / directory should never grow in size. It is a static directory and contains /dev and /etc. /dev may increase by a few bytes as device files are add, while /etc might increase when config files are added and/or edited. The 3 primary sysadmin errors causing / to fill up are:

1. root 'droppings' because root' HOME directory defaults to /, a very bad choice. root's HOME should be moved to /home/root and the passwd file updated accordingly. This prevents mistakes from taking out the entire opsystem (reboot).

2. Applications install in non-standard directories (ie, the app is stored in /app_name or similar) and this is very bad. Move (or remove & reinstall) apps in /opt where they belong.

3. A misspelled devicefile used for a backup automatically becomes an ordinary file as in:

tar cvf /dev/rmt/om /opt

where om should have been 0m (zero versus the letter o). There must be no ordinary files in the /dev directory (hint find /dev -type f)

All Unix systems require active management of /tmp and /var as poor user habits and/or applications will easily fill up these filesystems with junk. And /var also has numerous log files that only grow loarger--these must be trimmed on a regular basis. Your system will continue to be very unstable unless disk space is monitored on a regular basis (hint: cron jobs).

Re: Simple question

Romildo — Fri, 07 Mar 2003 20:03:42 GMT

Hi,
Thank Bill and Clay, you are right, the sybase process was unstable and the DBA adm kill then with ???kill -9???, before clean shared memory with ???ipcrm??? and try to start instances, but it didn??t go on. Then the reboot on the server was need.

Thanks
Romildo