- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Urgent Help Please. /tmp 100% full
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 05:40 AM
08-01-2013 05:40 AM
Hello Experts,
Need urgent help please.
Server Details:HP-UX rx3600 B.11.31 U ia64
I got below message on the sql prompt as well as on user prompt.
vxfs: msgcnt 27130 mesg 001: V-2-1: vx_nospace - /dev/vg00/lvol4 file system full (2 block extent)
when i look for dbf it shows that /tmp directory gets 100% full, tried to free some space but still it is 99 to 100% usage.
HP_perf_vmstat is getting updated frequently, can we create a new file with the same name and drop the existing one by taking the backup of it.
-rw-r--r-- 1 oracle oinstall 429617270 Aug 1 12:51 HP_perf_vmstat
I will be really thankful if you could provide the solution/workaround to fix it as it is production database server.
Thanks & Best Regards,
Syed
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 06:02 AM
08-01-2013 06:02 AM
Re: Urgent Help Please. /tmp 100% full
I erased all unnecessary files. In spite of that the /tmp dir is still showing %100 full. What could be a problem?
Please advise.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 06:23 AM
08-01-2013 06:23 AM
Re: Urgent Help Please. /tmp 100% full
If the files that you removed happen to be in use, then the space will not be reclaimed until the process(es) using those files stop.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 06:40 AM
08-01-2013 06:40 AM
Re: Urgent Help Please. /tmp 100% full
Thanks for the reply.
Please help me out to check for those process and kill it to release the space at earliest.
Regards
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 07:00 AM
08-01-2013 07:00 AM
Re: Urgent Help Please. /tmp 100% full
You deleted an open file, better keep it and delete the contents only.
Hope this helps!
Regards
Torsten.
__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.
__________________________________________________
No support by private messages. Please ask the forum!
If you feel this was helpful please click the KUDOS! thumb below!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 07:24 AM
08-01-2013 07:24 AM
Re: Urgent Help Please. /tmp 100% full
Dear Torsten,
I have already deleted the files after taking backup. Please advise what i have do now as am continously getting this below message
vxfs: msgcnt 28998 mesg 001: V-2-1: vx_nospace - /dev/vg00/lvol4 file system full (2 block extent)
now there is 0 space available in the /tmp.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 07:38 AM
08-01-2013 07:38 AM
Re: Urgent Help Please. /tmp 100% full
Hope this helps!
Regards
Torsten.
__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.
__________________________________________________
No support by private messages. Please ask the forum!
If you feel this was helpful please click the KUDOS! thumb below!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 07:48 AM
08-01-2013 07:48 AM
Re: Urgent Help Please. /tmp 100% full
#fuser -cu /tmp
/tmp: 1493o(root) 1483o(root) 1475o(root) 1467o(root) 13615c(root) 2094o(root) 2042o(root) 2320o(oracle) 2334o(oracle) 14313c(oracle) 14194o(oracle) 28666c(root) 14107c(oracle)
from the above how do i know which pid is of that deleted file i.e. of "HP_perf_vmstat", or do i need to kill all those process?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 08:18 AM
08-01-2013 08:18 AM
Re: Urgent Help Please. /tmp 100% full
Awaiting for the reply!!!
Please help me out to solve the issue!!!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 08:21 AM
08-01-2013 08:21 AM
Re: Urgent Help Please. /tmp 100% full
Hope this helps!
Regards
Torsten.
__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.
__________________________________________________
No support by private messages. Please ask the forum!
If you feel this was helpful please click the KUDOS! thumb below!
- Tags:
- lsof
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 09:10 AM
08-01-2013 09:10 AM
Re: Urgent Help Please. /tmp 100% full
No, it is not installed in the server.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 10:11 AM
08-01-2013 10:11 AM
Re: Urgent Help Please. /tmp 100% full
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 10:46 AM
08-01-2013 10:46 AM
SolutionFrom your earlier post:
===
#fuser -cu /tmp
/tmp: 1493o(root) 1483o(root) 1475o(root) 1467o(root) 13615c(root) 2094o(root) 2042o(root) 2320o(oracle) 2334o(oracle) 14313c(oracle) 14194o(oracle) 28666c(root) 14107c(oracle)
from the above how do i know which pid is of that deleted file i.e. of "HP_perf_vmstat", or do i need to kill all those process?
===
Since you dont have lsof, you will need to look at each individual process to determine what it is running. For example, take the first entry (after /tmp) :
1493o(root)
All you need from that are the digits - 1493. Issue a ps command: ps -ef | grep 1493
Look at what it gives back. Does this look like the process that is/was running "HP_perf_vmstat"?
If so, kill that process: kill -15 1493 or kill -9 1493.
If not, look at the next process in the list. Repeat with each process ID until you find the process that needs to be stopped.
Be sure to carefully examine the process before killing it AND be sure you really want to kill it. Killing the wrong process may lead to other issues.
As an alternative - was that command kicked off as part of something else?
IE is there an overall process/application that can be shutdown: /sbin/init.d/someapplication stop? This would be the cleaner way if it is an option.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 01:27 PM
08-01-2013 01:27 PM
Re: Urgent Help Please. /tmp 100% full
Hi sbrew,
Thanks for your detail steps. Following are the results of all those processes, but i didn' see any of them shows about the process which i was looking for i.e. "HP_perf_vmstat".
mcmsdbl1@root@/>#fuser -cu /tmp
/tmp: 1493o(root) 1483o(root) 1475o(root) 1467o(root) 13615c(root) 2094o(root) 2042o(root) 2320o(oracle) 2334o(oracle) 14313c(oracle) 14194o(oracle) 14107c(oracle)
mcmsdbl1@root@/>#ps -ef | grep 1493
root 1493 1 0 Jun 21 ? 0:00 /usr/sbin/trapdestagt
root 11506 8306 1 22:57:32 pts/2 0:00 grep 1493
mcmsdbl1@root@/>#ps -ef | grep 1483
root 1483 1 0 Jun 21 ? 6:42 /usr/sbin/mib2agt
root 11517 8306 0 22:57:46 pts/2 0:00 grep 1483
mcmsdbl1@root@/>#ps -ef | grep 1475
root 1475 1 0 Jun 21 ? 9:03 /usr/sbin/ipv6agt
root 11519 8306 0 22:57:54 pts/2 0:00 grep 1475
mcmsdbl1@root@/>#ps -ef | grep 1467
root 1467 1 0 Jun 21 ? 0:00 /usr/sbin/hp_unixagt
root 11523 8306 1 22:58:01 pts/2 0:00 grep 1467
mcmsdbl1@root@/>#ps -ef | grep 13615
root 13615 13612 0 11:46:41 pts/1 0:05 -sh
root 11544 8306 0 22:58:24 pts/2 0:00 grep 13615
oracle 28942 13615 0 18:01:37 pts/1 0:00 -sh
mcmsdbl1@root@/>#ps -ef | grep 2094
root 2094 1 0 Jun 21 ? 0:00 /bin/sh /sbin/init.d/init.ohasd run
root 11546 8306 0 22:58:38 pts/2 0:00 grep 2094
mcmsdbl1@root@/>#ps -ef | grep 2042
root 2042 1 0 Jun 21 ? 425:59 /u01/app/11.2.0/grid/bin/ohasd.bin reboot
root 11548 8306 0 22:58:47 pts/2 0:00 grep 2042
mcmsdbl1@root@/>#ps -ef | grep 2320
oracle 2320 1 0 Jun 21 ? 29:20 /u01/app/11.2.0/grid/bin/gpnpd.bin
root 11558 8306 0 22:58:55 pts/2 0:00 grep 2320
mcmsdbl1@root@/>#ps -ef | grep 2334
oracle 2334 1 0 Jun 21 ? 420:07 /u01/app/11.2.0/grid/bin/ocssd.bin
root 11560 8306 0 22:59:06 pts/2 0:00 grep 2334
mcmsdbl1@root@/>#ps -ef | grep 14313
oracle 14313 14107 0 Jul 22 pts/4 0:00 sqlplus / as sysdba
root 11584 8306 0 22:59:28 pts/2 0:00 grep 14313
oracle 2768 14313 0 Jul 23 pts/4 0:00 /sbin/sh
mcmsdbl1@root@/>#ps -ef | grep 14194
oracle 14194 1 0 Jul 22 pts/4 9:56 /usr/bin/vmstat -d -n 1
root 11587 8306 0 22:59:40 pts/2 0:00 grep 14194
mcmsdbl1@root@/>#ps -ef | grep 14107
oracle 14313 14107 0 Jul 22 pts/4 0:00 sqlplus / as sysdba
root 11617 8306 0 23:00:00 pts/2 0:00 grep 14107
oracle 14107 9275 0 Jul 22 pts/4 0:00 /sbin/sh
mcmsdbl1@root@/>#
still i see the below message continously going on
vxfs: msgcnt 36206 mesg 001: V-2-1: vx_nospace - /dev/vg00/lvol4 file system full (2 block extent)
server started giving slow response now.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 01:55 PM
08-01-2013 01:55 PM
Re: Urgent Help Please. /tmp 100% full
Of all the processes listed, I would look at the following:
mcmsdbl1@root@/>#ps -ef | grep 14194
oracle 14194 1 0 Jul 22 pts/4 9:56 /usr/bin/vmstat -d -n 1
it isrunning vmstat and circumstantial evidence (the process name) relates it to hp_perf_vmstat AND it appears to have been running since July 22. Since its being run under the oracle ID, you should contact your DBAs and see what they can tell you about it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 02:07 PM
08-01-2013 02:07 PM
Re: Urgent Help Please. /tmp 100% full
on a related note, how big is your /tmp mount point?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 03:13 PM
08-01-2013 03:13 PM
Re: Urgent Help Please. /tmp 100% full
Hey sbrews,
Thanks a ton for your great assistance to get my issue resolve. After killing that process it releases the space, now the current status of /tmp is as below.
/dev/vg00/lvol4 524288 98688 423416 19% /tmp
about the size of mb is as above in kb.
Thanks & Best Regards
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 03:21 PM
08-01-2013 03:21 PM
Re: Urgent Help Please. /tmp 100% full
FYI, Below the current file system layout.
File System layout
LVM Device file mount point size fs type
/dev/vg00:
/dev/vg00/lvol1 /stand 1792 vxfs
/dev/vg00/lvol2 swap 8192
/dev/vg00/lvol3 / 1024 vxfs
/dev/vg00/lvol4 /tmp 2048 vxfs
/dev/vg00/lvol5 /home 128 vxfs
/dev/vg00/lvol6 /opt 16448 vxfs
/dev/vg00/lvol7 /usr 5824 vxfs
/dev/vg00/lvol8 /var 16384 vxfs
/dev/vg00/lvo19 /u01 61440 vxfs
/dev/vg00/lvo20 /backup 102400vxfs
/dev/vg00/lvo22 57344 unused
/dev/vg00 unallocated 6016
If possible can i insrease the size of /tmp by using some space from the unallocated size i.e. of around 5 to 6gb.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 03:24 PM
08-01-2013 03:24 PM
Re: Urgent Help Please. /tmp 100% full
your /tmp is only 512M in size, you should look into making it larger. Overall size depends on how your system is being used. I tend to go with 2G as a minimum and depending on system use, I may make it larger.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 03:27 PM
08-01-2013 03:27 PM
Re: Urgent Help Please. /tmp 100% full
Check this page - it has some examples of adding extending file systems, etc.
http://www.unixguide.net/hp/hplvmtasks.shtml
Also, if you google for hpux file system resizing, you should find lots of other examples.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 04:33 PM
08-01-2013 04:33 PM
Re: Urgent Help Please. /tmp 100% full
Thanks for the information!!!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2013 09:12 PM
08-01-2013 09:12 PM
Re: Urgent Help Please. /tmp 100% full
>how big is your /tmp mount point?
Also, how big is /var/tmp? That's what should be used instead of /tmp, if on a different filesystem.