- Community Home
- >
- Servers and Operating Systems
- >
- Legacy
- >
- Operating System - Tru64 Unix
- >
- Re: swap space full
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-01-2008 10:27 AM
тАО07-01-2008 10:27 AM
need some guidance from all the gurus out there. I'm running out of idea on how to solve this.
i have a gs140 server with 14 x 700mhz w/ 4GB memory and 8GB swap space.It runs TRU64 v4.0F
My application team will executes routine job which process data files. That count 8-16 jobs daily.
Things which puzzle me, the servers suddenly consumes all the available swapspace and server stop responding.
I went to check server during one midnight and still able to login at the main console.
But application team complain they can't login (telnet). I tested and confirmed telnet is useless.
I think swap space is the main cause. An output from `swapon -s`.
Swap partition /dev/vol/rootdg/swapvol:
Allocated space: 1048512 pages (8191MB)
In-use space: 1048497 pages ( 99%)
Free space: 15 pages ( 0%)
Swap partition /dev/vol/rootdg/swapvol1:
Allocated space: 1048512 pages (8191MB)
In-use space: 1048506 pages ( 99%)
Free space: 6 pages ( 0%)
Total swap allocation:
Allocated space: 2097024 pages (16383MB)
In-use space: 2097003 pages ( 99%)
Available space: 21 pages ( 0%)
As a result of this, I add 8GB more of swap space. Now the server has 16GB of swapspace.
But it happened again. This time, i refer System configuration and tuning guide manual.
I execute the following. Pls refer attached file.
Can anyone guide what cause of this. Do i need to tune any parameters.
I know this is an indication of lack of memory space but this doesnt happen in other server which has the same specs and memory.
Thanks in advance.
Rgds
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-01-2008 10:35 AM
тАО07-01-2008 10:35 AM
Re: swap space full
these are the sysconfigtab parameters.
lsm:
lsm_rootdev_is_volume = 1
lsm_swapdev_is_volume = 1
ipc:
msg-max=32768
msg-mnb=65535
msg-mni=128
msg-tql=256
shm-max=2139095040
shm-mni=1024
shm-seg=512
sem-mni=512
sem-msl=512
sem-opm=512
sem-ume=512
num-of-sems=512
max-kernel-ports=16739
port-hash-max-num=8381950
port-reserved-max-num=16739
set-max-num=49413
proc:
max-proc-per-user=2048
max-threads-per-user=4096
per-proc-stack-size=2147483648
max-per-proc-stack-size=21447483648
per-proc-data-size=68719476736
max-per-proc-data-size=68719476736
max-per-proc-address-space=68719476736
per-proc-address-space=68719476736
maxusers=2048
task-max=16405
thread-max=32898
num-wait-queues=1024
num-timeout-hash-queues=1024
vm:
ubc-minpercent=5
ubc-maxpercent=10
ubc-borrowpercent=10
vm-maxvas=16106127360
vm-vpagemax=131072
I modified all parameters to the following today:
> per-proc-stack-size=4294967296
> max-per-proc-stack-size=4294967296
> per-proc-data-size=4294967296
> max-per-proc-data-size=4294967296
> max-per-proc-address-space=4294967296
> per-proc-address-space=4294967296
> ubc-minpercent=10
> ubc-maxpercent=50
> ubc-borrowpercent=20
> vm-maxvas=4294967296
> vm-vpagemax=1966080
After applied the parameters.This time its getting worse, while accessing the server via telnet, it hangs!
I had to access the server from main console, copy the original sysconfigtab file and reboot the server straight away.
When the server startup, app team submit same process. It hangs this time but for only for a few seconds. Then it resumes normally.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-01-2008 07:27 PM
тАО07-01-2008 07:27 PM
Re: swap space full
And if its OK then there must be some problem with application and will try to see the problem from service end.
Can you or ask application tem to check what is happening in application when it is starting , was that aplication running fine before??
BR,
Kapil
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-01-2008 11:29 PM
тАО07-01-2008 11:29 PM
Re: swap space full
this only results in the system becomming even slower beacuse of excessive swapping from memory to disk and back.
allocating more swapspace then 1,5 a 2 times the physical memory has only advantage when there are a lot of "sleeping" processes.
if you run a process wich really needs this much memory you should add physical memory.
this to investigate are processes allocating exessive memory.
- a single process allocating lots of memory
- too many processes running at the same time.
Most common cause i've run into is a periodically scheduled job (or one of its child's), that creates a temporary file with non-unique filename.
when the first instance still runs when the seccond instance starts, they want to access the same file.
This results in problems like
- locking
- one of them processing the wrong data (file from other instance).
as result they both run when a third instance starts etc...
when reducing the swap-space instead of increasing , the faulty job will run into problems earlier wich may help in locating the fault.
a command like this command might help:
ps -e -o vsz,pcpu,pid,rss,majflt,minflt,pmem,command|grep -v COMMAND|sort -nr|head
(vsz, pid and command could be enough).
beware! the sort order is not as we really want it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-02-2008 05:48 AM
тАО07-02-2008 05:48 AM
Re: swap space full
Tuning the system is not going to help much. You need to add more physical memory or reduce the workload or both. Adding more physical memory will probably do wonders for your system performance as well.
Martin
A quick resolution to technical issues for your HPE products is just a click away HPE Support Center
See Self Help Post for more details
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-02-2008 08:22 AM
тАО07-02-2008 08:22 AM
Re: swap space full
The process list from your first attachment shows that there are many processes all trying to grab massive amounts of memory.
For example:
adm 2792 0.0 2.4 3.46G 98M ?? S Jun 14 7-15:28:10 /runss17/
adm/MSP4.7/corp/bin_d/streaming Generic/generic_xgsm2
That's one process of many, that looks like it's trying to grab 3.46GB of memory.
You need to ask the application guys why this and similar processes are trying to use so much space. You might alleviate the problem by adding more physical memory, but I would suggest there is probably a serious issue with the application somewhere...
Cheers,
Rob
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-02-2008 10:08 AM
тАО07-02-2008 10:08 AM
SolutionYou appear to be running with 'eager' swap allocation, which pre-allocates more or less the full VSZ of each running process from the available swap space. That is why you are running out of swap..... because of the numerous 3G+ VSZs.
You might consider running with lazy swap allocation IF your current process load is fairly static.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-02-2008 10:59 PM
тАО07-02-2008 10:59 PM
Re: swap space full
cause of the long text in the first two posts i missed the attachment.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-03-2008 03:39 AM
тАО07-03-2008 03:39 AM
Re: swap space full
But I agree, the basic idea is that the application appears to be trying to hog too much memory.
Martin
A quick resolution to technical issues for your HPE products is just a click away HPE Support Center
See Self Help Post for more details
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-03-2008 01:41 PM
тАО07-03-2008 01:41 PM
Re: swap space full
per-proc-stack-size=2147483648
max-per-proc-stack-size=21447483648
I suspect it is a consequence of the very high stack size settings? WHY is that made so high at 2GB? That is about 1000 times larger than many applications require.
That would certainly explain the 3GB + VSZ's
Setting it to 4G as you did, 'clearly' (to me :-) will make matters worse. try 1GB!
Check out vmstat -P
I suspect it is consuming very many ptepages.
And check out the lzay vs eager spaw comments and concerns.
hth,
Hein