- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Re: Memory usage
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-07-2007 03:28 AM
тАО12-07-2007 03:28 AM
I try to calculate the amount of the memory that is used by all processes in the system.
I try this:
# UNIX95=1 ps -e -o vsz | awk 'BEGIN {sum=0} {sum+=$1} END {printf("%d\n",sum)}'
However ths command displays something unreasonable for me. The output is: 10794960 (~10GB!).
And this:
#ipcs -mob | tail +3 | awk 'BEGIN {sum2=0} {sum2+=$NF} END {printf("%d\n",sum2/1024/1024)}'
adds to my calculation ~1GB
The total memory usage in the system is ~11GB
But the server has only 2 GB of RAM and 4 GB of swap:
#swapinfo -t
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4194304 585804 3608500 14% 0 - 1 /dev/vg00/lvol2
reserve - 1876880 -1876880
memory 2085648 759992 1325656 36%
total 6279952 3222676 3057276 51% - 0 -
No swapping in the system (sar -w ...; vmstat ... display not swapouts)
So, where am I wrong?
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-07-2007 03:30 AM
тАО12-07-2007 03:30 AM
Re: Memory usage
HP-UX B.11.23 ia64
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-07-2007 04:28 AM
тАО12-07-2007 04:28 AM
Re: Memory usage
And you don't need to add shared memory in again unless you want to add in only segments where the NATTCH is 0 [those wouldn't be in the ps output, anything which is attached will].
It isn't going to be that unexpected to get a value higher than the total swap size from a VSZ summation -- first, not everything has to reserve swap. Anything backed by a normal file where the changes are shared across all attachers (even if the VA is private like Text) just uses the file in the case of paging, so no swap is needed or acquired. Second, the swap-backed shared values get a little finessed (to try to count the pages used across those attached), so on a running system that's easily a little off by the time you walk the whole process space.
In your case, I'd bet on file-backed objects adding in your additional 4Gb of virtual address space.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-07-2007 05:28 AM
тАО12-07-2007 05:28 AM
Re: Memory usage
As I see in the "man ps" the sz is "The size in physical pages of the core image of the process, including text, data, and stack space. Physical page size is defined by _SC_PAGE_SIZE in the header file
And vsz is "The size in kilobytes (1024 byte units) of the core image of the process. See column sz, above."
And no references about attachments to shared segments mentioned.
Even more:
getconf _SC_PAGE_SIZE
from the shell dives to me "4096"
For some processes (init for example) vsz/sz=4
But for some other processes (oracle) vsz/sz=14
And may be other ratios exists.
I suspect the "chatr" can be be used to change the default page_size.
So the question is open still:
How can I calculate the memory consumption for some process group or for all processes?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-07-2007 06:19 AM
тАО12-07-2007 06:19 AM
Re: Memory usage
# cat processmem
#!/bin/sh
# processmem - display memory claimed by a process
#
if [ $# -lt 1 -o \( $# -gt 1 -a $# -lt 4 \) ]
then
echo "Usage:"
echo "processmem \"process\""
echo "Example:"
echo "processmem rpc"
exit 1
fi
echo " "
PROCESS=$1
mps=0
#for sz in `ps -elf | grep $PROCESS | grep -v grep | awk '{print $10}'`
for sz in `UNIX95= ps -e -o vsz=Kbytes -o ruser -o pid,args=Command-Line | sort -rnk1 | grep -v Kbytes | grep $PROCESS | awk '{print $1}'`
do
mps=`expr $mps + $sz`
done
#echo `expr $mps \* 4096`
echo "\nMemory claimed by $PROCESS: $mps Kbytes.\n"
# cat usermem
#!/bin/ksh
# usermem - display memory claimed by a user
# gwild 06232003
#
if [ $# -lt 1 -o \( $# -gt 1 -a $# -lt 4 \) ]
then
echo "Usage:"
echo "usermem \"userid\""
echo "Example:"
echo "usermem gwild"
exit 1
fi
echo " "
USER=$1
t=0
for j in `UNIX95= ps -e -o vsz=Kbytes -o ruser -o pid,args=Command-Line | sort -rnk1 | grep -v Kbytes | grep " $USER " | awk '{print $1}'`
do
t=`expr $t + $j`
done
echo "\nMemory claimed by $USER: $t Kbytes.\n"
Rgds...Geoff
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-07-2007 06:41 AM
тАО12-07-2007 06:41 AM
Re: Memory usage
As I understud your scrips do the same things I did above: simple summarize of the SZ and VSZ columns of the ps output.
And as I mentioned above the SZ column is not always equal to 4096 for the different processes. Even though "getcont _SC_PAGE_SIZE" claims that PAGE_SISE IS 4096.
So, it is NOT correct to multiply SZ by 4096
Am I wrong?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-07-2007 06:54 AM
тАО12-07-2007 06:54 AM
Re: Memory usage
i.e. If you're using 4 pages (sz == 4) on most systems that translates to consuming 16kb of physical memory (4 * 4096 bytes).
And the chatr (or vps_* tunables) is not changing the system page size -- that affects the page size hint used for variable translation sizes. The system base page size determines the smallest possible size which can be used for a translation ever anywhere. [It also affects the per-page metadata within the kernel, obviously]. Variable translations are ones which are composed of a group of said pages but use a single translation. For example -- if you're allocating 16 4kb pages in a single contiguous range, you can represent that with a single 64kb translation instead of 16 unique translations. This keeps TLB miss rates down.
And yeah -- sz is physical, vsz is virtual. That's what I said in the first place.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-07-2007 06:55 AM
тАО12-07-2007 06:55 AM
SolutionDo you want to know "real" memory used - as in ram? Or do you want to count "virtual" memory?
If you want "How much total ram is in use", then use the attached C program called memdetail.
You can compile with the native compiler...
Output like:
# memdetail
Memory Stat total used avail %used
physical 16128.0 14724.9 1403.1 91%
active virtual 18692.0 6386.9 12305.1 34%
active real 11970.9 3868.4 8102.5 32%
memory swap 16128.0 3883.2 12244.8 24%
device swap 26528.0 17482.7 9045.3 66%
Rgds...Geoff
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-07-2007 07:26 AM
тАО12-07-2007 07:26 AM
Re: Memory usage
"And yeah -- sz is physical, vsz is virtual. That's what I said in the first place."
OK
"sz is physical" measured in what? KB?
I'm stupid, sorry.))
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-07-2007 08:02 AM
тАО12-07-2007 08:02 AM
Re: Memory usage
Besides Geoff's always excellent scripts/techniques, you could try the attached
[compile as:
cc +DD32 -D_PSTAT64 -o object_dump object_dump.c
or
cc +DD64 -o object_dump object_dump.c
[Rename the binary anything you want, if you wish].
Caveat: I don't have 11.11 or 11.0 systems, so I may have compilation issues there. This was written on r11.31 IPF.
If you just want the system wide totals:
# ./object_dump
VIRT/PHYS/LOCKED/SWAP summaries in pages.
System page size is 4096 or 0x1000 bytes.
System Wide Mb in Use:
Virt: 7221 Phys: 746 Swap: 321
If you want to know a particular process:
# ./object_dump -p 699
VIRT/PHYS/LOCKED/SWAP summaries in pages.
System page size is 4096 or 0x1000 bytes.
PID 699 Mb in Use:
Virt: 274 Phys: 6 Swap: 1
If you need to see details:
# ./object_dump -v -p 699
VIRT/PHYS/LOCKED/SWAP summaries in pages.
System page size is 4096 or 0x1000 bytes.
PID 699:
UNUSED TYPE consumes 0 VIRT, 0 PHYS, 0 LOCKED and 0 SWAP.
UAREA consumes 18 VIRT, 16 PHYS, 0 LOCKED and 18 SWAP.
TEXT consumes 50 VIRT, 46 PHYS, 0 LOCKED and 0 SWAP.
DATA/HEAP consumes 200 VIRT, 142 PHYS, 0 LOCKED and 200 SWAP.
MAIN STACK consumes 65536 VIRT, 8 PHYS, 0 LOCKED and 8 SWAP.
SYSV SHMEM consumes 0 VIRT, 0 PHYS, 0 LOCKED and 0 SWAP.
NULL DEREF consumes 1 VIRT, 1 PHYS, 0 LOCKED and 1 SWAP.
MEM MAPPED I/O consumes 0 VIRT, 0 PHYS, 0 LOCKED and 0 SWAP.
MMAP consumes 2321 VIRT, 1439 PHYS, 0 LOCKED and 108 SWAP.
GRAPHICS SPECIFIC consumes 0 VIRT, 0 PHYS, 0 LOCKED and 0 SWAP.
GRAPHICS SPECIFIC consumes 0 VIRT, 0 PHYS, 0 LOCKED and 0 SWAP.
RSE STACK consumes 2048 VIRT, 1 PHYS, 0 LOCKED and 2 SWAP.
PID 699 Mb in Use:
Virt: 274 Phys: 6 Swap: 1
If you want a LOT of detail, add another -v option [order doesn't matter] like -v -v -pid 699. Verbosity works on the system-wide option as well, but that will be a LOT of data.
It wasn't hard to add group data (-g option):
# ./object_dump -g 1644
VIRT/PHYS/LOCKED/SWAP summaries in pages.
System page size is 4096 or 0x1000 bytes.
PID 1644 Mb in Use:
Virt: 313 Phys: 29 Swap: 11
PID 1647 Mb in Use:
Virt: 367 Phys: 43 Swap: 26
PID 1646 Mb in Use:
Virt: 317 Phys: 25 Swap: 7
PID 1645 Mb in Use:
Virt: 296 Phys: 16 Swap: 1
PID 6753 Mb in Use:
Virt: 307 Phys: 23 Swap: 2
Group Wide Mb in Use:
Virt: 1603 Phys: 138 Swap: 50