High memory utilization//need to tune

[mrmo07]

hi experts !!

my systems (rx6600-hpuxv3) having high memory utization.  please help me to tune the memory utilization. sharing some output from one db host . please let me know which info you need for good solutins,

 Glance 11.00.044               17:45:42   dcpdb2     ia64                                                        Current  Avg  High
------------------------------------------------------------------------------------------------------------------------------------
CPU  Util   SSU  U                                                                                                 |  7%    6%   11%
Disk Util   FF                                                                                                     |  2%    3%   14%
Mem  Util   S                  SU                                                       UF                 F       | 95%   95%   95%
Swap Util   U       UR                                 R                                                           | 43%   43%   43%
------------------------------------------------------------------------------------------------------------------------------------
                                                           MEMORY REPORT                                                Users=    1
Event         Current   Cumulative   Current Rate   **bleep** Rate   High Rate
--------------------------------------------------------------------------------
Page Faults      1006        48649       189.8      126.6       571.4
Page In           480           41        90.5        0.1       265.3
Page Out            0            0         0.0        0.0         0.0
KB Paged In       0kb        164kb         0.0        0.4        21.2
KB Paged Out      0kb          0kb         0.0        0.0         0.0
Reactivations       0            0         0.0        0.0         0.0
Deactivations       0            0         0.0        0.0         0.0
KB Deactivated    0kb          0kb         0.0        0.0         0.0
VM Reads            0           17         0.0        0.0         0.7
VM Writes           0            0         0.0        0.0         0.0
Total VM :  20.1gb   Sys Mem  :   4.7gb   User Mem:  13.5gb   Phys Mem :  24.0gb
Active VM:  10.9gb   Buf Cache:     0mb   Free Mem:   1.2gb   FileCache:   4.5gb
MemFS Blk Cnt:                0   MemFS Swp Cnt:                0

========================

root@dcpdb2 [/tmp/morshed]# ./kmeminfo -static
tool: kmeminfo 7.09 - HP CONFIDENTIAL
unix: /stand/vmunix 11.31 64bit IA64 on host "dcpdb2"
core: /dev/kmem live
link: Wed Jul 25 11:33:56 EDT 2012
boot: Tue Apr  8 09:25:24 2014
time: Wed Sep 10 17:49:23 2014
nbpg: 4096 bytes


----------------------------------------------------------------------
Physical memory usage summary (in page/byte/percent):

Physical memory       =  6283593   24.0g 100%
Free memory           =   273898    1.0g   4%
User processes        =        0    0.0b   0%  details with -user
System                =       64  256.0k   0%
  Kernel              =        0    0.0b   0%  kernel text and data
    Dynamic Arenas    =   316354    1.2g   5%  details with -arena
      vm_alias_table_ =    35483  138.6m   1%
      vx_global_kmcac =    34806  136.0m   1%
      spinlock_arena  =    30899  120.7m   0%
      vm_pfn2v_arena  =    24920   97.3m   0%
      BTREE_NODE_OLA_ =    17575   68.7m   0%
      Other arenas    =   172671  674.5m   3%  details with -arena
    Super page pool   =   414552    1.6g   7%  details with -kas
    Static Tables     =   367482    1.4g   6%  details with -static
      pfdat           =   306816    1.2g   5%
      vhpt            =    32768  128.0m   1%
      text            =     9116   35.6m   0%  vmunix text section
      inode           =     7616   29.8m   0%
      bss             =     6762   26.4m   0%  vmunix bss section
      Other tables    =     4402   17.2m   0%  details with -static
  Buffer cache        =       30  120.0k   0%  details with -bufcache
  UFC file mrg        =  1196877    4.6g  19%
  UFC meta mrg        =       34  136.0k   0%

----------------------------------------------------------------------
System tables:

pfdat                 =   306816    1.2g   5%
vhpt                  =    32768  128.0m   1%
text                  =     9116   35.6m   0%  vmunix text section
inode                 =     7616   29.8m   0%
bss                   =     6762   26.4m   0%  vmunix bss section
data                  =     2057    8.0m   0%  vmunix data section
ncache                =     1400    5.5m   0%
nbuf                  =      512    2.0m   0%  bufcache headers
nc_hash               =      256    1.0m   0%
pfn2v_array           =      128  512.0k   0%
bufhash               =       16   64.0k   0%  bufcache hash headers
nc_lru                =       10   40.0k   0%
sextmap               =        3   12.0k   0%
spu_info[7].pdkptr    =        2    8.0k   0%
spu_info[6].pdkptr    =        2    8.0k   0%
spu_info[5].pdkptr    =        2    8.0k   0%
spu_info[4].pdkptr    =        2    8.0k   0%
spu_info[3].pdkptr    =        2    8.0k   0%
spu_info[2].pdkptr    =        2    8.0k   0%
spu_info[1].pdkptr    =        2    8.0k   0%
io_mem_tbl            =        1    4.0k   0%
root@dcpdb2 [/tmp/morshed]#

Regards,

Morshed.

[Patrick Wallek]

Before you think about "tuning" your memory utilization (which you can't really do) you first have to understand what processes are using the memory.

According to glance the majority of your memory usage appears to be in the user space, which means user processes.  If you are running a database, like Oracle, then the Oracle SGA and other areas will be counted among this. 

Keep in mind that 95% memory usage is not necessarily a bad thing.  You have paid for and installed the memory, so why not use it?  Now high memory usage CAN be a bad thing if you are constantly hitting the 100% mark and then paging out to disk on a regular basis.

Are you paging out to disk?  What does 'swapinfo -tam' show?

[mrmo07]

Hi patrick !!

thanks for the feedback. actullay alarming is migration is going on here. only 10% user migrated to this system. soon many others users will use this system. what will happen then ? yes that was from DB server. RAC is configured.   my applicaton server also using more mermoy. output below: i actually need to know  who is actullay using this memroy and is it normal ?

output from DB server:

root@dcpdb2 [/]# swapinfo -tam
             Mb      Mb      Mb   PCT  START/      Mb
TYPE      AVAIL    USED    FREE  USED   LIMIT RESERVE  PRI  NAME
dev       32768      14   32754    0%       0       -    1  /dev/vg00/lvol2
reserve       -   18955  -18955
memory    23345    4923   18422   21%
total     56113   23892   32221   43%       -       0    -

output from app:

 Glance 11.00.044               09:39:21 dcpapp01     ia64                                                        Current  Avg  High
------------------------------------------------------------------------------------------------------------------------------------
CPU  Util                                                                                                          |  1%    2%    5%
Disk Util   F                                                                                                      |  1%    2%    3%
Mem  Util   S                                  SU  UF                                   F                          | 75%   75%   75%
Swap Util   U                    UR R                                                                              | 25%   25%   25%
------------------------------------------------------------------------------------------------------------------------------------
                                                           MEMORY REPORT                                                Users=    3
Event         Current   Cumulative   Current Rate   **bleep** Rate   High Rate
--------------------------------------------------------------------------------
Page Faults       588         3330       113.0       45.8       292.8
Page In           161           48        30.9        0.6        41.7
Page Out            0            0         0.0        0.0         0.0
KB Paged In       0kb        192kb         0.0        2.6       135.2
KB Paged Out      0kb          0kb         0.0        0.0         0.0
Reactivations       0            0         0.0        0.0         0.0
Deactivations       0            0         0.0        0.0         0.0
KB Deactivated    0kb          0kb         0.0        0.0         0.0
VM Reads            0            0         0.0        0.0         0.0
VM Writes           0            0         0.0        0.0         0.0
Total VM :   1.7gb   Sys Mem  :   8.5gb   User Mem:   954mb   Phys Mem :  24.0gb
Active VM:   1.3gb   Buf Cache:     0mb   Free Mem:   6.0gb   FileCache:   8.6gb
MemFS Blk Cnt:                0   MemFS Swp Cnt:                0

root@dcpapp01 [/]# swapinfo -tam
             Mb      Mb      Mb   PCT  START/      Mb
TYPE      AVAIL    USED    FREE  USED   LIMIT RESERVE  PRI  NAME
dev       16384       0   16384    0%       0       -    1  /dev/vg00/lvol2
reserve       -    1218   -1218
memory    23345    8532   14813   37%
total     39729    9750   29979   25%       -       0    -

Regards,

Morshed.

[mrmo07]

HI ,

filecache=8.7 gb.

can i reduce this.

FYI,

filecache_max                Auto        12239650816  -           12239650816  74.5%     fs_bufcache
filecache_min                Auto        1223962624   -           1223962624   -         fs_bufcache

Regards,

MOrshed.

[Dennis Handly]

>filecache=8.7 gb.   can I reduce this?

Yes but no need to do that.  The OS will automatically reduce the cache when it needs memory.