filecache_min max error

[zxcv]

Hi guys ,

i have 11.31 O.S with server memory getting thrashed .

from forums i found out kernel param filecache_min and max.

i tried to change the values but getting following error;

root #/tmp >kctune filecache_min=5% filecache_max=10%
==> Update the automatic 'backup' configuration first? y
* The automatic 'backup' configuration has been updated.
* Future operations will update the backup without prompting.
NOTE: The requested changes could not be applied to the currently
running system, for the following reasons:
- Failed to resize up (grow) the Filedata Memory Resource
Group.Error code 12.
* The requested changes have been saved, and will take effect at
next boot.
Tunable Value Expression Changes
filecache_max (now) 134217728 134217728 Imm (auto disabled)
(next boot) (auto) 10%
filecache_min (now) 134217728 134217728 Imm (auto disabled)
(next boot) (auto) 5%

Cant it be done online ?

Whats dat Rc code 12 ?

[Duncan Edmonstone]

Are you up to date on patching? I can see mention of an error similar to this in some of the VM cumulative patches - latest is PHKL_42850 - might want to ensure you have that and dependencies?

A Memory Resource Group is a function of PRM, but I suspect that PRM may still do some work kernel-side even when you aren't using it - nevertheless it would be interesting to see the output of "prmconfig" - I think its likely you will just get back "Cannot display configuration. PRM is not configured. (PRM-226)", but you never know...

FInaly - the message about not growing a memory space is perfectly legal in some cirumstances, particularly if you are under memory pressure - are you sure you weren't _increasing_ filecache_min ? Maybe it was laready set lower than 5% ?? The output of "kctune -v filecache_max filecache_min" would tell us (it would be useful to know how much physical memory you have as well - "machinfo" should tell you that

[Dennis Handly]

>I tried to change the values but getting following error;

You may need to change them one at a time.

What does "swapinfo -tam" show?

>The output of "kctune -v filecache_max filecache_min" would tell us

Aren't they the "now" values of: 134,217,728

[zxcv]

Hi Dennis,

root #/tmp >kctune -v filecache_max filecache_min
Tunable filecache_max
Description Maximum amount of physical memory to be used for caching file I/O data
Module fs_bufcache
Current Value 134217728
Value at Next Boot (auto) [10%]
Value at Last Boot 134217728
Default Value 17067343872 (automatic)
Can Change Immediately (Automatic Tuning Disabled)

Tunable filecache_min
Description Minimum guaranteed physical memory used for caching file I/O data
Module fs_bufcache
Current Value 134217728
Value at Next Boot (auto) [5%]
Value at Last Boot 134217728
Default Value 1706688512 (automatic)
Can Change Immediately (Automatic Tuning Disabled)

root #/tmp >
root #/tmp >
root #/tmp >
root #/tmp >swlist -l patch|grep -i 428
PHKL_42850.ADMN-ENG-A-MAN 1.0 OS-Core.ADMN-ENG-A-MAN applied
PHKL_42850.CORE-ENG-A-MAN 1.0 OS-Core.CORE-ENG-A-MAN applied
PHKL_42850.CORE2-KRN 1.0 OS-Core.CORE2-KRN applied
PHKL_42850.KERN-ENG-A-MAN 1.0 OS-Core.KERN-ENG-A-MAN applied
# PHKL_42850 1.0 vm cumulative patch
# PHKL_42850.ADMN-ENG-A-MAN 1.0 OS-Core.ADMN-ENG-A-MAN applied
# PHKL_42850.C-INC 1.0 ProgSupport.C-INC applied
# PHKL_42850.C2-INC 1.0 ProgSupport.C2-INC applied
# PHKL_42850.CORE-ENG-A-MAN 1.0 OS-Core.CORE-ENG-A-MAN applied
# PHKL_42850.CORE2-KRN 1.0 OS-Core.CORE2-KRN applied
# PHKL_42850.KERN-ENG-A-MAN 1.0 OS-Core.KERN-ENG-A-MAN applied
# PHKL_42850.PAUX-ENG-A-MAN 1.0 ProgSupport.PAUX-ENG-A-MAN applied
PHKL_42850.C-INC 1.0 ProgSupport.C-INC applied
PHKL_42850.C2-INC 1.0 ProgSupport.C2-INC applied
PHKL_42850.PAUX-ENG-A-MAN 1.0 ProgSupport.PAUX-ENG-A-MAN applied
root #/tmp >sw
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 49152 233 48919 0% 0 - 1 /dev/vg00/lvol2
dev 16384 237 16147 1% 0 - 1 /dev/vg00/swap1
reserve - 545 -545
memory 32553 30180 2373 93%
total 98089 31195 66894 32% - 0 -

root #/tmp >./kmeminfo
tool: kmeminfo 8.07 - libp4 9.333 - HP CONFIDENTIAL
unix: /stand/current/vmunix 11.31 64bit IA64 on host "dratmsvr"
core: /dev/kmem live
link: Sun Sep 02 14:56:42 IST 2012
boot: Mon Sep 3 15:02:20 2012
time: Mon Oct 29 10:55:02 2012
nbpg: 65536 bytes

----------------------------------------------------------------------
Physical memory usage summary (in page/byte/percent):

Physical memory = 522455 31.9g 100%
Free memory = 33712 2.1g 6%
User processes = 443622 27.1g 85% details with -user
System = 37843 2.3g 7%
Kernel = 37840 2.3g 7% kernel text and data
Dynamic Arenas = 25652 1.6g 5% details with -arena
spinlock_arena = 4115 257.2m 1%
FCACHE_ARENA = 4042 252.6m 1%
vx_global_kmcac = 2546 159.1m 0%
misc region are = 1816 113.5m 0%
ktune_tunable_a = 1744 109.0m 0%
Other arenas = 11389 711.8m 2% details with -arena
Super page pool = 1273 79.6m 0% details with -kas
Static Tables = 3572 223.2m 1% details with -static
pfdat = 1594 99.6m 0%
text = 579 36.2m 0% vmunix text section
inode = 476 29.8m 0%
bss = 424 26.5m 0% vmunix bss section
vhpt = 256 16.0m 0%
Other tables = 242 15.1m 0% details with -static
Buffer cache = 3 192.0k 0% details with -bufcache
UFC file mrg = 2028 126.8m 0%

[zxcv]

Hi Duncan,

root #/tmp >prmconfig
Could not find configuration file name. (PRM-846)

PRM CPU scheduler state: Enabled
No Group records in the configuration file. (PRM-803)

PRM application manager state: Not Running

Disk manager state: Disabled

[Duncan Edmonstone]

Physical memory = 522455 31.9g

So you have about 52GB of memory on this system - 10% would be 5.2GB, 5% would be 2.6GB.

Tunable filecache_max
Description Maximum amount of physical memory to be used for caching file I/O data
Module fs_bufcache
Current Value 134217728
Value at Next Boot (auto) [10%]
Value at Last Boot 134217728
Default Value 17067343872 (automatic)
Can Change Immediately (Automatic Tuning Disabled)

Tunable filecache_min
Description Minimum guaranteed physical memory used for caching file I/O data
Module fs_bufcache
Current Value 134217728

So filecache_max and filecache_min are both currently set to 134217728 (134217728 / 1024 / 1024 = 128MB)

If filecache_min and filecache_max were set to 128MB and you tried to change them to 2.6GB and 5.2GB respectively when there was significant memory pressure, I suspect the system was doing the right thing when it told you " Failed to resize up (grow) the Filedata Memory Resource Group".

Dennis,

> >The output of "kctune -v filecache_max filecache_min" would tell us

>  

> Aren't they the "now" values of: 134,217,728

Yes, they are... I'm one of those people who shouts to my wife "have you seen the car keys" when they are right in front of me!

[Dennis Handly]

>with server memory getting thrashed.

Why do you think this?  You have vmstat(1) output? (po/pi values)

>Current Value 134,217,728

Yes, it matches what I said.  And you are increasing filecache_max by 30 times.

>memory 32553 30180 2373 93%

So you are nearly using all of pseudo-swap but probably not actively paging.

>So you have about 52 GB of memory on this system - 10% would be 5.2 GB, 5% would be 2.6 GB.

Huh?  I see 32GB and 522,455 pages of 64 KB each.

[zxcv]

Hi Duncan ,

Even after rebooting the server ,glance is showing 94% memory utilzation.

root #/tmp >kcusage -l

Tunable: filecache_max
Usage: 160432128
Setting: 3413468774
Percentage: 4.7

Tunable: maxdsiz
Usage: 67633152
Setting: 2147483647
Percentage: 3.1

Tunable: maxdsiz_64bit
Usage: 48889856
Setting: 34359738368
Percentage: 0.1

Tunable: maxfiles_lim
Usage: 76
Setting: 63488
Percentage: 0.1

Tunable: maxssiz
Usage: 196608
Setting: 134217728
Percentage: 0.1

Tunable: maxssiz_64bit
Usage: 131072
Setting: 1073741824
Percentage: 0.0

Tunable: maxtsiz
Usage: 8650752
Setting: 1073741824
Percentage: 0.8

Tunable: maxtsiz_64bit
Usage: 2097152
Setting: 8589934592
Percentage: 0.0

Tunable: maxuprc
Usage: 12
Setting: 27000
Percentage: 0.0

Tunable: max_thread_proc
Usage: 101
Setting: 3000
Percentage: 3.4

Tunable: msgmbs
Usage: 0
Setting: 8
Percentage: 0.0

Tunable: msgmni
Usage: 8
Setting: 4096
Percentage: 0.2

Tunable: msgtql
Usage: 0
Setting: 4096
Percentage: 0.0

Tunable: nflocks
Usage: 35
Setting: 4096
Percentage: 0.9

Tunable: ninode
Usage: 1330
Setting: 165888
Percentage: 0.8

Tunable: nkthread
Usage: 655
Setting: 250000
Percentage: 0.3

Tunable: nproc
Usage: 196
Setting: 30000
Percentage: 0.7

Tunable: npty
Usage: 0
Setting: 200
Percentage: 0.0

Tunable: nstrpty
Usage: 1
Setting: 200
Percentage: 0.5

Tunable: nstrtel
Usage: 0
Setting: 60
Percentage: 0.0

Tunable: nswapdev
Usage: 2
Setting: 32
Percentage: 6.2

Tunable: nswapfs
Usage: 0
Setting: 32
Percentage: 0.0

Tunable: semmni
Usage: 22
Setting: 8192
Percentage: 0.3

Tunable: semmns
Usage: 143
Setting: 60000
Percentage: 0.2

Tunable: shmmax
Usage: 199163840
Setting: 4398046511104
Percentage: 0.0

Tunable: shmmni
Usage: 16
Setting: 4096
Percentage: 0.4

Tunable: shmseg
Usage: 2
Setting: 512
Percentage: 0.4

root #/tmp >kctune -v filecache_max filecache_min
Tunable filecache_max
Description Maximum amount of physical memory to be used for caching file I/O data
Module fs_bufcache
Current Value 3413468774 [10%]
Value at Next Boot (auto) [10%]
Value at Last Boot 3413468774
Default Value 17067343872 (automatic)
Can Change Immediately (Automatic Tuning Disabled)

Tunable filecache_min
Description Minimum guaranteed physical memory used for caching file I/O data
Module fs_bufcache
Current Value 1706734387 [5%]
Value at Next Boot (auto) [5%]
Value at Last Boot 1706734387
Default Value 1706688512 (automatic)
Can Change Immediately (Automatic Tuning Disabled)

May i know whats thrashing my memory ??

Not a single database/application is running now .

[zxcv]

Pls check glance snapshot attached herewith.

[zxcv]

Hi guys,

This box had hpvm running on it.

We later decided not to use it and configure it as standalone box.

Please check kernel attached file.