topic Re: filecache_min max error in Operating System - HP-UX

filecache_min max error

zxcv — Sun, 28 Oct 2012 06:19:07 GMT

Hi guys ,

i have 11.31 O.S with server memory getting thrashed .

from forums i found out kernel param filecache_min and max.

i tried to change the values but getting following error;

root #/tmp >kctune filecache_min=5% filecache_max=10%
==> Update the automatic 'backup' configuration first? y
* The automatic 'backup' configuration has been updated.
* Future operations will update the backup without prompting.
NOTE: The requested changes could not be applied to the currently
running system, for the following reasons:
- Failed to resize up (grow) the Filedata Memory Resource
Group.Error code 12.
* The requested changes have been saved, and will take effect at
next boot.
Tunable Value Expression Changes
filecache_max (now) 134217728 134217728 Imm (auto disabled)
(next boot) (auto) 10%
filecache_min (now) 134217728 134217728 Imm (auto disabled)
(next boot) (auto) 5%

Cant it be done online ?

Whats dat Rc code 12 ?

Re: filecache_min max error

Duncan Edmonstone — Sun, 28 Oct 2012 20:22:52 GMT

Are you up to date on patching? I can see mention of an error similar to this in some of the VM cumulative patches - latest is PHKL_42850 - might want to ensure you have that and dependencies?

A Memory Resource Group is a function of PRM, but I suspect that PRM may still do some work kernel-side even when you aren't using it - nevertheless it would be interesting to see the output of "prmconfig" - I think its likely you will just get back "Cannot display configuration. PRM is not configured. (PRM-226)", but you never know...

FInaly - the message about not growing a memory space is perfectly legal in some cirumstances, particularly if you are under memory pressure - are you sure you weren't _increasing_ filecache_min ? Maybe it was laready set lower than 5% ?? The output of "kctune -v filecache_max filecache_min" would tell us (it would be useful to know how much physical memory you have as well - "machinfo" should tell you that

Re: filecache_min max error

Dennis Handly — Mon, 29 Oct 2012 03:29:37 GMT

>I tried to change the values but getting following error;

You may need to change them one at a time.

What does "swapinfo -tam" show?

>The output of "kctune -v filecache_max filecache_min" would tell us

Aren't they the "now" values of: 134,217,728

Re: filecache_min max error

zxcv — Mon, 29 Oct 2012 05:24:41 GMT

Hi Dennis,

root #/tmp >kctune -v filecache_max filecache_min
Tunable filecache_max
Description Maximum amount of physical memory to be used for caching file I/O data
Module fs_bufcache
Current Value 134217728
Value at Next Boot (auto) [10%]
Value at Last Boot 134217728
Default Value 17067343872 (automatic)
Can Change Immediately (Automatic Tuning Disabled)

Tunable filecache_min
Description Minimum guaranteed physical memory used for caching file I/O data
Module fs_bufcache
Current Value 134217728
Value at Next Boot (auto) [5%]
Value at Last Boot 134217728
Default Value 1706688512 (automatic)
Can Change Immediately (Automatic Tuning Disabled)

root #/tmp >
root #/tmp >
root #/tmp >
root #/tmp >swlist -l patch|grep -i 428
PHKL_42850.ADMN-ENG-A-MAN 1.0 OS-Core.ADMN-ENG-A-MAN applied
PHKL_42850.CORE-ENG-A-MAN 1.0 OS-Core.CORE-ENG-A-MAN applied
PHKL_42850.CORE2-KRN 1.0 OS-Core.CORE2-KRN applied
PHKL_42850.KERN-ENG-A-MAN 1.0 OS-Core.KERN-ENG-A-MAN applied
# PHKL_42850 1.0 vm cumulative patch
# PHKL_42850.ADMN-ENG-A-MAN 1.0 OS-Core.ADMN-ENG-A-MAN applied
# PHKL_42850.C-INC 1.0 ProgSupport.C-INC applied
# PHKL_42850.C2-INC 1.0 ProgSupport.C2-INC applied
# PHKL_42850.CORE-ENG-A-MAN 1.0 OS-Core.CORE-ENG-A-MAN applied
# PHKL_42850.CORE2-KRN 1.0 OS-Core.CORE2-KRN applied
# PHKL_42850.KERN-ENG-A-MAN 1.0 OS-Core.KERN-ENG-A-MAN applied
# PHKL_42850.PAUX-ENG-A-MAN 1.0 ProgSupport.PAUX-ENG-A-MAN applied
PHKL_42850.C-INC 1.0 ProgSupport.C-INC applied
PHKL_42850.C2-INC 1.0 ProgSupport.C2-INC applied
PHKL_42850.PAUX-ENG-A-MAN 1.0 ProgSupport.PAUX-ENG-A-MAN applied
root #/tmp >sw
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 49152 233 48919 0% 0 - 1 /dev/vg00/lvol2
dev 16384 237 16147 1% 0 - 1 /dev/vg00/swap1
reserve - 545 -545
memory 32553 30180 2373 93%
total 98089 31195 66894 32% - 0 -

root #/tmp >./kmeminfo
tool: kmeminfo 8.07 - libp4 9.333 - HP CONFIDENTIAL
unix: /stand/current/vmunix 11.31 64bit IA64 on host "dratmsvr"
core: /dev/kmem live
link: Sun Sep 02 14:56:42 IST 2012
boot: Mon Sep 3 15:02:20 2012
time: Mon Oct 29 10:55:02 2012
nbpg: 65536 bytes

----------------------------------------------------------------------
Physical memory usage summary (in page/byte/percent):

Physical memory = 522455 31.9g 100%
Free memory = 33712 2.1g 6%
User processes = 443622 27.1g 85% details with -user
System = 37843 2.3g 7%
Kernel = 37840 2.3g 7% kernel text and data
Dynamic Arenas = 25652 1.6g 5% details with -arena
spinlock_arena = 4115 257.2m 1%
FCACHE_ARENA = 4042 252.6m 1%
vx_global_kmcac = 2546 159.1m 0%
misc region are = 1816 113.5m 0%
ktune_tunable_a = 1744 109.0m 0%
Other arenas = 11389 711.8m 2% details with -arena
Super page pool = 1273 79.6m 0% details with -kas
Static Tables = 3572 223.2m 1% details with -static
pfdat = 1594 99.6m 0%
text = 579 36.2m 0% vmunix text section
inode = 476 29.8m 0%
bss = 424 26.5m 0% vmunix bss section
vhpt = 256 16.0m 0%
Other tables = 242 15.1m 0% details with -static
Buffer cache = 3 192.0k 0% details with -bufcache
UFC file mrg = 2028 126.8m 0%

Re: filecache_min max error

zxcv — Mon, 29 Oct 2012 05:35:15 GMT

Hi Duncan,

root #/tmp >prmconfig
Could not find configuration file name. (PRM-846)

PRM CPU scheduler state: Enabled
No Group records in the configuration file. (PRM-803)

PRM application manager state: Not Running

Disk manager state: Disabled

Re: filecache_min max error

Duncan Edmonstone — Mon, 29 Oct 2012 07:16:15 GMT

Physical memory = 522455 31.9g

So you have about 52GB of memory on this system - 10% would be 5.2GB, 5% would be 2.6GB.

Tunable filecache_max
Description Maximum amount of physical memory to be used for caching file I/O data
Module fs_bufcache
Current Value 134217728
Value at Next Boot (auto) [10%]
Value at Last Boot 134217728
Default Value 17067343872 (automatic)
Can Change Immediately (Automatic Tuning Disabled)

Tunable filecache_min
Description Minimum guaranteed physical memory used for caching file I/O data
Module fs_bufcache
Current Value 134217728

So filecache_max and filecache_min are both currently set to 134217728 (134217728 / 1024 / 1024 = 128MB)

If filecache_min and filecache_max were set to 128MB and you tried to change them to 2.6GB and 5.2GB respectively when there was significant memory pressure, I suspect the system was doing the right thing when it told you " Failed to resize up (grow) the Filedata Memory Resource Group".

Dennis,

> >The output of "kctune -v filecache_max filecache_min" would tell us

> Aren't they the "now" values of: 134,217,728

Yes, they are... I'm one of those people who shouts to my wife "have you seen the car keys" when they are right in front of me!

Re: filecache_min max error

Dennis Handly — Mon, 29 Oct 2012 10:21:38 GMT

>with server memory getting thrashed.

Why do you think this? You have vmstat(1) output? (po/pi values)

>Current Value 134,217,728

Yes, it matches what I said. And you are increasing filecache_max by 30 times.

>memory 32553 30180 2373 93%

So you are nearly using all of pseudo-swap but probably not actively paging.

>So you have about 52 GB of memory on this system - 10% would be 5.2 GB, 5% would be 2.6 GB.

Huh? I see 32GB and 522,455 pages of 64 KB each.

Re: filecache_min max error

zxcv — Mon, 29 Oct 2012 11:08:29 GMT

Hi Duncan ,

Even after rebooting the server ,glance is showing 94% memory utilzation.

root #/tmp >kcusage -l

Tunable: filecache_max
Usage: 160432128
Setting: 3413468774
Percentage: 4.7

Tunable: maxdsiz
Usage: 67633152
Setting: 2147483647
Percentage: 3.1

Tunable: maxdsiz_64bit
Usage: 48889856
Setting: 34359738368
Percentage: 0.1

Tunable: maxfiles_lim
Usage: 76
Setting: 63488
Percentage: 0.1

Tunable: maxssiz
Usage: 196608
Setting: 134217728
Percentage: 0.1

Tunable: maxssiz_64bit
Usage: 131072
Setting: 1073741824
Percentage: 0.0

Tunable: maxtsiz
Usage: 8650752
Setting: 1073741824
Percentage: 0.8

Tunable: maxtsiz_64bit
Usage: 2097152
Setting: 8589934592
Percentage: 0.0

Tunable: maxuprc
Usage: 12
Setting: 27000
Percentage: 0.0

Tunable: max_thread_proc
Usage: 101
Setting: 3000
Percentage: 3.4

Tunable: msgmbs
Usage: 0
Setting: 8
Percentage: 0.0

Tunable: msgmni
Usage: 8
Setting: 4096
Percentage: 0.2

Tunable: msgtql
Usage: 0
Setting: 4096
Percentage: 0.0

Tunable: nflocks
Usage: 35
Setting: 4096
Percentage: 0.9

Tunable: ninode
Usage: 1330
Setting: 165888
Percentage: 0.8

Tunable: nkthread
Usage: 655
Setting: 250000
Percentage: 0.3

Tunable: nproc
Usage: 196
Setting: 30000
Percentage: 0.7

Tunable: npty
Usage: 0
Setting: 200
Percentage: 0.0

Tunable: nstrpty
Usage: 1
Setting: 200
Percentage: 0.5

Tunable: nstrtel
Usage: 0
Setting: 60
Percentage: 0.0

Tunable: nswapdev
Usage: 2
Setting: 32
Percentage: 6.2

Tunable: nswapfs
Usage: 0
Setting: 32
Percentage: 0.0

Tunable: semmni
Usage: 22
Setting: 8192
Percentage: 0.3

Tunable: semmns
Usage: 143
Setting: 60000
Percentage: 0.2

Tunable: shmmax
Usage: 199163840
Setting: 4398046511104
Percentage: 0.0

Tunable: shmmni
Usage: 16
Setting: 4096
Percentage: 0.4

Tunable: shmseg
Usage: 2
Setting: 512
Percentage: 0.4

root #/tmp >kctune -v filecache_max filecache_min
Tunable filecache_max
Description Maximum amount of physical memory to be used for caching file I/O data
Module fs_bufcache
Current Value 3413468774 [10%]
Value at Next Boot (auto) [10%]
Value at Last Boot 3413468774
Default Value 17067343872 (automatic)
Can Change Immediately (Automatic Tuning Disabled)

Tunable filecache_min
Description Minimum guaranteed physical memory used for caching file I/O data
Module fs_bufcache
Current Value 1706734387 [5%]
Value at Next Boot (auto) [5%]
Value at Last Boot 1706734387
Default Value 1706688512 (automatic)
Can Change Immediately (Automatic Tuning Disabled)

May i know whats thrashing my memory ??

Not a single database/application is running now .

Re: filecache_min max error

zxcv — Mon, 29 Oct 2012 11:11:12 GMT

Pls check glance snapshot attached herewith.

Re: filecache_min max error

zxcv — Mon, 29 Oct 2012 11:51:22 GMT

Hi guys,

This box had hpvm running on it.

We later decided not to use it and configure it as standalone box.

Please check kernel attached file.

Re: filecache_min max error

zxcv — Mon, 29 Oct 2012 12:50:04 GMT

Hi Duncan / Dennis ,

I have resolved the problem.

As you can see that hpvm was ocupying all the memory , i just changed the values startup disable and automatic memory allocation to "0" in /etc/rc.config.d/hpvmconf file .

Thanks for your help.

Re: filecache_min max error

Dave Olker — Mon, 29 Oct 2012 13:38:22 GMT

Which version of 11.31 are you running and which version of HPVM? My guess is you're running the March 2012 or later Fusion and HPVM v6.1 or later.

Dave

Re: filecache_min max error

Duncan Edmonstone — Mon, 29 Oct 2012 15:57:37 GMT

Dennis,

> >So you have about 52 GB of memory on this system - 10% would be 5.2 GB, 5% would be 2.6 GB.

> Huh? I see 32GB and 522,455 pages of 64 KB each.

I was reading this from the OP's kmeminfo output:

>> Physical memory = 522455 31.9g 100%

Is that wrong?

Re: filecache_min max error

Dave Olker — Mon, 29 Oct 2012 16:31:31 GMT

@Duncan Edmonstone wrote:
I was reading this from the OP's kmeminfo output:
>> Physical memory = 522455 31.9g 100%
Is that wrong?

Hi Duncan,

The kmeminfo output displays the physical memory in three formats - number of pages, bytes, percent. Looking at his output:

Physical memory usage summary (in page/byte/percent):

Physical memory = 522455 31.9g 100%

There are 522455 memory pages, 32GB of RAM in the system. That being the case, we know that the system is using 64K memory pages (32 * 1024 * 1024) = 33,554,432 bytes / 522,455 pages = 64. This makes sense on an Integrity VM host because we automatically tune the base_pagesize=64 on systems configured as VM Hosts or VSPs.

My guess is this system is running March 2012 release or newer and Integrity VM v6.1 or newer because at that release we automatically grab 92% of the available memory when HPVM starts and assign it to the HPVM/vPar memory pool. This is expected behavior that is documented in the Integrity VM/vPars v6.1 Admin Guide.

Hope this helps,

Dave

Re: filecache_min max error

Duncan Edmonstone — Mon, 29 Oct 2012 16:53:17 GMT

hmmm - it would appear I have lost the ability to read! Not sure how I was reading that as 52GB???

Re: filecache_min max error

Dennis Handly — Mon, 29 Oct 2012 19:33:27 GMT

>I was reading this from the OP's kmeminfo output:

Yes but it also says:

(in page/byte/percent):

nbpg: 65536 bytes

And 32 GB matches the pseudo-swap total.

Re: filecache_min max error

jody_mckinzie — Wed, 07 Nov 2012 15:47:45 GMT

I did it like this:

kctune filecache_min=5%

kctune filecache_max=10%

Re: filecache_min max error

zxcv — Thu, 08 Nov 2012 05:17:29 GMT

Thanks Dave , i always had a doubt as to how the calculation was done.