topic Re: calculate decimal value for a tunable in Operating System - Linux

calculate digital value for a tunable

totoperdu — Fri, 30 Mar 2007 04:01:22 GMT

Hello,

i search a way for calculate values of tunable parameters for integration in a script, my problem is from formula.

i've tried sysdef but there's not all values (maxusers, nstrtel, etc...)

i've tried to calculate the values with awk but there is no recusivity.

ie. If NPROC is a degital value recovered from kmtune i know how to calculate :
((16*NPROC)+2048)
but not if NPROC is another formula, recursivity problem.

do you have any scripts or commands which is able to return the digital values of tunables?

for hexa values i use:
echo "0x04000000=D" |adb

Regards,
Cedrick Gaillard

Re: calculate decimal value for a tunable

Dennis Handly — Fri, 07 Oct 2011 11:05:31 GMT

>i've tried to calculate the values with awk but there is no recusivity.

awk allows recursion for functions. Just convert every variable into a function. Then it is evaluated just in time.

Or convert them to C macros and compile with the bundled C compiler. Just print the macro values out.
Macro processing allows nesting.

Or give up on 11.11 go to 11.23 where kctune gives everything in terms of numbers.

Re: calculate digital value for a tunable

Ralph Grothe — Fri, 30 Mar 2007 07:37:54 GMT

I was also tired as you of kmtune's display of formulars.
But more than that I needed them to be evaluated automatically when having automated kernel generation from a conf file of new to establish kernel tunables.
I have done this by a wee Perl script.
Because it partly is not very portable (due to lack of time) and thus was never meant for publication I can only post here some more general snippets just to give you an idea
(I'm sure you will do better).

(I urge you, if you want to make use of it to use the strict and warnings pragma, which I omit here for brevity)
First this is a sub that simply parses the output of kmtune and pushes it into a LoL.

sub get_tunables {
my (@dump, %parms);
chomp(@dump = qx(/usr/sbin/kmtune));
splice @dump, 0, 2;
map {
$parms{$_->[0]} = {current => $_->[1] =~ /^\d+$/ ? $_->[1] : undef,
dynamic => $_->[2] eq 'Y' ? 1 : 0,
pending => $_->[3]}
} map [split], @dump;
my @formula_pending = grep $parms{$_}{pending} =~ m@[()/*+-]@, keys %parms;
return %parms;
}

would be used like so

my %parms = get_tunables();

Because the LoL admitedly isn't quite handy,
but more than that because on my servers only the pending values displayed by kmtune exhibit the formula noise, one could strip this column into a separate hash that I name %pending here.

%pending = map {$_=>$parms{$_}{pending}} keys %parms;

Then I defined another sub that actually does a recursive evaluation of formular tainted tunables which it expects to be passed as first arg as a string scalar.
The second arg is hash of tunables,
i.e. in the example above %pending;

sub eval_formula {
my ($expr, %parms) = @_;
my $eval_this;
my $parm = '';
foreach my $char ($expr =~ /./g) {
if ($char =~ m@[\d\s()*/+-]@) {
if (exists $parms{"\L$parm\E"}) {
$eval_this .= $parms{"\L$parm\E"};
$parm = '';
}
$eval_this .= $char;
}
else {
$parm .= $char;
}
}
if ($eval_this =~ m{^[\d\s()*/+-]+$}) {
$parm = eval $eval_this;
}
else {
$parm = eval_formula($eval_this, %parms);
}
return int($parm + 0.5);
}

To reduce invocation of above eval_formula() function to only those tunables tainted one could grep those into a separate hash,
that I call %eval_these here.

%eval_these = map { $_ => $pending{$_} } grep $pending{$_}=~m|[()/*+-]|, keys %pending;

To show you its contents here is a dump from a debugger session of it.

DB<12> |x \%eval_these
0 HASH(0x800000010050bef0)
'bufpages' => '(NBUF*2)'
'dnlc_hash_locks' => '4*128'
'effective_maxpid' => '((NPROC<=30000)?30000:(NPROC*5/4))'
'ksi_alloc_max' => '(NPROC*8)'
'msgmap' => '(2+MSGTQL)'
'ncallout' => '(16+NKTHREAD)'
'nclist' => '(100+16*MAXUSERS)'
'ncsize' => '(NINODE+VX_NCSIZE)+(8*DNLC_HASH_LOCKS)'
'netisr_priority' => '-1'
'nfile' => '(16*(NPROC+16+MAXUSERS)/10+32+2*(NPTY+NSTRPTY+NSTRTEL))'
'ninode' => '((NPROC+16+MAXUSERS)+32+(2*NPTY))'
'nkthread' => '(((NPROC*7)/4)+16)'
'nproc' => '(20+8*MAXUSERS)'
'nsysmap' => '((NPROC)>800?2*(NPROC):800)'
'nsysmap64' => '((NPROC)>800?2*(NPROC):800)'
'semmap' => '(SEMMNI+2)'
'timeslice' => '(100/10)'
'vol_dcm_replay_size' => '(256*1024)'
'vol_max_nmpool_sz' => '(4*1024*1024)'
'vol_max_rdback_sz' => '(4*1024*1024)'
'vol_max_vol' => '(8*1024*1024)'
'vol_min_lowmem_sz' => '(512*1024)'
'vol_nm_hb_timeout' => '(10)'
'vol_vvr_transport' => '(1)'
'vol_vvr_use_nat' => '(0)'
'voliomem_chunk_size' => '(64*1024)'
'voliomem_maxpool_sz' => '(4*1024*1024)'
DB<13>

Ugly, isn't it?

Now I have a problem which my code doesn't handle yet.
Take a look at the value of nsysmap.
This contains a shorthand if else block as it seems.
Luckily Perl has almost an identical so called ternary operator, viz. " expr ? val1 : val2 ".

'nsysmap' => '((NPROC)>800?2*(NPROC):800)'

So you can easily fix this.
I for now simply wipe these inerfering tunable from my hash of tainted params
(this is rather sloppy and should be done in a more maintainable manner).

delete @eval_these{grep($eval_these{$_}=~/\?/,keys %eval_these)};

Also note, that you may have to convert hex numbers.
If these were given as literals Perl would convert them automatically.
But since they are read in from a pipe to kmtune one needs to use the Perl functions oct() or hex().

map $pending{$_} = hex($pending{$_}),
grep $pending{$_} =~ /^0[xX]/, keys %pending;

Finally, one can evaluate all in %eval_these in a loop.

foreach my $eval_this (keys %eval_these) {
$evaled{$eval_this} =
eval_formula($eval_these{$eval_this}, %pending);
}

Here the contents of %evaled after that loop in a debugger session.

DB<11> !10
x \%evaled
0 HASH(0x800000010058c488)
'bufpages' => 0
'dnlc_hash_locks' => 512
'ksi_alloc_max' => 2208
'msgmap' => 42
'ncallout' => 515
'nclist' => 612
'ncsize' => 5596
'netisr_priority' => 0
'nfile' => 910
'ninode' => 476
'nkthread' => 499
'nproc' => 276
'semmap' => 66
'timeslice' => 10
'vol_dcm_replay_size' => 262144
'vol_max_nmpool_sz' => 4194304
'vol_max_rdback_sz' => 4194304
'vol_max_vol' => 8388608
'vol_min_lowmem_sz' => 524288
'vol_nm_hb_timeout' => 10
'vol_vvr_transport' => 1
'vol_vvr_use_nat' => 0
'voliomem_chunk_size' => 65536
'voliomem_maxpool_sz' => 4194304
DB<12>

Finally, if you wish you can rejoin the hashes like
(note, not pretty efficient but the hashes are rather small)

%numeric_pending = (%pending, %evaled);

My sample snippets are far from production proof but I hope that they suffice as an example how this could be dealt with in Perl.

HTH

Ralph

Re: calculate digital value for a tunable

totoperdu — Mon, 02 Apr 2007 04:49:50 GMT

Thanks for your inputs.

Denis, i've very few knoledge in awk functions but according to the man page, it seems not so complicated, i'll try it.

Ralph, your solution is great because almost ready to use, i like it ;)

for your value:
'nsysmap' => '((NPROC)>800?2*(NPROC):800)'
awk can count it very well:
#> awk -v NPROC=8212 'BEGIN{print "NSYSMAP:"((NPROC)>800?2*(NPROC):800)}'
NSYSMAP:16424

i've wrote a script that can calculate the kmtune values but the formula behind, this is:
---<----------
# i start by unset all future var.
kmtune |egrep -v "^Parameter|^=====|^$|\?" | \
while read param val
do
unset $param
done

NOTGOOD=false

while [ -n "$NOTGOOD" ]
do
NOTGOOD=
kmtune |egrep -v "^Parameter|^=====|^$|\?" | \
awk '{print tolower($1)"="tolower($2)}' | \
while IFS='=' read -r param val
do
# if value start by 0X, it's hexa, i need to convert:
if [ "$(echo $val |egrep -ci "^0x")" = "1" ]
then
val="$(echo "${val}=D" |adb)"
fi

#if value is alpha, it's a formula i nees to calculate
if [ "$(echo $val |egrep -c "[[:alpha:]_]")" = "1" ]
then
NOTGOOD=
for PARAM in $(echo $val |sed 's/[^[:alpha:]_]/ /g')
do
# if i've not the var value, i restart the loop
if [ -z "$(eval echo \$$PARAM)" ]
then
NOTGOOD=true
break
fi
done
if [ -z "$NOTGOOD" ]
then
eval "$param=\$(($val))"
fi
else
eval $param="\$val"
fi
done
done

# easy to test
echo $nfile
echo $msgmap
---<----------

i now need to change the shell calcul by awk for calculate formula like ((NPROC)>800?2*(NPROC):800) and all will be fine.

thanks all for your help/ideas.

Cheers,
Cedrick Gaillard