topic Re: recommended size cache_mem in Operating System - Linux

recommended size cache_mem

Maaz — Fri, 07 Sep 2007 07:04:14 GMT

what would be the recommended size of cache_mem in squid.conf on a dedicated Gateway Server(squid2.6, and iptables), having 1 Gig DDR2 RAM ?

On a 20 users network, What would be the recommended size of hard drive cache size(cache_dir ufs /path/to/cache_dir SIZE 16 256) in squid.cong on a dedicated Gateway Server(squid2.6, and iptables), having 1 Gig DDR2 RAM ?

Regards
Maaz

Re: recommended size cache_mem

Steven E. Protter — Fri, 07 Sep 2007 07:28:41 GMT

Shalom,

If the server id dedicated for one purpose, squid, you should max out the memory setting.

Here is how I'd calculate.

Start the server. Shut off squid.

free

Leave a few hundred meg, say 200 free for user processes and give the rest to squid. Don't push the system to the point where it pages processes in and out of swap.

SEP

Re: recommended size cache_mem

Ivan Ferreira — Fri, 07 Sep 2007 10:57:43 GMT

Based on http://wiki.squid-cache.org/SquidFaq/SquidMemory:

>>> As a rule of thumb on Squid uses approximately 10 MB of RAM per GB of the total of all cache_dirs (more on 64 bit servers such as Alpha), plus your cache_mem setting and about an additional 10-20MB. It is recommended to have at least twice this amount of physical RAM available on your Squid server.

So, if you have 1 GB RAM, squid should be configured to use about 500 MB (OS normally uses very small amount of memmory).

The cache_dir depends of your available space, and for each GB, 10 MB or RAM is used.

So, the formula should be:

((cache_dir/1024) * 10 MB ) + cache_mem MB + 20 MB <= 500 MB

Re: recommended size cache_mem

Maaz — Wed, 12 Sep 2007 00:58:04 GMT

Thanks Dear SEP and Ivan Ferreira for help.

Ok, cache_mem is 300 now, and cache_dir is 2000(2 GB).

another issue, now when I run free -m I found memory usage 679 MB, and this machine is a dedicated Gateway machine and on the average with only 10 users simultaneously connected to the internet, but when I run top,

1, i dont find squid in the list.. why ?

95% of processes thats in the list(top output) consumes only the 0.0 %MEM, and only 2 or 3 process consumes 1.x or 3.x %MEM.

2, then why "free -m" shows MEM used as 679 MB, as per the output of top it should be very less ?

3, whats the proper way of checking which process is consuming/using how much MEM ?

Regards
Maaz

Re: recommended size cache_mem

Maaz — Wed, 12 Sep 2007 01:02:02 GMT

4, is there any command to check that how much mem is using by squid ? other than the formula(or manual calculaion) that Ivan provide ?

Regards
Maaz

Re: recommended size cache_mem

Stuart Browne — Wed, 12 Sep 2007 03:13:11 GMT

Ok, for 1, 3 & 4, 'ps -elf' is your friend. In particular:

ps -elf | grep squid

The 10th column is your memory usage in kilobytes.

In top, you can sort by memory usage by using 'M' (not m). This should also show you 'squid' near the top.

As for question 2, the memory 'used' in top is the same as the memory 'used' in free. These values are grabbed from '/proc/meminfo'. The tools just format it differently.

Re: recommended size cache_mem

Maaz — Wed, 12 Sep 2007 05:41:24 GMT

thanks Dear Stuart Browne for help

I ran the following command

# ps -elf |grep squid
1 S root 2363 1 0 76 0 - 2987 wait 08:21 ? 00:00:00 /etc/init.d/squid start
4 S nobody 3259 2363 0 76 0 - 5594 - 08:21 ? 00:00:05 (squid) start

as per the output ( 2987 + 5594 = 8581 ) and then ( 8581 / 1000 = 8 MB ) its clear that squid is using only 8 MB isn't ?

then which process/software is using 679 MB of Memory on this dedicated Gateway server ?

top also doesnt show me any memory hungry process, so who is using 679-715 MB memory

Re: recommended size cache_mem

Ivan Ferreira — Wed, 12 Sep 2007 05:59:33 GMT

I think that the best way to check the memory information is using cachemgr.cgi as described in the link previously posted.

Re: recommended size cache_mem

Maaz — Wed, 12 Sep 2007 06:13:43 GMT

and why top doesnt show me the squid in its output ?

ok, one more thing bind is also running on this machine as a cache-only server, and bind is also not visible in top output .. why ?

Regards
Maaz

Re: recommended size cache_mem

Ivan Ferreira — Wed, 12 Sep 2007 10:58:19 GMT

How are you sorting the output of top? Can your share the output? Try using top -b

Re: recommended size cache_mem

Maaz — Thu, 13 Sep 2007 00:51:36 GMT

Thanks Dear Ivan for help
Yes when I ran "top -b" it shows the squid and named both, once again thanks for sugestions.

But again I'll ask that "top -b" or "ps -elf |grep squid" shows MEM used by squid is too low/little, ok, but "free -m" shows me the mem used almost 700 MB.
So which process is responsible for this much(700 MB) mem usage ?

attach are the "top -b" and "ps -elf" output

Regards
Maaz

Re: recommended size cache_mem

Stuart Browne — Thu, 13 Sep 2007 03:26:57 GMT

First off, there's just over 400MB used by 'cached' and 'buffers'. This is the kernel using memory to buffer disk IO to make things quicker for users.

2nd largest portion of it is used by X & Gnome, coming in at about 300 or so MB.

As for Squid, it will slowly grow as it gets used. If it isn't used, there's nothing to keep in memory, so it doesn't bother wasting it.

Re: recommended size cache_mem

Maaz — Sat, 22 Sep 2007 01:55:58 GMT

Dear Stuart Thanks for your reply, and sorry for acknowledging too late.

> there's just over 400MB used by 'cached' and 'buffers'....
if you can please list those proceses(someone not all) that demonstrate the buffer and cached .. I mean those enteries in the output of "ps -elf" and top, that demonstrate the kernel buffer and cache.

Regards
Maaz