swap recomendation for RHEL6 on DL585 G7 with 380GB RAM

[Christian Masopust]

I've to install RHEL6 on a DL585G7 with 380GB RAM. Now the question arises how much swap should be configured.

In the past I installed a DL585G7 having 128GB RAM with 2 140GB SATA disks used for swap. What we saw then was that the system stopped responding when swap usage was about 80GB !! (system usage was that appr. 10 of the 48 cores were used by 100%).

Any idea why my 128GB-system freezes when swap usage is above 80GB?

And... more important: which recomendation for my 380GB-system?

Thanks a lot,
christian

[Randy Jones_3]

The amount of swap space you need for a system like that with huge RAM is very dependent on your application profile. The recommendations I have seen generally come out in the range of "same size as RAM" down to "half the size of RAM" but those are only very general guidelines and you may be able to get away with far less.

An example: I have a 64 GB database server with 20 GB of swap disk. Our DBAs manage the application carefully to keep everything active in RAM because they have experience with poor performance when they push RAM allocation to the point where swap space is commonly used. On that system now, we rarely see more than 2-4 GB of swap space used.

If you had swap usage of 80 GB on a 128 GB system then I think you were heavily oversubscribed and will see better results with your 380 GB system. Are you sure your system actually froze, or did it just slow to the point of uselessness because applications were blocked while swap I/O was being done?

On a system with huge RAM you should make sure you understand the Linux parameter "vm.swappiness". Run sar and watch your si and so values, and very slowly adjust swappiness. The results can be surprising, but it depends on your application.

[iinfi1]

i believe recommendations like swap being twice the size of RAM or 1.5 times the size of RAM are 5-6 years old.
they dont apply for the new generation servers which come with over 80GB of RAM.
In my opinion when you go above 14 GB of RAM, you swap need not be more than 4-5GB.
you will have to size the load on the server accordingly.

i have question. how do we see how much swap is being utilized. vmstat report? the so column?