ls and high CPU SYS load

[Pedro Cirne]

Hi,

I've a server with a CPU bottleneck on SYS CPU, I've been monitoring this and arrive to the conclusion that the cause for the high SYS CPU load is an huge amount of "ls" (more than 1000/min) that some user scripts execute on a filesystem with more than 100.000 files.

Is there a way to tune this from the OS side? How can I improve performance?

Thks

Pedro

[Patrick Wallek]

The way to tune this is to better control what those scripts are doing.

1000 'ls' command per minute on a filesystem with 100,000+ files is going to hamper performance.

The first step would be to look at what exactly those scripts are doing and see if there is a way to rewrite the scripts to be more efficient and much much much more frugal in the use of the 'ls' command.

[A. Clay Stephenson]

You might see some small improvements by increasing the buffer cache or by increasing vx_ncsize but bad code is bad code and the real way to fix this is to change the way the scripts work. 1000 ls/min seems a bit excessive -- the overhead of forking that many processes but the other processes that these scripts fork probably accounts for a large portion of your load.

If you are willing to accept a bit of staleness, you might be able to do something like this. Save the results of an ls to a cache file and let the scripts read from this file; a timestamp could be used to determine if the file is too old and if so refresh the cached version.

[David Child_1]

Pedro,

I believe you are going to have a hard time tuning this out. Might I ask if the files in this file system are created/deleted often? If so you may look at performing regular defragmentation of the file system (I assume vxfs).

fsadm -d -D -e -E /filesystem

Of course you should also look at limiting those 'ls' commands.

David

This could speed up performance in this case

[Bill Hassell]

You really must change those scripts--they are killing your filesystem performance. I can't imagine any useful purpose in doing an ls on a 10k filesystem. Perhaps the script is looking for a file by using grep (ouch!). hrow out the grep and simply put the string matches in an echo, for example:

Old way: ls /bad_directory | grep *my*file*

Better way: echo /bad_directory/*my*file*

This allows the shell to do the searching. But in all cases, the rapid ls in a way-too-large set of files is crushing your system for no good reason. If you post relevent portions of some of these scripts, we can probably fix them up quite handily.

[Pedro Cirne]

Hi,

Thank you all.

Enjoy :)

Pedro