- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - Linux
- >
- Re: Performance - please suggest
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО03-09-2008 06:14 PM
тАО03-09-2008 06:14 PM
Performance - please suggest
The user is executing below find command on a NFS/NAS share. This command is executed on a RHEL4 U3 64bit Virtual machine(1CPU, 1GB RAM).
find -name out.\*.gz -mtime -5 -exec zcat {} \; | grep -c "date_posted"
The time required to complete this is 28 minutes.
Please suggest/recommend what can be done to get this done quickly.
Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО03-10-2008 06:19 AM
тАО03-10-2008 06:19 AM
Re: Performance - please suggest
There probably is no way to speed this up.
Your speed is effected most by NFS read/seek time.
The inquiry itself is complex, if you could simplify it and make it less CPU intensive that would help.
What are you trying to accomplish? check the contents of some zcat'd files for certain content? That is hard on the disk and the CPU.
SEP
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО03-10-2008 09:04 AM
тАО03-10-2008 09:04 AM
Re: Performance - please suggest
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО03-10-2008 11:32 AM
тАО03-10-2008 11:32 AM
Re: Performance - please suggest
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО03-10-2008 02:19 PM
тАО03-10-2008 02:19 PM
Re: Performance - please suggest
Also, you could split your command in more than one, if you have enought cpu available and your network/disk bandwidth allows it.
For example:
RES1=`find /path1 -name out.\*.gz -mtime -5 -exec zcat {} \; | grep -c "date_posted"`
RES2=`find /path2 -name out.\*.gz -mtime -5 -exec zcat {} \; | grep -c "date_posted"`
RES3=`find /path3 -name out.\*.gz -mtime -5 -exec zcat {} \; | grep -c "date_posted"`
expr $RES1 + $RES2 + $RES3
Using this approach, ensure that your find command does not overlaps parent/child directories.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО03-11-2008 04:41 AM
тАО03-11-2008 04:41 AM
Re: Performance - please suggest
Thanks for all your inputs.
We want to run the find / grep commands on the NFS/NAS mounts.
During this operation, I want to measure the network traffic. I want to measure what is the download speed.
I guess there will be a lot of data being downloaded from the NFS/NAS share on to the temporary local filesystems for processing (correct me if I am wrong).
I guess the time that this operation takes will depend on the network speed, local memory and cpu capabilities (correct me if I am wrong).
Please suggest,
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО03-11-2008 05:48 AM
тАО03-11-2008 05:48 AM
Re: Performance - please suggest
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО03-11-2008 08:08 PM
тАО03-11-2008 08:08 PM
Re: Performance - please suggest
find -name out.\*.gz -mtime -5 -exec zcat {} \; > /dev/null
Why should a simple grep be slower than an NFS transfer? First, zcat may expand the files a lot after the NFS transfer. Second, grep in RHEL4U3 has a really bad performance problem with locales that have utf8 character encoding.
See the comment about "Back-ported egf-speedup patch (bug #179636)." in
http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/release-notes/as-x86/RELEASE-NOTES-U4-x86-en.html
and the defect report at
https://bugzilla.redhat.com/show_bug.cgi?id=69900
You can work around the grep performance problem by setting "export LANG=C" before running grep.