- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - Linux
- >
- Re: low performance when running 'grep' and 'awk' ...
Operating System - Linux
1752605
Members
4919
Online
108788
Solutions
Forums
Categories
Company
Local Language
юдл
back
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
юдл
back
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Blogs
Information
Community
Resources
Community Language
Language
Forums
Blogs
Go to solution
Topic Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-31-2007 08:54 AM
тАО01-31-2007 08:54 AM
Our application algorithm has to parse 12 times over a large text file (size=148MB).
We ran the same script over the same file on different machines and strangelly the worst performante was produced on our brand new server: an HP Proliant DL380G5 w/ 10krpm SAS disks. I suppose that it must be a question of tunning the kernel. What can i do to enhance the application processing time on this server?
TEST RESULTS (same script, same large file, different servers):
i) Laptop PC w/ Celeron processor runnig GentoOS (BEST PERFORM) - 1'38"
ii) HP Proliant DL380G3 Xeon 3GHz running RHEL3.6 - 6'14"
iii) HP Proliant DL380G5 2xXeon 2-core 3,4GHz running RHEL4.4 - 11'26"
We ran the same script over the same file on different machines and strangelly the worst performante was produced on our brand new server: an HP Proliant DL380G5 w/ 10krpm SAS disks. I suppose that it must be a question of tunning the kernel. What can i do to enhance the application processing time on this server?
TEST RESULTS (same script, same large file, different servers):
i) Laptop PC w/ Celeron processor runnig GentoOS (BEST PERFORM) - 1'38"
ii) HP Proliant DL380G3 Xeon 3GHz running RHEL3.6 - 6'14"
iii) HP Proliant DL380G5 2xXeon 2-core 3,4GHz running RHEL4.4 - 11'26"
Solved! Go to Solution.
3 REPLIES 3
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-31-2007 07:02 PM
тАО01-31-2007 07:02 PM
Solution
This is a long shot by what is environment variable LANG set to?
In the past it's been reported that grep, sort and friends can be slow if LANG is set to one of the UTF variants.
Try
$ LANG=C grep ...
In the past it's been reported that grep, sort and friends can be slow if LANG is set to one of the UTF variants.
Try
$ LANG=C grep ...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-31-2007 10:30 PM
тАО01-31-2007 10:30 PM
Re: low performance when running 'grep' and 'awk' scripting over large text file
Shalom,
Interesting results.
The Os is different.
The processor is different.
The amount ofmemory is likely different.
The Patch level is different.
The storage(disk) performance is likely different.
Any of the factors above could impact the results.
More memory and a larger buffer cache will normally improve performance in such sitations. There are a lot of variables here.
SEP
Interesting results.
The Os is different.
The processor is different.
The amount ofmemory is likely different.
The Patch level is different.
The storage(disk) performance is likely different.
Any of the factors above could impact the results.
More memory and a larger buffer cache will normally improve performance in such sitations. There are a lot of variables here.
SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-31-2007 11:41 PM
тАО01-31-2007 11:41 PM
Re: low performance when running 'grep' and 'awk' scripting over large text file
BINGO! That is impressive: setting var env LANG, the execution time has reduced from 11' to 48". Thank you Ermin Borovac.
The opinions expressed above are the personal opinions of the authors, not of Hewlett Packard Enterprise. By using this site, you accept the Terms of Use and Rules of Participation.
News and Events
Support
© Copyright 2024 Hewlett Packard Enterprise Development LP