- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Re: select random lines from a very large file
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-10-2011 10:48 AM
тАО02-10-2011 10:48 AM
Code snippet:
LOG=/tmp/mylog
total=`wc -l "$LOG" | awk {'print $1'}`
# echo "total: $total"
x=0
while [ $x -lt 50 ]
do
# Get a random number:
rand=$(((($RANDOM*$total)/32767)+1))
# echo "Random line number is $rand"
x=`echo "$x + 1" | bc`
done
Solved! Go to Solution.
- Tags:
- random numbers
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-10-2011 12:08 PM
тАО02-10-2011 12:08 PM
Re: select random lines from a very large file
You could do:
# rand=$(perl -le '$x=1;$y=2_000_000;$n=$x+int rand($y-$x+1);print $n')
...which will return a random integer between 'x' and 'y' or in this case between 1 and 2,000,000 inclusive.
Regards!
...JRF...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-10-2011 12:59 PM
тАО02-10-2011 12:59 PM
Re: select random lines from a very large file
Your perl command works good on a command line, but when I paste it into my script, I get this:
Undefined subroutine &main::RAND called at
I've tried a few variations (including full path to perl), but the error persist.
- Mike P
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-10-2011 01:11 PM
тАО02-10-2011 01:11 PM
Re: select random lines from a very large file
> Undefined subroutine &main::RAND called at
...means you didn't copy-and-paste because 'rand' isn't uppercase in Perl :-)
Regards!
...JRF...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-10-2011 02:16 PM
тАО02-10-2011 02:16 PM
Re: select random lines from a very large file
Right on. After placing the perl line, I had done a global replacement on rand to RAND (because I like upper-case variables). Of course, it change the perl rand to RAND.
Thanks, I've got this running in my script and randomly selecting lines in a 2 million line file.
- Mike
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-10-2011 02:23 PM - last edited on тАО09-20-2011 10:19 AM by Kevin_Paul
тАО02-10-2011 02:23 PM - last edited on тАО09-20-2011 10:19 AM by Kevin_Paul
SolutionHi (again) Mike:
> Thanks
If you are happy with the answers you received, please don't forget to assign points:
http://h30499.www3.hp.com/t5/help/faqpage/faq-category-id/kudos#kudos
Regards!
...JRF...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-10-2011 03:41 PM
тАО02-10-2011 03:41 PM
Re: select random lines from a very large file
I generally wait a bit for multiple answers before assigning points.
But, I'm quite happy to have my script running (it's doing a cksum on randomly selected identical files in two locations). So, you've got 10 points from me.
Thanks again,
Mike
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-12-2011 07:40 PM
тАО02-12-2011 07:40 PM
Re: select random lines from a very large file
How often do you do this? Do you care about performance?
Perhaps you should select files based on the next full line after a random byte offset.
>x=`echo "$x + 1" | bc`
You can use: (( x += 1 ))
Or if you use: typeset -i x
x=$x+1