the trailboss abuses his CodeRanch power for his other stuff (power corrupts. absolute power corrupts absolutely is kinda neat!)
permaculture light bulbs permaculture electric heat permaculture cast iron permaculture wood burning stove permaculture solar food dehydrators
The moose likes Meaningless Drivel and the fly likes cat|cut|cut|sort|uniq|sort > file Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Other » Meaningless Drivel
Bookmark "cat|cut|cut|sort|uniq|sort > file" Watch "cat|cut|cut|sort|uniq|sort > file" New topic
Author

cat|cut|cut|sort|uniq|sort > file

David O'Meara
Rancher

Joined: Mar 06, 2001
Posts: 13459

So it turns out running this on 24Gb takes a while.

Who knew?
David O'Meara
Rancher

Joined: Mar 06, 2001
Posts: 13459

actually I've been running the following on the combined log files for the month from 100 client sites. Personally I think the problem is too much logging.

cat *.log* |cut -d' ' -f7|cut -d'?' -f1|sort |uniq -c|sort -n>file.count
David O'Meara
Rancher

Joined: Mar 06, 2001
Posts: 13459

Ho hum.

I was hoping to have a look at the results, but I might go to bed.
Doug Slattery
Ranch Hand

Joined: Sep 15, 2007
Posts: 294
Maybe this will save you a second or two . (Warning, I didn't test it, but I think you get the idea...)

sed -e 's/^\([^ ]* \)\{6\}\([^?]*\)?/\2/' < *.log* > reallybig.log; sort -n reallybig.log | uniq -c > file.count

Aloha,
Doug

-- Nothing is impossible if I'mPossible
Pat Farrell
Rancher

Joined: Aug 11, 2007
Posts: 4650
    
    5

When N is big, even N ln(N) is big
Doug Slattery
Ranch Hand

Joined: Sep 15, 2007
Posts: 294
yes, but at least now, it's not 2N

Aloha,
Doug

-- Nothing is impossible if I'mPossible
David O'Meara
Rancher

Joined: Mar 06, 2001
Posts: 13459

keeping in mind we're in MD and not Linux, I'll just say "One day I need to look at sed"
 
It is sorta covered in the JavaRanch Style Guide.
 
subject: cat|cut|cut|sort|uniq|sort > file