IntelliJ Java IDE
The moose likes Linux / UNIX and the fly likes Remove unwanted text from a big text file Big Moose Saloon
  Search | Java FAQ | Recent Topics
Register / Login
JavaRanch » Java Forums » Engineering » Linux / UNIX
Reply Bookmark "Remove unwanted text from a big text file" Watch "Remove unwanted text from a big text file" New topic
Author

Remove unwanted text from a big text file

Sam Saha
Ranch Hand

Joined: Jan 23, 2010
Posts: 76
I have a log file which contains a big text lines. I wanted to keep the important lines and remove the unwanted lines from the big text file. I am new in UNIX. I would appreciate if someone can help me write UNIX script to remove the unwanted line from the text file.

Actual Text is like this in the log file:



I wanted the output to be like this:

Andrew Monkhouse
author and jackaroo
Marshal Commander

Joined: Mar 28, 2003
Posts: 9982

How did you go with Peter's starting suggestions? No point in us rehashing old ground.

Personally, I think that this is a job that calls for Global Regular Expression Print(ing) - grep. Peter's suggestion is closer to what you originally requested, insofar as you would end up with a program that is relatively easy to read and maintain, even by people who don't do much shell scripting. Whereas using grep will result in a more concise script. If you are interested in going the grep route, you may want to look at the Wikipedia article on grep, especially the examples.


The Sun Certified Java Developer Exam with J2SE 5: paper version from Amazon, PDF from Apress, Online reference: Books 24x7 Personal blog
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 12513

It's worth noting, however, that for the sample message, that the app has had a simple data validation error. For a production app, that should have been intercepted and logged more concisely, since there's no need for a stack trace and there is conversely the possibility that a sufficiently-aware "catch" clause in a Struts action processor might be able to provide more useful context to the message that actually got logged. Assuming that invalid data even needs logging in the first place.


One of the most odious afflictions that Business has inflicted on the modern English language is "pro-active". Most of the time it's simply redundantly used in place of the simple old word "active". And a good deal of the rest of the time it means "You're not overworked enough yet, so go out and find more!"
Duc Quoc
Greenhorn

Joined: Dec 15, 2011
Posts: 2
Sam Saha wrote:I have a log file which contains a big text lines. I wanted to keep the important lines and remove the unwanted lines from the big text file.
... ... ...
I wanted the output to be like this:


From the shell prompt (console), you can use simple grep command like this: (assumed the log file is "server.log" )



It will filter out all the lines with "at " strings, hence results to your expected text.

To write the output text to a new logfile (says, it's "newfile.log"), simply add like following:


We can also achieve the expected result with sed, awk, perl, vim, ... but I think grep is simple enough :-)
 
 
subject: Remove unwanted text from a big text file
 
Threads others viewed
Needs help for grouping data
NoSuchMethodException: does not contain handler
Need help for Shell Script to reduce a file
BIRT Exception messge when click to view report
Transport endpoint is not connected
MyEclipse, The Clear Choice