aspose file tools*
The moose likes Other Open Source Projects and the fly likes Need to search large log files present in a remote unix server Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Products » Other Open Source Projects
Bookmark "Need to search large log files present in a remote unix server " Watch "Need to search large log files present in a remote unix server " New topic
Author

Need to search large log files present in a remote unix server

Sivapraveen Chandana
Greenhorn

Joined: Jan 01, 2008
Posts: 16
Hi all,
I need to search large log files present in a remote unix server. I know that I can use SFTP APIs like Jcraft's jsch api to connect to the remote server but then copying those log files to my local machine would not be a good idea since each of them is at least 500 MB in size.
I need to search through 5 or 6 such log files based on the search string input by the user and then copy relevant search data and display it to the user.
Since, the log files are too big in size, I am thinking of the possibility of running a shell script in the remote server that can do the task of searching and displaying relevant results, through my java program.
As of now, this process is done manually through the Putty client and I need to automate it.
Is there a better way of achieving what I need to do?

Please advise.
Thanks in advance..


praveen
Martijn Verburg
author
Bartender

Joined: Jun 24, 2003
Posts: 3274
    
    5

Hmm, interesting problem. Have yo considered looking at open source search engines that can search through text. For example Lucene?

Jsch IIRC also gives you ways of interacting through an SSH session, so in theory you could open up a session and execute a script.

Hope that helps a little!


Cheers, Martijn - Blog,
Twitter, PCGen, Ikasan, My The Well-Grounded Java Developer book!,
My start-up.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Need to search large log files present in a remote unix server