• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Liutauras Vilda
  • Paul Clapham
Sheriffs:
  • paul wheaton
  • Tim Cooke
  • Henry Wong
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Piet Souris
Bartenders:
  • Mike London

Need to search large log files present in a remote unix server

 
Greenhorn
Posts: 16
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi all,
I need to search large log files present in a remote unix server. I know that I can use SFTP APIs like Jcraft's jsch api to connect to the remote server but then copying those log files to my local machine would not be a good idea since each of them is at least 500 MB in size.
I need to search through 5 or 6 such log files based on the search string input by the user and then copy relevant search data and display it to the user.
Since, the log files are too big in size, I am thinking of the possibility of running a shell script in the remote server that can do the task of searching and displaying relevant results, through my java program.
As of now, this process is done manually through the Putty client and I need to automate it.
Is there a better way of achieving what I need to do?

Please advise.
Thanks in advance..
 
author
Posts: 3285
13
Mac OS X Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hmm, interesting problem. Have yo considered looking at open source search engines that can search through text. For example Lucene?

Jsch IIRC also gives you ways of interacting through an SSH session, so in theory you could open up a session and execute a script.

Hope that helps a little!
 
Don't get me started about those stupid light bulbs.
reply
    Bookmark Topic Watch Topic
  • New Topic