• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Task to parse a remote log file

 
Ramaswamy Srinivasan
Ranch Hand
Posts: 295
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hey all,

I have a task to perform, as outlined below

1. Read a log file, on a remote machine.
2. Parse it and pick up few values (I have the code for this step).
3. Store the values in a List and use for further processing.

Now, my questions are

1. Is there any mechanism to read a remote file?
2. Do I essentially have to download it to my local machine?
3. We cannot run to code on the server, due to security constraints.

Kindly suggest, whether java is the best suited language to implement this kind of a solution.

Many thanks for your support.
 
K. Tsang
Bartender
Pie
Posts: 3228
12
Android Java
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
For your 1st question how are you connecting to remote server RMI, sockets, others?

2nd question no, if you can access it meaning able to read it and storing it in a list in memory then once you done reading your list object will have the data from the remote file. Then you can use the list to produce a local copy if needed and do manipulation.

3rd question kind of answered. if you can connect to remote server then you are set. BTW RMI can use security manager to do a whole lots of stuff and depending on what security constraints you are referring to, it's hard to say.
 
Rob Spoor
Sheriff
Pie
Posts: 20396
47
Chrome Eclipse IDE Java Windows
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
1) You can use FTP, Windows file sharing (a.k.a. SMB), or if it's read-only perhaps even HTTP. You can also use RMI but that would require some more work, but it's not impossible (my own FileTransferRemote and FileTransferRemoteObject have proven that ).

2) in the end yes, you will need to download it. Perhaps not as you mean with transferring the file first, then reading, but even if you are reading it directly from the remote location you are still downloading its contents when you read it.
 
Ramaswamy Srinivasan
Ranch Hand
Posts: 295
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks a lot Tsang and Rob for your replies. Your inputs are highly helpful

@ Rob,

Could you kindly elaborate a bit about the SMB you are talking about?

1. What could be the best point of start for it? Where could I download it from?
2. Also, one more point about this SMB -- does it allow you to read a remote file line by line? Or, download it?

Thanks!

 
Rob Spoor
Sheriff
Pie
Posts: 20396
47
Chrome Eclipse IDE Java Windows
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You don't need to download SMB for Windows; it's already built in. It's the official name for Windows file sharing. For Linux and Unix you should check out your distribution's package system for "samba".

With Java you can use File with FileReader with files located on shares; for instance, new File("\\\\machine\\share\\myfile.txt"). Note the 4 backslashes at the start; that's because the two backslashes from the protocol are escaped for Java.
Now this will only work if you have access to that share without an additional need for a username and password. Otherwise you should check out JCIFS.

As for accessing Windows shares from Linux / Unix, I haven't tried that ever before so I couldn't tell you how that works.


If you cannot get the Windows shares working you can also use FTP. With a third party library like Apache Commons Net or JvFTP you can access files on an FTP server, and get InputStreams and OutputStreams to those.
 
Randi Randwa
Greenhorn
Posts: 7
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator


It is a good idea never to install anything on the server for security reasons. But, you can access server log files (for reading).

To process a log file, you can use something similar to http://www.biterscripting.com/SS_WebLogParser.html . You can run this script directly on a remote file as follows. Enter the following command in biterscripting.




or



Or, if you have mapped the shared folder with a drive name X:



You can even access the log file over the internet




Of course, you can also use it on a local file

You can even access the log file over the internet




 
Ramaswamy Srinivasan
Ranch Hand
Posts: 295
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Randi Randwa wrote:

It is a good idea never to install anything on the server for security reasons. But, you can access server log files (for reading).

To process a log file, you can use something similar to http://www.biterscripting.com/SS_WebLogParser.html . You can run this script directly on a remote file as follows. Enter the following command in biterscripting.




or





Thanks Randi, Looks like we've got something interesting here. However, is there any constraints for this to work? In terms of ports being open, or a VPN access or something like that? Why I ask this question is because, the remote computer is on a different Domain than mine!!

Thanks a lot
 
Randi Randwa
Greenhorn
Posts: 7
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

No special setup should be needed.

You will need read access to the remote file being read. Try first, just to read the file with the simple cat command as follows - try the various addressing formats I specified in the previous post.




or



If the command succeeds, you have read access. Then you can use any scripts with that file.

 
I agree. Here's the link: http://aspose.com/file-tools
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic