aspose file tools*
The moose likes Web Services and the fly likes Downloading large files Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Java » Web Services
Bookmark "Downloading large files" Watch "Downloading large files" New topic
Author

Downloading large files

Saurabh Pillai
Ranch Hand

Joined: Sep 12, 2008
Posts: 509
We are saving large files (typically around 50-100 MB) in our database. I want to know what kind of web service framework we should be using? Web service client (consumer) would be in C# .NET.
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 42278
    
  64
None. WS are not made to handle large chunks of data. There are, on the other hand, a number of options that are better suited to that task,like FTP, SFTP, or even straight HTTP. You can still use a WS for communicating the details needed to access the file, though.

Saving large files in the DB is also something that you should think hard about - it might be better to store just the metadata in the DB, and keep the files in a file system.


Ping & DNS - my free Android networking tools app
Abhay Agarwal
Ranch Hand

Joined: Feb 29, 2008
Posts: 1162
    
    1

Hi Saurabh

Agreed with Ulf.
Follwoing Ulf advicve can help in easier management of these large files later on

thanks


Oracle Java Web Service Developer (1z0-897), Oracle certified Java 7 Programmer, SCJA 1.0, SCJP 5.0, SCWCD 5.0, Oracle SQL Fundamentals I
Saurabh Pillai
Ranch Hand

Joined: Sep 12, 2008
Posts: 509
Ulf Dittmer wrote:None. WS are not made to handle large chunks of data. There are, on the other hand, a number of options that are better suited to that task,like FTP, SFTP, or even straight HTTP. You can still use a WS for communicating the details needed to access the file, though.


So after conversation with our network admin, we decided to use HTTPS. Now I have few more issues. We are going to have Tomcat clustering (i.e. multiple tomcat instance). Now typical URL to access the file would be https://www.mycompany.com/appname/.../video/<filename>. Obviously, video folder would be under webapps folder under tomcat. Now consider this scenario,

- First Request comes to server on tomcat instance 1 to download xyz.mov file. Server pulls it from database and saves it under webapps/.../video/ folder. And send OK response to caller.

- Caller sends another request (https://www.mycompany.com/appname/.../video/xyz.mov) to download the file which goes to say tomcat instance 2 because of load balancing. and it won't find the xyz.mov file there.

How do I resolve this issue? Please let me know if I am not enough clear.
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 42278
    
  64
You could use a shared file system where files reside in one place, but are accessed from several web apps. It would be better not have those directories underneath Tomcat's web app directory hierarchy anyway. If the Tomcats are fronted by an Apache you could have the Apache serve those files out of one of its directories - it would be better suited for that task.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Downloading large files