aspose file tools*
The moose likes Web Services and the fly likes XML Web Service response -- maximum size? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of EJB 3 in Action this week in the EJB and other Java EE Technologies forum!
JavaRanch » Java Forums » Java » Web Services
Bookmark "XML Web Service response -- maximum size?" Watch "XML Web Service response -- maximum size?" New topic
Author

XML Web Service response -- maximum size?

Steve Buck
Ranch Hand

Joined: Nov 23, 2004
Posts: 45
I'm curious to know what the maximum size of an xml web service response is.

Could I pass back 10MB? 100MB?
James Clinton
Ranch Hand

Joined: Jun 23, 2003
Posts: 190
AFAIK - there is none. However browsers do have a timeout limit, IE6 is 60mins, but this can be changed.
Steve Buck
Ranch Hand

Joined: Nov 23, 2004
Posts: 45
But you don't use a browser to access an xml web service I thought?
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12675
    
    5
I have seen discussion of SOAP messages with megabytes of data attached - I don't think the standards mention a limit. However there is a practical consideration - FTP could be a better protocol for large file transfer. Suppose your SOAP message supplies the caller with complete information to set up a FTP transfer - then the client can set up the transfer as a background process.
Bill


Java Resources at www.wbrogden.com
James Clinton
Ranch Hand

Joined: Jun 23, 2003
Posts: 190
But you don't use a browser to access an xml web service I thought?


of course not, but you may have a web based app which invokes a number of web services behind the scenes.
[ January 25, 2005: Message edited by: James Clinton ]
Steve Buck
Ranch Hand

Joined: Nov 23, 2004
Posts: 45
Originally posted by William Brogden:
I have seen discussion of SOAP messages with megabytes of data attached - I don't think the standards mention a limit. However there is a practical consideration - FTP could be a better protocol for large file transfer. Suppose your SOAP message supplies the caller with complete information to set up a FTP transfer - then the client can set up the transfer as a background process.
Bill


The problem with this method is that I was going to do the business logic within an EJB method. Inside EJB as per the specification you cannot use file I/O (prohibited). If it uses FTP it must then read the file. If it uses an xml web service it can just apply business logic on the data then put in database or whatever.
Steve Buck
Ranch Hand

Joined: Nov 23, 2004
Posts: 45
How many pieces/responses would you suggest I break up 1GB of XML into?

I want to keep the overhead of breaking it up to a minimal, is there any way to suggest how much more it would cost to initiate an HTTPS request between the client and server? Maybe 10 kilobytes?

I was thinking maybe break it up into 10MB chunks. The extra overhead for that would be pretty minimal, right? (I mean as far as the client and server communication goes for initiating, setting up and tearing down HTTPS etc)

Thanks again.
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12675
    
    5
1GB of XML
Burn it to a dvd and mail the dvd?
(ahem) sorry
What an interesting problem - I think you are going to have to dig into some interesting areas to do this efficiently. The following thoughts occur:
1. Resumable transfer - your protocol must be able to cope with broken connections.
2. Multiple threading - a single thread wastes significant time in handshakes - separate threads could work on different chunks that are later assembled.
3. I wonder if JXTA file sharing project could help http://www.jxta.org/
4. I seem to recall there is a Java based music file "sharing" program, I wonder what they used.
5. Your whole create/transfer process may do better outside the J2EE environment.
Bill
Steve Buck
Ranch Hand

Joined: Nov 23, 2004
Posts: 45
Thanks William!

Basically I'm just attempting to duplicate thousands of records at one site to another. I can't just do database-to-database either as I need to apply some business rules for that site (where the Java comes in) and the databases will likely not even be the same product.

Basically anything I do it has to be contained with J2EE :/ Unfortunate because using something outside the container would be easier and faster--but I want it to be completely contained inside.

There won't be any reassembly of the XML responses needed on the other end--it can just apply business rules to those records and insert the valid ones in the database as it completes receiving each XML web service response.

I just need to know if I can really count on doing 10MB chunk transfers or not (can both sides of the equation handle it)

I realize there are other more preferably methods (believe me ) but I just need some sort of "validation" from an expert on whether or not this is a good idea (the overhead of xml and so forth aside)

Thanks!
[ January 31, 2005: Message edited by: Steve Buck ]
JeanLouis Marechaux
Ranch Hand

Joined: Nov 12, 2001
Posts: 906
Steve,

maybe you should also consider compressing the data you send.
It's amazing to see how thin a soap message can be once compressed

and even if it is not the question here, maybe web services is not the good technology to transfert huge volumes :roll:
[ February 02, 2005: Message edited by: Jean-Louis Marechaux ]

/ JeanLouis<br /><i>"software development has been, is, and will remain fundamentally hard" (Grady Booch)</i><br /> <br />Take a look at <a href="http://www.epfwiki.net/wikis/openup/" target="_blank" rel="nofollow">Agile OpenUP</a> in the Eclipse community
Steve Buck
Ranch Hand

Joined: Nov 23, 2004
Posts: 45
Originally posted by Jean-Louis Marechaux:
Steve,

maybe you should also consider compressing the data you send.
It's amazing to see how thin a soap message can be once compressed

and even if it is not the question here, maybe web services is not the good technology to transfert huge volumes :roll:

[ February 02, 2005: Message edited by: Jean-Louis Marechaux ]


It isn't the ideal technology, of course not! The problem is that I'd like to start using XML Web Services so that any other person who wants to take advantage of my system can easily do just that...

The alternative, of course, is to use an external measure for the data exchange that is more efficient and correct, but still provide XML & open functionality on the front end of things (where the client and server communicate... getUsers(), doThis() etc). That would mean that only my systems could communicate amongst one another for data exchange, but others could still use and extract data through the other means on the front end.

Tough choice!

As far as compression methods--yes I have been reading up on these a fair bit. I'm confident that when transfering thousands of records in XML the patterns will be so significant (most elements will be the same) that the compression will be really good. I would prototype and benchmark but I'm so limited at this point (in terms of J2EE web service programming knowledge) that its difficult.

Cheers.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: XML Web Service response -- maximum size?
 
Similar Threads
Response XML Size Test
WS-I Validation Issue
SOAP Array Response In Web Service Using Java.
limitting the size of SOAP response
web service low response