File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Sending big file using web service

 
Giang Truong
Greenhorn
Posts: 4
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have to send a big base64 file to a vendor. I use the wsdl2java tool to convert vendor's WSDL to Java stub classes and call a web service method to upload the file. The file content is inserted into the SOAP message as a member of a complex type.

When the file is big, 10MB or so, I run out heap space.
Normally, using MTOM will solve this kind of problem, however, MTOM requires that both client and server implement MTOM. In my case, the vendor will not implement MTOM.

Is there any alternatives that I can send the file? I am thinking about streaming the SOAP message, so instead of loading the whole file into the message, I will write one chunk at a time till I complete the file. IS THIS POSSIBLE at all?

Thanks,
 
Raja Pal
Ranch Hand
Posts: 92
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Just being a little pragmatic, did you evaluate that this situation actually needs a web-service - I mean the overhead of SOAP and schema typing a complex type, etc. Have you explored a simple change of options with the method invocation/messages still being sent the SOAP way(wsdl) but the file(and I have a hunch this is just some serialized data stream) can be sent using good old FTP, which is most cases can be supported by the vendor end of the service channel? If this is ruled out, then you may consider breaking the SOAP packet size to less granular chunks.
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13045
6
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
SOAP encoding large objects is a really bad idea - thats why SAAJ SOAP API for Attachements in Java was invented.

The overhead of SOAP encoding and decoding can be avoided by using SAAJ.

As Raja Pal said - maybe you should step back and consider a simpler solution.

Bill
 
Giang Truong
Greenhorn
Posts: 4
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Raja, William:

Thanks for your response.
As Raja suggested, there are other ways to achieve the goal like ftp, https, but web service does have some extra benefits.
I try to push Web Service as much as I can but it looks like I have reached the limit.

William:
Can you elaborate a bit more about the SAAJ? Do you have any code skeleton that I can follow?

Thanks,


 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13045
6
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Can you elaborate a bit more about the SAAJ? Do you have any code skeleton that I can follow?


No, the link I cited should get you started. Use Google to find SAAJ examples for whichever SOAP toolkit you are using.

Bill
 
Ivan Krizsan
Ranch Hand
Posts: 2198
1
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi!
Section 5.7 of my electronic book (freely downloadable here http://www.slideshare.net/krizsan/scdjws-5-study-notes-3085287) shows a complete example on using SAAJ.
Section 10.3 contains a complete example (client and service) showing how to use MTOM.
If I recall correctly, it automatically handles placing the data in an attachment. I may be wrong, though.
Disclaimer: All examples uses the Metro JAX-WS reference implementation.

I recall having read an article about being able to stream very large attachments with Metro and SOAP web services, but I cannot seem to find it at the moment.
Best wishes!
 
Giang Truong
Greenhorn
Posts: 4
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ivan's doc is incredible. Lots of details and explanations.

Here is what I have found. Correct me if I make any mistake.
I changed my code to use SAAJ, and the setContent() helps a lot. Normally, in order to create the SOAP message, I had to maintain 2 copies of a string: one is the Base64 string representing the file, and the other is the Base64 string of the file inside the SOAP message.

If I send a 20Mb word doc, it is equivalent to 26Mb Base64 string. In order to keep one copy of the 26Mb Base64 string, the heap space grows to around 120Mb.
2 copies will take up 240Mb heap space easily. The setContent() maintains only one copy of the string in form of byte[] and thus reducing the heap space to only 120Mb.

However, 120Mb heap space is till a lot.

The word streaming in SAAJ gives me the impression that I don't have to keep a full copy of the SOAP message in memory and that I can keep the SOAP message in a FileInputStream (or something similar) and send the content of a stream one chunk at a time. So far, it is not the case.

Thanks


 
Giang Truong
Greenhorn
Posts: 4
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I just found this piece of code

http://www.java-tips.org/other-api-tips/httpclient/how-to-send-an-xml-document-to-a-remote-web-server-using-http-7.html

In the strURL is the URL of the web service

This block of code will solve the heap memory issue



And, the Post Method getResponseBodyAsString() contains the web service response.

This is so simple.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic