aspose file tools*
The moose likes Sockets and Internet Protocols and the fly likes Byte Array as HTTP Payload Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Sockets and Internet Protocols
Bookmark "Byte Array as HTTP Payload" Watch "Byte Array as HTTP Payload" New topic
Author

Byte Array as HTTP Payload

Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9350
    
    2

It might be a silly question, but I would like to know if there is a possibility to just send array of bytes as HTTP payload for a POST request? On the server, I would then convert these bytes to a String representation. Any thoughts?


SCJP 1.4, SCWCD 1.4 - Hints for you, Certified Scrum Master
Did a rm -R / to find out that I lost my entire Linux installation!
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18541
    
    8

Perhaps I'm misunderstanding your question, but the payload for a POST request is just a series of bytes, wrapped in a bunch of HTTP-specific protocol. So in one way you can't send anything except an array of bytes. Perhaps you could explain what other options you were trying to eliminate, or something?
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16014
    
  20

Paul is correct. It's just a matter of getting everyone talking the same language.

The Internet was originally created as a connected set of (mostly mainframe) computers from different vendors with different OS's and different hardware. Some spoke ASCII, some spoke EBCDIC. Some were bytewise-continuous and some were bytewise-discontinuous in one of several different forms (if you prefer, you can use the more popular terms "big-endian" and "little-endian").

To make it all work, the original protocols - http, smtp, telnet and so forth were all text-based, and as data went from system to system, character-set translation was applied so that the meaning didn't get lost.

In the case of stuff that absolutely, positively had to make the trip without being mangled by character-set code page translations, a special encoding called MIME (Multipurpose Internet Mail Extensions). Using MIME, you can transmit pure binary data over a text-only link because all of the bytes are base-64 encoded into characters.

So, let's take a classic case of sending an binary POST payload: uploading a JPEG image file. The form that is being posted is configured to respond with a multi-part MIME encoded POST payload. The actual binary image data has been converted to base-64 characters and wrapped in special delimiting strings that help the server automatically extract and decode the image. In addition to the image itself, the posting typically also includes some text name/value pairs such as the uploaded filename, plus any other control values that were contained within the form being submitted.

On the reverse trip, a binary payload such as an image isn't embedded with a form, so the MIME type is typically sent in an HTTP Content-Header element. The client, however, can choose to determine data types via other means if it wants to. On almost every system you'll see a table or tables that determines what to do with downloaded data based on its URL extension, 'magic numbers' in its content or however the client wishes.


Customer surveys are for companies who didn't pay proper attention to begin with.
Jayesh A Lalwani
Bartender

Joined: Jan 17, 2008
Posts: 2330
    
  28

Not only is this possible, my first book on Servlets had a whole chapter devoted to it. It was called Servlet tunneling. The idea was that you could send a serialized object through HTTP. Actually, my first "web" project way back in the year 2000 (atleast it was around 2000) was building a photo upload applet

This technique was superceded with RMI over HTTP, and then RMI fell out of favor, because it didn't interoperate with other languages, and then SOAP came up, and then SOAP fell out because well.. it sucked, and REST came in.. *pant* *pant*

So, yeah, you can do it, but the reason we don;t do it is because we have something better. You can still use it, though. There are some places where it makes a lot of sense. If you want to compact your payload a lot, it makes sense to use some sort of binary representation.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16014
    
  20

The difference between HTTP Tunnelling and and something like a form-based file upload is, by the way, really just a matter of convention. The form upload is a protocol that's built into the client, whereas tunnelling is the more general concept. I used to do tunnelling from an applet, but of course a stand-alone Java application can also serve as a tunnelling client too.

Compression is often provided as part of HTTP services, but your mileage may vary, including whether the available compression schemes are optimal for your payload. Same for encryption.
Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9350
    
    2

Paul Clapham wrote:Perhaps I'm misunderstanding your question, but the payload for a POST request is just a series of bytes, wrapped in a bunch of HTTP-specific protocol. So in one way you can't send anything except an array of bytes. Perhaps you could explain what other options you were trying to eliminate, or something?


What I'm doing is that I'm having a String in JSON format. I'm converting this String to a Byte Array by zipping it using the java.util.zip and using this zipped Byte Array as the Payload to a Post HTTP call. Here is how the client call looks like (It is Scala code):



The HTTP Call


When I tried to call the server, I get the following exception:



Any ideas on what might have gone wrong?

Jayesh A Lalwani
Bartender

Joined: Jan 17, 2008
Posts: 2330
    
  28

Is the request getting to the server? Is it unzipping it? It could be that there is a bug on the server.
Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9350
    
    2

Here is the source code to the entire client and server stuff that I was trying. I'm not using Servlets anymore. So I don't think that it is relevant to the Servlets forum anymore. Kindly move it as it sees fit to any of the other forums.

http://stackoverflow.com/questions/21885825/play-framework-zipped-byte-stream-handling
Jayesh A Lalwani
Bartender

Joined: Jan 17, 2008
Posts: 2330
    
  28

I don;t think you should do baos.toString, although I might be wrong. You should just send the bytes over your HTTP request. DO you have a link to the documentation of Http.postData? I'm pretty sure it's overloaded to take a byte array
Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9350
    
    2

Jayesh A Lalwani wrote:I don;t think you should do baos.toString, although I might be wrong. You should just send the bytes over your HTTP request. DO you have a link to the documentation of Http.postData? I'm pretty sure it's overloaded to take a byte array


Yes, it does take byte array and I tried that as well. But however, when I tried that what I noticed is that, on the server, I print the request headers. In that one of them is the content-length and I see the content length to be of 4187752 instead of 37254 which is the size after GZip compression.



Why should the content-length on the server be equal to the size of the uncompressed data? This is really puzzling me!
Bear Bibeault
Author and ninkuma
Marshal

Joined: Jan 10, 2002
Posts: 60997
    
  65

Why are you compressing it in the first place rather than taking advantage of the builtin gzip compression?


[Asking smart questions] [Bear's FrontMan] [About Bear] [Books by Bear]
Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9350
    
    2

Bear Bibeault wrote:Why are you compressing it in the first place rather than taking advantage of the builtin gzip compression?


Built in gzip compression of what? You mean the built in gzip compression and decompression mechanism offered by Play framework that I use? Of-course I would use that on the server side to decompress the requests and compress the response back. I have to do the reverse operations on the client. So I still would need to compress, decompress on the client.
Bear Bibeault
Author and ninkuma
Marshal

Joined: Jan 10, 2002
Posts: 60997
    
  65

You may be trying to re-invent the wheel:

https://developers.google.com/speed/articles/gzip

http://www.playframework.com/documentation/2.2.x/api/scala/index.html#play.filters.gzip.GzipFilter

What's your client? Does it not support gzip'd requests?
Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9350
    
    2

Bear Bibeault wrote:You may be trying to re-invent the wheel:

https://developers.google.com/speed/articles/gzip

http://www.playframework.com/documentation/2.2.x/api/scala/index.html#play.filters.gzip.GzipFilter

What's your client? Does it not support gzip'd requests?


My client can be anything from an excel file with a plugin to make http calls to a javascript.

I did look into the Play framework's documentation. The Play version that I use is 2.1.1 which unfortunately does not support Gzip's yet. I have managed to get around with that. As mentioned in my post above, I fail to understand why the compressed byte array will not be sent to the server?
Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9350
    
    2

What should be the Content-Type header value in my scenario? I have set it to application/json. Is that correct even though I'm sending a byte array as payload?
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Byte Array as HTTP Payload