aspose file tools*
The moose likes Java in General and the fly likes Build a file in Memory Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of The Java EE 7 Tutorial Volume 1 or Volume 2 this week in the Java EE forum
or jQuery UI in Action in the JavaScript forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "Build a file in Memory" Watch "Build a file in Memory" New topic
Author

Build a file in Memory

Richard Butterwood
Ranch Hand

Joined: Nov 07, 2005
Posts: 45
An XML file is returned to me via a http connection. Currently I write the XML file out to a temporary file on the OS. After the file is completed I return the temporary filename which another uses to manipulate the XML file.

Is it possible to build the file in memory and just return the file object?

If so, can you supply example code?

Thanks!
Scott Selikoff
Saloon Keeper

Joined: Oct 23, 2005
Posts: 3704
    
    5

Sure it is, if you are receiving information using Http request, build the file as a stream then output the file to the user using Http response. In order to transmit just the file you have to change the content type of the stream to direct the browser to download this file.
[ November 17, 2005: Message edited by: Scott Selikoff ]

My Blog: Down Home Country Coding with Scott Selikoff
Richard Butterwood
Ranch Hand

Joined: Nov 07, 2005
Posts: 45
Why do I need to use http respone? At this point I have finished with http and have the XML file in a nice little file object.

What I want is to close the input and output stream and just return the file object.

Can this be done?
Ernest Friedman-Hill
author and iconoclast
Marshal

Joined: Jul 08, 2003
Posts: 24183
    
  34

Yes -- well, sort of. You can't return a java.io.File object without creating an actual file on disk, because that's what a File is. But you can return, for example, a Reader from which the other code can read the XML. If you've got the downloaded XML in a String, then just directly construct a java.io.StringReader from it, and return that. Otherwise, the class ByteArrayInputStream lets you create an InputStream that returns an arbitrary collection of bytes -- use InputStreamReader to turn that into a Reader if you need it.

Does that answer your question?


[Jess in Action][AskingGoodQuestions]
Richard Butterwood
Ranch Hand

Joined: Nov 07, 2005
Posts: 45
Hi Ernest,

Thank you for the reply and yes it does answer my question.

Is there a size limit in a string buffer or string reader? Some of these XML files are going to be very big.
Ernest Friedman-Hill
author and iconoclast
Marshal

Joined: Jul 08, 2003
Posts: 24183
    
  34

No limit other than available memory.
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18541
    
    8

The size limit for a string or an array is 2^31-1 which is about 2 billion. However the practical limit is the amount of memory you can have. It would be preferable to store the XML as a byte array rather than as a String, since unless your XML is mostly Asian characters the bytes will only take up half as much room as the chars.

But you have a process that just reads the XML from the HTTP connection and stores the data, then passes a reference to the data to some other process that parses the data? Why? Why not just let the second process read its data directly from the HTTP connection?
Richard Butterwood
Ranch Hand

Joined: Nov 07, 2005
Posts: 45
Here is the code that I use to build the XML file:

OutputStream outStream = new FileOutputStream( XMLFile );
InputStream inStream = connection.getInputStream();
int c;
while ((c = inStream.read()) != -1)
outStream.write(c);
inStream.close();
outStream.close();

Are you saying that I should return "connection.getInputStream()" to the calling program?

I am using XML Beans to parse the XML. I suspect that XML Beans requires a physical file to use.
Clifford Garvis
Greenhorn

Joined: Nov 17, 2005
Posts: 2
Do you want it in memory to speed up performance? If you want it in memory you could pull it into a MappedByteBuffer
Richard Butterwood
Ranch Hand

Joined: Nov 07, 2005
Posts: 45
Yes, I am going down this road for performace reasons. It seems a waste to dump it out to a file, just to read it back in again.

Do you think performance would be a hit doing it like this?
Clifford Garvis
Greenhorn

Joined: Nov 17, 2005
Posts: 2
It all depends on how much work you are really doing... unless you are reading in very large files and doing alot of I/O and modifications I don't think you will see much difference. Doing it in memory is faster but you probably have alot more space available on your storage device then you do in memory.
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18541
    
    8

Are you saying that I should return "connection.getInputStream()" to the calling program?
Exactly.
I am using XML Beans to parse the XML. I suspect that XML Beans requires a physical file to use.
This is the XML Beans from Apache? Given that it's written by Apache I expected that there would be overloaded methods that allowed you to read XML from a file, a reader, an input stream, and so on. Any competently written XML software should have those, restricting you to input from a file would be poor design. And when I looked at the API documentation for the product I saw that the various parse() methods did indeed have those overloaded methods. You can even parse directly from a URL.
Jeff Albertson
Ranch Hand

Joined: Sep 16, 2005
Posts: 1780
On XMLBeans: yes, the various parse methods of Factories can take a File,
InputStream, Reader, URL, etc... and conversely, the save methods can
take a File, OutputStream, Writer etc...

You should realize that XMLBean is built on top of DOM, so it is not
promising better speed or memory footprint than DOM. Have you
considered something lighter weight, like SAX, XSLT or StAX?


There is no emoticon for what I am feeling!
Richard Butterwood
Ranch Hand

Joined: Nov 07, 2005
Posts: 45
Some of the XML "Responses" will be very small, but some will be very large.

Are you saying that XML Beans will be too much of overhead to use?

If performance becomes an issue I will investigate XLST.
Jeff Albertson
Ranch Hand

Joined: Sep 16, 2005
Posts: 1780
Why not write some test cases and see if the performance is satisfactory?
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Build a file in Memory