This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes Tomcat and the fly likes Adding images available to the container without writing them to the file system Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Products » Tomcat
Bookmark "Adding images available to the container without writing them to the file system" Watch "Adding images available to the container without writing them to the file system" New topic
Author

Adding images available to the container without writing them to the file system

Ronald Schild
Ranch Hand

Joined: Jun 09, 2008
Posts: 117
Hello all,

I've written a fun fileuploader and it works like a charm. Files uploaded are images of products that various pages must contain. When such an image is uploaded I store it in a database and I write it to a directory in my webapp where Tomcat can find it. What I want is to not write it to the file system, but keep it in memory and make it available that way. How do I tell Tomcat that there is a new image available in memory, so that it hits when requested? (add it to its cache? or can I offer it to Apache somehow, it's the webserver's job to respond to such resource requests right?)

Any reference to the right docs is appreciated, thanks in advance!


Java hobbyist.
Kathleen Angeles
Ranch Hand

Joined: Aug 06, 2012
Posts: 122

What happens to the images in memory if you restart your server? Are you going to reload all images from the db during the startup of the server?

It sounds like reinventing the cache system.

You may opt to use a cache implementation out there (e.g. ehcache).
Jayesh A Lalwani
Bartender

Joined: Jan 17, 2008
Posts: 2274
    
  28

I would strongly suggest that you don't cache images in memory if you want your application to be scalable.

You can implement a servlet that can retrieve the images from the database and serve them up to the browser. You don't have to keep all the images in memory. It is possible to do this

However, practically, serving them up from the file system is going to be much more efficient. Why do you not want to keep the images in the file system?
Kathleen Angeles
Ranch Hand

Joined: Aug 06, 2012
Posts: 122

If fast delivery of images is essential, CDNs (Content Delivery Network) can help non-intrusively speed up downloading of your images worldwide.
Ronald Schild
Ranch Hand

Joined: Jun 09, 2008
Posts: 117
Kathleen Angeles wrote:What happens to the images in memory if you restart your server? Are you going to reload all images from the db during the startup of the server?

It sounds like reinventing the cache system.

You may opt to use a cache implementation out there (e.g. ehcache).


All image data is persistent in a relational db (MySQL in this case). I access it with Spring's jdbc DAO and eventually want my cache of objects in ehcache as you describe.

Whenever a page is called that requires product data, this is loaded into memory from the DB in the form of beans. Having the image data in memory, I now write it to file and use the filename in my responses. But the container (or webserver?) must load the data from the file and store it in memory somewhere. I want to skip that file writing part and go from memory to memory.

Thinking about ehcache (unexperienced) I would figure I could create the cache with the data, but then I'd still wonder how to make Tomcat/Apache look in that cache for the images without having a filesystem as a necessary in-between (I want to be ready in case my webapp is not allowed writing to any filesystem and may only use memory and the persistent source).

Thanks for replies so far :-)

Edited:
It's not that I don't want to use a disk, I just want to gracefully degrade when none is available.
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18541
    
    8

I don't see why you should be implementing a cache for these images when you could perfectly well make use of the caching feature of the browsers who access your application.

You can override the "long getLastModified(HttpServletRequest req)" method of the servlet which returns the image to return the time that the image was uploaded; this will cause browsers to cache the image.
Ronald Schild
Ranch Hand

Joined: Jun 09, 2008
Posts: 117
Paul Clapham wrote:I don't see why you should be implementing a cache for these images when you could perfectly well make use of the caching feature of the browsers who access your application.

You can override the "long getLastModified(HttpServletRequest req)" method of the servlet which returns the image to return the time that the image was uploaded; this will cause browsers to cache the image.


It's all a server side cache issue to prevent DB lookups. I don't want to touch the browser cache (yet). I'm not trying to make a form of an image cache to try and make quicker image retrieval across the net. I want a cache that stores product information after it has been retrieved from the DB (this works).

Let's say I've gotten the right bean from the cache that represents a product. I use these values in my view components - but I don't know how to write an img ref to a response, with the filename as identifier, and then make the container access the image data I have available by offering it from memory, skipping writing the file onto the filesystem (for in case my app does not have the rights/something goes wrong..). So I'm wondering if there is a more transparant way to tell the container 'there is a new image resource' that it can use when resolving such an img ref.
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18541
    
    8

I see, I didn't read your original post carefully enough. So you didn't write a servlet to handle requests for the uploaded images. Did you consider doing that and reject that on the basis that you could somehow persuade the container to handle those requests? You might find that writing the image-serving servlet makes many of your questions go away, or at least allows you to reformulate them into something which is easier to answer.
Kathleen Angeles
Ranch Hand

Joined: Aug 06, 2012
Posts: 122

Ronald,

Now I see better what you are asking.

The image-serving servlet, a very common thing that almost everyone who gets images from the database uses, as mentioned by Paul and Jayesh is what you are looking at. You dont need to write to a file at all.

Depending on if you wish to do it using pure servlet, or through struts, or through Spring MVC, etc. they work the same way, with the expected differences in the coding. E.g. you will do it through Spring MVC, you can google like 'spring mvc controller return image jpg'.

Just a quick overview on what this image-serving servlet strategy is all about.

Instead of a url like 'http://a.com/pic.jpg' in your img html tag, you will use a url that points to this image-serving servlet like 'http://a.com/giveMeMyPic.do'

If the image differs depending on some key, for example employee id, the img html tag will have url like 'http://a.com/giveMeEmployeePic.do?employeeId=12321' .

------------

You may not be interested, but I want to mention that a caching of reads on the db will be very helpful. Ehcache can be very easily integrated into your Spring system (1 jar+a few annotations+a few lines of xml). Images are usually big in size. Caching can mean, reading a 5kb image from MYSQL once only even if it is needed 100000 times. Ehcache has a lot of settings you can take advantage of.
Ronald Schild
Ranch Hand

Joined: Jun 09, 2008
Posts: 117
Paul Clapham wrote:I see, I didn't read your original post carefully enough. So you didn't write a servlet to handle requests for the uploaded images. Did you consider doing that and reject that on the basis that you could somehow persuade the container to handle those requests? You might find that writing the image-serving servlet makes many of your questions go away, or at least allows you to reformulate them into something which is easier to answer.


I think I see what you mean. A custom servlet for images, that purely deals with image requests? As in it would write the byte array to the response directly, dealing with any image requests that result from view components. This servlet would be mapped to any available image type (*.jpg etc.) in the web.xml.


Thank you, that will work. Is this also the way to go when using JSF? I'm thinking many web applications would need an image servlet.

Ronald Schild
Ranch Hand

Joined: Jun 09, 2008
Posts: 117
Kathleen,

That already answers the question, thanks :-) I will definitely use ehcache and now also know what to look for in the Spring environment.
Kathleen Angeles
Ranch Hand

Joined: Aug 06, 2012
Posts: 122

Last note if you dont mind.

Jayesh has brought up a very good point. When you scale up in number or size of images, reading from the DB, even with the help of cache, can take a toll on your memory and cpu.

If you have a powerful system (cpu, etc.), then this would not be an issue. But as you gain more customers (which we want) visiting and using your website, you need to think of these stuff. Otherwise, you will need more expensive host servers which will lower your profit.

My website has around 500 images. I resorted to making duplicate .jpg files of those in the database, and serve the .jpg files, instead of reading from the database everytime. This is to save cpu and memory.
Jayesh A Lalwani
Bartender

Joined: Jan 17, 2008
Posts: 2274
    
  28

Right, the database makes a very poor file storage system. No matter how you try to scale up your architecture, you will find the database will keep getting in the way. Most people who serve up images through Servlets usually serve up dynamically generated images.

If you are serving static images, your best bet is always going to be to store them on the file system, and serve them up right through Apache. The way I would do it is put the files in the file system, and put the relative path to the file in the database. You can then construct the HREF to the image using the file name in the DB
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 15960
    
  19

Jayesh A Lalwani wrote:
If you are serving static images, your best bet is always going to be to store them on the file system, and serve them up right through Apache.


Not necessarily. The original reason for doing that was that Apache could serve static content much more efficiently than Tomcat could. That was about 4 Tomcat generations ago, however and it's no longer true.

There are still times when Apache might be a good choice, but not for performance sake alone. And by splitting services between 2 different servers/webapps, you will be complicating the overall app design.


Customer surveys are for companies who didn't pay proper attention to begin with.
 
It is sorta covered in the JavaRanch Style Guide.
 
subject: Adding images available to the container without writing them to the file system
 
Similar Threads
Loading problem with t:inputFileUpload
Beginner at Image manipulation...
junk characters in netui:form with enctype="multipart/form-data"
Performance issue with large data(such as images)
Interrupting big file uploads to save traffic