How to allocate the memory from OS instead of increasing the JVM’s heap size?

[Linda Wu]

I need to detect whether the file I am attaching to an email is exceeding the server limit. I am not allowed to increase the JVM heap size to do this since it is going to affect the application performance.

If I don’t increase the JVM heap size, I will run into OutOfMemoryError directly.

I would like to know how do allocate the memory from OS instead of increasing the JVM’s heap size?

I can read the file size before uploading the file to the server. However, the server team does not have an API in place to tell me what is the server policy in terms of what is the max file limit. I have to stream the file to the server and have it determine whether the file has exceeded the limit. This is not efficient. Since I can't increase the JVM heap size, how could I allocate the operating system memory to do this?

Thanks a lot.

[Paul Clapham]

The OS is the only source of memory for the application. So whether you say initially that you're going to need X megabytes or you allocate memory up to X megabytes later on, there's no difference. You're going to be using X megabytes at that point.

So if using more than X megabytes will affect the application performance, but you need to use more than X megabytes at some point, then you have a choice to make. Affect the application performance or run out of memory.

However if you're only uploading a file to a server, you should really consider not storing the entire file in memory. That would make your problem vanish.

[Linda Wu]

Hello, Paul:

Thanks for your quick response on this.

Here is the code. Could you let me know how I could NOT storing all the file in the memory in order to avoid this issue?

Content getContent(String fileName) {
        Content myContent = new Content();
        try {
            File file = new File(fileName);
            String fname;
            	fname = file.getName();
            
            myContent.setFileName(fname);
            FileInputStream fis = new FileInputStream(file);
            myContent.setInputStream(fis);
       
        } catch (FileNotFoundException e) {
			.............
		}	


        return myContent;
        
    }

After this, the MimeMultipart will write the content to the outputstream. The information will be posted to the server.


Thanks again!

[NK: Added code tags. Please UseCodeTags while posting code]

[Satya Maheshwari]

Could you post more code to understand what you are trying to do?
Also please use code tags for posting code.

[Rok Štelcer]

Hi,

According to your description of a problem, I'm not sure that there even is a solution. ;)

Well, to summarize what you wrote:

server guys are to lazy to expose the API for the max. file size limit, which I guess is set via some variable

you can't increase the heap of JVM, thus resulting in OutOfMemory

if your application survives the load of a file, it will then uploaded the file to the server, who will in worst case say:"Good job, however the file size is too large."

Did I get it right? ;)

Well in general, you have two possibilities:

compressing data with e.g. zip (add and compress)
This would be the best solution, if you're operating with a flat files.
In case this is a classic client/server web application, this can be achieved with Filters.

send data in chunks
Read e.g. 64k of data and then flush it. Of course, in such case the server would have to have the knowledge to handle such requests.

Hope this helps.

One more thing.
Why again would increase of JVM heap size slower down the application?


Regards,
Rok