File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Beginning Java and the fly likes Java Memory Question Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Beginning Java
Bookmark "Java Memory Question" Watch "Java Memory Question" New topic
Author

Java Memory Question

Ralf Obermayer
Greenhorn

Joined: May 15, 2008
Posts: 3
Hi there,

i am wondering why java tries to reserve
this large amount of memory in my program
- of course resulting in an out of memory
heap size error.
Domain : There are 500000 users who each rated
some items and i need to load those 100mio
ratings into java.
Now there is a class containing byte rating;
and short itemID;. Shouldn't this result in a
heap size requirement of approx. 100mio*3bytes = 300MB?
Even -Xmx1600m does not seem to be enough.
Can anybody help me please?

Thanks in advance!
scimer
Henry Wong
author
Sheriff

Joined: Sep 28, 2004
Posts: 18978
    
  40

Now there is a class containing byte rating;
and short itemID;. Shouldn't this result in a
heap size requirement of approx. 100mio*3bytes = 300MB?
Even -Xmx1600m does not seem to be enough.


How do you know that your objects only take 3 bytes? Which BTW, without even seeing the code, I find that hard to believe.

Henry


Books: Java Threads, 3rd Edition, Jini in a Nutshell, and Java Gems (contributor)
Ralf Obermayer
Greenhorn

Joined: May 15, 2008
Posts: 3
Hi Henry,

the code is simple :

This is the class containing
the rating a user gave for
the specified itemID.


And here is the relevant part of my main class.
It's just a test to create those 200*NUM_USERS = 100mio
ratings which will be filled in later.



Thanks,
scimer

P.S. : Added my name.
Henry Wong
author
Sheriff

Joined: Sep 28, 2004
Posts: 18978
    
  40

the code is simple :

This is the class containing
the rating a user gave for
the specified itemID.


What about the parts of the objects used for bookkeeping? Like how does Java know what type of object it is? What about the objects that it inherits from? The Object class does have stuff like serialization ids, monitors, etc.

And here is the relevant part of my main class.
It's just a test to create those 200*NUM_USERS = 100mio
ratings which will be filled in later.


You do know that arrays are objects too, right? You do know that arrays of arrays are objects too, right?


Regardless, this is all moot. The size of an object is an implementation detail. You are not supposed to care about the actual size an object takes -- which may be different between JVMs.

Henry
Ralf Obermayer
Greenhorn

Joined: May 15, 2008
Posts: 3
Yes, i know that memory for some bookkeeping is needed
but i couldn't believe that it's a factor even larger than 5!

Apparently i'll have to use two arrays with primitive datatypes
(one array for the ratings and one for the corresponding itemIDs)
though i'm not comfortable about not being able to unite them into
one datastructure.

Thanks,
Ralf
Henry Wong
author
Sheriff

Joined: Sep 28, 2004
Posts: 18978
    
  40

Yes, i know that memory for some bookkeeping is needed
but i couldn't believe that it's a factor even larger than 5!


Don't think of it as a "multiplication factor". Think of it as a fixed cost. Just how much do you think it should take?

How about the reference to the Class object (4 bytes for a 32 bit JVM. 8 bytes for 64 bit)? How about the identity hashmap (4 bytes)? The monitor used to synchronization, wait, and notify (I have no idea)? Just because you don't use a feature doesn't mean its size is zero.

Henry
arulk pillai
Author
Ranch Hand

Joined: May 31, 2007
Posts: 3252
Try calculating as shown below



500+ Java Interview Questions and Answers | Java job hunting know how & Java resumes
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Java Memory Question