File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
Win a copy of Clojure in Action this week in the Clojure forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Java Memory Question

 
Ralf Obermayer
Greenhorn
Posts: 3
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi there,

i am wondering why java tries to reserve
this large amount of memory in my program
- of course resulting in an out of memory
heap size error.
Domain : There are 500000 users who each rated
some items and i need to load those 100mio
ratings into java.
Now there is a class containing byte rating;
and short itemID;. Shouldn't this result in a
heap size requirement of approx. 100mio*3bytes = 300MB?
Even -Xmx1600m does not seem to be enough.
Can anybody help me please?

Thanks in advance!
scimer
 
Henry Wong
author
Marshal
Pie
Posts: 20835
75
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Now there is a class containing byte rating;
and short itemID;. Shouldn't this result in a
heap size requirement of approx. 100mio*3bytes = 300MB?
Even -Xmx1600m does not seem to be enough.


How do you know that your objects only take 3 bytes? Which BTW, without even seeing the code, I find that hard to believe.

Henry
 
Ralf Obermayer
Greenhorn
Posts: 3
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Henry,

the code is simple :

This is the class containing
the rating a user gave for
the specified itemID.


And here is the relevant part of my main class.
It's just a test to create those 200*NUM_USERS = 100mio
ratings which will be filled in later.



Thanks,
scimer

P.S. : Added my name.
 
Henry Wong
author
Marshal
Pie
Posts: 20835
75
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
the code is simple :

This is the class containing
the rating a user gave for
the specified itemID.


What about the parts of the objects used for bookkeeping? Like how does Java know what type of object it is? What about the objects that it inherits from? The Object class does have stuff like serialization ids, monitors, etc.

And here is the relevant part of my main class.
It's just a test to create those 200*NUM_USERS = 100mio
ratings which will be filled in later.


You do know that arrays are objects too, right? You do know that arrays of arrays are objects too, right?


Regardless, this is all moot. The size of an object is an implementation detail. You are not supposed to care about the actual size an object takes -- which may be different between JVMs.

Henry
 
Ralf Obermayer
Greenhorn
Posts: 3
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yes, i know that memory for some bookkeeping is needed
but i couldn't believe that it's a factor even larger than 5!

Apparently i'll have to use two arrays with primitive datatypes
(one array for the ratings and one for the corresponding itemIDs)
though i'm not comfortable about not being able to unite them into
one datastructure.

Thanks,
Ralf
 
Henry Wong
author
Marshal
Pie
Posts: 20835
75
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yes, i know that memory for some bookkeeping is needed
but i couldn't believe that it's a factor even larger than 5!


Don't think of it as a "multiplication factor". Think of it as a fixed cost. Just how much do you think it should take?

How about the reference to the Class object (4 bytes for a 32 bit JVM. 8 bytes for 64 bit)? How about the identity hashmap (4 bytes)? The monitor used to synchronization, wait, and notify (I have no idea)? Just because you don't use a feature doesn't mean its size is zero.

Henry
 
arulk pillai
Author
Ranch Hand
Posts: 3380
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Try calculating as shown below

 
I agree. Here's the link: http://aspose.com/file-tools
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic