Help needed JVM crashes while running the following program

[Anand Natraj]

Hi,

I have a a 10 threads each generating 1 billion object at a time. But my JNI crashes when i run the thread and insert a System.out.println statement in my finalize method. when i remove the print stament the threads are running properly to completion. but i want the print statement to be present in the finalize method because i have have to integrate the JNi code and will making call to the native methods from the finalize methods. Please find below the code snippet:

Class 1:

public class simpleTest extends Thread
{
 
public void test()
{
TestJava tv = new TestJava();
}
 
 
public int threadId = 0;
 
public void run()
{
int count = 1000000000;
for (int x=0;x<count;x++)
{
test();
}
}
 
 
public static void main(String args[])
{
 
simpleTest test1 = new simpleTest(); test1.threadId = 1; test1.start();
 
simpleTest test2 = new simpleTest(); test2.threadId = 2; test2.start();
 
simpleTest test3 = new simpleTest(); test3.threadId = 3; test3.start();
 
simpleTest test4 = new simpleTest(); test4.threadId = 4; test4.start();
 
simpleTest test5 = new simpleTest(); test5.threadId = 5; test5.start();
 
simpleTest test6 = new simpleTest(); test6.threadId = 6; test6.start();
 
simpleTest test7 = new simpleTest(); test7.threadId = 7; test7.start();
 
simpleTest test8 = new simpleTest(); test8.threadId = 8; test8.start();
 
simpleTest test9 = new simpleTest(); test9.threadId = 9; test9.start();
 
simpleTest test10 = new simpleTest(); test10.threadId = 10; test10.start();
 
}
 
};

Class 2:

public class TestJava
{
public TestJava(){

}

 
// Print something in here and it will blow.
protected void finalize(){

System.out.println("Inside finalize");
}
}

Thanks in advance.


Regards,
Anand
[ February 23, 2007: Message edited by: Jim Yingst ]

[Stan James]

finalize() is pretty unreliable. Have you tried just doing it as run() completes?

public void run()
{
   int count = 1000000000;
   for (int x=0;x<count;x++) 
   {
      test();
   }
   cleanUp(); // HERE?
}

[ February 12, 2007: Message edited by: Stan James ]

[Anand Natraj]

Hi James,

Thanks a lot for the reply.

I am going to use JNI and make some native calls to the 'C++' code. So i have to explicitly call a native in the finalize methos which inturn will call the destructor of the C++ code. So i have to call that method only in the finalize method. Is there any other way to avoid the JVM crash.

Thanks.

Regards,
Anand

[Ernest Friedman-Hill]

Are you being very careful with local vs. global references in the JNI code? Finalize will be called from a different thread than "new", in each case. You can't share jobject references created in one thread with code in another thread.

[Anand Natraj]

Hi,

Thanks for your reply..

Thanks for your suggestion on sharing jobject.

But my problem is the code i have listed itself is crashing the JVM. Please suggest me a way to avoid the JVM crash without modifying the code in finalize method. I tried sychronizing the run method and also synchronizing the test method. But the JVM still crashes..

Thanks in advance

Regards,
Anand

[Pradyumna Singh]

Hi,

i ran the program, the error is heap overflow.

case 1: When there is no print statement in finalize
the objects are being created but garbage collector runs at apropriate times and frees the memory in the heap, so there is no problem.

case 2: When something is being printed in finalize
objects are being created at the same rate but they are not being released at the same rate as in first case. This is because before being released, something is being printed at the console which is much much slower as compared to other operations beng performed in the program. so the garbage collector is not able to free the memory at the same rate as last case. so there is heap overflow in this case.

One solution to this is to increase the maximum heap size of the java virtual machine.

But i think, if in the finalize, you are just releasing some memory and not doing any input/output operations, then you will never face the problem.

thanks



[ February 22, 2007: Message edited by: chilled blue ]

[Peter Chase]

That's very interesting. I wouldn't have thought of that. I guess one has to be very careful about how long it takes to execute any code that one puts in finalize(). Or just never override finalize()...

Would using PhantomReference and a ReferenceQueue be a good alternative? One could have a thread draining the queue and printing out statistics about how often it did that. This wouldn't slow down the finalisation and deallocation of the test objects. But I guess one might end up running out of heap due to not draining the queue fast enough...

[Ilja Preuss]

Anand, what exactly do you mean when you say the JVM crashes? Do you get any error message?

[Anand Natraj]

Hi All,

Thanks a lot for your comments/suggestions on the issue.

1. By JVM crash, i mean the Java process(am using WindowsXP)is ending abruptly with the error message of OutOfMemory Exception.

2. I tried to run the program without doing any IO operations inside the finalize method. But still the JVM crashes.

3. I tried increasing the heap size to 256 MB and also tried with parallelGC, this only delays the JVM crash but not protecting the JVM from crashing.

Can any one suggest me a way to avoid this JVM crash

Thanks in advance

Regards,
Anand

[Anand Natraj]

Hi,

Please find attached the updated version of the program(Without doing any IO operations):

program 1:

public class simpleTest extends Thread
{
 
public TestJava tv;
 
int i;
 
public void test()
{
tv = new TestJava();
}
 
 
public int threadId = 0;
 
public void run()
{
int count = 1000000000;
 
for (int x=0;x<count;x++)
{
test();
}
i= tv.getInt();
 
System.out.println("The value of i is: "+i);
}
 
 
public static void main(String args[])
{
 
simpleTest test1 = new simpleTest(); test1.threadId = 1; test1.start();
 
simpleTest test2 = new simpleTest(); test2.threadId = 2; test2.start();
 
simpleTest test3 = new simpleTest(); test3.threadId = 3; test3.start();
 
simpleTest test4 = new simpleTest(); test4.threadId = 4; test4.start();
 
simpleTest test5 = new simpleTest(); test5.threadId = 5; test5.start();
 
simpleTest test6 = new simpleTest(); test6.threadId = 6; test6.start();
 
simpleTest test7 = new simpleTest(); test7.threadId = 7; test7.start();
 
simpleTest test8 = new simpleTest(); test8.threadId = 8; test8.start();
 
simpleTest test9 = new simpleTest(); test9.threadId = 9; test9.start();
 
simpleTest test10 = new simpleTest(); test10.threadId = 10; test10.start();
 
}
 
};

program2:

public class TestJava
{
int i;
public TestJava()
{
 
}
 
 
public int getInt(){
 
return i;
 
}
 
// Print something in here and it will blow.
protected void finalize()
{
//System.out.println("Just a string");
i++;
}
}

Thanks and Regards,
Anand
[ February 23, 2007: Message edited by: Jim Yingst ]

[Mr. C Lamont Gilbert]

Good thing you asked. Yes this is terrible and unreliable way to close any external resource. I know this has been used by folks and some teach that is the purpose of finalizers. But its not. They are for testing only. And as you can see from your tests finalizers disrupt the normal collection of objects. In your case, and object can not be collected totally until it gets access to system.out. Which means where normally 1 run of the GC can clear several billion objects, in your case, it can only clear those objects that have already spit their data out. I doubt if that will be anywhere near a billion print statements... (System.out is an exclusive resource)


The person that suggested you use phantom reference and ReferenceQueue is dead on. This is the ONLY way to achieve what you are doing. Please adjust your code accordingly. And whoever told you to use finalizers, go back and tell them they mislead you and give them the proper answer.


Nevertheless, a phantom reference is an object too. And it won't be collectable until after its done causing the print to be performed. Your still going to run out of memory...


try this for starters

 
public void test()
{
   try {
      tv = new TestJava();
   }
   catch(Exception ex){
       System.err.println("Oops, failed to allocate memory");  
   }
}

[ February 23, 2007: Message edited by: Mr. C Lamont Gilbert ]

[Jim Yingst]

Anand, I added code tags to your posts above for readability.

[B][CLG]: try this for starters

    public void test()
    {
       try {
          tv = new TestJava();
       }
       catch(Exception ex){
           System.err.println("Oops, failed to allocate memory");  
       }
    }

[/B]
Um... ewww. That assumes that any error thrown will be an OutOfMemoryError. Which could be badly misleading in some cases. Surely we could either (a) catch OutOfMemoryError specifically, or (b) print the exception and stack trace as part of the error message? I really don't like to encourage the use of misleading or uninformative error messages.

Anand, even if you don't use System.out in the finalize method, any use of finalize() tends to slow down garbage collection considerably. So you may still run into problems because you're creating objects faster than they can be collected. I would also suggest that, contrary to popular opinion, an OutOfMemoryError often is a recoverable error. In your code, OOME is somewhat expected, and an appropriate response would be to simply wait a short while, and try again. In fact this strategy can be employed much more efficiently by checking the available heap space before creating the object. That way the JVM can spend less time throwing and catching exceptions, and more time doing GC. Try something like this:

    public void test() {
        boolean done = false;
        while (!done) {
            if (Runtime.getRuntime().freeMemory() > 1000) {
                try {
                    tv = new TestJava();
                    done = true;
                } catch (OutOfMemoryError e) {
                    System.out.println("in test(): " + e);
                }
                if (!done) {
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                        System.out.println("in test(): " + e);
                    }
                }
            }
        }
    }

Of course the two 1000s above may be replaced with whatever other numbers you think are appropriate.

[CLG]: I know this has been used by folks and some teach that is the purpose of finalizers. But its not.

Or to put it another way, I think it was the original intended use of finalizers, but it turns out that they have many problems and do a poor job of this sort of thing, so you should avoid relying on them if at all possible.
[ February 23, 2007: Message edited by: Jim Yingst ]

[Mr. C Lamont Gilbert]

It was just a quick and dirty start point. You are welcome to substitute the proper exception if you wish.

I think the References were added in 1.2 in the javadoc. So I think you are correct. its strange that Sun did not just deprecate the method.

[Rahul Toaikani]

So Anand,

Any new findings on this? Was it resolved?

[M Easter]

Originally posted by chilled blue:

case 2: When something is being printed in finalize
objects are being created at the same rate but they are not being released at the same rate as in first case. This is because before being released, something is being printed at the console which is much much slower as compared to other operations beng performed in the program. so the garbage collector is not able to free the memory at the same rate as last case. so there is heap overflow in this case.
[ February 22, 2007: Message edited by: chilled blue ]

I think this is excellent analysis. I just want to highlight the point here that the GC is running as a single thread and competes with the rest of the system. I once saw a speaker (Glenn Vanderburg) who related a story about a demo at Java One where a sophisticated Swing GUI was (intentionally) brought to its knees by the addition of one line in the finalize() method.
[ March 09, 2007: Message edited by: M Easter ]

[Mr. C Lamont Gilbert]

Wouldn't matter even with multi-threaded GC. System.out is an exclusive resource. Any thread trying to access it must wait its turn.

[Anand Natraj]

Hi All,

I am exteremly sorry for replying so late. Thank for all who have helped me for my better understaning.I tried runing the code posted by Jim Yingst, the code ran for more than 12 hrs,it wasn't crashing the JVM nor it did the program run to completion. I had to kill the program manually. I think i need run the code for 2 days for it to complete. I will test hte same and and post the reply soon.

Thanks and Regards,
Anand

[sreekanth nair]

I do not know what is your query is. Any way increasing your Memory Heap size will solve the problem. If you are not sure how to go about it just search in google

[Anand Natraj]

Hi Sreekanth,

Increasing the heap size doesn't solve the problem. Please run the sample code and check with the increased memory size.

[sreekanth nair]

i do not know by which way you have tried improving Heap Size, In my System that code working perfectly all rite, If you want to know just reduce the count value to 9 digit and check again. so you will understand.