• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

OutOfMemoryError while running DataClassTest

 
Ranch Hand
Posts: 160
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It will not terminate successfully.

and this occur in error log.. sometimes it will say "java.lang.OutOfMemoryError: unable to create new native thread"


org.eclipse.swt.SWTException: Failed to execute runnable (org.eclipse.swt.SWTError: No more handles)
at org.eclipse.swt.SWT.error(SWT.java:4083)
at org.eclipse.swt.SWT.error(SWT.java:3998)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:137)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4041)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3660)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2640)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2604)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2438)
at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:671)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:664)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:115)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:620)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:575)
at org.eclipse.equinox.launcher.Main.run(Main.java:1408)
Caused by: org.eclipse.swt.SWTError: No more handles
at org.eclipse.swt.SWT.error(SWT.java:4109)
at org.eclipse.swt.SWT.error(SWT.java:3998)
at org.eclipse.swt.SWT.error(SWT.java:3969)
at org.eclipse.swt.browser.IE.create(IE.java:265)
at org.eclipse.swt.browser.Browser.<init>(Browser.java:119)
at org.eclipse.jface.internal.text.html.BrowserInformationControl.createContent(BrowserInformationControl.java:218)
at org.eclipse.jface.text.AbstractInformationControl.create(AbstractInformationControl.java:448)
at org.eclipse.jface.internal.text.html.BrowserInformationControl.<init>(BrowserInformationControl.java:197)
at org.eclipse.jdt.internal.ui.text.java.hover.JavadocHover$2.<init>(JavadocHover.java:383)
at org.eclipse.jdt.internal.ui.text.java.hover.JavadocHover$HoverControlCreator.doCreateInformationControl(JavadocHover.java:383)
at org.eclipse.jface.text.AbstractReusableInformationControlCreator.createInformationControl(AbstractReusableInformationControlCreator.java:45)
at org.eclipse.jface.text.AbstractInformationControlManager.getInformationControl(AbstractInformationControlManager.java:808)
at org.eclipse.jface.text.AbstractInformationControlManager.internalShowInformationControl(AbstractInformationControlManager.java:1157)
at org.eclipse.jface.text.AbstractInformationControlManager.presentInformation(AbstractInformationControlManager.java:1139)
at org.eclipse.jface.text.AbstractHoverInformationControlManager.presentInformation(AbstractHoverInformationControlManager.java:902)
at org.eclipse.jface.text.TextViewerHoverManager.doPresentInformation(TextViewerHoverManager.java:243)
at org.eclipse.jface.text.TextViewerHoverManager$5.run(TextViewerHoverManager.java:233)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
... 22 more
 
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
This seems to be a problem in eclipse, try a fresh installation.
 
Ixus See
Ranch Hand
Posts: 160
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
key guys, I am really stuck I don't know how to resolve this issue...

I can run the loop 100~500 times with no problem but anything more it will not terminate successfully.

I implemented the following

1) Using cache for read methods (find, criteria and etc)
2) Write to file for write methods(update create, delete)
3) I use the follow code for my locking

[edit] do not post actual code snippets


when there is a need to read from cache.. I will

[edit] do not post actual code snippets

when there is a need to writo file.. I will

[edit] do not post actual code snippets

I Don't know what to do anymore Please help me!!! if anyone can kindly help me do a code review, it would be greatly appreciated.. obviously I'm desperate that is why I am asking for help here.

Thanks
 
Ixus See
Ranch Hand
Posts: 160
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Roberto Perillo wrote:

Roel De Nijs wrote:You say you can only increase counter to 100. What happens with your program if you increase the counter to 500 or 1000? Giving us such information is much more important than the modified DataClassTest.



Agreed. For instance, do you run into a deadlock? That's weird, because that would (could) happen when running the program 100x as well.



yes I agree! I think there is something wrong with eclipse. I even refactor to store String[] in cache and NOT record objects, hoping it will use less memory.

I already spend 16 hours on it.. I think I won't make it in time to submit my assignment. sux

JConsole detected zero dead lock. I think is really eclipse.

Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Unknown Source)
at Test.DataClassTest.startTests(DataClassTest.java:59)
at Test.DataClassTest.main(DataClassTest.java:40)

 
Ranch Hand
Posts: 590
Eclipse IDE Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
This test is a useful resource. I'd suggest creating posts for individual issues with running the test in a separate thread - unless they turn out to be general issues. Keep this thread for any problems\updates\suggestions etc. with the original test. That will make the thread more useful, rather than polluting it posts that are just specific to an individual.
 
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Looks like thread leaking. Threads are created but then stuck somewhere. Maybe because they never run to completion...
 
Ixus See
Ranch Hand
Posts: 160
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
if they don't run into completion, I should face the problem with 100 iteration too?
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Not at all.

Let's say the JVM (and/or the OS) allow 400 native threads to be alive at any given time. Per iteration you create 5 threads (create, find, delete rec1, update rec1, update random rec). Maybe only the threads for updating and deleting do not run to completion. So when using 100 iterations, 300 threads are never completed and thus alive. That's under the limit of 400, so no problem. These are just arbitrary numbers and they will depend on the OS you are using. And/or maybe just 1 kind of thread (the delete rec1) may run forever. So it's a combination.

But I would assume it has something to do with the update and delete threads (because managing these 2 is one of the hardest parts of the assignment), so if you disable them and just run the test with the create and find threads I think you'll be able to increase the counter to 500, 1000 and even many more.
 
Ixus See
Ranch Hand
Posts: 160
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I didn't disable any of them with regards to this current test. You mean if the threads are alive, the program will still terminate successfully?

Since it might be OS dependent, does that mean I can hand these codes up ?

I have spend 2 days trying to speed up the method, I feel I am getting diminishing returns on my effort because it might be introducing more issues to the codes.

 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You don't get it at all! If your program works as it should, you can increase the counter until 100000 (and even further) without getting any exception at all. In fact my application can run on a Linux, Windows or any other machine with any number of iterations.

You definitely can't, because with 1000 iterations you have an OutOfMemoryError. So you have to pinpoint the cause of this problem before you can try to solve the issue. Therefor you need to do some debugging. That's normally something a developer would do on his own initiative, not something that someone else must suggest before you take any (appropriate) action. To be frankly it sounds like this assignment is the first application you are developing. If you have a problem with your code, you debug to find the cause and then you solve it appropriately.

Based on the error message you have threads which do not run to completion, because you have a flaw (or many flaws) in your Data class. So in order to be able to make appropriate changes you have to know which method(s) has (have) bugs. Roberto's Data Class Test is an excellent resource for analyzing your Data class and spotting any problems. A possible route you could take for debugging, could be: you run the test with 1000 iterations of nothing more than the FindThread which is executed (all other threads disabled). If you are able to run the program without getting the error, the find-method is fine. This scenario is repeated for every type of thread in the test. A next step then would be to start combining these threads (findthread together with createthread).

You definitely need to solve this issue (or issues) if you want to pass this certification. If you have threads not running to completion you will almost certainly get failed.
 
Ixus See
Ranch Hand
Posts: 160
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hmmmm

I did try debugging, but I cannot seems to find out the reason why....
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you can't find the cause, we can't help you
 
Ixus See
Ranch Hand
Posts: 160
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I don't want to be insulting but you have push my button a few times over and over again..

firstly I am no Java black belt nor did I score full marks for my SCJD.

BUT I have been developing applications since 18... I'm 28

I'm just not good with concurrency as i develop mostly banking related application, which requires more algorithm and etc. We also use ADA for real time critical system, basically I am not the person handling the database.


anyway I have been under a lot of stress lately, partly because someone in the family have pass on and its administration of assets is a problem too.

Oh yes there are 2 types of team lead/developers, one just like to pass general remarks when people ask for help and another provide solutions and hope the other guy improve on the solution. I am the one that provide the solution and possible fault in my solution and my colleagues will improve on it, therefore with each iteration, the product improves.

I guess you are the first type, I respect your decision and partly it might due to the rules of this exam.

oh yes I spend 3 full days trying to resolve this issue too! THANK YOU!
 
Ixus See
Ranch Hand
Posts: 160
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
org.eclipse.swt.SWTException: Failed to execute runnable (org.eclipse.swt.SWTError: No more handles)
at org.eclipse.swt.SWT.error(SWT.java:4083)
at org.eclipse.swt.SWT.error(SWT.java:3998)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:137)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4041)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3660)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2640)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2604)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2438)
at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:671)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:664)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:115)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:620)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:575)
at org.eclipse.equinox.launcher.Main.run(Main.java:1408)
Caused by: org.eclipse.swt.SWTError: No more handles
at org.eclipse.swt.SWT.error(SWT.java:4109)
at org.eclipse.swt.SWT.error(SWT.java:3998)
at org.eclipse.swt.SWT.error(SWT.java:3969)
at org.eclipse.swt.browser.IE.create(IE.java:265)
at org.eclipse.swt.browser.Browser.<init>(Browser.java:119)
at org.eclipse.jface.internal.text.html.BrowserInformationControl.createContent(BrowserInformationControl.java:218)
at org.eclipse.jface.text.AbstractInformationControl.create(AbstractInformationControl.java:448)
at org.eclipse.jface.internal.text.html.BrowserInformationControl.<init>(BrowserInformationControl.java:197)
at org.eclipse.jdt.internal.ui.text.java.hover.JavadocHover$2.<init>(JavadocHover.java:383)
at org.eclipse.jdt.internal.ui.text.java.hover.JavadocHover$HoverControlCreator.doCreateInformationControl(JavadocHover.java:383)
at org.eclipse.jface.text.AbstractReusableInformationControlCreator.createInformationControl(AbstractReusableInformationControlCreator.java:45)
at org.eclipse.jface.text.AbstractInformationControlManager.getInformationControl(AbstractInformationControlManager.java:808)
at org.eclipse.jface.text.AbstractInformationControlManager.internalShowInformationControl(AbstractInformationControlManager.java:1157)
at org.eclipse.jface.text.AbstractInformationControlManager.presentInformation(AbstractInformationControlManager.java:1139)
at org.eclipse.jface.text.AbstractHoverInformationControlManager.presentInformation(AbstractHoverInformationControlManager.java:902)
at org.eclipse.jface.text.TextViewerHoverManager.doPresentInformation(TextViewerHoverManager.java:243)
at org.eclipse.jface.text.TextViewerHoverManager$5.run(TextViewerHoverManager.java:233)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
... 22 more

hmmm
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Seems odd to me you need a browser for this assignment. So what kind of eclipse version are you using? Is it a fresh install? And what actions did you take for this stack trace to be shown in your (eclipse) logs?
 
Ixus See
Ranch Hand
Posts: 160
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Eclipse Version: Helios Service Release 2

After I set the iteration at 550 and that error came out.
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
And are you sure this stack trace and the program run are related? Based on the stack trace something went wrong when the javadoc was shown on hover in a browser (based on the classes JavadocHover and Browser in the stack trace) and that's something I would think you don't need for running the DataClassTest
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
First of all I'm not a Java black belt and that's completely not needed to pass this certification (even with full marks). When I started with this assignment I had no practical experience with threading and multi-threaded application (except for a few classes I wrote to pass the SCJP certification), but during the assignment I learnt a whole lot about this topic (also thanks to this forum). In the big Java world of multi-threading applications I'm still a greenhorn.

I'm sad to here someone in the family has deceased.

The purpose of this certification is (like I already indicated) is to proof you have the set of skills a true developer needs to have. I guess you have other expectations of help on this forum (asking your question and getting code that you can directly copy/paste into your application), but that's not how it works and certainly not in this forum (you can't simply share actual code snippets of your solution). So everybody on this forum (I'm not the only one) gives hints, tips, pointers,... and it's up to you to do something with the given advice.

I gave you a few posts back a route you could take for debugging your Data class with the help of Roberto's DataClassTest. You said you did some debugging, but did you do what I suggested?
 
Ixus See
Ranch Hand
Posts: 160
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Roel De Nijs wrote:First of all I'm not a Java black belt and that's completely not needed to pass this certification (even with full marks). When I started with this assignment I had no practical experience with threading and multi-threaded application (except for a few classes I wrote to pass the SCJP certification), but during the assignment I learnt a whole lot about this topic (also thanks to this forum). In the big Java world of multi-threading applications I'm still a greenhorn.

I'm sad to here someone in the family has deceased.

The purpose of this certification is (like I already indicated) is to proof you have the set of skills a true developer needs to have. I guess you have other expectations of help on this forum (asking your question and getting code that you can directly copy/paste into your application), but that's not how it works and certainly not in this forum (you can't simply share actual code snippets of your solution). So everybody on this forum (I'm not the only one) gives hints, tips, pointers,... and it's up to you to do something with the given advice.

I gave you a few posts back a route you could take for debugging your Data class with the help of Roberto's DataClassTest. You said you did some debugging, but did you do what I suggested?



I'm quoting yes I DID as you say..

I run each function individually for 1000 times.. each of them give me the same problem and unable to create native thread issue.

I just switch to Eclipse classic and it doesn't pause or die on me anymore, I'm able to run up to 750.

anything above will have problem... When I look at debug mode, a lot of thread end up in garbage collected and i hit out of memory

Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Unknown Source)
at Test.DataClassTest.startTests(DataClassTest.java:61)
at Test.DataClassTest.main(DataClassTest.java:40)

same issue but my eclipse no longer hangs at times.
 
Ixus See
Ranch Hand
Posts: 160
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
what is the pseudocode code for LOCK unlock update and delete..

let me check it against mine.

like you say.. I suspect there is something to do with delete and update on record 1 over 1000 times each...
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That's weird and completely against my expectations. But we don't give up

Replace the startTests() method with this one:


This code will start 1000 threads, each thread will read record 1. You should be able to run that one without any problem. If this one throws the same exception, that would be
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Ixus See wrote:I suspect there is something to do with delete and update on record 1 over 1000 times each...


If only that would be the reason then running the DataClassTest with only the FindThread enabled would run without any problem. But you indicated in your previous post that each run give you the same problem
 
Ixus See
Ranch Hand
Posts: 160
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
actually after running eclipse classic, I can run 1000 times with no problems(each at a time)

but if I do a combination there will be problem..

by the way.. all the update1, delete1 will only call lock method right? (that means I am just testing lock function?)

and since my lock method check if the record 1 exist... it will just keep throwing record not found exception.

and would not even go into update or delete function or even attempt to lock record 1.

so what is the point here.. sorry just have to ask again.

partial code

[edit] do not post actual code snippets
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
So when you run DataClassTest with each type of thread separately you can increase the number of iterations to 1000 without a problem. You could even increase to 10000 (and even more, I tested it with my application: 100 000 times reading record 1 succeeds in just 27 seconds)

I would expect that combining the FindThread and CreateThread would work too. But when you start combining update/delete thread you come into problem. Can you confirm?
 
Ranch Hand
Posts: 54
Eclipse IDE Spring Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Have you tried running the application through the command line without eclipse?
 
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You need to start to simplify the situation until you can track down the problem.

  • I second the comment to take Eclipse out of the loop and run from the command line
  • Are you setting the memory explicitly using Xmx? It may be that your jvm has a very low default value
  • It has been suggested that threads may not be closing. You can find this out by adding logging when the thread is created and logging when the thread is destroyed. If all the threads that you create are shutdown then you an eliminate this as a cause.
  • Run it in the debugger and in the middle of the run pause all the threads. Then look at the threads and see if any are blocking. By analysing the stack traces of the threads you will be able to determine if you have any deadlocks or other locking problems


  •  
    Ixus See
    Ranch Hand
    Posts: 160
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Roel De Nijs wrote:That's weird and completely against my expectations. But we don't give up

    Replace the startTests() method with this one:


    This code will start 1000 threads, each thread will read record 1. You should be able to run that one without any problem. If this one throws the same exception, that would be



    OH MY GOODNESS THIS HAS PROBLEM TOO! 1100 and it dies =x Yes I'm quoting, anyway I need to go work and hammer people for not conforming to frame work again .. can you believe that is my job?

    I will try just using command line later... I'm thinking of encapsulating all the method in a synchronized method. maybe that will help too.
     
    Ixus See
    Ranch Hand
    Posts: 160
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    i added System.gc() at the end of every function in DataClassTest

    I'm able to clear up to 1000...

    If I increase to 3000, I will get out of memory: unable to create new native thread on the Main. I highly suspect this is due to the cache I am using to store all the records.. imaigne the amount of records there is..


    can someone explain to me the purpose or running delete1 and update1, both of them will throw RNFE at lock();

    I don't see the purpose of this...
     
    Ixus See
    Ranch Hand
    Posts: 160
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    I did some modification to my lock and unlock

    1) lock(int recNo) method

    I move recordExist(recNo) out of the sychronized (lockRecord) { }

    to remove any possible concurrency issue

    2) unlock(int recNo)

    I remove the check for RNFE for unlock, I don't see the point in that...
    How do you guys implement the RNFE for unlock? just check and throw RNFE?

    if the record is in the lock list, would it be forever lock?

    3) I move the updateRecord and deleteRecord from FILE out of the sychronized(records) { }

    both the updateRecord and deleteRecord from file are synchronized methods.

    The reason is I don't wish to delay the release of records, writing to file is independent of updating the record cache.


    with these modificaitons, I am able to hit 3000 with no outofmemory : unable to create native thread issue. i think the main issue is the amount of thread that stays alive is too much, even with GC at the end of dataclasstest methods.

    any above, the debug will suspend and hit out of memory. and require me to press play to continue

    I am just worried that my modification might break some functionality, thats why I say there might be diminishing returns on my effort to resolve this issue

    DO YOU GUYS HAVE ANY VIOLENT OBJECTION? If not please forever hold your peace ;)

     
    Roel De Nijs
    Sheriff
    Posts: 11604
    178
    Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Which machine are you running on? And which Java version are you using?

    Ixus See wrote:i added System.gc() at the end of every function in DataClassTest


    That's something you certainly won't to submit, because System.gc() is just a hint, you don't have any guarantee that it's get executed and when it's executed. So it might work on your pc, but not on the accessor's.

    Ixus See wrote:If I increase to 3000, I will get out of memory: unable to create new native thread on the Main. I highly suspect this is due to the cache I am using to store all the records.. imaigne the amount of records there is..


    Not correct. If the issue is related to the amount of records in your cache, then you would get an "OutOfMemoryError: Java heap space". But the description tells you it's thread related. So that's why I suggested to start testing with the easiest method in the Data class, namely the read-method. That's simply how you debug: start with the simplest thing and with each step you add a bit of complexity until you have encountered the issue. I can run the test with the code snippet I posted few posts back (thread which just reads record 1) with a counter of 100000 without any problem (on a WinXP machine using JDK 1.6.0_17). What about you?

    Ixus See wrote:can someone explain to me the purpose or running delete1 and update1, both of them will throw RNFE at lock();


    I will answer this question here.
     
    Ixus See
    Ranch Hand
    Posts: 160
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Window 7, Java JDK 1.5

    For read(1) -----------------

    I will hit outofmemory(unable to create native thread) @ 10K if I do it without System.Gc();

    but if I use System.gC() at the end of thread/method, I can do up to 100 000...so does it mean the GC is not fast enough?

    Yes my find method sychronize the records and my dataclass is a singleton
     
    Roel De Nijs
    Sheriff
    Posts: 11604
    178
    Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    You can't use Java 1.5 anymore for this assignment. That's (clearly) stated in the instructions. So start by downloading and installing the latest version of JDK 1.6

    I would assume your read-method looks like this: if recNo exists return the String[] from the cache, otherwise throw a RNFE. You don't want to use System.gc() in your code and certainly not rely on it for correct behavior of your application.
     
    Ixus See
    Ranch Hand
    Posts: 160
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    I was using JDK 1.6 then I downgraded to 1.5 because I think I register my assignment as 5.0.. which part of the instruction says I must use 1.6?

    [edit] removed actual code snippet

    For me is get record if it is null throw new RNFE else return record.get(recNo)

    I use null as deleted record, since if a record doesn't exist it should return null.. it should works both ways.

    Reference: http://download.oracle.com/javase/1,5.0/docs/api/java/util/LinkedHashMap.html#get(java.lang.Object)
     
    Roel De Nijs
    Sheriff
    Posts: 11604
    178
    Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Ixus See wrote:which part of the instruction says I must use 1.6?


    Read the Target Platform and Execution section of your instructions.

    Regarding your code (I had to remove it, because we try to prevent posts with actual code, even something as small and easy as the read-method): that looks just fine. No real comments on that one. So it's strange that you can't run the test with a counter of 100000. Maybe it's something related to Win7, I'll try to run my program on my laptop this evening and post back the results of this test.
     
    Ixus See
    Ranch Hand
    Posts: 160
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Roel De Nijs wrote:

    Ixus See wrote:which part of the instruction says I must use 1.6?


    Read the Target Platform and Execution section of your instructions.

    Regarding your code (I had to remove it, because we try to prevent posts with actual code, even something as small and easy as the read-method): that looks just fine. No real comments on that one. So it's strange that you can't run the test with a counter of 100000. Maybe it's something related to Win7, I'll try to run my program on my laptop this evening and post back the results of this test.



    I just optimize it, I really hate to do optimization.. it will store result into variable and test for positive result first.

    anyway Can i safely say that this problem is related to java/eclipse or whatsoever and has nothing to do with the assignment and my codes should be fine?
     
    Roel De Nijs
    Sheriff
    Posts: 11604
    178
    Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Ixus See wrote:anyway Can i safely say that this problem is related to java/eclipse or whatsoever and has nothing to do with the assignment and my codes should be fine?


    That's impossible for me to say. If you are confident about your update and delete methods both competing for lock on the same record (which can be easily tested with a small program, I posted it here), you can convince yourself that your code is ok and you can move on.
     
    Ranch Hand
    Posts: 43
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hello Roel,

    first of all i would want to thank you, Roel, Roberto Perillo and other guys for help that you all provide to support candidates to pass developer exam.
    Thank you especially for your patience !...
    I'm working on UrlyBird 1.1.1 assignment and if you, Roel, want to know - i read almost all your posts that you submit after 17 giugno 2009...

    I think that OutOfMemoryError is not unusual for the DataClassTest of Roberto - if the index of startTests's for-loop is bigger than 3000 (for my JVM on my PC) you program will terminate without deadlock, but in infinite list of screen messages you will find something like this :

    Exception in thread "main" 7588 trying to lock record #1 on UpdatingRecord1Thread
    java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method)
    at java.lang.Thread.start(Thread.java:640)
    at suncertify.tests.DataClassTest.startTests(DataClassTest.java:106)
    at suncertify.tests.DataClassTest.main(DataClassTest.java:74)


    Modified version of Roberto's program adapted to my case is :

     
    Ixus See
    Ranch Hand
    Posts: 160
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    thank you! I believe as long what we get is OutOfMemory, unable to create native I think is okie

    if it is heapsize overload.. I would think there is some issues with our cache management.
     
    Roel De Nijs
    Sheriff
    Posts: 11604
    178
    Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Dmitri Cherkas wrote:and if you, Roel, want to know - i read almost all your posts that you submit after 17 giugno 2009...


    Then you had a whole lot of reading to do
     
    Dmitri Cherkas
    Ranch Hand
    Posts: 43
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Yes it was a pleasant and useful reading, thank you...

    I ask myself - hundreds of people are used Roberto's program in preparing to developer exam... many guys are used the program with more than 10 000 iterations (you, Roel (King of Belgium ), tested with 20 000 iteration)... Is it possible that nobody is find out that after, say, 5000 iterations the main thread breaked and no more threads are created ?... I think this is no use to say "I tested with 1 000 000 index of for-loop." if threads quantity actually was no more than 8000...

    Am i right ?
     
    Water proof donuts! Eat them while reading this tiny ad:
    a bit of art, as a gift, the permaculture playing cards
    https://gardener-gift.com
    reply
      Bookmark Topic Watch Topic
    • New Topic