Hi ALl: I wrote a Multi-Thread program to fetch web data. and I need it run 24 hr a day. But there always be problems: 1.It will make CPU usage up to 100%, until the page is completely downloaded. 2.Memory is never enough, the usage keep growing as long as the program is running, and finally get an outOfMemory Exception. I am using J2SE 1.4.0_01 on Windows 2000 Server. It starts with 5 threads to get web page, 1 thread to extract URL link in every downloaded page and another thread to write to Database.When I stop the Thread, the usage of memory stop growing, but never going down unless I quit the program. Any help will be appreciated. Thanks.
i have encountered that problem too..but it was resolved that by making my code more precise..Just look at your code that would make your thread continously make a loop, change it by sleeping the threads or using wait() and notify().
Joined: Nov 14, 2002
Thanks... Yes, I have made all threads sleeping after every page done. It can slow down the usage of CPU, but not memory. So I still get an outOfMemory Exception if it run for a long time.
The soul is dyed the color of its thoughts. Think only on those things that are in line with your principles and can bear the light of day. The content of your character is your choice. Day by day, what you do is who you become. Your integrity is your destiny - it is the light that guides your way. - Heraclitus
Two suggestions: 1: Make sure there are no references to the objects you no longer need. For example, if you store a reference in a container, the memory will not be released until there are no more references to a particular object. 2: Once you reset all of the references, invoke: System.gc() or Runtime.getRuntime().gc(). This should suggest JVM to kick off a garbage collector.
To the 100% CPU "problem" ... why would you want a program that uses less? Would you want it to run longer than necessary? Well, maybe if you want to do anything else at the same time. I have a little download program that runs up to 100% CPU (3.1 ghz desktop PC) with about 4 threads. I gave some thought to interrupting it so I could browse the Ranch at the same time but decided to just let it have the machine for as long as it takes. Actually Windows is good enough to let me do other things, just slightly slower than usual.
A good question is never answered. It is not a bolt to be tightened into place but a seed to be planted and to bear more seed toward the hope of greening the landscape of the idea. John Ciardi