• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Performance Tunning

 
Greenhorn
Posts: 13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hi

in my code i am using hashmap to store data and process it.
previously the processing was done in 30 minutes while now it takes 3 hours. this is degrading the performance.

the details are like

previous data count

x=40000
y=95000 after processing.

Now
x=84000
y=253000 after processing.

It processes the data and inserts in the oracle table. Is it the problem with of concurrent access or the code is not optimum.
How can i increase the performance.
 
Ranch Hand
Posts: 214
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I doubt your problem has to do with the HashMap, it sounds more like you're using a quadratic complexity (or worse) processing algorithm. So that would be the first place to look I think, but without knowing more, its hard to tell
 
sunil oza
Greenhorn
Posts: 13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hi
thanks for the reply
is the data count creating the problem or is there any other problem. because the code is working fine and it is not giving any error or any exception.
Actually it is fetching data from a table and does the calculation depending on the variables in the table.

After processing it inserts in a table in oracle and fires the update query to update a particular column.

Can there be the issue of concurrent access.

Please suggest any way to optimise the code.
 
JavaMonitor Support
Posts: 251
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Dear sunil,

We cannot optimise your code, since we don't have it.

1) Add print statements to the code to see what operation takes up so much time. Also, count the number of database accesses you perform.

2) Ask yourself: am I doing stuff in Java code that I should let Oracle do? For example this is wrong:



And this is right:


In general, let the database do operations on sets and collections and let Java do operations on individual entities. That way, you use the strengths of each.
 
Ranch Hand
Posts: 433
Netbeans IDE
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Kees Jan Koster wrote:
1) Add print statements to the code to see what operation takes up so much time.


If he adds print-statements to a routine which takes now already 3 hours to complete, he has to wait an eternity... Beside that I have my doubts that he wants to crawl through thousands of line of output. A profiler is here the tool of choice. But beside that you are right with your suggestions.
 
Greenhorn
Posts: 29
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If there's a way to use a representative test data set, which will take a fraction of the three hours to run, that would be one possibility to examine the data. Also, you can collect processing times at various benchmarks, especially when running through loops, and sum them all together to show a summary level view of where the processing time is being spent. That helps to narrow your focus.
 
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you are loading values from a db table into a HashMap, spawn a few threads and try picking up the data in chunks inside each thread. It would certainly be faster than what you currently have..
 
Ranch Hand
Posts: 96
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
We need a little more detail.

Some questions:
1. How big are the rows? Give us an idea of the table schema, or at least the subset of data you're pulling in.
2. Are you retrieving and storing all 84,000 rows at once? Or are you doing it a little bit at a time?
Based on your second post it sounds like you're trying to do it all at once, but let's be sure.
3. Give a rough sketch of what your processing is doing. Not interested in the business logic, of course, but let's be sure complexity isn't an issue.


 
sunil oza
Greenhorn
Posts: 13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hi
I have tried to find out the part of code taking more time in execution using loggers.
But it didn't help. The only thing i noticed is one record is processed 3 times for 3 different calculations.
The calculation is done on the basis of the configuration variables in one of the configuration table.
The value of configuration variables is set by the users in another module. so can't change them.
The table from which it fetches the data for processing has the column index_val and index_code which are important. there are 21 index_codes.
Also suggest, can implemention of multithreading help.
for e.g. create 21 theads one for each index and start all the treads in paralles and invoke the method where the calcualation is done from the run method. and use join().
 
Climb the rope! CLIMB THE ROPE! You too tiny ad:
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic