wood burning stoves 2.0*
The moose likes Performance and the fly likes File Operations in Core Java Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of OCM Java EE 6 Enterprise Architect Exam Guide this week in the OCMJEA forum!
JavaRanch » Java Forums » Java » Performance
Bookmark "File Operations in Core Java" Watch "File Operations in Core Java" New topic
Author

File Operations in Core Java

Srikanth Yerasuri
Greenhorn

Joined: Aug 14, 2013
Posts: 3
Hi Guys,

I want to know about the performance of Java while doing file operations.

We have to develop a program to read the tag value files and perform some good amount of business logic along with some DB transactions and write the output files.

Currently there is a huge code which is written in PRO*C in order to do this activity. Another important thing is PRO*C is reading all the data directly from database.

In the new program which we want to develop on Java needs to read the data from a file. However there can be some DB transactions in order to consult some information.

Can you please let me know if it is wise to shift to Java along with PROs and CONs.

Appreciate you help.

Regards,
Srikanth
Srikanth Yerasuri
Greenhorn

Joined: Aug 14, 2013
Posts: 3
The size of input file can be around 50 GB.
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12769
    
    5
Java file operations are heavily optimized and integrated with the OS so file IO will not be a problem.

As usual, your DB operations will be the biggest CPU time consumer.

Java Threads will let you separate the various functions - for example one input file reading Thread to ensure that records are always available for the computing Thread(s) to work on, and output writing Threads.

What is unsatisfactory about your existing PRO C program?

Bill
Srikanth Yerasuri
Greenhorn

Joined: Aug 14, 2013
Posts: 3
Thanks for the reply Bill...

Currently the only problem is we don't have resources who can develop in PRO*C.

This PRO*C code is being run as 16 instances and the whole instances on average take 3 hours to complete processing data from database of around 30GB.

If we migrate from PRO*C to Java, we should be able to achieve atleast this performance.

What are the important points to be taken care while doing the development in Java. As I told in my previous post, we should currently read data from files unlike reading from DB in case of PRO*c code.

Regards,
Srikanth
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12769
    
    5
Important points..... hmm

Be prepared to measure / profile your Java code.

Nothing wastes more time and creates more bugs then trying to create "optimized" code rather than understandable code.

16 instances - doing intense DB work, yow! Did somebody optimize that or what?

My first step would be to write the class for reading from the input files and parsing the data only - time that on a full size file to get a good starting point. I am betting you will be pleasantly surprised.

Bill
Srikanth Yerasuri
Greenhorn

Joined: Aug 14, 2013
Posts: 1
Hi Bill,

They are preparing a cursor in PRO*C code and for each record in the DB, they are loading information from different tables and then they are processing and writing in to the output file.

There is a huge DB activity. That is the reason they would have chosen PRO*C for development.

But now, as we have to read from Files (of course for some information, we need to query DB as well) I would like to go for Java.

Also can you suggest some tips if we have some DB operations in Java. Also please suggest some memory related things which needs to be taken care.

Regards,
Srikanth
Ivan Jozsef Balazs
Rancher

Joined: May 22, 2012
Posts: 867
    
    5

They are preparing a cursor in PRO*C code and for each record in the DB, they are loading information from different tables and then they are processing and writing in to the output file.

There is a huge DB activity. That is the reason they would have chosen PRO*C for development.



Well, this activity is inside the DB, and from this aspect it should not matter whether that tremendous DB activity is triggered from PRO*C or Java or whatnot.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: File Operations in Core Java