aspose file tools*
The moose likes JDBC and the fly likes parallel insertion in oracle Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of EJB 3 in Action this week in the EJB and other Java EE Technologies forum!
JavaRanch » Java Forums » Databases » JDBC
Bookmark "parallel insertion in oracle " Watch "parallel insertion in oracle " New topic
Author

parallel insertion in oracle

Adi Kulkarni
Ranch Hand

Joined: Mar 12, 2009
Posts: 86
Hello all,

I need to insert huge csv file into an oracle database using JDBC . I read somewhere that parallel insert is possible only when the query contains

insert into <tablename> select * from <tablename1> i.e. we can parallely insert from one table to other.

insert into <tablename> values ... ... cannot do a parallel insert.

Now my csv file is very big say around a 1-2 gb. Inserting serially would take day(s) to load it to database which isnt acceptable. Now can I use threading in this case and how ? I have to improve the performance drastically.

Thanks,
Aditya


SCJP 1.5
Paul Sturrock
Bartender

Joined: Apr 14, 2004
Posts: 10336

The SQL you have doesn't demonstrate a parallel insert and wont really help you speed up your bulk data load.

SQL Loader lets you perform parallel loads. This is where you multi thread the loading task accross available CPUs and disks. Have a read of the SQL Loader documentation They contain a huge wealth of possible performance improvements.


JavaRanch FAQ HowToAskQuestionsOnJavaRanch
Adi Kulkarni
Ranch Hand

Joined: Mar 12, 2009
Posts: 86
thanks Paul,

I could not understand, can you elaborate if possible ?

Thanks,
Aditya
Paul Sturrock
Bartender

Joined: Apr 14, 2004
Posts: 10336

SQL Loader is a tool that comes with Oracle whose purpose is to load bulk data in an efficient and configurable way. See the documentation, which also covers parallel loading.
Adi Kulkarni
Ranch Hand

Joined: Mar 12, 2009
Posts: 86
Hi Paul / all,

I tried using sql loader for the same. The command run at the console is freezing/hanging indefinitely and no log file is also being populated.

I have tried default options as well still the command is freezing at the console. Can anyone help me on this ?

Thanks,
Aditya
Paul Sturrock
Bartender

Joined: Apr 14, 2004
Posts: 10336

have you specified a log fiel? SQL Loader ony writes to a log file once a job is complete. How long did you wait to see if the job completed (bulk loading takes time).
Adi Kulkarni
Ranch Hand

Joined: Mar 12, 2009
Posts: 86
The file was only 33 MB big and I waited for more thatn an hour for that. Also to test I tried it with file of size 2 kb. Still it hangs

My control file is:



The command on the console was :

sqlldr userid=username/password@service control=dlc.ctl
Adi Kulkarni
Ranch Hand

Joined: Mar 12, 2009
Posts: 86
Hi Paul,

Can you help me with this ?

Thanks,
Aditya
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: parallel insertion in oracle
 
Similar Threads
comparing csv file values with database attributes.
read large CSV file and update the database
Suggestion Needed on tuning the performance.
how to load a cvs file into a oracle 9i database using java
using JFileChooser