This week's book giveaway is in the Mac OS forum.
We're giving away four copies of a choice of "Take Control of Upgrading to Yosemite" or "Take Control of Automating Your Mac" and have Joe Kissell on-line!
See this thread for details.
The moose likes JDBC and the fly likes parallel insertion in oracle Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


JavaRanch » Java Forums » Databases » JDBC
Bookmark "parallel insertion in oracle " Watch "parallel insertion in oracle " New topic
Author

parallel insertion in oracle

Adi Kulkarni
Ranch Hand

Joined: Mar 12, 2009
Posts: 86
Hello all,

I need to insert huge csv file into an oracle database using JDBC . I read somewhere that parallel insert is possible only when the query contains

insert into <tablename> select * from <tablename1> i.e. we can parallely insert from one table to other.

insert into <tablename> values ... ... cannot do a parallel insert.

Now my csv file is very big say around a 1-2 gb. Inserting serially would take day(s) to load it to database which isnt acceptable. Now can I use threading in this case and how ? I have to improve the performance drastically.

Thanks,
Aditya


SCJP 1.5
Paul Sturrock
Bartender

Joined: Apr 14, 2004
Posts: 10336

The SQL you have doesn't demonstrate a parallel insert and wont really help you speed up your bulk data load.

SQL Loader lets you perform parallel loads. This is where you multi thread the loading task accross available CPUs and disks. Have a read of the SQL Loader documentation They contain a huge wealth of possible performance improvements.


JavaRanch FAQ HowToAskQuestionsOnJavaRanch
Adi Kulkarni
Ranch Hand

Joined: Mar 12, 2009
Posts: 86
thanks Paul,

I could not understand, can you elaborate if possible ?

Thanks,
Aditya
Paul Sturrock
Bartender

Joined: Apr 14, 2004
Posts: 10336

SQL Loader is a tool that comes with Oracle whose purpose is to load bulk data in an efficient and configurable way. See the documentation, which also covers parallel loading.
Adi Kulkarni
Ranch Hand

Joined: Mar 12, 2009
Posts: 86
Hi Paul / all,

I tried using sql loader for the same. The command run at the console is freezing/hanging indefinitely and no log file is also being populated.

I have tried default options as well still the command is freezing at the console. Can anyone help me on this ?

Thanks,
Aditya
Paul Sturrock
Bartender

Joined: Apr 14, 2004
Posts: 10336

have you specified a log fiel? SQL Loader ony writes to a log file once a job is complete. How long did you wait to see if the job completed (bulk loading takes time).
Adi Kulkarni
Ranch Hand

Joined: Mar 12, 2009
Posts: 86
The file was only 33 MB big and I waited for more thatn an hour for that. Also to test I tried it with file of size 2 kb. Still it hangs

My control file is:



The command on the console was :

sqlldr userid=username/password@service control=dlc.ctl
Adi Kulkarni
Ranch Hand

Joined: Mar 12, 2009
Posts: 86
Hi Paul,

Can you help me with this ?

Thanks,
Aditya
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: parallel insertion in oracle