aspose file tools*
The moose likes Performance and the fly likes Can threading help improve I/O performance Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of EJB 3 in Action this week in the EJB and other Java EE Technologies forum!
JavaRanch » Java Forums » Java » Performance
Bookmark "Can threading help improve I/O performance" Watch "Can threading help improve I/O performance" New topic
Author

Can threading help improve I/O performance

Rajagopal Manohar
Ranch Hand

Joined: Nov 26, 2004
Posts: 183
Hi All,
I need to copy a lot of files form a network location to a local machine. Aroung 500 MB.
Initially I tried to improve the performance using buffering, it worked unfortunately it did not help much when buffering went beyond 10000 bytes.

Then I decided to use threads to import different files parallely to futher improve performance. However it seems to have an opposite effect and it now takes more time.

So does threading really help in case of I/O operations?

ps: can any one suggest other ways to improve I/O performance.

Edited to add: I am using java 1.3
Regards,
Rajagopal
[ June 22, 2005: Message edited by: Rajagopal Manohar ]
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12675
    
    5
Threading can help if your problem involves both IO and computation since the computation thread can work while the IO thread is waiting for the operating system to provide the next buffer read. With dual-core or dual-processor systems this can be significant.
Reading and writing binary data will be MUCH faster than trying to work with character streams - so don't use Reader and Writer classes.
Buffers should be some multiple of the operating system file blocks size.
If you can contrive to execute the operating system copy with Runtime.exec it will save the time involved in moving data between operating system buffers and Java program buffers at the expense of setting up the extra process.
Bill


Java Resources at www.wbrogden.com
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8707
    
    6

Originally posted by William Brogden:

If you can contrive to execute the operating system copy with Runtime.exec it will save the time involved in moving data between operating system buffers and Java program buffers at the expense of setting up the extra process.


You can also do a direct copy using NIO:
Top Ten New Things You Can Do With NIO: #3: Direct Channel Transfers


"blabbing like a narcissistic fool with a superiority complex" ~ N.A.
[How To Ask Questions On JavaRanch]
Rajagopal Manohar
Ranch Hand

Joined: Nov 26, 2004
Posts: 183
Thanks for replies



but it does not do anything.
when I try some thing like notepad the application opens.
cant we just directly execute a xcopy using this method.

I am working from within an IDE

You can also do a direct copy using NIO:
Top Ten New Things You Can Do With NIO: #3: Direct Channel Transfers


Unfortunately I use java 1.3 and will not be able to use nio .

please help me in making use of xcopy and Runtime.exec()
[ June 22, 2005: Message edited by: Rajagopal Manohar ]
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8707
    
    6

Originally posted by Rajagopal Manohar:

cant we just directly execute a xcopy using this method.

No. XCOPY is a command, not a program. Have a look at
When Runtime.exec won't, the section marked " Assuming a command is an executable program "
Rajagopal Manohar
Ranch Hand

Joined: Nov 26, 2004
Posts: 183
No. XCOPY is a command, not a program. Have a look at
When Runtime.exec won't, the section marked " Assuming a command is an executable program "


Thank you very much.
Stefan Wagner
Ranch Hand

Joined: Jun 02, 2003
Posts: 1923

I guess you could improve performance by threads with different physical drives.
Operating on a single drive, the reading head has to jump from file to file which is causing the opposite effect.


http://home.arcor.de/hirnstrom/bewerbung
 
Don't get me started about those stupid light bulbs.
 
subject: Can threading help improve I/O performance
 
Similar Threads
Why we need Buffer ?
How does something get Double Buffered
NIO and performance question
How to prevent a MS DOS window popping up when I run cmd.exe from java
Performance improvement beyond BufferedReader & BufferedInputStream