• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

How to define compute-intensive and IO-intensive tasks?

 
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I want to use a thread pool to parse multiple XML concurrently,which take String as input.
I've be told that the size of the pool should be tuned differently for compute-intensive and IO-intensive tasks,but how to clearly define whether a task is compute-intensive or IO-intensive? I used a SAXParser to parser the XML and i can't tell exactly which kind it belongs to.
Please help.
 
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Some general inputs you might be able to use in your decision making.

I/O intensive tasks are quite straightforward to identify - ones which are reading or writing data to an external resource(disk/db)
So for these tasks the bottleneck is the socket(db connections/file handles and similar resources)
If your I/O task use the same resources your threads should be optimized to the number of these resources available to you and the pooling mechanisms used to access these resources.

Another factor to consider in case of file/db operations is that even though you can get 10 connections but if connection is hitting the same table/file then that table lock will become the bottleneck and spawning 10 threads to use 10 connections will not yield any speed improvements. In some cases it may even degrade performance due to increased concurrency operations.

Again you have mentioned you are using SAXParser so the parsing will be compute intensive but reading the xml and loading into the SAXParser will be I/O. Reading and loading XML will also depend where are you getting the XML from, is it from a file(i/o) is it from a web service (then I am not sure).

In Compute intensive tasks you are not reading or writing but processing data. They are more amenable to parallelization, meaning if you can break them up into independent units of work. Then you can spawn more threads if more cores/cpus are available increase the scalability. Parsing would definitely be compute intensive.
 
wd nanji
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks a lot, I'm clear now
 
Consider Paul's rocket mass heater.
reply
    Bookmark Topic Watch Topic
  • New Topic