File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Threads and Synchronization and the fly likes Thread to create and load large list of images Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Threads and Synchronization
Bookmark "Thread to create and load large list of images" Watch "Thread to create and load large list of images" New topic
Author

Thread to create and load large list of images

Yahya Elyasse
Ranch Hand

Joined: Jul 07, 2005
Posts: 510

Hi ,
In my java application i use a java thread to create and display thumbnails of images selected from a given machine folder.
in thread's run() method i do all the job : create thumbnails , use a JLabel(new ImageIcon(thumbnail)) and add this JLabel to a JPanel.i do this in a loop for every image in my list.
now I've opened one folder with 272 photos, 660MB, it took about 4 minutes to load all the images. I think it's too much time in a 3.0GHz 2GB RAM PC.
I want to speed up this process.
Can i use multiple threads? usually when I have to fill a large list, first I get the total items and set the list to this size, then I divide the list in three or four portions (if is bigger than a size) and every portion is loaded in separated threads.
is that a good approach ?
any other suggestions to increase performance ofmy large photos loading ?

thank you.
Satya Maheshwari
Ranch Hand

Joined: Jan 01, 2007
Posts: 368
I think your approach should be correct. We use multiple threads to do a big task which could be split in smaller independent tasks, and each thread takes care of the smaller independent task. This would work as long as you take care of concurrency issues. Please let us know, what sort of performance gain you achieved using multi threading.


Thanks and Regards
Yahya Elyasse
Ranch Hand

Joined: Jul 07, 2005
Posts: 510

i had a discussion with a colleage and that's what he said about this issue :

1 -- the significant portion of your time is in loading the files--waiting for I/O to be done
... more threads will not speed this up unless you have multiple busses and you have the images
... on multiple drives at least some of which are connected via different busses.
2 -- do you have a multi-threaded machine? If you do not have at least a dual-core cpu, then your threading
... is cooperative sharing of the same CPU. Multiple thread will help separate your processing from your loading
... tasks and give you a boost there, but multiple processing thread will aid your speed. 2 threads
... one for processing and one for loading is about good for a single threaded CPU.
3 -- I would load your directory entries of the images into a list of some type and then remove the image as
... it is assigned to a worker--but only if the CPU is multi-threaded.


my colleague's point of view scared me
does it mean that spliting images list into smaller lists and spawning a separate thread for each list portion won't increase the speed of loading and displaying the large images list?

is it possible at all using 2 threads : one for loading image and one for displaying it ? can someone give me an example if that's a possible way to increase performance ?

please guys if someone has a good idea to boost the speed of this process that will make me a happy man
my boss is one of those guys obsessed by speed !
-------------------------------------------
to board moderator : can you please move this thread to discussion board "performance" ?
i think it is better to move it there right ?
Nitesh Kant
Bartender

Joined: Feb 25, 2007
Posts: 1638

Firstly, I must say that your colleague has given you an excellent advice. You will be good listening to him for most of your problems

I am not sure about the I/O processing dependencies because i dont have much knowedge about the file system.

It is true that blindly increasing the number of threads will not increase speed. In fact it may bring it down accounting to thread scheduling overheads of sharing the same CPU for high number of threads.

As told by your colleague, the optimal number of threads depends on the number of CPU. Since a major chunk of your work is I/O bound, so theoretically, if you use the number of threads equal to the number of processors, you will be wasting CPU as for a lot of time your threads will be doing nothing and waiting for I/O (reading image data from the disk).

I am not sure whether you can separate out the image loading from disk and processing of image data. But if you can then you can possibly have two threadpools:
  • One for reading images from disk. (You can have a higher number of threads here.)
  • Other for processing the image data once read completely. (The number of threads at max must be equal to the number of processors.)

  • Using the above design also will need some profiling to find out the optimal number of threads.

    On the other hand you can have a single threadpool with the number of threads greater than the number of processors(since many threads will be waiting for I/O at different time instances) and a single thread doing the loading and processing. This approach is similar to the one you currently have but the number of threads must be governed by the number of processors.
    [ June 16, 2008: Message edited by: Nitesh Kant ]

    apigee, a better way to API!
    Yahya Elyasse
    Ranch Hand

    Joined: Jul 07, 2005
    Posts: 510

    I have a question that i want you experts in software performance to shade some light on :
    i start a java thread to load & process 272 photos with size 660MB. it took about 4 minutes to load& process all the images. do you think this is too much slow in a 3.0GHz 2GB RAM PC?

    a person is trying to convince me that this software is slow .what can you guys say about that ?

    thank you.
    Ernest Friedman-Hill
    author and iconoclast
    Marshal

    Joined: Jul 08, 2003
    Posts: 24183
        
      34

    Have you done any profiling? The thing to do would be to measure how long it takes to do each step of the process. You might find that computing the thumbnail image takes the longest, or painting the screen, or something else unexpected. Find out where the time is going by measuring it, and then you'll know where to concentrate your efforts.

    Regarding the multithreading: there is one advantage to multithreading which is that if you spawn the work off to one or more background threads, your GUI can become responsive before all the images are loaded. Furthermore, you might consider breaking things into pages: display the first 20 images, say, and immediately begin loading the next 20 in the background. The user perception will be that the application is much faster, even though the same amount of time passes before they're seen every image.


    [Jess in Action][AskingGoodQuestions]
     
    It is sorta covered in the JavaRanch Style Guide.
     
    subject: Thread to create and load large list of images