This week's book giveaway is in the OO, Patterns, UML and Refactoring forum. We're giving away four copies of Refactoring for Software Design Smells: Managing Technical Debt and have Girish Suryanarayana, Ganesh Samarthyam & Tushar Sharma on-line! See this thread for details.
To write a program that can do a matrix multiplication with multiple threads efficiently, you'll have to look at the algorithm for the matrix multiplication and split it into tasks that can performed independently (i.e. one task should not depend on the outcome of the other task). So have a look at the algorithm and think about how you can split it into tasks that can be performed independently and at the same time.
After you have made a plan, you'll have to implement it. For that you'll have to learn how to program multiple threads in Java. The Java Tutorial has a chapter about concurrency and multi-threading: Lession: Concurrency.