One thing you'll want to do is keep track of the highest sum so far. As you
test each sub-rectangle, if its sum is higher make it the new "highest". To help keep track of sub-rectangles, why not make a little class?
(Some would tell you that since it has no methods it's a data structure and not a very nice class, but it will do the job.)
There is a neat approach to writing programs called Test Driven or Test First Development. What if you wrote this test:
Could you write computeSum to make that work? (I believe you already have!) Then try more complex matrices and rectangles to make certain that compute is golden.
How about:
Then bigger matrices with more rectangles. This would separately validate your logic for finding rectangles. I didn't follow every line of what you had, but it looked a bit suspect. I'd be happier with a test in hand to make sure it's golden, too.
Finally, loop through the Set of candidates and keep track of the highest sum found.
Now, since you're brand new to
Java, suggesting additional classes and using Set (from the collection framework in the JDK) may be a bit advanced. Was that useful or just overwhelming?