I can think of two approaches. Either use OpenOffice in server mode to open the file, and get at the cell contents using its Java API.
Or write an RTF grammar for a lexer or parser. It wouldn't need to cover all of RTF, just the parts you're interested in. (The JavaCC parser actually has two RTF grammars available already, so that may be a good start.)