JSP would be the way to go if the XHTML is to be served to a browser. Any actual file I/O (and further processing, if there is any) should be done in a backing bean or a servlet, though, not in the JSP itself.
If this was my problem I would get an example of the desired XHTML text and study it to find regular features which could turn into Java classes and methods. Since an XHTML page forms a DOM there will be a logical hierarchy.
Try to locate the point at which a line of CSV gets turned into XHTML - is this a line in a Table or what.
Is the final result entirely derived from the CSV or is their other content which must be included?