There's a little bit of confusion with "xhtml". Traditionally, xhtml is strict-form XML HTML as defined by the W3C. However, a View Definition Language - also known as View Template Language - was created as part of the Facelets project, and the VDL files are also in XML format and carry the ".xhtml" file extension, even though they are not, strictly speaking XHTML.
If you want to output XHTML, that's just a matter of coding your view template properly. To the best of my knowledge, JSF's HTML renderers all create HTML that conforms to the XHTML standard.
On the other hand, if what you really want is to define views using VDL, your webapp must include the facelets jar available from the Facelets project, and you must configure Facelets in your web.xml.
Facelets has proven to be very useful. So useful, in fact, that VTL is the standard for defining Views in JSF2, Facelets support is built in to the JSF2 standard, and JSP Views are deprecated.
An IDE is no substitute for an Intelligent Developer.
In fact, if you're just getting started do you have a pressing reason for using JSF 1.2 such as an employer
requirement? If not it would save you a fair amount of messing about by just using JSF 2. Another benefit
is that you can use a Java EE 6 server.