You cannot change to JSP anyway. JSP is not supported in JSF2.
Your problem comes from not protecting your resources properly.
Any resource within a web application (WAR/EAR) can be directly accessed via a URL request unless it meets 1 of 2 constraints:
1. The resource is located under the WEB-INF directory. Anything in WEB-INF or its children will not be used to resolve a URL request, per the J2EE standard.
2. The resource is protected by the container security system via a suitable access control rule defined in WEB-INF/web.xml.
In the usual course of events, you would employ constraint #2 to forbid direct access to "*.xhtml" URLs. Since the generally-accepted web.xml config options for JSF formulate JSF URLS in the format of "*.jsf" or "/faces/*", the webapp container (WAS) will ensure that no one can retrieve the raw xhtml sources.
An IDE is no substitute for an Intelligent Developer.