JSF2, as I said, does not support JSPs. However "xhtml" is merely the conventional file extension for View Templates. If someone wanted to, they could pick "jspx" instead. The actual template would be JSPX in name only, however - it still won't compile down to a real JSP Servlet anymore than an "xhtml" view is technically what the W3C defines as XHTML. In practical use, I wouldn't pick "jspx" as an View Template extension, however, since that would obscure reall JSPXs and thereby remove one of the options available for parts of the webapp that don't need JSF services.
JSF-specific code basically is employing any function or data structure other than those in the javax.faces.model packages. There are times when such stuff is necessary, but in most cases, I stuff those functions over in a utility service class. That way, the majority of the backing beans are mostly POJO and I can more easily
test them offline.
Along that vein, there's apparently a lot of very obsolete code out there - plus newer bad code - that over-uses/abuses bindings and listeners. There's almost never a need for an actionListener, for example, The simple action method is POJO and can do almost everything an actionListener normally does.
As far as your actual problem goes, the only thing that makes me suspicious is that you appear to be managing the dialog via backing bean actions. Unless IceFaces is a lot different than RichFaces, most of those functions are client-side (javascript).