This week's book giveaway is in the OCAJP 8 forum. We're giving away four copies of OCA Java SE 8 Programmer I Study Guide and have Edward Finegan & Robert Liguori on-line! See this thread for details.
A solid understanding of the basics is always beneficial. Web apps are built on HTTP, and that fact tends to come back to bite you in the most inconvenient moments. Servlets let you get pretty close to HTTP, so I think yes, it does pay off to learn about them. Some facts of JSPs are also much easier to understand if one has a grasp of how they are compiled into servlets.
I'd definitely say learn the basics though I prefer to use a framework.
I think most frameworks refer back to the servlet design/logic basics either to explain the inner workings of the framework itself (struts documentation describes its controller servlet and compares JSP include logic with Tiles) or how the framework aims to overcome perceived problems with Servlets and JSP.
Do all Java web developers eventually gravitate to a framework though ?
I think you definitely need to know servlets. Sometimes you need to write a very simple app where the overhead of a framework is simply not worth it. Other times you need to access the servlet layer directly from a framework. J2EE is and probably always will be either directly based on or very closely tied with the servlet API and I don't see how you can avoid knowing such basics as a good engineer.
Sometime, you also need to get your hands dirty and do some low level stuff. And then, the best technology often remains the servlet and plain old Java. Some examples might be : - serving binary content, for instance images. Applications might be watermarking images, applying transformation on images or generating images on the fly. - generating views that are highly recursive. For instance, rendering a complex navigation bar is a pain in JSP or any view technology, but very straightforward in Java (through include mechanisme) or a custom tag. The first is often easier when you need a real controller to prepare the structure before rendering it in a "Java view". - doing some very specific stuff that falls outside a framework.
Besides, some framework do not protect you from the details of how servlet works. For instance, Spring MVC controllers are pretty close to a servlet. I don't think you can do anything serious in them if you have no idea on how a servlet works.