I was wondering what your thoughts are on Container based testing approaches like Arquillian (http://www.jboss.org/arquillian.html) and where they fit into your testing strategy. Is it something that you use alongside other testing frameworks (Junit/TestNG), a replacement for them for testing out certain integration points, or a competing platform where you choose one or the other to accomplish your tests.
If your answer is the last, I'd also be curious to know what you thing the advantages and disadvantages are - and which one produces more of a real-world test that leaves less surprises for when code is deployed to a real environment.
I haven't used container testing for a long time, and I haven't (yet! even though I definitely plan to) used Arquillian, which seems to be a very powerful and useful testing tool (or more than this). Because of this, I do not want to pretend I can answer your question as it deserves to be answered. All I can say here is that, from different kind of tests that your application should undergo, Arquillian seems to address a very important and difficult part. It promises (and from what I heard also delivers) a way to test the integration aspect with a short feedback loop and in a very realistic environment (less mocking more real things).
Addressing your other question, I think that every application benefits from different kind of tests (unit, integration, end-to-end) but depending on the type of application the usefulness of each of these types vary. I can imagine some simple web apps being successfully tested with almost only end-to-end tests, and a complicated library with not much frontend which is tested by unit and integration tests.
Currently I test my applications on three levels:
* unit tests (with mocks) which verify business logic of classes
* integration tests (with Spring context and real database) - verification of things like correctness of database queries,
* end to end tests - applications set up with Cargo, database filled with data, real requests via user interface
Book author: Practical Unit Testing with TestNG and Mockito
I've used Arquillian alongside with standard JUnit tests in a recent project and I found it quite helpful when it comes to working out the initial skeleton of the web application, just as described in the book Growing Object-Oriented Software, Guided by Tests. Arquillian seems to ideally fit into this kind of testing as it let's you build and test archives on-the-fly and defer final implementations of interfaces to a later phase when you're confident that the API is sensible.
One thing that Arquillian couldn't protect us from, were the errors that cropped out during actual implementation of APIs and frontend/backend integration but that's mainly because our infrastructure is not capable of providing full, proper testing environment (with databases, LDAPs, etc.) yet.
On the other hand, providing frontend developers with "fake" Arquillian web application archives made it possible to parallelize the work on frontend and backend. Which was a very good thing IMHO. :-)