I'm in the thinking about starting to plan a possible prototype of a system for regression testing my Servlets.
I've finally made it past command line and Java programs that can do Shibboleth Authentication. Shib is very cool from the server side and a downright PIA from the client side if not using a browser.
What I have in the prototype stage is a set of bash scripts that use curl to pull stuff, and starting on some Java stuff that goes deeper. This handles redirects (3xx status codes) and reports errors on anything that doesn't return 200.
I'm thinking that the servlets could return something other than 200 if something other than "here's what you asked for" needs to be tagged. My question is what would be the best return code?
Specifically my app "visualizes data from a large data store". Requests can be something like make these 10 plots on these 20 datasets. As an aside, my biggest current problem is that it's way too easy to ask for something that takes a month to calculate, but that's a topic for another thread.
What I'd like to do is make it easy for regression tests to be able to tell the difference between "here's everything you requested", "here's part of what your requested", "some 500 code", and "WTF do you want"?
It would be nice if those returns could be coded in the HTML status but I'm loathe to invent my own numbers.
What would you use?
It's not what your program can do, it's what your users do with the program.
There's a useful little Apache project called JMeter that can probably do what you're looking for, so you wouldn't have to roll-your-own test tooling. It's usually used for load and performance testing, but you can use it for functional/regression tests as well. A web test plan allows you to configure basically anything you need for the HTTP request, and you can run all sorts of assertions against the HTTP response. You wouldn't have to be limited to response codes, as you could verify the response body quite thoroughly.
Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life.
I've been reading the jMeter documentation and working on my first test plan. I'm still trying to figure out how to do SAML (Shibboleth) or Kerberos (SPNEGO) authentication without putting my password in the test plan.
It does seem very powerful although I can't quite get started.