I'm working with a friend on an application he's building and he's asked me to do some Automated Testing for him with JMeter. I've never used it before and I feel like I may be overcomplicating things. He wants a bunch of tests done, with very specific criteria, that involve different search parameters added onto the URL. For example, one test would add &hotel=mariotte&pricerange=200-400 and Check that the number of rooms returned was what you'd expect given that price range, and that the value of Room 1 was "The Apple Suite" (information is gathered from an external source and returned as JSON to the app to then be processed). A lot of the tests are like this; very specific. My question is, doesn't this defeat the purpose of automation? Doesn't slightly changing the URL and parameters each time with a specific requirement for the test to pass make it very tedious? Almost as much so as manual testing? I see the point in automating a login, however, where you would have a CSV file with usernames and passwords and you'd set the number of threads you want and each would use a line and try to login...But for something this specific? I'm also finding it hard to automate.
For example, what I'm doing is this right now:
Thread Group --> HTTPRequest element with its own User Parameters element (to specify the additional parameters to tack onto the URL and store things like a variable for the expected Room Name and Result Size) --> A BSF PostProcessor that checks, using Java, if the returned JSON values match the values from the User Parameters.
I feel like I'm overcomplicating things here. Is there an easier way to build something like this? Also...I'm not understanding how flexible these tests need to be. Should a user be able to change, for example, the variable for Result Size and one of the URL parameters to a different value so they can go into JMeter and change the test to different values and it will still execute? Or do JMeter tests just test ONE specific case (ex. Room name is 'Apple Suite' and number of results was 1)? I'm very confused. Any help you can provide would go a long way. Thank you.
However your friend's approach seems to be more functional than load testing his concern on consistency of data being returned can be understood. The only thing I would recommend is switching from BSF approach of JSON validation and extraction to doing it via appropriate JMeter Plugins (you'll need Extras with Libs set), in particular:
Thanks for your reply. Maybe you can help me with this. I've changed from using separate Thread Groups for each test, and now am trying to use one Thread Group with a bunch of Simple Controllers with the test names. Inside these, I have each HTTPRequest, each with it's own User Parameters (for the specific "expected" return values and result sizes for that particular test that come back as JSON) and BSF PostProcessor (to process the JSON). In the top-level Thread Group, I have a User Defined Variables section that holds generic things that all tests use (ex. server name). My problem is, when it executes the User Parameters are acting all wonky. I've done tests to make sure that the variables have been set properly for that specific test, and that the response value is correct for that specific test...however when I use an IF statement in that test's BSF PostProcessor to check the equality of the expected and returned sizes, it always passes...even if I set the expected variable to a different number.
I know both numbers are set properly, so this is very bizarre. The exact same IF statement works in a different workspace where each test is it's own Thread Group. Can someone help me? Thank you much.