• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Jeanne Boyarsky
  • Tim Cooke
Sheriffs:
  • Liutauras Vilda
  • paul wheaton
  • Henry Wong
Saloon Keepers:
  • Tim Moores
  • Tim Holloway
  • Stephan van Hulst
  • Carey Brown
  • Frits Walraven
Bartenders:
  • Piet Souris
  • Himai Minh

Need to Unit test an app that submits a form to a webpage

 
Ranch Hand
Posts: 167
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hey All,
I'm developing an app (a bot really) that retrieves information from webpages by submitting forms to the page. I'm wondering how can I unit test this feature.
Just to make it clear, I do not wish to unit test the servlet/jsp that the form is being submitted to. Rather I am trying to unit test the bot that will do the submitting. Any help would be appreciated.
Sincerely,
Jon McDonald
 
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Some issues/hints...
* Network code can be vulnerable to thread timing issues which can make your tests fail intermittently every five or fifty or five hundred times.
* If you don't control the target website, there tends to be a bit of trial and error in finding the correct url and html formats... tests are a good way of being systematic about this trial and error... if a page breaks your bot in trials, you add it to your tests and fix your bot until it passes the tests.
* There's a question of how heavy do you have to make the test framework... if scale wasn't an issue, we might test our cnn.com headline scraping bots by simulating the whole cnn.com website in our test harness.
* Your test code might be more robust and manageable if you test the following areas of functionality separately (assuming that your bot has a structure along these lines...)
- mapping of information request to url
- the http client itself (mapping of url to html page)
- mapping of html page to list of headlines or whatever (html parsing)
 
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Jon McDonald:
I'm developing an app (a bot really) that retrieves information from webpages by submitting forms to the page. I'm wondering how can I unit test this feature.


Basically (and generally), divide and conquer: Identify the different steps you need to do and test them separately.
For example, you probably need to parse HTML pages for the information you want. You can develop and test this parser without any thought on sending forms over the net etc.
Did that help?
 
Jon McDonald
Ranch Hand
Posts: 167
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the info guys!
I think I'll go the divide and conquer route. It should also make for more reusable code that way.
Jon
 
Ilja Preuss
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Jon McDonald:
It should also make for more reusable code that way.


Yes!
In fact, that is one of the reasons why many developers nowadays find that taking testing into account from early on (often even before writing the actual production code) makes for better designs: for code to be testable, it needs to be well decoupled.
See http://c2.com/cgi/wiki?TestDrivenDevelopment
 
You totally ruined the moon. You're gonna hafta pay for that you know. This tiny ad agrees:
free, earth-friendly heat - a kickstarter for putting coin in your pocket while saving the earth
https://coderanch.com/t/751654/free-earth-friendly-heat-kickstarter
reply
    Bookmark Topic Watch Topic
  • New Topic