• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

HTMLUnit unable to process Javascript?

 
Mike Cheung
Ranch Hand
Posts: 113
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi, am trying to see if I can make a GWT site more crawlable by detecting whenever a web bot visits, I'd invoke HtmlUnit to download (internally) the web page and return to the bot.
Just tried running it against SmartClient's demo site using the following code but it gave me a lot of errors, ranging from a complain of 'text/javascript' being obsolete to CSS problems, etc.
Has anyone tried using HTMLUnit to check a site with Javascript successfully? If yes any idea why this is happening?







 
Ulf Dittmer
Rancher
Posts: 42967
73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The only error I see is the UnknownHostException - make sure that host is not blocked from the machine where this code runs. That seems a network issue, though, not anything to do with HtmlUnit.

That the Assert in line 15 fails would obviously be expected.
 
Mike Cheung
Ranch Hand
Posts: 113
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks but that URL was copied from my browser window after having found the site and loaded it in the browser so it's accessible.

And isn't it throwing warnings about JavaScript? So it definitely have loaded some of the JavaScripts.

It is also complaining about CSS with the following...
 
Ulf Dittmer
Rancher
Posts: 42967
73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If the server sends malformed CSS then there's nothing HtmUnit can do about that. But for the purposes HtmlUnit is used CSS problems are generally inconseqeuntial. I think there's some configuration setting through which CSS error reporting can be turned off.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic