File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Other Open Source Projects and the fly likes HTMLUnit unable to process Javascript? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Products » Other Open Source Projects
Bookmark "HTMLUnit unable to process Javascript?" Watch "HTMLUnit unable to process Javascript?" New topic
Author

HTMLUnit unable to process Javascript?

Mike Cheung
Ranch Hand

Joined: Feb 01, 2013
Posts: 58
Hi, am trying to see if I can make a GWT site more crawlable by detecting whenever a web bot visits, I'd invoke HtmlUnit to download (internally) the web page and return to the bot.
Just tried running it against SmartClient's demo site using the following code but it gave me a lot of errors, ranging from a complain of 'text/javascript' being obsolete to CSS problems, etc.
Has anyone tried using HTMLUnit to check a site with Javascript successfully? If yes any idea why this is happening?







Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 41073
    
  43
The only error I see is the UnknownHostException - make sure that host is not blocked from the machine where this code runs. That seems a network issue, though, not anything to do with HtmlUnit.

That the Assert in line 15 fails would obviously be expected.


Ping & DNS - my free Android networking tools app
Mike Cheung
Ranch Hand

Joined: Feb 01, 2013
Posts: 58
Thanks but that URL was copied from my browser window after having found the site and loaded it in the browser so it's accessible.

And isn't it throwing warnings about JavaScript? So it definitely have loaded some of the JavaScripts.

It is also complaining about CSS with the following...
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 41073
    
  43
If the server sends malformed CSS then there's nothing HtmUnit can do about that. But for the purposes HtmlUnit is used CSS problems are generally inconseqeuntial. I think there's some configuration setting through which CSS error reporting can be turned off.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
 
subject: HTMLUnit unable to process Javascript?
 
Similar Threads
htmlunit - first use - I get big bunch of Warnings when I load the page. Is that normal?
Retrieving word lists using htmlunit and xpath
How to login to javaranch with HtmlUnit?
JsfUnit testing
text/javascript obsolete?