posted 9 years ago
Hi, am trying to see if I can make a GWT site more crawlable by detecting whenever a web bot visits, I'd invoke HtmlUnit to download (internally) the web page and return to the bot.
Just tried running it against SmartClient's demo site using the following code but it gave me a lot of errors, ranging from a complain of 'text/javascript' being obsolete to CSS problems, etc.
Has anyone tried using HTMLUnit to check a site with Javascript successfully? If yes any idea why this is happening?