• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

emulating a synchronous html request

 
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hello, i found this forum entry:
https://coderanch.com/t/517587/oa/Web-Crawler-Java

and looked over this link:
http://java-source.net/open-source/html-parsers

I am looking for something unusual. I wish to automatically parse the HTML (and possibly get a screenshot) of some websites that seem to be circumventing asynchronous HTML requests.

Some sites are built so that if you use one of the many asynchronous methods that are provided in many programming examples that you always return a page that doesn't match what is currently there. It is like they are "masking" their page from automated parsing.

(If you attempt to do a GetURL, or LoadStrings or all of the other methods that I have seen, you make an asynchronous request. The sites in question thwart it by rendering content you don't want.)

Is it possible to emulate a synchronous HTML request and still get the benefits of HTML parsing and/or screenshots? Thanks.

 
Fred Scott
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hi,

i hope i'm not bumping this thread too soon. However, does anyone have any thoughts about my question?

thanks,
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic