• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

How to capture response sent from a website

 
Ranch Hand
Posts: 44
Eclipse IDE Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello everyone,

I tried to search this topic on the internet but couldn't get any proper solution. I want to open a page through my Servlet and capture response that is received from the server so that I could alter/retrieve the data I get from the response. I am not talking about filtering the data sent from my own servlet but getting the response from other websites.

Any suggested reading would be greatly appreciated.

Basically, I was trying to extract all the hrefs present on a page through FileIO. But that can be tedious process as I had to download the pages before-hand. Is there any other way to do this? Please advice.
 
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Sure, look at the java.net package, specifically the URLConnection and HttpURLConnection classes.

The concept used to be called "Screen scraping" - back when terminal displays were simple. Now I think "web crawler" is what you want.

Of course, once you have the initial HTML text you still have a lot of work to do to extract all the linked resources.

Bill
 
Consider Paul's rocket mass heater.
reply
    Bookmark Topic Watch Topic
  • New Topic