• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
  • Campbell Ritchie
  • Ron McLeod
  • Rob Spoor
  • Tim Cooke
  • Junilu Lacar
  • Henry Wong
  • Liutauras Vilda
  • Jeanne Boyarsky
Saloon Keepers:
  • Jesse Silverman
  • Tim Holloway
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
  • Al Hobbs
  • Mikalai Zaikin
  • Piet Souris

How to capture response sent from a website

Ranch Hand
Posts: 44
Eclipse IDE Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello everyone,

I tried to search this topic on the internet but couldn't get any proper solution. I want to open a page through my Servlet and capture response that is received from the server so that I could alter/retrieve the data I get from the response. I am not talking about filtering the data sent from my own servlet but getting the response from other websites.

Any suggested reading would be greatly appreciated.

Basically, I was trying to extract all the hrefs present on a page through FileIO. But that can be tedious process as I had to download the pages before-hand. Is there any other way to do this? Please advice.
Author and all-around good cowpoke
Posts: 13078
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Sure, look at the java.net package, specifically the URLConnection and HttpURLConnection classes.

The concept used to be called "Screen scraping" - back when terminal displays were simple. Now I think "web crawler" is what you want.

Of course, once you have the initial HTML text you still have a lot of work to do to extract all the linked resources.

snakes are really good at eating slugs. And you wouldn't think it, but so are tiny ads:
Thread Boost feature
    Bookmark Topic Watch Topic
  • New Topic