Meaningless Drivel is fun!
The moose likes Other JSE/JEE APIs and the fly likes How to capture response sent from a website Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

Win a copy of Java Interview Guide this week in the Jobs Discussion forum!
JavaRanch » Java Forums » Java » Other JSE/JEE APIs
Bookmark "How to capture response sent from a website" Watch "How to capture response sent from a website" New topic

How to capture response sent from a website

Nelo Angelo
Ranch Hand

Joined: Jul 25, 2011
Posts: 44

Hello everyone,

I tried to search this topic on the internet but couldn't get any proper solution. I want to open a page through my Servlet and capture response that is received from the server so that I could alter/retrieve the data I get from the response. I am not talking about filtering the data sent from my own servlet but getting the response from other websites.

Any suggested reading would be greatly appreciated.

Basically, I was trying to extract all the hrefs present on a page through FileIO. But that can be tedious process as I had to download the pages before-hand. Is there any other way to do this? Please advice.

I love java but she hates me... :'(
William Brogden
Author and all-around good cowpoke

Joined: Mar 22, 2000
Posts: 13037
Sure, look at the package, specifically the URLConnection and HttpURLConnection classes.

The concept used to be called "Screen scraping" - back when terminal displays were simple. Now I think "web crawler" is what you want.

Of course, once you have the initial HTML text you still have a lot of work to do to extract all the linked resources.

I agree. Here's the link:
subject: How to capture response sent from a website
It's not a secret anymore!