This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes I/O and Streams and the fly likes Reading HTML Surcecode for a URL  Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Java » I/O and Streams
Bookmark "Reading HTML Surcecode for a URL  " Watch "Reading HTML Surcecode for a URL  " New topic
Author

Reading HTML Surcecode for a URL

omkar wadkar
Greenhorn

Joined: Jun 13, 2010
Posts: 6
I am trying to make a HTML page scrapping application .In order to do so I want to read the HTML source for a given URL .
I am using following code


The problem is when i am passing http://www.nextag.com/camera as a search parameter ,Its not showing the entire source .
<head>,<title> are all missing .I am wondering why it behaving when I am putting search parameter along with URL??




Devaka Cooray
ExamLab Creator
Saloon Keeper

Joined: Jul 29, 2008
Posts: 3023
    
  35

It is because http://www.nextag.com/camera has a redirection to http://www.nextag.com/camera/search-html


Author of ExamLab ExamLab - a free SCJP / OCPJP exam simulator
What would SCJP exam questions look like? -- Home -- Twitter -- How to Ask a Question
Rob Spoor
Sheriff

Joined: Oct 27, 2005
Posts: 19656
    
  18

You may want to turn on following redirects for the URLConnection.


SCJP 1.4 - SCJP 6 - SCWCD 5 - OCEEJBD 6
How To Ask Questions How To Answer Questions
omkar wadkar
Greenhorn

Joined: Jun 13, 2010
Posts: 6
Thanks you guys

But my application is taking search parameter as input ,
So I will have to append the search String to the URL .
Any suggestions about it ??
Rob Spoor
Sheriff

Joined: Oct 27, 2005
Posts: 19656
    
  18

The URLEncoder.encode method transforms the value so it can be used in a URL. For instance, space becomes either + or %20, & becomes %26 etc.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Reading HTML Surcecode for a URL
 
Similar Threads
url reader applet
Connection timeout in URL connection
How to read text content not source code from webpage in java ?
how to open a web page and checks the http responce
Java I/O class