File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
The moose likes Sockets and Internet Protocols and the fly likes problem with search engines Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Sockets and Internet Protocols
Bookmark "problem with search engines" Watch "problem with search engines" New topic

problem with search engines

vas vas

Joined: Dec 13, 2004
Posts: 19
Hello All,

I am having a problem with search engines.
I am able to connect to google and yahoo thro java and retrieve the html code.

say,for google the url is

I am getting the html code when "java" is the query.

But when I click next in google(for next set of results) , the url is ..

When I try to connect to this url , i am getting malformed url exceptions.

It is the same with yahoo.

Hope u understood my problem.
How can i get the html code from this url???

Can anyone help me??
pascal betz
Ranch Hand

Joined: Jun 19, 2001
Posts: 547
perhaps you can show what you coded so far ?

Jeroen Wenting
Ranch Hand

Joined: Oct 12, 2000
Posts: 5093
read up on URL encoding.

Ernest Friedman-Hill
author and iconoclast

Joined: Jul 08, 2003
Posts: 24199

We have a whole forum devoted to Sockets and Internet Protocols which I think is a god fit for your problem. I'm moving this thread over there.

[Jess in Action][AskingGoodQuestions]
vas vas

Joined: Dec 13, 2004
Posts: 19
Hi, this is my code.I am trying to display the html code of google search results.I am using command line arguments to give input.

i have executed like

java Demo


class Demo {

public static void main(String[] args) throws Exception {
URL url = new URL(args[0]);
URLConnection conn = url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = in.readLine()) != null)

Why this is not working?But when i substitute the query in place of args[0] in the code,it is working.Is there anything to do with encoding?

Can anyone help me?
David Harkness
Ranch Hand

Joined: Aug 07, 2003
Posts: 1646
Try printing out args[0] before you create the URL from it. Perhaps your command shell is munging the &s or some other characters.
vas vas

Joined: Dec 13, 2004
Posts: 19
What you said is exactly right.

when I execute like java Demo "url" it is working.May be the problem is with "&".

thank you so much.I appreciate your help.
I agree. Here's the link:
subject: problem with search engines
It's not a secret anymore!