wood burning stoves*
The moose likes Sockets and Internet Protocols and the fly likes problem with search engines Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Sockets and Internet Protocols
Bookmark "problem with search engines" Watch "problem with search engines" New topic
Author

problem with search engines

vas vas
Greenhorn

Joined: Dec 13, 2004
Posts: 19
Hello All,

I am having a problem with search engines.
I am able to connect to google and yahoo thro java and retrieve the html code.

say,for google the url is http://www.google.com/search?q=java

I am getting the html code when "java" is the query.

But when I click next in google(for next set of results) , the url is ..


http://www.google.com/search?q=java&hl=en&lr=&start=10&sa=N

When I try to connect to this url , i am getting malformed url exceptions.

It is the same with yahoo.

Hope u understood my problem.
How can i get the html code from this url???

Can anyone help me??
pascal betz
Ranch Hand

Joined: Jun 19, 2001
Posts: 547
perhaps you can show what you coded so far ?

pascal
Jeroen Wenting
Ranch Hand

Joined: Oct 12, 2000
Posts: 5093
read up on URL encoding.


42
Ernest Friedman-Hill
author and iconoclast
Marshal

Joined: Jul 08, 2003
Posts: 24183
    
  34

We have a whole forum devoted to Sockets and Internet Protocols which I think is a god fit for your problem. I'm moving this thread over there.


[Jess in Action][AskingGoodQuestions]
vas vas
Greenhorn

Joined: Dec 13, 2004
Posts: 19
Hi, this is my code.I am trying to display the html code of google search results.I am using command line arguments to give input.

i have executed like

java Demo http://www.google.com/search?q=java&hl=en&lr=&start=10&sa=N

import java.net.*;
import java.io.*;

class Demo {

public static void main(String[] args) throws Exception {
URL url = new URL(args[0]);
URLConnection conn = url.openConnection();
conn.setRequestProperty("User-Agent","");
conn.connect();
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = in.readLine()) != null)
{
System.out.print(line);
}
}
}



Why this is not working?But when i substitute the query in place of args[0] in the code,it is working.Is there anything to do with encoding?

Can anyone help me?
David Harkness
Ranch Hand

Joined: Aug 07, 2003
Posts: 1646
Try printing out args[0] before you create the URL from it. Perhaps your command shell is munging the &s or some other characters.
vas vas
Greenhorn

Joined: Dec 13, 2004
Posts: 19
What you said is exactly right.

when I execute like java Demo "url" it is working.May be the problem is with "&".

thank you so much.I appreciate your help.
 
wood burning stoves
 
subject: problem with search engines
 
Similar Threads
The page cannot be displayed
Who is your worst enemy??
Browser Back button problem using struts.
a VB Question
problem with search engines