• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

problem with search engines

 
vas vas
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello All,

I am having a problem with search engines.
I am able to connect to google and yahoo thro java and retrieve the html code.

say,for google the url is http://www.google.com/search?q=java

I am getting the html code when "java" is the query.

But when I click next in google(for next set of results) , the url is ..


http://www.google.com/search?q=java&hl=en&lr=&start=10&sa=N

When I try to connect to this url , i am getting malformed url exceptions.

It is the same with yahoo.

Hope u understood my problem.
How can i get the html code from this url???

Can anyone help me??
 
pascal betz
Ranch Hand
Posts: 547
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
perhaps you can show what you coded so far ?

pascal
 
Jeroen Wenting
Ranch Hand
Posts: 5093
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
read up on URL encoding.
 
Ernest Friedman-Hill
author and iconoclast
Marshal
Pie
Posts: 24208
35
Chrome Eclipse IDE Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
We have a whole forum devoted to Sockets and Internet Protocols which I think is a god fit for your problem. I'm moving this thread over there.
 
vas vas
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi, this is my code.I am trying to display the html code of google search results.I am using command line arguments to give input.

i have executed like

java Demo http://www.google.com/search?q=java&hl=en&lr=&start=10&sa=N

import java.net.*;
import java.io.*;

class Demo {

public static void main(String[] args) throws Exception {
URL url = new URL(args[0]);
URLConnection conn = url.openConnection();
conn.setRequestProperty("User-Agent","");
conn.connect();
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = in.readLine()) != null)
{
System.out.print(line);
}
}
}



Why this is not working?But when i substitute the query in place of args[0] in the code,it is working.Is there anything to do with encoding?

Can anyone help me?
 
David Harkness
Ranch Hand
Posts: 1646
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Try printing out args[0] before you create the URL from it. Perhaps your command shell is munging the &s or some other characters.
 
vas vas
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What you said is exactly right.

when I execute like java Demo "url" it is working.May be the problem is with "&".

thank you so much.I appreciate your help.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic