Win a copy of Svelte and Sapper in Action this week in the JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Paul Clapham
  • Bear Bibeault
  • Junilu Lacar
Sheriffs:
  • Jeanne Boyarsky
  • Tim Cooke
  • Henry Wong
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • salvin francis
  • Frits Walraven
Bartenders:
  • Scott Selikoff
  • Piet Souris
  • Carey Brown

problem with search engines

 
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello All,

I am having a problem with search engines.
I am able to connect to google and yahoo thro java and retrieve the html code.

say,for google the url is http://www.google.com/search?q=java

I am getting the html code when "java" is the query.

But when I click next in google(for next set of results) , the url is ..


http://www.google.com/search?q=java&hl=en&lr=&start=10&sa=N

When I try to connect to this url , i am getting malformed url exceptions.

It is the same with yahoo.

Hope u understood my problem.
How can i get the html code from this url???

Can anyone help me??
 
Ranch Hand
Posts: 547
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
perhaps you can show what you coded so far ?

pascal
 
Ranch Hand
Posts: 5093
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
read up on URL encoding.
 
author and iconoclast
Posts: 24203
43
Mac OS X Eclipse IDE Chrome
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
We have a whole forum devoted to Sockets and Internet Protocols which I think is a god fit for your problem. I'm moving this thread over there.
 
vas vas
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi, this is my code.I am trying to display the html code of google search results.I am using command line arguments to give input.

i have executed like

java Demo http://www.google.com/search?q=java&hl=en&lr=&start=10&sa=N

import java.net.*;
import java.io.*;

class Demo {

public static void main(String[] args) throws Exception {
URL url = new URL(args[0]);
URLConnection conn = url.openConnection();
conn.setRequestProperty("User-Agent","");
conn.connect();
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = in.readLine()) != null)
{
System.out.print(line);
}
}
}



Why this is not working?But when i substitute the query in place of args[0] in the code,it is working.Is there anything to do with encoding?

Can anyone help me?
 
Ranch Hand
Posts: 1646
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Try printing out args[0] before you create the URL from it. Perhaps your command shell is munging the &s or some other characters.
 
vas vas
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What you said is exactly right.

when I execute like java Demo "url" it is working.May be the problem is with "&".

thank you so much.I appreciate your help.
 
Clowns were never meant to be THAT big! We must destroy it with this tiny ad:
Thread Boost feature
https://coderanch.com/t/674455/Thread-Boost-feature
    Bookmark Topic Watch Topic
  • New Topic