• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

HttpURLConnection

 
Ranch Hand
Posts: 150
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,

I want to read content of the web site through java program. I am using HttpURLConnection class. But I am unable to view the contents. I got code from net "Html source viewer". But what if website is in php, jsp etc.
Any pointers ??

Thanks,
Anand
 
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Reading the content of a web page could be as simple as:


It doesn't matter whether the web site is written in JSP, PHP or Fortran, because either way you're going to get HTML delivered back to you.
[ March 22, 2007: Message edited by: Ulf Dittmer ]
 
Anand Loni
Ranch Hand
Posts: 150
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Ulf,

Solution given by you is not working. See the stack trace below :
java.net.ConnectException: Connection timed out: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.Socket.connect(Socket.java:507)
at java.net.Socket.connect(Socket.java:457)
at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:365)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:477)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:214)
at sun.net.www.http.HttpClient.New(HttpClient.java:287)
at sun.net.www.http.HttpClient.New(HttpClient.java:299)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:792)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:744)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:669)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:913)
at java.net.URLConnection.getContent(URLConnection.java:682)
at java.net.URL.getContent(URL.java:1021)
at Tp.main(Tp.java:20)
 
Java Cowboy
Posts: 16084
88
Android Scala IntelliJ IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Are you behind a proxy?

Connect through a Proxy
Java Tip 42: Write Java apps that work with proxy-based firewalls

The official documentation: Networking Properties
 
Anand Loni
Ranch Hand
Posts: 150
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks a lot, Jesper.
 
Ranch Hand
Posts: 32
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am not able to execute below program. Please help.
I have set my proxies as class variables which i did not specify here.
When i used this URL in browser it is working perfectly fine.
https://chart.googleapis.com/chart?cht=lc&chtt=This%20is%20%7C%20my%20graph&chs=300x300&chxt=y&chd=t%3A40%2C20%2C50%2C20%2C200





When executed this code. It is throwing ConnectionRefused Exception..


java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(Unknown Source)
at com.sun.net.ssl.internal.ssl.BaseSSLSocketImpl.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.<init>(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.New(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(Unknown Source)
at com.perse.swing.browser.SSlPost.getPOSTUrl(SSlPost.java:108)
at com.perse.swing.browser.SSlPost.main(SSlPost.java:35)


 
Ulf Dittmer
Rancher
Posts: 43081
77
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
What does "not able to execute" mean? What, exactly, are you trying to do, and what, exactly, is the result? TellTheDetails
 
Sri Palem
Ranch Hand
Posts: 32
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I edited my previous post and pasted exception there. Please have a look.
 
Ulf Dittmer
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Does the proxy handle HTTPS? Have you tried with HTTP instead of HTTPS?

Also, use System.setProperty("...", "...") instead of your approach.
 
Sri Palem
Ranch Hand
Posts: 32
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thank you so much . I used http instead of https and it worked.
 
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
connectionURL = new URL("http://" + localAddr + reportUrl);

logger.debug("ReportThread ConnectionURL: " + connectionURL+connectionURL.getPort());
HttpURLConnection httpURLConnection;
try {
httpURLConnection = (HttpURLConnection) connectionURL.openConnection();
httpURLConnection.setDoInput(true);
httpURLConnection.setDoOutput(true);
httpURLConnection.setRequestMethod("GET");
OutputStream outputStream = httpURLConnection.getOutputStream();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(httpURLConnection.getInputStream(),"ISO-8859-1"));






here am getting below exception why it is getting and also the host name how i will get..?

java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.Socket.connect(Socket.java:524)
at java.net.Socket.connect(Socket.java:474)
at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:388)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:523)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:231)
at sun.net.www.http.HttpClient.New(HttpClient.java:304)
at sun.net.www.http.HttpClient.New(HttpClient.java:321)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:839)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:791)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:716)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:883)
at com.ys.drx.ui.web.reports.actions.ReportThread.run(ReportThread.java:61)
 
soumya Uppula
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

soumya Uppula wrote:connectionURL = new URL("http://" + localAddr + reportUrl);

logger.debug("ReportThread ConnectionURL: " + connectionURL+connectionURL.getPort());
HttpURLConnection httpURLConnection;
try {
httpURLConnection = (HttpURLConnection) connectionURL.openConnection();
httpURLConnection.setDoInput(true);
httpURLConnection.setDoOutput(true);
httpURLConnection.setRequestMethod("GET");
OutputStream outputStream = httpURLConnection.getOutputStream();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(httpURLConnection.getInputStream(),"ISO-8859-1"));






here am getting below exception why it is getting and also the host name how i will get..?

java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.Socket.connect(Socket.java:524)
at java.net.Socket.connect(Socket.java:474)
at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:388)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:523)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:231)
at sun.net.www.http.HttpClient.New(HttpClient.java:304)
at sun.net.www.http.HttpClient.New(HttpClient.java:321)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:839)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:791)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:716)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:883)
at com.ys.drx.ui.web.reports.actions.ReportThread.run(ReportThread.java:61)

Untitled.png
[Thumbnail for Untitled.png]
 
Saloon Keeper
Posts: 7582
176
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Some questions: 1) Is the URL correct? 2) Is it reachable from wherever this code is run? 3) Does or does it not work with a publicly reachable URL you can share with us?
 
reply
    Bookmark Topic Watch Topic
  • New Topic