Win a copy of Clojure in Action this week in the Clojure forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Getting web pages from an app.

 
Bob Burnquist
Greenhorn
Posts: 13
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
My question is, what is the best way to call a number of web pages from a java class.

I put this in the servlets section but then realisied my code does not need to be a servlet as this application will be called by a cron job.

It needs to go through a list and make requests to a lot of websites so that the urls can be cached server-side. Im not interested in rendering any of these websites, as long as the pages are sent from the server.

This easy to do through sockets?

Thanks loads.
[ March 10, 2006: Message edited by: Sebastian Patten ]
 
Vishnu Prakash
Ranch Hand
Posts: 1026
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Looks more appropriate to in servlet forum.
 
Stan James
(instanceof Sidekick)
Ranch Hand
Posts: 8791
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If you just want a Java client that can pull pages from web sites and write them to disk it's pretty simple to do. Look into the URL class which can get you a URLConnection. From there you get the stream and read whatever is at that address. For HTML you can probably use a buffered reader with no problem. For images or other binary files stick with binary streams.

Take a look at the doc and see what you can put together. If you get stuck, show us some code and we'll try to get you going again.

Keep it fun!
 
I agree. Here's the link: http://aspose.com/file-tools
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic