Granny's Programming Pearls
"inside of every large program is a small program struggling to get out"
The moose likes Sockets and Internet Protocols and the fly likes Java client querying web page Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Sockets and Internet Protocols
Bookmark "Java client querying web page" Watch "Java client querying web page" New topic

Java client querying web page

Bill Denniston

Joined: Feb 18, 2006
Posts: 5
I'm a greenhorn. I'm an experienced (actually, just old) programmer, but new to Java. I've read Head First Java and am trying to muddle through some simple but real initial applications.

I'm looking for examples of simple java client apps that query (i.e. submit POST requests with form parameters to) web pages. I can find lots about writing servlets to respond to such requests, but haven't found anything that shows how to emulate a browser sending the request. (I have found examples of that perform GET operations by simply opening a URL stream.) Any suggestions on how to explore this functionality would be appreaciated. Thanks.
Bear Bibeault
Author and ninkuma

Joined: Jan 10, 2002
Posts: 63844

Check out the method.

[Asking smart questions] [About Bear] [Books by Bear]
William Brogden
Author and all-around good cowpoke

Joined: Mar 22, 2000
Posts: 13036
You might find the HttpClient toolkit useful for emulating the browser side of web application interaction. For example, it can handle cookies like a browser does.
Frank Carver

Joined: Jan 07, 1999
Posts: 6920
As an alternative, I often use HTTPUnit for this, as described in part 5 of my article series in the JavaRanch Newsletter.

Read about me at ~ Raspberry Alpha Omega ~ Frank's Punchbarrel Blog
Ulf Dittmer

Joined: Mar 22, 2005
Posts: 42965
I second Franks suggestion of HttpUnit. Don't let the fact that it has its roots in unit testing turn you off - it is a very handy tool for accessing web pages, and it can even deal with simple JavaScript. It has its own API that lets you access the contents of the web page, but also presents that web page as a DOM object, so there are a number of possibilities for further processing.
Ben Souther

Joined: Dec 11, 2004
Posts: 13410

Since this has more to do with writing client code in Java than servlets, I'm going to move it to the "Sockets And Internet Protocols" forum.

Java API J2EE API Servlet Spec JSP Spec How to ask a question... Simple Servlet Examples jsonf
I agree. Here's the link:
subject: Java client querying web page
It's not a secret anymore!