The java.net.URL is the package we use to read from and write to existing web pages. I have a Java program that imports the java.net.URL and reads and extracts some information from web pages. However, it is only the latter half of the task. How does a Java program interact with; for example, cgi-bin scripts, on the server side provide that I have no idea about the contents of the cgi-bin scripts? What I mean is that: Many HTML pages contain forms-- text fields and other GUI objects that let us enter data to send to the server. After we type in the required information and initiate the query by clicking a button, the Web browser writes the data to the URL over the network. At the other end, a cgi-bin script (usually) on the server receives the data, processes it, and then sends us "a response", usually in the form of a new HTML page. As I stated in the beginning of this post, I have prepared a Java program to read the responding HTML page.
My question is regarding the first half of the process. How do I feed data in my Java program that initiate the query (instead of manually entering data in the text field) over the network for the host server to process? And then the Java program continues its work in reading the HTML page returned by the server? Provided that I do not know what kind of the program the hosting server uses to process the query.
To make a request to a URL, URLConnection object. Well, the same can also be used to submit data to the url. The simplest way would be to submit to the URL using query strings.. you know the "?name=..." kinda format after the url. If there is anything more specific, please let me know. Regds Lupo
How do I feed data in my Java program that initiate the query (instead of manually entering data in the text field) over the network for the host server to process?
This is more or less straightforward:
Joined: Nov 19, 2000
Thanks for your attention. The example can be viewed in the link below: http://www.netsol.com/cgi-bin/whois/whois At this web site, we can enter a domain name; say, yahoo.com, in the text field and click on the GO button. Then this HTML web page http://www.netsol.com/cgi-bin/whois/whois?STRING=yahoo.com&SearchType=do&STRING2.x=32&STRING2.y=10 is returned by the host server. I am supposed to get the e-mail address of the Administrative Contact in that page. I have to put the whole process in a Java program because I have to get the e-mail address of the Administrative Contact for a long list of domain names. I have already prepared a Java program to retrieve the e-mail address of the Administrative Contact at the http://www.netsol.com/cgi-bin/whois/whois?STRING=yahoo.com&SearchType=do&STRING2.x=32&STRING2.y=10 I do not know the program that the host server uses to process the inquiry. My question is 1. how to feed domain names (of course one at a time in a loop) in a Java program for http://www.netsol.com/cgi-bin/whois/whois to receive as input? And 2. how do I know the URL of the HTML web page that is to be returned by the host server so that I can use my Java program to scan for the e-mail address of the Administrative Contact? I am also thinking about logging into a remote web site that uses basic authentication or form based authentication and hold the session with a Java program. Thanks in advance. [ January 02, 2003: Message edited by: JiaPei Jen ] [ January 02, 2003: Message edited by: JiaPei Jen ]
Joined: Oct 08, 2001
I do not know the program that the host server uses to process the inquiry.
You don't need to know it whatsoever, all you need is an URL.
Hello, Eugene, thanks for your responses to my questions. I tried you code; however, something gets screwed up: http://www.netsol.com/cgi-bin/whois/whois is the web site where I enter the query data in the text field. Suppose I enter; say, yahoo.com, and click on the GO button, I get this HTML web page with the link shown below in the address bar: